# Generate random 10% of file to be used in testing with Ruby

I'm new to Ruby, but not to programming. I just need a simple script that, given a text file, will pull out around 10% of the lines at random. Below is what I came up with based upon a Python script I wrote. What do I need to do differently to make it more Ruby-like?

prng = Random.new

File.open('english-words-partial.txt', 'w') {  |f|
if prng.rand >= 0.9
f.write(line)
end
end
}

• "...pull out around 10% of the lines at random" will not do. You must define precisely what that means. You could say, for example, 8%-12% of the lines, but that still doesn't do it, because it lacks the criterion for choosing the percentage in that range. That's your job. You could say a random percentage based on a particular probability distribution (e.g., uniform between 8% and 12%), but what's the point? Better to ask for a specific percentage selected at random (e.g., 10%), and you determine that percentage elsewhere in your code. – Cary Swoveland Jul 14 '14 at 20:59
• The percentage wasn't very important for my use case. I had a large file of data that I was running unit tests against and it was taking a long time to run. My solution was to use a portion of the file (around 10%) to speed up the tests. Since I am learning ruby right now, I took this as an opportunity to see how best to write the script. – pgreen2 Jul 14 '14 at 21:04
• I understand your motivation, but that is beside the point. Inserting the word, "around" makes the question unnecessarily vague. It's like me asking, "what is six plus seven, approximately?". Is fifteen close enough? Ten? Who knows? – Cary Swoveland Jul 14 '14 at 21:10

Here's my shot at it:

File.open('english-words-partial.txt', 'w') do |file|
File.foreach('english-words-full.txt') do |line|
file.puts(line) if rand(10) == 0
end
end


In Ruby, you can add an if statement to the end of a line, to conditionally execute that line. This also works with the unless statement.

Also, the rand function can take a number, and produce that many possible numbers from it. So rand(10) will give you a random number from 0 to 9

In Ruby, you usually want to use { } for 1 line blocks, and do/end for multiline blocks, but they both act slightly different. Theres more on that here: https://stackoverflow.com/questions/5587264/do-end-vs-curly-braces-for-blocks-in-ruby

File.foreach('...') is just a bit shorter and neater then File.readlines('...').each (thanks to @Flambino for this)

And lastly, you can use IO methods on a file object like print and puts, so I liked that better but file.write works too. Thats really personal preference.

• I'd recommend File.foreach instead of File.readlines(...).each given the usage – Flambino Jul 3 '14 at 7:45
• @addison, +1 for proper block usage; +1 for add if to the end of the line; while I like the string formatting with %, it feels like too much here. – pgreen2 Jul 3 '14 at 11:37
• @Flambino, great suggestion to use foreach, I didn't notice that. – pgreen2 Jul 3 '14 at 11:38
• if 0 == rand(10) – Nakilon Jul 4 '14 at 6:43
• Your method can return zero lines, all lines or any number in between. What is your argument for that satisfying the requirement that it, "pull out around 10% of the lines at random"? – Cary Swoveland Jul 14 '14 at 20:53

This is even more random (since the order will change):

File.open('english-words-partial.txt', 'w') {  |f|
f.puts full.sample(full.size/10)
end

• Thank you for the alternative, I would like to add that I agree that yours changes the order of the lines, it also reads the full file into memory which may or may not be desirable. – pgreen2 Jul 5 '14 at 3:57
• There appears to be a rogue downvoter in our midst. – Cary Swoveland Jul 14 '14 at 19:25

Here's a way that uses the method Array#sample and several class methods for reading and writing text files that are defined in the IO class. I have made the following assumptions:

• a specified number of lines are to be chosen at random
• the lines chosen at random are to be kept in order
• the entire file is not to be read into memory
• lines to be written to file are not accumulated in memory before being written

Code

def doit(fname_in, fname_out, nbr)
n_lines = File.foreach(fname_in).reduce(0) { |c, _| c+1 }
return nil if n_lines.zero? || nbr.zero?
ndx = [*(0...n_lines)].sample(nbr).sort
f_out = File.open(FNAME_OUT, 'w')
offset = ndx.shift
File.foreach(fname_in).with_index do |line, i|
if i == offset
f_out.write(line)
offset = ndx.shift
end
break unless offset
end
f_out.close
end


Example

FNAME_IN, FNAME_OUT = 'infile', 'outfile'


Let's first create an input file:

lines =<<_
Now
is
the
time
for
all
good
Rubyists
to
put
aside
work
and
watch
the
World
Cup
final
between
Germany
and
Argentina
_

File.write(FNAME_IN, lines) #=> 116


We can confirm the contents of this file with:

File.foreach(FNAME_IN) { |l| puts l }


Now try it:

doit(FNAME_IN, FNAME_OUT, 10)


Let's see what was written. (Drum roll.)

File.foreach(FNAME_OUT) { |l| puts l }
Now
is
the
for
to
aside
watch
World
Cup
final