3
\$\begingroup\$

I currently have the following class that takes a custom file object as an argument and makes a copy of the file locally.

class FileCopier
 def initialize(file_info_object)
  @file_info_object = file_info_object
 end

 def copy
  File.open(local_file_path, 'wb') do |f|
   read_file_in_chunks do |chunk|
     f.write chunk
   end
  end
 end

private

 def local_file_path
  File.join(Rails.root,'tmp/blah.txt')
 end

 def read_file_in_chunks(chunksize=10343)
  index = 0
  size = @file_info_object.size
  path = @file_info_object.path

  while index < filesize
   yield Client.read(path, offset: index, length: chunksize)
   index += chunksize
  end

 end

end

The copy method seems to be fairly straight forward but I was wondering if there is a way to simplify it. It seems difficult to test to me at the moment.

\$\endgroup\$
  • 1
    \$\begingroup\$ Could you tell us more about CustomFileReader? \$\endgroup\$ – 200_success Jan 10 '15 at 7:15
  • \$\begingroup\$ I should have been more specific. My apologies. The CustomFileReader has now been renamed to 'Client'. Basically the client is an HDFS client that is reading from an HDFS directory. I am using the webHDFS gem for the client. webHDFS gem \$\endgroup\$ – Graham Jackson Jan 10 '15 at 14:51
  • \$\begingroup\$ One constraint I have is that I can't connect to HDFS locally. What would be a good way to test the export method? (Rspec 3) \$\endgroup\$ – Graham Jackson Jan 12 '15 at 17:14
1
\$\begingroup\$

Some notes:

  • Use 2-space indentation.
  • f.write chunk. You use parens in other places, be consistent.

Other than these minor issues, I think your code is pretty good. I'd just refactor read_file_in_chunks to use a enumerator instead (more versatile) and a range.

def read_file_in_chunks(chunksize=10343)
  Enumerator.new do |yielder|
    (0...@file_info_object.size).step(chunksize).each do |offset|
      data = Client.read(@file_info_object.path, offset: offset, length: chunksize)
      yielder.yield(data)
    end
  end
end

Using lazy collections (Ruby >= 2.0) it can be simplified even more:

def read_file_in_chunks(chunksize=10343)
  (0...@file_info_object.size).step(chunksize).lazy.map do |offset|
    Client.read(@file_info_object.path, offset: offset, length: chunksize)
  end
end
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.