1
\$\begingroup\$

I'm writing a Java-Helper library. I want to include a method to zip files together, and this is what I've come up with. I'm using the java.util.zip.* library (ZipOutputStream and ZipEntry specifically).

/**
* This uses the java.util.zip library to zip the given files to the given destination.
*
* @param destination
* @param files
* @throws FileNotFoundException
* @throws IOException
*/
public static void zipFiles(File destination, File... files) throws FileNotFoundException, IOException {
  try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(destination))) {
    byte[] buffer = new byte[1024];
    for (int i = 0; i < files.length; i++) {
      try (FileInputStream in = new FileInputStream(files[i])) {
        out.putNextEntry(new ZipEntry(files[i].getName()));

        // Transfer bytes from the file to the ZIP file
        int length;
        while ((length = in.read(buffer)) > 0) {
          out.write(buffer, 0, length);
        }

        // Complete the entry
        out.closeEntry();
      }
    }
  }
}

Any pointers?

Also, from the code, is this a good buffer size?

byte[] buffer = new byte[1024];

I've seen a lot of disks with a blocksize of 512 KB. Would that be a more "optimal" size?

\$\endgroup\$
1
  • 1
    \$\begingroup\$ Why don't you benchmark that? \$\endgroup\$
    – ZeroOne
    Commented May 11, 2012 at 11:25

2 Answers 2

Reset to default
0
\$\begingroup\$

Well, I benchmarked it and found that a buffer size of 10485760 (10 MB) was fastest... Kind of surprised about that and I'm thinking that's not the most efficient. I think I'll leave it at 1 MB unless someone else has any thoughts.

\$\endgroup\$
4
  • 1
    \$\begingroup\$ Interesting result. Thinking of it, it isn't too easy to come up with a good benchmark for a case like this. You are both reading a file and writing one, and I'd imagine the optimum buffer size is different for reading and writing. The block size of your file system and the hardware you are using are likely to have an influence. You would probably get different results for regular old HDs and SSDs. Also, disks have buffers of fast memory into which they store writes that come in too fast and into which they may prefetch data from adjacent locations when you request data from one location. \$\endgroup\$
    – ZeroOne
    Commented May 11, 2012 at 14:03
  • \$\begingroup\$ Because this is supposed to be for developers who may work on all different types of systems do you think it would be helpful to include the buffer size as a parameter? That way they can benchmark it themselves. \$\endgroup\$
    – kentcdodds
    Commented May 11, 2012 at 14:06
  • 3
    \$\begingroup\$ Unfortunately, I highly doubt anyone would do the benchmarking, unless it would be absolutely critical for the system that the compression algorithm worked as fast as possible. The general principle is that you shouldn't optimize a part of code unless you are sure that that part is the current bottleneck of the system. How big were the differences in your benchmark anyway? Within 5, 10, 50 or 100% of speed up? This seems to be a classical example of a leaky abstraction that the programmer shouldn't need to worry about. \$\endgroup\$
    – ZeroOne
    Commented May 11, 2012 at 14:25
  • \$\begingroup\$ Well, I'm not that surprised by that result. HDs tend to have quite large writer buffers nowadays, so that the actual blocksize does not matter. On the other hand, we have so much memory today, that 10 MB is nothing, esp. not causing memory and garbage allocation problems. Thus, getting as close as possible to your disks write buffer size should be the best. Nevertheless, this is micro-management and as ZeroOne already mentioned, is probably not worth much. \$\endgroup\$
    – mtj
    Commented Jul 10, 2017 at 6:01
0
\$\begingroup\$

The code looks perfect.

The only thing I would do differently is to leave out the FileNotFoundException.

If you are going to write some Javadoc it should mention that the files are stored with only their name, omitting all information about the directories they come from. If you need that feature, passing the files as a LinkedHashMap<String, File> would be my first choice.

Regarding the buffer size, I would probably settle on 8 kB, without measuring. It's small enough not to cause a garbage collection because of a fragmented heap.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.