0
\$\begingroup\$

I'm trying to read a file and print its byte in hex form, using the below code. The code those it but i think optimization can be done to make it read and print faster, because if i remove the print the timestamp is 53ms+/-Without Print, but printing, it takes 18 minutes to read a file of 29mb. With print

import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.util.concurrent.TimeUnit;

public class cOnvert {
public static void main(String[] args) {
    long startTime = System.nanoTime();
    try {
        try (RandomAccessFile aFile = new RandomAccessFile("filePath", "r")) {
            FileChannel inChannel = aFile.getChannel();
            MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());
            String set;
            for (int i = 0; i < buffer.limit(); i++) {
                byte read = buffer.get();
                set = (String.format("%02X ", (byte) read));
                System.out.print(set);
            }
        }
    } catch (IOException ioe) {
    }
    long endTime = System.nanoTime();
    long elapsedTimeInMillis = TimeUnit.MILLISECONDS.convert((endTime - startTime), TimeUnit.NANOSECONDS);
    System.out.println("Total elapsed time: " + elapsedTimeInMillis + " ms");
}
}

Thank you.

\$\endgroup\$
0
\$\begingroup\$

Writing to the console is incredibly slow.... it also bit me in the past :)

Console is not really meant to print such big data anyway. What's your use case ?


Performance tuning

To reduce the impact on performance, you should try to reduce the number of call to System.out.print(set) to only one, at the end of the run....

You may end with a very big String however, so you should use a StringBuilder instead.

Combining both those things should reduce the time taken by your application by a very noticeable amount and you should end up with something looking like this :

StringBuilder sb = new StringBuilder();
// some more code
try (RandomAccessFile aFile = new RandomAccessFile("filePath", "r")) {
        FileChannel inChannel = aFile.getChannel();
        MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());
        for (int i = 0; i < buffer.limit(); i++) {
            final byte read = buffer.get();
            sb.append((String.format("%02X ", read)));
            sb.append("\n");
        }
}

To ((very) slighlty) improve performance you may give your StringBuilder a big starting size like this :

StringBuilder sb = new StringBuilder(1_000_000);

Of course, you should prefer a well named constant over this ugly 1000000.

Other things in the code

Now that we are done with the performance aspect :
1. the name aFile is not really good looking IMO.
2. there is two nested try clause within each other but it's useless here.
3. please never user empty catch block :'(

After all those tweaks : it takes my computer 1 minute and a half to read a 28.5 mb file.

\$\endgroup\$
  • \$\begingroup\$ Thank you worked like magic @Ronan. Kindly check [search.maven.org/… The author claims to be able to read over 5 exabyte of file, how did they achieve it with out memory leaks. \$\endgroup\$ – king amada Jan 21 '18 at 5:26
  • \$\begingroup\$ Also it doesn't read a big file, i tried it on 120mb and i OutOfMemory error. \$\endgroup\$ – king amada Jan 21 '18 at 6:59
  • \$\begingroup\$ @kingamada sorry didn't saw your messages. Not really sure honestly :p StringBuilder won't be able to store a file over ~2 Go anyway so exabyte is obviously out of the question. By having an array of string you may theorically store millions elements of 2 Go string... but you'd never have enough RAM for this.... To help with your problem that is within the mb realm : launch your application with the xmx argument : stackoverflow.com/questions/14763079/… \$\endgroup\$ – Ronan Dhellemmes Jan 23 '18 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.