Below is the code for user recommendations using mahout.

DataModel dm = new FileDataModel(new File(inputFile));
UserSimilarity sim = new LogLikelihoodSimilarity(dm);
UserNeighborhood neighborhood = new NearestNUserNeighborhood(100, sim,
GenericUserBasedRecommender recommender = new GenericUserBasedRecommender(
        dm, neighborhood, sim);

After the recommendations are generated, I am trying to write it to a file like this:

FileWriter writer = new FileWriter(outputFile);
for (LongPrimitiveIterator userIterator = dm.getItemIDs(); userIterator.hasNext();) {
long user = (long) userIterator.next();
List<RecommendedItem> recs = recommender.recommend(user, numOfRec );
    for (RecommendedItem item : recs) {
        writer.write(user + "," + item.getItemID() + ","
                + item.getValue()+"\n");

This code to write to file is taking lot of time. How can I speed up the write operations?

I tried with BufferedWriter, but was unable to gain speed-up.

  • \$\begingroup\$ State only code purpose in the title. \$\endgroup\$
    – Caridorc
    Nov 9, 2015 at 13:34
  • \$\begingroup\$ Are you sure it's actual the writing and not the method recommend that is taking that long? \$\endgroup\$
    – RoToRa
    Nov 9, 2015 at 13:54
  • \$\begingroup\$ @RoToRa, yes the recommend method is taking more time. Also as stated by ratchet freak, BufferedWriter saved considerable time. \$\endgroup\$ Nov 27, 2015 at 9:22

1 Answer 1


You can gain some efficiency by wrapping the writer in a BufferedWriter.

If you are using java 7 or better you should use try-with-resources to auto close the writer, otherwise you should use a try-finally:

try(BufferedWriter writer = new BufferedWriter(new FileWriter(outputFile))){
    //the for loops

with try-finally:

BufferedWriter writer = new BufferedWriter(new FileWriter(outputFile));
    //the for loops

This ensures that the writer is closed should an exception occur.

Second appending Strings for output is not the most performant thing that should be done instead pass each string separately:

writer.newLine();//only available in BufferedWriter

As a more general suggestion there is a better method than reading the entire file doing some processing and then writing the output. Instead you can read only as far as you need to process a part of the data and then write the result out again. Whether you can depends on what you are actually doing with the data. This is only possible if you have a 1-pass transform.

  • \$\begingroup\$ Would BufferedWriter with strings passed separately outperform a StringBuilder with one call to writer.write? I feel the amount of separate write calls would hinder performance, and if they could be reduced to 1 write call, it would be more efficient. \$\endgroup\$
    – Clark Kent
    Nov 9, 2015 at 16:04
  • 2
    \$\begingroup\$ @SaviourSelf But a BufferedWriter will flush its own buffer regularly as it gets full instead of allocating more memory and copying everything over. \$\endgroup\$ Nov 9, 2015 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.