2
\$\begingroup\$
def getLinesFromFile(file:File, linesToGet:Int): List[String] ={
    val bufferedSource = Source.fromFile(file)
    val lb = ListBuffer[String]()
    try {
      val lines = bufferedSource.getLines()
      var counter = 0
      while (lines.hasNext && counter < linesToGet) {
        counter += 1
        lb.append(lines.next())
      }
    } finally {
      bufferedSource.close()
    }
    lb.toList
  }

I am trying to retrieve a certain amount of lines from a read text file, and return each line as a List back to the user. Validation for the file actually being a .txt file is done prior to this call, and I can say with certainty that each line won't be longer than @500 characters.

My primary concern is I'm wondering if there any reason this code would create any memory leaks?

\$\endgroup\$
2
\$\begingroup\$

That implementation is very procedural. getLines() returns an Iterator, which supports a take(n) method.

def getLinesFromFile(file:File, linesToGet:Int): List[String] = {
  val bufferedSource = Source.fromFile(file)
  try {
    bufferedSource.getLines.take(linesToGet).toList
  } finally {
    bufferedSource.close
  }
}
\$\endgroup\$
  • \$\begingroup\$ That's a good point, changed! Thanks. However, not quite where I was looking for help. Does calling getLines load the entire file into memory? And at what point is the buffered source fully closed, and no longer needed in memory? I'm worried that even though I'm calling close, it isn't actually closing until some time later, perhaps via the GC, is this right? \$\endgroup\$ – NateH06 Feb 2 '18 at 21:32
  • \$\begingroup\$ From the Scala FAQ: "Iterators are non-strict: they are computed on-demand instead of in advance. … An iterator can only be used once because it is a traversal pointer into a collection, and not a collection in itself." It's just an elegant way to do what your counter did. Question is, does BufferedSource load the entire file into memory? It shouldn't. If you are really stingy, you could set the buffer size when calling Source.fromFile(). If you're paranoid, run your code under strace to see what data got read. \$\endgroup\$ – 200_success Feb 2 '18 at 21:45
  • \$\begingroup\$ What makes you think that calling close() doesn't close the file handle? \$\endgroup\$ – 200_success Feb 2 '18 at 21:45
  • \$\begingroup\$ Thanks, that's very helpful. What's making me paranoid is my system at large, when reading several very large files (1 gig each or more, sometimes 5 at a time), my entire drive slows to a crawl about 70% of the way through, and persists even though I have confirmation the program has closed after a System.exit(1) . And I'm just not sure what it would be, but an issue like this was the source of the problem in the past. \$\endgroup\$ – NateH06 Feb 2 '18 at 21:57
  • \$\begingroup\$ Well, you did ask for a List, which does store all its data. If your system runs out of physical memory, it starts swapping memory out to disk ("thrashing"). Then, after the memory hog program ends, the OS would tend to swap in the data to memory again, which is also I/O intensive. But that's my speculation based on your description. You shouldn't speculate either; use a memory profiler if you want to understand what your problem is. \$\endgroup\$ – 200_success Feb 2 '18 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.