Reading lines of integers into a list of lists

Question

I have a text file containing integers in different lines which I want to read into a list of lists.

So

01
11 14
45 50 09

should become

((1)
 (11 14)
 (45 50 9))

The real file is longer so I want to read it in using CL instead of transforming it using Emacs' macros etc. It is important that every line becomes exactly one list and the data is stored as integers, i.e. 09 becomes 9.

I ended up with the following (using the external library split-sequence):

(defun read-data (&optional (file "data.txt"))
  "Returns the numerical data in FILE as a list of lists."
  (with-open-file (data file)
    (loop :for line = (read-line data nil nil)
          :while line
          :collect (map 'list #'parse-integer (split-sequence #\Space line)))))

which is great because it does the job, I needed only two minutes for it and I knew exactly what I wanted to do. Nevertheless, I found the chain read-line, split-sequence and map a bit too much for this rather simple job. Is there a simpler way just using read which I am just missing?

Answer 1

A simple possibility could be to enclose each row between parentheses and use read-from-string (assuming that the file does not contain incorrect data):

(defun read-data (&optional (file "data.txt"))
  "Returns the numerical data in FILE as a list of lists."
  (with-open-file (data file)
    (let (*read-eval*)   ; set *read-eval* to NIL to avoid malicious code in the input file
      (loop :for line = (read-line data nil nil)
            :while line
            :collect (read-from-string (concatenate 'string "(" line ")"))))))

Answer 2

A way to read a single line of integers without splitting it into strings:

(defun read-line-of-integers (line &aux (start 0) item (end (length line)))
  (loop while (< start end)
        do (setf (values item start)
                 (parse-integer line :start start :junk-allowed t))
        when (integerp item)
        collect item))

PARSE-INTEGER takes an argument where it should start to read in the string. It also returns two values: the number read and the end in the string where it stopped reading. We can use that in a loop to move forward in the string from one number to another.

CL-USER 33 > (read-line-of-integers " 2 10 3 11    ")
(2 10 3 11)

Not splitting a line into strings means that the garbage collector has less work.

Another thing to consider for production code: read-line is relatively slow, since it conses a new string for each line. For faster code one needs a version which reads a line directly into a pre-allocated buffer.