Efficient way of reading in list of string csvs' into lists

Question

I have a list of strings. Each string is a comma separated list of numbers. I want to quickly convert each of these strings into a list. So far I have done the following:

My Solution So Far

def convertCommaSeperatedStringToList(data):

    s_lists = []

    for s in data:
        s_list =  [int(x) for x in s.split(',')]
        s_lists.append(s_list)

Example Dataset

# Here we have 600 very long lists to process
import numpy as np

data = []

for x in range(0,600):
    a = (np.random.randint(1000,size=100000)).tolist()
    b = ','.join(str(x) for x in a)
    data.append(b)

Just some background if you're wondering why I would have such a wierd list, I am reading this list from a downstream database.

Answer 1

Fastest way would be to use a list-comprehension here. This would avoid the repetitive lookups to list.append and will use the LIST_APPEND byte code.

s_lists = [[int(x) for x in row.split(',')] for row in data]

Note that for function names underscores are preferred over camel-case.

Answer 2

As mentioned in other answers, list comprehension is the fastest way to solve this. However, I'd just like to point out that you should also consider using a generator function if it suits your needs to prevent the need to load the entire dataset.

def convertCommaSeparatedStringToList(data):
    for s in data:
        yield [int(x) for x in s.split(',')]