My script looks like this:
with open('toy.json', 'rb') as inpt: lines = [json.loads(line) for line in inpt] for line in lines: records = [item['hash'] for item in lines] for item in records: print item
What it does is read in data where each line is valid JSON, but the file as a whole is not valid JSON. The reason for that is because it's an aggregated dump from a web service.
The data looks, more or less, like this:
The code works, it allows me to interact with the data as JSON, but it's so slow that it's essentially useless. Is there a good way to speed up this process?