The code below works pretty well - it takes about 4 seconds (this seems too slow for me) to find a group that's located in the tail area of the text file I'm searching. The text file has ≃ 250,000 lines with 20 elements per line. I cut my teeth on other programming languages and picked up Python just for this current project I'm working on, so I really am a neophyte when it comes to python efficiency.
with open(file) as infile: datadictionary = csv.DictReader(infile, dialect='excel-tab', quoting=csv.QUOTE_NONE) for key, group in itertools.groupby(datadictionary, key=lambda x:x[patient_number_field_header] == patient_id): if key: super_list = group break
- 'patient_id' is a string of digits
- file is a text file
I'm wondering what you think - how can I make this more efficient? Am I "doing it wrong"?