General remarks
pep 8
for your names and code style, try to follow pep-8
lower_case
for variable and function names
- spaces around operators
main guard
put the calling of your functions after a if __name__ == '__main__':
, so you can load the script from somewhere else without it executing the code immediately
looping
Don't loop over indices. Code like for i in range(nr_files):
is a lot cleaner using enumerate
: for i, filename in enumerate(file_names)
.
I suggest you check out the excellent 'Looping like a Pro' talk by David Baumgold
functions
Instead of having 1 function to load the files, loop over them and pick the correct element, seasiest would be to split if into different functions:
- takes the list of files, and passes them on one by one to the parse
- parse a single file
Generators
The most pythonic an efficient approach to do this would be to use generators, pathlib.Path
and the built-in csv
module
My solution
parse one file
This function takes a filehandle, and parses the requested element from the line
import csv
from pathlib import Path
def parse_file(filehandle, field_name):
kwargs = { # https://docs.python.org/3/library/csv.html#csv-fmt-params
'delimiter': ',',
'skipinitialspace': True,
# ...
}
# field_name = b # or the column name
reader = csv.DictReader(filehandle, **kwargs) # or csv.reader if there is no header, and it might be faster
for line in reader:
yield line[field_name]
This can be easily tested like this:
from io import StringIO
csv_str = '''a, b, c
0, 1, 2
3, 4, 5'''
with StringIO(csv_str, newline='') as file:
print(list(parse_file(file, 'b')))
['1', '4']
parse multiple file
def parse_files(files):
for file in files:
try:
with filename.open('r', newline='', ) as csv_file:
yield list(parse_file(csv_file))
except FileNotFoundError:
print("ERROR: FILES ARE MISSING!!!!")
raise
Now we have a good method to parse the information, we just need to call it with the subsequent files
main
def main(files):
results = list(parse_files(files))
return results
if __name__ == '__main__':
files= [Path("DOCS/1.csv"),Path("DOCS/2.csv")]
main(files)