Although I've been programming a fair while I'm learning Python and would like to know how I can make my code more "Pythonic".
I'm have a CSV file of eBay categories, containing the name and numeric code which I want to import in to my database. Unfortunately the file does not include a 'parent' category ID for each row, instead it uses the ordinal position of the name in row to indicate the nesting level:
L1,L2,L3,L4,L5,L6,Category ID Antiques,,,,,,20081 ,Antique Clocks,,,,,13851 ,,Bracket Clocks,,,,100904 ,,,Pre-1900,,,66840 ,,,Post-1900,,,96762 ,,Cuckoo Clocks,,,,13850 ,,Longcase/ Grandfather Clocks,,,,100905 ,,,Pre-1900,,,60249 ,,,Post-1900,,,96763 ,Antique Furniture,,,,,20091 ,,Armoires/ Wardrobes,,,,98436
I've written a Python script to process the file, which is working correclty and generating the output I need, but I would like to know how it can be improved.
The script below is slightly changed from the one I am using, with simplified output (making it less verbose) but if anyone is interested in the other version which generates a JSON file for importing in to a Django site database, let me know.
I wanted to process the file line-by-line and avoid loading all the data in to memory, in case the file being processed is huge. So it uses a stack to record the parent category ID, other than that very little information is stored.
import csv
class Stack:
def __init__(self):
self.items = []
def push(self, o):
self.items.append(o)
def pop(self, n=1):
while n > 1:
self.items.pop()
n -= 1
return self.items.pop()
def peek(self, n=1):
return self.items[-n] if len(self.items) >= n else 'null'
def extract_info(row):
"""
:param row: data extracted from the csv file
:return: level (number), the name of the category (string), category ID (number)
Steps through the csv row until the first populated field (category name)
and then returns the nesting level, category name and ID (which is always in
the last field of the row.
"""
level = 0
for field in row:
if field:
return level, field, row[-1]
level += 1
id_stack = Stack()
with open('UK_New_Structure_(Oct2015).csv', 'rb') as csvfile:
rdr = csv.reader(csvfile, delimiter=',', quotechar='"')
# Skip three header rows
next(rdr)
next(rdr)
next(rdr)
first_row = next(rdr)
parent_id = 'null' # top level categories do not have a parent
level, name, cat_id = extract_info(first_row)
print level, name, cat_id, parent_id
id_stack.push(cat_id) # Save first parent ID
last_level = -1
for row in rdr:
level, name, cat_id = extract_info(row)
if level == last_level: # remove the newly added parent id, last item was not a parent
id_stack.pop()
parent_id = id_stack.peek()
id_stack.push(cat_id)
elif level < last_level:
id_stack.pop(last_level - level + 1)
parent_id = id_stack.peek()
id_stack.push(cat_id)
last_level = level
elif level > last_level:
parent_id = id_stack.peek()
id_stack.push(cat_id)
last_level = level
print level, '..' * level, name, cat_id, parent_id
I am using Python 2.7.10, but if there are better ways to do this in Python 3, then I'm interested in those as well.