Suppose I need to process a list of list of list of many objects, where the data usually comes from third party APIs.
For the sake of an example, lets assume we have many users, and for each user we get many days of data, and for each day you get many events. We want to process all of these objects and store in our database
I thought about some differents design patterns I could use, but I'm not sure what is considered to be the best approach, if any.
These are my thoughts:
1) Nested iterations and do the side-effect (save to database) at the lowest-level loop.
for user in users:
for day in user.days:
for event in day.event:
processed_object = process_event(event)
save_to_db(processed_object)
2) Similar to one, except that each nested iteration is 'hidden' in another method.
for user in users:
process_user(user)
def process_user(user):
for day in user.days:
process_day(day)
def process_day(day):
for event in day.events:
process_event(event)
def process_event(event):
save_to_db(event)
3) Return everything from deep in the nest and then do the side-effects afterwards.
def process_users(users):
processed = []
for user in users:
processed.append(process_user(user))
return processed
def process_user(user):
processed = []
for day in user.days:
processed.append(process_day(day))
return processed
def process_day(day):
processed = []
for event in day.events:
processed.append(process_event)
return processed
processed_events = process_users(users)
for event in processed_events:
save_to_db(event)
Is there a general, agreed on approach for these type of programs, where side-effects are involved?
itertools.chain.from_iterable
could be useful to flatten the iterations. \$\endgroup\$ – Bakuriu Jul 20 '13 at 15:54