# Regrouping list by nested groups

I have a list of units (as in entities in a game):

workers = [engineer1, engineer2, scientist1, scientist2]


These workers have a couple of properties I want to group them by: city and specialization. This grouping is nested, first we group by specializations and then by cities. Then we throw the list of workers that match the filter. Note that for each level we also include relevant information.

Rearranging the initial array into the desired grouped collection, we'd end up with the following (example included):

{
"workers": [
{
"name": "Engineer",
"cities": [
{
"name": "Paris",
"workers": [
{
"name": "Homer Simpson",
"building_points": 3
}
],
"population": 2244000
},
{
"name": "Berlin",
"workers": [
{
"name": "Donald Duck",
"building_points": 4
}
],
"population": 3500000
}
],
"description": "Boosts production"
},
{
"name": "Scientist",
"cities": [
{
"name": "Paris",
"workers": [
{
"name": "Marie Curie",
"building_points": 7
},
{
"name": "Tyrion Lannister",
"building_points": 7
}
],
"population": 2244000
}
],
"description": "Boosts research"
}
]
}


Note that this structure is strictly a requirement. It could probably be rearranged into a more proper way, but that's just how it must be.

Notice we first have a parent "worker" key under which everything is placed. Then we group by specialization including the name and description attributes, then we group by city including the name and population attributes, and finally we have list of workers that satisfy the given specialization and city, also including specific information like name and building_points.

I am relatively new to Python so I am inexperienced in using its full functional potential. I tried using some of its grouping and one-lining tools like map, groupby, lambdas and list comprehensions, but I couldn't simplify the operations past the first level of nesting.

Here is my actual attempt:

grouped_workers = defaultdict(list)

workers_by_spec = {spec: list(workers) for spec, workers in groupby(workers, lambda w: w.specialization)}

for spec, spec_workers in workers_by_spec.items():
workers_by_city = {city: list(workers) for city, workers in groupby(spec_workers, lambda w: w.city)}

cities = []
for city, city_workers in workers_by_city.items():
cities.append({
"name": city.name,
"population": city.population,
"workers": [{"name": worker.name, "building_points": worker.building_points} for worker in city_workers]
})

grouped_workers["workers"].append({"name": spec.name, "description": spec.description, "cities": cities})


Something tells me Python is the right language to perform these instructions in a more elegant, smooth, clear and compact way. Here's where I would appreciate tips that'd help me use the potential this language has to do the same I am already doing but the right way.

In case any of you want to try out the actual code I am going to include the classes (and the dummy objects), but note that these are not what I am trying to improve, but only the way I restructure the initial worker list.

class Unit:
def __init__(self, name):
self.name = name;

class City:
def __init__(self, name, population):
self.name = name
self.population = population

class Specialization:
def __init__(self, name, description):
self.name = name
self.description = description

class Worker(Unit):
def __init__(self, name, building_points, city, specialization):
super().__init__(name)
self.building_points = building_points
self.city = city
self.specialization = specialization

def __str__(self, *args, **kwargs):
return "Worker [name=%s, bp=%d, city=%s, spec=%s]" % (
self.name, self.building_points, self.city, self.specialization)

paris = City("Paris", 2244000)
berlin = City("Berlin", 3500000)
engineer = Specialization("Engineer", "Boosts production")
scientist = Specialization("Scientist", "Boosts research")

engineer1 = Worker("Homer Simpson", 3, paris, engineer)
engineer2 = Worker("Donald Duck", 4, berlin, engineer)
scientist1 = Worker("Marie Curie", 7, paris, scientist)
scientist2 = Worker("Tyrion Lannister", 7, paris, scientist)

• You said that you wanted to be able to change the key in the last question. But they're all "name" again, do you still need to change it? It's just that I didn't see any mention of it. – Peilonrayz Sep 20 '16 at 9:53
• They're not all "name"... there are more specific attributes for each nested group. What's the deal about all of them having the same "name" attribute? – dabadaba Sep 20 '16 at 10:11

This is my attempt at trying to convert it to one large list comprehension, I got pretty far but things just stopped going my way... to many nested loops (and not enough pre-planning) and hard to keep track of conceptually.

grouped_workers = defaultdict(list)
specializations = list(x[0] for x in groupby(list_of_workers, lambda x :x.specialization.name))
cities = list(set(x[0] for x in groupby(list_of_workers, lambda x :x.city.name)))
for i in list_of_workers:
grouped_workers['workers'] = [{'name':x,'cities':[{'name':y,'workers':[{'name':p.name,'building_points':p.building_points} for p in list_of_workers if p.specialization.name == x and p.city.name ==y]} for y in cities ]} for x in specializations]


output:

{'workers': [{'cities': [{'name': 'Berlin',
'workers': [{'building_points': 4,
'name': 'Donald Duck'}]},
{'name': 'Paris',
'workers': [{'building_points': 3,
'name': 'Homer Simpson'}]}],
'name': 'Engineer'},
{'cities': [{'name': 'Berlin', 'workers': []},
{'name': 'Paris',
'workers': [{'building_points': 7,
'name': 'Marie Curie'},
{'building_points': 7,
'name': 'Tyrion Lannister'}]}],
'name': 'Scientist'}]}


if you were open to trying a different data structure, then maybe there would be a simpler answer, but your data structure needs to be specific, and so your code is specific as well so you already have the best answer that matches the structure you need. And alternative and more "pythonic" approach would be to use a smaller list comprehension in a function to filter by common searches for what you need when you need it, eg. Another thing to note is that you already have the data stored in objects, why do you need to extract the attributes into an dictionary, firstly its using more memory to store it in two places, and secondly its no longer in object-oriented programming and so you loose all the advantages of having classes in the first place.

def group_workers_by_city(list_of_workers,cityname = None):
return [x for x in list_of_workers if x.city.name == cityname or cityname == None]

def group_workers_by_class(list_of_workers,specialization):
return [x for x in list_of_workers if x.specialization == specialization]

engineer = Specialization("Engineer", "Boosts production")

print(group_workers_by_city(list_of_workers,'Paris'))
print(group_workers_by_class(list_of_workers,engineer))
print(group_workers_by_city(group_workers_by_class(list_of_workers,engineer),'Paris'))


This implies that you have your Specialization class has an __eq__ overload.

• to satisfy your curiosity, answering "why do you need to extract the attributes into an dictionary,": because the data will actually be consumed by JavaScript, it won't be treated in Python anymore. – dabadaba Sep 20 '16 at 17:41