Sorting nested dictionary by price - without extra memory

Question

I am working on parsing an orders.csv file with products and prices by different vendors which is kind of a time-series data.
The output of that process looks something like below, with order total by product and vendor -

source_dict = {
    'amazon': {'macbookpro': {'total': 1200}, 'ps5': {'total': 600}},
    'walmart': {'ps5': {'total': 750}},
    'walgreens': {'sonyheadset': {'total': 900}}
}

I need to write this data to another csv file in descending order by total. I am able to sort the items by total, but needs a new dictionary to store the sorted order -

new_dict = {}

for domain, products in source_dict.items():
    for product in products:
        total = products[product]['total']
        new_dict[domain+product] = {'source': domain, 'product': product, 'total': total}

sorted_items = sorted(new_dict.values(), key=lambda x: x["total"], reverse=True)

for item in sorted_items:
    print(f"{item['source']},{item['product']},{item['total']}")

In the above code I am basically using two different temporary objects "new_dict", "sorted_items" to store and sort the data items.

Can I simply sort the "source_dict" by "total" eliminating "new_dict"?

Any suggestions are highly appreciated.

Best,
Sri

Answer 1

No, you cannot sort source_dict by total, since an element of source_dict could have both the lowest and highest price and need to be at both the start and end of the sorting.

We can get close to what you want, but first ...

---

You are constructing new_dict as a dictionary, but never using it as one. You are just using it as a container, and getting new_dict.values().

Worse, you've got the possibility of losing values because you assign to new_dict[domain+product]. If ever "walmar" had an "tps5" product, it would conflict with "walmart" having a "ps5"!

Instead you could just maintain a list of items...

items = []
for domain, products in ...:
    for product in ...:
        ...
        items.append(...)

sorted_items = sorted(items, ...)

---

You are constructing a 3-element dictionary with fixed keys in a fixed order:

{'source': domain, 'product': product, 'total': total}

You could simply use a tuple: (domain, product, total), although a NamedTuple would be better:

from typing import NamedTuple

class Item(NamedTuple):
    source: str
    product: str
    total: int        # Or maybe float?

items = []
for domain, products in ...:
    for product in ...:
        ...
        items.append(Item(domain, product, total))

sorted_items = sorted(items, key=lambda item: item.total, reverse=True)

for item in sorted_items:
    print(f"{item.source},{item.product},{item.total}")

---

Finally, items (or new_dict in your case) was a temporary container only needed for the storing the list items to be sorted. You don't need to store that temporary in a named variable; you can generate the list on the fly:

sorted_items = sorted((Item(domain, product, details['total'])
                       for domain, products in source_dict.items()
                       for product, details in products.items()
                      ), key=lambda item: item.total, reverse=True)