Merge two dict in separate list by id

Question

I have following data structure

>>> a = [{'id': 1, 'name': 'a'}, {'id': 2, 'name': 'b'}]
>>> b = [{'id': 1, 'age': 12}, {'id': 2, 'age': 21}]

I want to merge the dictionaries present in list b with list a by matching id. (a and b have the same number of entries, but the order can be different.) So I want following:

[{'age': 12, 'id': 1, 'name': 'a'}, {'age': 21, 'id': 2, 'name': 'b'}]

I am doing it by following:

>>> for ta in a: 
...   for tb in b:
...     if ta['id'] == tb['id']:
...       ta.update(tb)
... 
>>> a
[{'age': 12, 'id': 1, 'name': 'a'}, {'age': 21, 'id': 2, 'name': 'b'}]
>>> 

Is there a better Pythonic way of doing it?

Answer 1

Assuming that no ids are missing in either list, you can just sort the lists by id, and then zip and merge. To sort by id, you can do:

import operator
sorting_key = operator.itemgetter("id")
a = sorted(a, key=sorting_key)
b = sorted(b, key=sorting_key)

Then, you can go though and update as you were doing before:

for i, j in zip(a, b):
    i.update(j)

Making your final code:

import operator
sorting_key = operator.itemgetter("id")
for i, j in zip(sorted(a, key=sorting_key), sorted(b, key=sorting_key)):
    i.update(j)

a will then be [{'name': 'a', 'age': 12, 'id': 1}, {'name': 'b', 'age': 21, 'id': 2}]. Note that this can be condensed into one line (excluding the import), but I wouldn't recommend it:

[i.update(j) for i, j in zip(sorted(a, key=operator.itemgetter("id")), sorted(b, key=operator.itemgetter("id")))]

Answer 2

You are saying

I want to merge the dictionary present

but what makes your task a bit difficult is that you are not handling dictonnaries but lists. Among other things, if you keep that idea that way, you'll have to iterate over n*m pairs (ta, tb) (where n and m are respective size of a and b).

A probably better idea would be to reorganise data in a and b to have them as an actual dict mapping id to some additional information. Then, merging will go through maximum n + m elements.

Answer 3

The canonical response is to make an id → dictionary mapping to remove the inner loop. This has two cases.

Firstly, if you just want to update a with the values in b, use

id_to_a = {dict_['id']: dict_ for dict_ in a}

for dict_ in b:
    id_to_a[dict_['id']].update(dict_)

If you want to support a more general case such as an n-way merge, use

from itertools import chain
from collections import defaultdict

def merge_by_key(dicts, key):
    merged = defaultdict(dict)

    for dict_ in dicts:
        merged[dict_[key]].update(dict_)

    return merged.values()

merge_by_key(chain(a, b), 'id')
#>>> dict_values([{'name': 'a', 'age': 12, 'id': 1}, {'name': 'b', 'age': 21, 'id': 2}])

Answer 4

I'd do something like:

import collections
result = collections.defaultdict(dict)
for d in a:
    result[d['id']].update(d)
for d in b:
    result[d['id']].update(d)

And you'd then be left with a dictionary of dictionaries grouped by id. You could finally do:

 result.values()

To get all merged dicts. This doesn't required sorting, so I believe it should be faster than the accepted answer.