Merge two list of dicts with same key

Question

I have two lists of dictionaries. Both lists always have the same ids. I want to get the following result:

{9: {'av': 4, 'nv': 45}, 10: {'av': 0, 'nv': 0}, 8: {'av': 0, 'nv': 30}}

The following code is working but I wonder if this is the best solution?

l1 = [{'id': 9, 'av': 4}, {'id': 10, 'av': 0}, {'id': 8, 'av': 0}]

l2 = [{'id': 9, 'nv': 45}, {'id': 10, 'nv': 0}, {'id': 8, 'nv': 30}]


l3 = {x['id']: {'av': x['av']} for x in l1}
l4 = {x['id']: {'nv': x['nv']} for x in l2}

{key: value.update(l4[key]) for key, value in l3.items()}

>> {9: {'av': 4, 'nv': 45}, 10: {'av': 0, 'nv': 0}, 8: {'av': 0, 'nv': 30}}

Answer 1

Your approach is rather good but your implementation is hardly extensible.

For starter, you don't need l4 as you can update l3 directly instead:

l3 = {x['id']: {'av': x['av']} for x in l1}
for d in l2:
    l3[d['id']].update(nv=d['nv'])

Second, you can pop the id so you don't have to know the other keys in the various dictionaries:

l3 = {d.pop('id'): d for d in l1}
for d in l2:
    l3[d.pop('id')].update(d)

But this approach has the drawback of modifying all input dictionaries. To mitigate that, we can start with an empty dictionary, update with every keys (including id) and pop the extra key afterwards. This is easily done using a defaultdict:

from collections import defaultdict


result = defaultdict(dict)
for sequence in (l1, l2):
    for dictionary in sequence:
        result[dictionary['id']].update(dictionary)
for dictionary in result.values():
    dictionary.pop('id')

There are few overheads using this approach compared to the first version, but it is way easier to generalize so you are able to merge more than 2 lists. Speaking of which, in such case, you should define a function taking a variable number of lists to merge:

import itertools
from collections import defaultdict


def merge(shared_key, *iterables)
    result = defaultdict(dict)
    for dictionary in itertools.chain.from_iterable(iterables):
        result[dictionary[shared_key]].update(dictionary)
    for dictionary in result.values():
        dictionary.pop(shared_key)
    return result

Usage be like

merge('id', l1, l2)

Answer 2

If you want to merge lists of dicts, you don't have to reinvent the wheel.

pandas might be a 800-pound gorilla but it's included in many distros, is well tested and documented.

You just need to initialize the dataframes, set their index and merge them:

import pandas as pd

l1 = [{'id': 9, 'av': 4}, {'id': 10, 'av': 0}, {'id': 8, 'av': 0}]
l2 = [{'id': 9, 'nv': 45}, {'id': 10, 'nv': 0}, {'id': 8, 'nv': 30}]

df1 = pd.DataFrame(l1).set_index('id')
df2 = pd.DataFrame(l2).set_index('id')
df = df1.merge(df2, left_index=True, right_index=True)
df.T.to_dict()
# {9: {'av': 4, 'nv': 45}, 10: {'av': 0, 'nv': 0}, 8: {'av': 0, 'nv': 30}}

Here's the corresponding console output:

>>> l1 = [{'id': 9, 'av': 4}, {'id': 10, 'av': 0}, {'id': 8, 'av': 0}]
>>> l2 = [{'id': 9, 'nv': 45}, {'id': 10, 'nv': 0}, {'id': 8, 'nv': 30}]
>>> import pandas as pd
>>> df1 = pd.DataFrame(l1).set_index('id')
>>> df1
    av
id    
9    4
10   0
8    0
>>> df2 = pd.DataFrame(l2).set_index('id')
>>> df2
    nv
id    
9   45
10   0
8   30
>>> df = df1.merge(df2, left_index=True, right_index=True)
>>> df
    av  nv
id        
9    4  45
10   0   0
8    0  30
>>> df.T.to_dict()
{9: {'av': 4, 'nv': 45}, 10: {'av': 0, 'nv': 0}, 8: {'av': 0, 'nv': 30}}