python3 function to merge dictionaries

Question

I have a bunch of JSON records, each record with a varying degree of completeness. In other words, record A may contain keys not in record B, and vice-versa. To get a better understanding of the data within, I have created a function to take N number of records and merge them together, creating one single frankenstein record containing all keys and a single value for each key.

import sys
import json
def frankenstein(out, in_dict, key=None):
    if isinstance(in_dict, dict):
        for k, v in in_dict.items():
            if isinstance(in_dict[k], list) and v:
                out.setdefault(k, [])
                frankenstein(out[k], v, k)
            elif isinstance(in_dict[k], dict) and v:
                out.setdefault(k, {})
                frankenstein(out[k], v, k)
            elif v:
                out[k] = v
    elif isinstance(in_dict, list):
        s = {}
        for item in in_dict:
            if isinstance(item, dict):
                frankenstein(s, item)
            elif not out:
                out.append(item)
            if s:
                if not out:
                    out.append(s)
                else:
                    frankenstein(s, out[0])
                    out[0] = s
if __name__ == '__main__':
    l = [
        {
            "name": "foo bar",
            "experience": [
                {
                    "company": {
                        "name": "oracle",
                        "hq": "123 main st",
                        "size": 100
                    },
                    "function": [
                        {
                            "name": "go getter"
                        }
                    ],
                    "location": {
                        "doubleday": "publisher"
                    },
                    "animal": "horse"
                }
            ],
            "skills": ["programming", "eating"]
        },
        {
            "name": "poo dar",
            "experience": [
                {
                    "company": {
                        "name": "microsoft",
                        "url": "foo.bar/com"
                    },
                    "function": [
                        {
                            "name": "bread",
                            "level": "really high"
                        }
                    ],
                    "solitary": {
                        "fat": "cat"
                    },
                    "health": "no good"
                }
            ],
            "skills": ["igz"]
        },
        {
            "name": "poo mar",
            "experience": [
                {
                    "function": [
                        {
                            "zoo": "creature"
                        }
                    ],
                    "location": {
                        "taste": "food"
                    },
                    "ping": {
                        "pong": "bong"
                    }
                }
            ],
            "skills": ["woots own"]
        }
    ]
    out = {}
    for item in l:
        frankenstein(out, item)
    print(json.dumps(out, indent=4))

This is the output from the code:

{
    "name": "poo mar",
    "experience": [
        {
            "function": [
                {
                    "name": "bread",
                    "level": "really high",
                    "zoo": "creature"
                }
            ],
            "location": {
                "taste": "food",
                "doubleday": "publisher"
            },
            "ping": {
                "pong": "bong"
            },
            "company": {
                "name": "oracle",
                "url": "foo.bar/com",
                "hq": "123 main st",
                "size": 100
            },
            "solitary": {
                "fat": "cat"
            },
            "health": "no good",
            "animal": "horse"
        }
    ],
    "skills": [
        "programming"
    ]
}

I've tested the function and it does work. What I'd like is some feedback with regards to the code. Am I doing this in the most efficient way possible? Are there better ways to do it?

Answer 1

The key parameter to frankenstein() does not appear to be used anywhere and can be eliminated.

Use blank lines to split the code up into smaller logical chunks. It aides reading and understanding the code.

in_dict is a misleading name, because it can be a list, dict, or something else altogether.

In the for k, v in in_dict.items(): loop, in_dict[k] and v are the same thing, v is tested 3 times, and the return value of setdefault() is discarded then looked up on the next line. It can be rewritten it like so:

    for k, v in in_dict.items():
        if not v:
            continue

        if isinstance(v, (list, dict)):
            out_k = out.setdefault(k, v.__class__())
            frankenstein(out_k, v)

        else:
            out[k] = v

The logic for handling lists seems convoluted. For example, it looks like s can get updated each time through the loop and also appended to out[k]. The question doesn't clearly state the rules for combining things, so maybe it's correct. A comment or doc-string explaining the purpose of the function and the rules for merging values would be helpful.