5
\$\begingroup\$

I have a .json file with a complex structure: dict in a dict, it has no constant structure and is dynamically changed.

The goal is to make new dict where the keys are numbers of hierarchy and values must be strings that are composed of all previous steps like

2 drinks
2.1 drinks coffee
2.1.1 drinks coffee instant 
2.1.2 drinks coffee real 
2.2 drinks tea 
2.3 drinks water 

I wrote a script, and it works. But as I am new to Python and in programming, please check it out and correct me in order to code may look and perform more elegantly and in a pythonic way.

Also I would like to decrease time of execution because the input .json file is going to be quite big.

Sample cat.json file:

{
"communication":
    {"mobile":{
        "vodafone":{"subscr":"","txt":"","mms":"","internet":"","calls":
    {"in":{"home":"","roaming":""},"out":{"home":"","roaming":""}}},
        "verizon":{"subscr":"","txt":"","mms":"","internet":"1Gb","calls":
    {"in":{"home":"","roaming":""},"out":{"home":"500 min","roaming":"Other country"}}}},
    "internet":"SomeProviderName"
    },
"food":{"dairy":{"cheese":"Gauda","milk":{"brand":"name","origin":"place"}}},
"drinks":
        {
        "water":"",
        "tea":"",
        "coffee":
                {
                "instant":"",
                "real":""
                }}}

Script:

#!/usr/bin/env python -tt
# -*- coding: utf-8 -*-
import json
filetoread='cat.json'
def load_existed(filetoread):

try:
    data=json.loads(open(filetoread).read())        
    return data

except ValueError:
    print 'data  loading error'

cat_data=load_existed(filetoread)

def walk_dict(d,mess,ln,new_dict,crumbs):
    inter=1
    lc=list(mess)
    last_crumb=crumbs.split( )

    for k,v in sorted(d.items(),key=lambda x: x[0]):
        if mess=='':
            mess=str(inter)
            lc=list(mess)
            last_crumb=crumbs.split( )
        if isinstance(v, dict) :
            ln=len(v)
            lc[len(lc)-1]=str(inter)
            mess="".join(lc)
            crumbs=" ".join(last_crumb)
            if len(crumbs.split( ))>0:
                crumbs=crumbs+" "+k
            else:
                crumbs=k
            #print mess,'-->',k,">"
            new_dict[mess]=crumbs
            mess=mess+'.1'
            walk_dict(v,mess,ln,new_dict,crumbs)
        else:
            if ln>0:
                ln=ln-1
                lc[len(lc)-1]=str(inter)
                mess="".join(lc)
                crumbs=" ".join(last_crumb)
                crumbs=crumbs+" "+k+" "+v
            #print  mess,'-->',"%s -> %s" % (k, v)
            new_dict[mess]=crumbs
        inter=inter+1
    return new_dict

new_data=walk_dict(cat_data,"",0,{},'')
print "-"*30
#print new_data
for v in sorted(new_data):
  print v, new_data[v]
\$\endgroup\$
5
\$\begingroup\$

Some notes about your code:

  • for readability, you should add spaces around operators
  • you should use with to open files; also, instead of first reading the file to a string and then using json.loads, just use json.load on the file itself
  • instead of running your own counter with inter, use enumerate
  • using key=lambda x: x[0] is pointless, it does not affect the sort order at all
  • you are constantly converting between strings and lists; stick to one representation, e.g. here: crumbs = " ".join(last_crumb) and right in the next line: if len(crumbs.split()) > 0:; instead, just test if last_crumb: or if crumbs:, whichever you prefer
  • lc[len(lc)-1] is the same as lc[-1]
  • there's some code duplication in the if/else; try to move that outside
  • your function seems to miss some of the items, e.g. you never output 1.2.2.4 communication mobile vodafone subscr; this is because you overwrite the ln parameter with ln = len(v); not sure why you need that ln parameter anyway...

Taking those into account, your walk_dict function can be simplified significantly:

def walk_dict(d, mess, new_dict, crumbs):
    lc = list(mess) if mess else ["1"]
    for inter, (k, v) in enumerate(sorted(d.items()), 1):
        lc[-1] = str(inter)
        mess = "".join(lc)
        if isinstance(v, dict) :
            crumbs2 = (crumbs + " " + k) if crumbs else k
            walk_dict(v, mess + '.1', new_dict, crumbs2)
        else:
            crumbs2 = crumbs + " " + k + " " + v
        new_dict[mess] = crumbs2
    return new_dict

But the way you convert mess into a list lc and then replace the last element is still -- quite fitting -- a "mess". Same with the way you are mixing "output parameters" and return values.

Here's how I would do it:

def walk_dict(d, key=None, parent=None):
    res = {}
    for i, e in enumerate(sorted(d), 1):
        k = (key    + "." + str(i)) if key    else str(i)
        p = (parent + " " +     e ) if parent else     e
        if isinstance(d[e], dict):
            res.update(walk_dict(d[e], k, p))
            res[k] = p
        else:
            res[k] = p + " " + str(d[e])
    return res

with open('data.json') as f:
    cat_data = json.load(f)
    new_data = walk_dict(cat_data)
    for v in sorted(new_data):
        print v, new_data[v]
| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Yeah, that is called "Feel the difference" in my language. I mean the function became three time less in lines and whole script became smaller twice. I didn't measure time performance, but I suppose the result would be at least as impressive as code improvement in lines. Needless to say about my bug that script even didn't work properly. So thank you so much. I wrote a bit more than 500 lines and it's just about 10%, I beleive, of my plan. And I think if I check all that is written It would be really similar to that sample. But it's a hobby, I do it for myself but I'd like to do it good. THNX! \$\endgroup\$ – Sergii Artele May 2 '15 at 20:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.