I made this and left it up on a gist a while back broken. I don't like broken code, so today I spent a while getting tests working, using more sensible names, and making tests pass. Its goal is to take inputs in the form of:
[
('root[subnode][name][]','that guy'),
('root[subnode][attribute][]','somevalue'),
('root[subnode][attribute][]','another value')
]
Then it would transform into:
{
'root': {
'subnode': {
'name': {
0: 'that guy'
},
'attribute': { # there is no way to guarantee order AFAIK
0: 'somevalue',
1: 'another value'
}
}
}
}
I'm looking for feedback to improve upon it. It passes PEP8 check and has inline doctests. The way it's designed at the moment from outside it can be made immutable by import deepcopy from copy
in the app / lib code using and passing deepcopy(input)
.
# FormData / Flat-to-Structured Dictionary Accessories For Python
# Copyright (C) 2015 Lewis Cowles
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import re
def _int_if_poss(s):
"""
Returns an integer Number if non-NaN input provided
>>> _int_if_poss(99)
99
>>> _int_if_poss(12.0)
12
>>> _int_if_poss("999999")
999999
>>> _int_if_poss("0")
0
>>> _int_if_poss("tudor dave")
'tudor dave'
>>> _int_if_poss("0tudor dave")
'0tudor dave'
>>> _int_if_poss("tudor dave22")
'tudor dave22'
>>> _int_if_poss("tudor99dave")
'tudor99dave'
"""
try:
return int(s)
except ValueError:
pass
return s
def _autoKey(collection):
"""
Generates an auto-key based upon length of collection
>>> _autoKey({})
0
>>> _autoKey({"name":"robert poulsen"})
1
>>> _autoKey({"name":"robert poulsen","rules":[
... "do not talk about fight club",
... "do not talk about fight club",
... "do not talk about fight club"]})
2
"""
return len(collection)
def _parse_to_dict_val(key, value, dictin):
"""
Parses depth, encoded names into a JSON'ish disctionary structure
>>> _parse_to_dict_val(
... "name[]",
... "starlord",
... {})['name'][0]
'starlord'
>>> _parse_to_dict_val(
... "name[accusor]",
... "ronin",
... {})['name']['accusor']
'ronin'
>>> _parse_to_dict_val(
... "characters[by name][accusor]",
... "ronin",
... {})['characters']['by name']['accusor']
'ronin'
>>> _parse_to_dict_val("name", "bob", {})
{'name': 'bob'}
"""
patt = re.compile(r'(?P<name>.*?)?[\[](?P<key>.*?)[\]](?P<leftover>.*?)$')
matcher = patt.match(key)
matched = matcher is not None
# Guard clause for non-nested
if not matched:
dictin[key] = value
return dictin
tmp = matcher.groupdict()
basename = _int_if_poss(str(tmp['name']))
subkey = str(tmp['key'])
leftover = str(tmp['leftover'])
if len(str(basename)) > 0:
# First off, we should ALWAYS have a matched name
dictin.setdefault(basename, {})
if len(leftover) == 0:
# Is this deeply nested or not
if len(subkey) == 0:
# For standard flat values and when no more remains (easy)
if "[" in key and "]" in key:
# in this case the key needs auto-key
autoKey = _autoKey(dictin[basename])
dictin[basename][autoKey] = value
else:
# no list / dict initialiser pair present
dictin[basename] = value
elif len(subkey) > 0:
# if nothing remains to be done, but we have a key, set a value
dictin[basename][subkey] = value
else:
# This is definitely deeply nested
if len(subkey) > 0:
# For n-length nesting by name (leftover becomes nested)
dictin[basename].setdefault(subkey, {})
_parse_to_dict_val((subkey+leftover), value, dictin[basename])
else:
# For n-length nesting without name (auto-key)
autoKey = _autoKey(dictin[basename])
dictin[basename].setdefault(autoKey, {})
_parse_to_dict_val(
(str(autoKey)+leftover),
value,
dictin[basename])
return dictin
def parse_to_dict_vals(listin):
"""
Parses dictionary for encoded keys signifying depth
>>> parse_to_dict_vals([
... ("name[]","starlord"),
... ("name[accusor]","ronin")
... ])['name'][0]
'starlord'
>>> parse_to_dict_vals({
... "name[]":"starlord",
... "name[accusor]":"ronin"
... })['name']['accusor']
'ronin'
>>> parse_to_dict_vals({
... "goodies[starlord][age]":"29",
... "goodies[starlord][planet]":"Earth"
... })['goodies']['starlord']['age']
'29'
>>> parse_to_dict_vals({"name":"bob"})
{'name': 'bob'}
>>> 'abilities' in parse_to_dict_vals([
... ("goodies[starlord][age]","29"),
... ("goodies[starlord][planet]","Earth"),
... ("goodies[Yondu][species]","Centaurian"),
... ("goodies[Yondu][abilities][]","Magic Arrow"),
... ("goodies[Yondu][abilities][]","Not Eating starlord")
... ])['goodies']['Yondu']
True
>>> parsed = parse_to_dict_vals([
... ("goodies[starlord][age]","29"),
... ("goodies[starlord][planet]","Earth"),
... ("goodies[Yondu][species]","Centaurian"),
... ("goodies[Yondu][abilities][]","Magic Arrow"),
... ("goodies[Yondu][abilities][]","Not Eating starlord")
... ])
>>> len(parsed['goodies']['Yondu']['abilities'])
2
>>> parse_to_dict_vals({"root[subnode][attribute][][]":"value"})
{'root': {'subnode': {'attribute': {0: {0: 'value'}}}}}
>>> parse_to_dict_vals({"root[subnode][attribute][][me]":"value"})
{'root': {'subnode': {'attribute': {0: {'me': 'value'}}}}}
"""
dictout = {}
if isinstance(listin, dict):
listin = listin.items()
for key, value in listin:
_parse_to_dict_val(key, value, dictout)
return dictout
if __name__ == "__main__":
import doctest
doctest.testmod()
Earlier on the #python
IRC someone mentioned using defaultdict
, which I didn't understand. Then my wife wanted to enjoy the sunshine, so I probably missed some feedback. Be as brutal as necessary.
I'm looking for ways to use more of the standard library, or alternative existing Python code, maybe ways to allow this to take and output different data structures if it becomes necessary.
For the fight-club and guardians of galaxy test data, I was just looking for something to structure.
Some have said I should just send in JSON. Whilst that is true, I'd prefer to have this work with form-data that can be sent by lots of utilities easily, CURL, web-browsers from IE5 - chrome nightly. JSON just isn't what this is designed for.
A living copy of the code is available here.
'name': {0: 'that guy'}
rather than'name': 'that guy'
? \$\endgroup\$