I have a very large dictionary that contains key-value pairs where the key is a string and the value a list. All lists have the same lengths. I want to split up the dataset (dictionary) in two based on the values in one of the lists. One of the lists will contain binary values (i.e. 0 or 1). I want to split up the dictionaries so that one of them contains all those where the binary value is 0 and the other where it is 1.
(It is easier to understand this when you mentally transform the dataset in a table where each index of the lists is a row, and where the dictionary values are column names.)
I guess I could first convert the dict to a Pandas dataframe and then subset the dataframe, and convert the resulting dataframes back to dicts, but I only want to do this if it proves to be faster than doing it with dictionaries only.
The following code works (test it here), but as said I'm not sure about efficiency. I'm only interested in Python >= 3.6
from collections import defaultdict
def split_d(d, key):
include = defaultdict(list)
exclude = defaultdict(list)
for i in range(0, len(d[key])):
binary = d[key][i]
for k, v in d.items():
if k == key:
continue
if binary == 0:
exclude[k].append(v[i])
elif binary == 1:
include[k].append(v[i])
else:
raise ValueError(f"Key {key} is not binary. Expected values 0 or 1")
return include, exclude
d = {
'prof': [1,0,0,1,1,1,1,1,0,1],
'val': [45,12,36,48,48,59,5,4,32,7]
}
print(split_d(d, 'prof'))
'val'
? \$\endgroup\$