The following code takes a dictionary of lists (indexed by keyword) and converts it into a list of list of dictionaries that contains all possible permutations of those lists. However, my solution seems hackish, and is certainly space inefficient.. I feel like I'm missing something obvious.
The use case is for enumerating all the possible hyperparameters for a neural network (think sklearn grid search). So I specify the options in a compact dictionary containing the name of the parameter, and a list of values to test, and then convert that into a list of single-entry-per-key dictionaries that contain all possible permutations of those configuration parameters.
It's important to note I'm trying to make this generic, so that there can be an arbitrary number of keys specified in the initial dictionary, with an arbitrary number of options for each key.
Here's the code:
import copy
config_overrides = {
'nodetype': ['srnn', 'lstm', 'gru'],
'nhidden': [1,2,3,4,5,6,8,10,12,16,20],
'opt': ['sgd', 'rmsprop', 'adagrad', 'adamax', 'adam', 'nadam', 'adadelta'],
'actfunc': ['softplus', 'softsign', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear', 'relu'],
'loss': ['mse', 'msle']
}
experiments = [{},]
for k,v in config_overrides.items():
new_values = len(v)
current_exp_len = len(experiments)
for _ in range(new_values-1):
experiments.extend(copy.deepcopy(experiments[:current_exp_len]))
for validx in range(len(v)):
for exp in experiments[validx*current_exp_len:(validx+1)*current_exp_len]:
exp[k] = v[validx]
print(len(experiments))
print([x for x in experiments[1034:1039]])
The output of the above program, which includes random outputs from the middle of the list, and shows it appears tobe working correctly (the above example should come out to 3234 permutations):
3234
[{'loss': 'mse', 'opt': 'adadelta', 'actfunc': 'sigmoid', 'nodetype': 'lstm', 'nhidden': 4}, {'loss': 'msle', 'opt': 'adadelta', 'actfunc': 'sigmoid', 'nodetype': 'lstm', 'nhidden': 4}, {'loss': 'mse', 'opt': 'sgd', 'actfunc': 'hard_sigmoid', 'nodetype': 'lstm', 'nhidden': 4}, {'loss': 'msle', 'opt': 'sgd', 'actfunc': 'hard_sigmoid', 'nodetype': 'lstm', 'nhidden': 4}, {'loss': 'mse', 'opt': 'rmsprop', 'actfunc': 'hard_sigmoid', 'nodetype': 'lstm', 'nhidden': 4}]
A few immediate questions:
- Is list.extend() the best way to expand the list?
- The deepcopy() is necessary, or else you end up with copies of the original dictionary. But that also means the strings are copied, redundantly. Not a huge deal, but doesn't feel right to me.
Other than that, it just seems like there should be a cleaner, less obtuse way of doing this..and maybe I'm just too tired to think of it. Any ideas?
in line 19: for exp in experiments[validx*current_exp_len(validx+1)*current_exp_len]: TypeError: 'int' object is not callable
\$\endgroup\$:
got eaten while I was formatting my code cut-n-paste I'll be sure to re-test the cut-n-paste code in the future. \$\endgroup\$