1
\$\begingroup\$

I have this stupid code block below to save multiple files in Python. It is kind of repetitive (but it works). I was wondering how to improve it, notably:

Is it worth it to write a function for two lines of code ? Is it possible to get the generic object name so that i don't have to specify it in the 'open' ? Are there other approaches / optimisations to make it better / looks nicer ?

with open(RES_PATH + 'res_folds_seeds.p', 'wb') as f:
    pickle.dump(res_folds_seeds, f)
    
with open(RES_PATH + 'res_folds.p', 'wb') as f:
    pickle.dump(res_folds, f)
    
with open(RES_PATH + 'res_folds_neutralised.p', 'wb') as f:
    pickle.dump(res_folds_neutralised, f)    
    
with open(RES_PATH + 'dict_coeffs.p', 'wb') as f:
    pickle.dump(dict_coeffs, f)
    
with open(RES_PATH + 'dict_coeffs_seeds.p', 'wb') as f:
    pickle.dump(dict_coeffs_seeds, f)
    
with open(RES_PATH + 'preds.p', 'wb') as f:
    pickle.dump(preds, f)
    
with open(RES_PATH + 'exps.p', 'wb') as f:
    pickle.dump(exps, f)
\$\endgroup\$

2 Answers 2

5
\$\begingroup\$

Is it worth it to write a function for two lines of code?

In this case, yes.

Are there other approaches / optimisations to make it better / looks nicer ?

Yes: use pathlib -

from pathlib import Path
import pickle
from typing import Any

RES_PATH = Path('.')


def pdump(stem: str, data: Any) -> None:
    path = (RES_PATH / stem).with_suffix('.p')
    with path.open('wb') as f:
        pickle.dump(obj=data, file=f)

You can then call this function with a sequence as Harith demonstrates.

Is it possible to get the generic object name so that i don't have to specify it in the 'open'?

Yes. With your current code you would need to use locals() (the bad kind of clever; I don't recommend doing this). A somewhat cleaner approach would put all of your objects in a parent class Persisted(NamedTuple), and then iterate over all of them.

from pathlib import Path
import pickle
from typing import Any, NamedTuple

RES_PATH = Path('.')


def pdump(stem: str, data: Any) -> None:
    path = (RES_PATH / stem).with_suffix('.p')
    with path.open('wb') as f:
        pickle.dump(obj=data, file=f)


Mystery = Any


class Persisted(NamedTuple):
    res_folds_seeds: Mystery
    res_folds: Mystery
    res_folds_neutralised: Mystery
    dict_coeffs: dict[Mystery, Mystery]
    preds: Mystery
    exps: Mystery

    def dump(self) -> None:
        for name in self.__annotations__.keys():
            pdump(stem=name, data=getattr(self, name))
\$\endgroup\$
4
\$\begingroup\$

You can simplify your code by creating a list of tuples where each tuple consists of the filename and the corresponding data, and then looping over the list and performing the operation in a loop like so:

files_data = [
   ('res_folds_seeds.p', res_folds_seeds),
   ('res_folds.p', res_folds),
   ('res_folds_neutralised.p', res_folds_neutralised),
   ('dict_coeffs.p', dict_coeffs),
   ('dict_coeffs_seeds.p', dict_coeffs_seeds),
   ('preds.p', preds),
   ('exps.p', exps)
]

for file_name, data in files_data:
   with open(RES_PATH + file_name, 'wb') as f:
       pickle.dump(data, f)
\$\endgroup\$
1
  • 1
    \$\begingroup\$ If the correspondence is trivial, you could just have a list of strings and convert those into a tuple of file name, variable. (The variable's value is globals()[string] if it's global.) \$\endgroup\$
    – tripleee
    Jan 1 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.