4
\$\begingroup\$

Functions with common and simple file operations have started to repeat in my code. Many like this one, but with subtle differences for example:

def do_something_with_file(file_name, access_mode, data):
    with open(file_name, access_mode) as f:
        f.do_something(data)

This is my solution so far:

import pickle
from functools import partial


def _open_file(func, access_mode, file_name, *args, **kwargs):
    with open(file_name, access_mode) as f:
        return func(f, *args, **kwargs)


open_file = partial(_open_file, lambda f: f, 'r')

read_lines = partial(_open_file, lambda f: list(f), 'r')

write_lines = partial(_open_file, lambda f, d: f.writelines(d), 'w')

load_pickle = partial(_open_file, lambda f: pickle.load(f), 'rb')

save_pickle = partial(_open_file, lambda f, d: pickle.dump(d, f), 'wb')

Example usage:

write_lines('file.txt', data)
pickle = load_pickle('file.pickle')

It seems readable enough for simple tasks and more complicated function definitions could also be passed to partial instead of simple anonymous functions like above.

Does anyone see any drawbacks of using this or has suggestions on how it could be improved?

\$\endgroup\$
4
  • 1
    \$\begingroup\$ Is anyone using a different approach? Probably. Is that really what you want to know though? Best-practices questions don't do well on Code Review. We can review your code, not determine whether it's 'the best thing to do'. \$\endgroup\$
    – Mast
    Apr 22, 2016 at 11:44
  • \$\begingroup\$ You are right. I have edited my question. \$\endgroup\$
    – inejc
    Apr 22, 2016 at 12:03
  • \$\begingroup\$ Just wanted to check, partial is functools.partial? \$\endgroup\$
    – Peilonrayz
    Apr 22, 2016 at 13:27
  • \$\begingroup\$ Yes. I've added the imports to the code snippet. \$\endgroup\$
    – inejc
    Apr 22, 2016 at 13:32

2 Answers 2

1
\$\begingroup\$

If it works for you why not.

The definitions could be further simplified too:

read_lines = partial(_open_file, list, 'r')
load_pickle = partial(_open_file, pickle.load, 'rb')

And possibly also extract the first two repeated tokens:

__open_file = partial(partial, _open_file)
read_lines = __open_file(list, 'r')
load_pickle = __open_file(pickle.load, 'rb')

With a bit more metaprogramming that could be done in a loop too and there are probably also ways to convert the other two lambdas using something from the operator module or so.

You see where I'm going with this? Pointfree style isn't exactly the most readable after all (actually the comparison with Pointfree style isn't the best, but it illustrates the point about combining functions with combinators for decreased readability quite well).

If I saw this in production I'd probably say that you aren't gonna need it. While interesting there's simply not that much added value in saving up the definitions IMO.

Also, how with open(...) etc. behaves is quite obvious - with these definitions not so much.

The underscore on _open_file suggests a private function, which it might not be, depending on your module layout. It looks like a useful enough combinator, so could also be exported for reuse.

open_file on the other hand looks useless: What do you want to use a closed file object for?

Biggest complaint: There are no docstrings. So, help(_open_file) basically gives me the function signature, whereas help(load_pickle) gives me the documentation for partial - which might be correct, but is also completely useless. In the interest of better understandable code I'd probably not recommend to use this approach unless you figure out a way to assign docstrings to those function-like objects (I could find one with a quick search, but that doesn't mean there isn't one).

\$\endgroup\$
2
\$\begingroup\$

The code is straighforward and reads good; there is not much to say per se. I would just advise to use f.readlines instead of list(f).

However, there exist a pattern in Python when you need to wraps functions using a common behavior: decorators.

I suggest having a decorator taking the open_mode as parameter:

import pickle
from functools import wraps

def opening(access_mode='r'):
    def decorator(func):
        @wraps(func)
        def wrapper(filename, *args, **kwargs):
            with open(filename, access_mode) as f:
                return func(f, *args, **kwargs)
        return wrapper
    return decorator

@opening()
def read_lines(file_):
    return file_.readlines()

@opening('w')
def write_lines(file_, data):
    file_.writelines(data)

@opening('rb')
def load_pickle(file_):
    pickle.load(file_)

@opening('wb')
def save_pickle(file_, data):
    pickle.dump(data, file_)

Usage is the same.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.