Python persistant cache for non-pythonic objects

Question

I am trying to implement a cache which can be persisted. Also, some of the functions that I am trying to cache takes arguments that are not python objects (external library objects). But they seem to have a hash value. My first try was to use lru_cache but it didn't have an option to persist across sessions. Then I stumbled upon this answer which explained a way to create a custom decorator to cache the function results. I made some minor changes to it and came up with this decorator (which can take arguments) now.

def cached(maxsize=None, hashkey=True, persist=False):
    """Decorator method to cache an expensive function

    Args:
        maxsize (int): The maximum number of cached results to store
            if None, the size is indefinite.
        hashkey (bool): If the arguments of the function are objects
            which cannot be pickled, use the hashkey option which will
            hash the arguments.
        persist (bool): Write the cached results to the disk so that
            it can be read in even after the session has ended.

    Returns:
        Decorator function
    """

    def outer(func):
        func.cache = {}
        func.cache_file = f"{func.__name__}.cache"

        @wraps(func)
        def wrapper(*args):
            if hashkey:
                key = hash(args)
            else:
                key = args
            try:
                return func.cache[key]
            except KeyError:
                try:
                    if persist:
                        with open(func.cache_file, "rb") as cachefile:
                            func.cache = pickle.load(cachefile)
                    return func.cache[key]
                except (KeyError, FileNotFound):
                    func.cache[key] = result = func(*args)
                    if maxsize and len(func.cache) > maxsize:
                        func.cache.pop(next(iter(func.cache))) # removes the first item
                    if persist:
                        with open(func.cache_file, "wb") as cachefile:
                            pickle.dump(func.cache, cachefile)
                    return result

        return wrapper

    return outer

The functions that I am planning to cache may be called at most 10000 times with a given cache file.

Here are my questions:

1. Is there anything obviously wrong with this code?
2. Is hashing the arguments a bad idea?
3. Are there any obvious places where I can store the cache file?

Answer 1

Explanation

As @RootTwo mentioned, there are two things that can be improved:

• Loading the file could be done whenever the decorator is being applied. There is no need to read from the cache file whenever a key is not found (unless you plan modifying contents from elsewhere in the code)

• Updating the cached file every miss is very inefficient

I would also argue that storing the cache file in the working directory is probably not the best idea: if you run your code from a different directory, cached data will not be available

That's regarding the code itself. Now, regarding code style, I would personally swith to the pathlib library, rather than using the raw open function.

Solution

Now, the solution I propose is as follows:

• Read the cache file when applying the decorator

• Write the cache file on exit (using module atexit)

import atexit
import pickle
from pathlib import Path
from functools import wraps


def cached(maxsize=None, hashkey=True, persist=False):
    """Decorator method to cache an expensive function

    Args:
        maxsize (int): The maximum number of cached results to store
            if None, the size is indefinite.
        hashkey (bool): If the arguments of the function are objects
            which cannot be pickled, use the hashkey option which will
            hash the arguments.
        persist (bool): Write the cached results to the disk so that
            it can be read in even after the session has ended.

    Returns:
        Decorator function
    """
    def outer(func):
        func.cache = {}
        func.cache_file_path = Path(f"{func.__name__}.cache")

        if persist:
            if func.cache_file_path.exists():
                with func.cache_file_path.open("rb") as cachefile:
                    func.cache = pickle.load(cachefile)

            def persits_cache_on_exit():
                with func.cache_file_path.open("wb") as cachefile:
                    pickle.dump(func.cache, cachefile)
            atexit.register(persits_cache_on_exit)

        @wraps(func)
        def wrapper(*args):
            if hashkey:
                key = hash(args)
            else:
                key = args

            try:
                return func.cache[key]
            except KeyError:
                func.cache[key] = result = func(*args)

                if maxsize and len(func.cache) > maxsize:
                    # removes the first item
                    func.cache.pop(next(iter(func.cache)))

                return result
        return wrapper
    return outer

Regarding where to store the cache files, if you intend to use this decorator in just one project, you could probably store cache files relative to the project files (If you do not know how to do this, I'd be happy to point you in the right direction). If you intend to use it elsewhere, you could maybe store it in a temporary directory such as /temp/ in *nix.

Answer 2

It seems rather inefficient to reload the cache file on every cache miss. The last part of wrapper() saves the file whenever the cache is updated, so the cache and file should always be in synch. Loading the file could be moved into outer().