Python Runtime Config

Question

I am implementing runtime configuration for a python package. I am doing this in mypkg.config.__init__.py. The idea is to load the configuration from a JSON file into a ConfigManagerClass that uses a Pydantic model to validate and store the configuration. The ConfigManager lets users of the package update the config from another JSON file of their choice when needed.

Here's the code:

"""Runtime configuration for the package."""
import importlib.resources as pkg_resources
import json
from mypkg.pydantic.schemas import RuntimeConfigSchema


class RuntimeConfigManager():
    with pkg_resources.open_text('mypkg.config', 'default.json') as f:
        _config_json = json.load(f)
        print('Loading default config')
        _config = RuntimeConfigSchema(**_config_json)
        print('Config loaded')

    @classmethod
    def set_config_file(cls, config_file):
        """Update runtime config from file."""
        print(f'Updating config from {config_file}')
        with open(config_file, 'r') as f:
            cls._config_json = json.load(f)
            cls._config = RuntimeConfigSchema(**cls._config_json)
            print('Config updated')
    
    @classmethod
    def get_config(cls):
        """Get current runtime config."""
        return cls._config

Is this idiomatic? Are there any failure cases to be aware of? The package is compatible with Python >=3.8.10, <3.10

Answer 1

Is this idiomatic?

Largely it is, yes.

clean import

We strive to minimize side effects at import time, deferring them until some caller actually calls into the library code.

        print('Loading default config')
        _config = ...
        print('Config loaded')

Signalling to other engineers that _config is private works nicely.

The pair of print()'s produce a side effect on stdout and there's no provision to turn that off. Prefer to use a logger at DEBUG priority.

A caller should be able to pull in this module, say from a unit test, and see the import succeed silently.

Similarly for set_config_file.

---

deprecated function

The package is compatible with Python >=3.8.10, <3.10

Thank you for calling that out.

I am reading the docs:

Deprecated since version 3.11: Calls to this function can be replaced by open().

It would be worth noting that in a # comment or """docstring""", perhaps at module level. I imagine you wrote this for a project that happens to be at cPython 3.8, so you just went with what works. Consider writing a different classfunction that a 3.12 app could successfully call. Consider writing down a sentence or two on "roadmap" or "transition plans". My concern is that it would be sad for a developer to take a dependency on this class, and then feel constrained to ancient interpreters as 3.15 and 3.16 roll out.

---

getter

    @classmethod
    def get_config(cls):
        """Get current runtime config."""
        return cls._config

This is nice enough. The _private class attribute is clear, and we offer no setter.

But are we really concerned that some engineer will feel emboldened to write crazy stuff onto the config? We're all adults here. My reading is that maybe we don't need this getter at all, and a public cls.config identifier might be the appropriate spelling.

Recall that if you change your mind later, you can always introduce a @property for the getter, with no change to app-level callers. The part of my proposal that's visible at the app level is whether access is (roughly) through config() or through config. That second form can still work out to a method call once a property is involved.

---

helper

I am looking at the _config_json class attribute and thinking that maybe we just created it by accident?

The cool thing about helper functions is that all local variables disappear once we return and they go out-of-scope. I'm thinking that maybe the with should call a helper which defines a config json temp var, and then the temp var evaporates upon return.

Down in set_config_file we assign cls._config_json for consistency. But it seems like it wants to be an ephemeral local variable there.

---

This code achieves its design goals.

I would be willing to delegate or accept maintenance tasks on it.