3
\$\begingroup\$

This is how I handle configuration in my python code, currently, but I guess it might look magic to people, and it is cumbersome to code. How can this be improved? Are there better solutions?

The purpose is to have a general working configuration stored in UPPER CASE in global variables at the beginning of the file. When first run, it creates a config.json file with that configuration. That can be edited by the user, and it would supersede the default configuration, if run again. In order to still benefit from typing hints in modern IDEs, the cfg[] dict is not used directly, but the configuration is written back to the global variables.

This is cumbersome to code, and it contains a lot of repetition. Every variable is touched multiple times.

How can I improve this system and make it more elegant and transparent?

import json
from typing import Any, Dict

CONFIG_FILE = 'config.json'
GLOBAL_CONFIG_EXAMPLE_PORT = 3


def read_config() -> Dict[str, Any]:
    try:
        with open(CONFIG_FILE) as config_file:
            return json.load(config_file)
    except FileNotFoundError:
        pass
    # generate standard config file
    cfg = {
        'example': {
            'port': GLOBAL_CONFIG_EXAMPLE_PORT,
        },
    }
    with open(CONFIG_FILE, 'w') as f:
        json.dump(cfg, f)

    set_global_variables(cfg)
    return cfg


def set_global_variables(cfg: Dict[str, Any]):
    global GLOBAL_CONFIG_EXAMPLE_PORT
    GLOBAL_CONFIG_EXAMPLE_PORT = cfg['example']['port']


def main():
    cfg: Dict[str, Any] = read_config()
    print(cfg['example']['port'], GLOBAL_CONFIG_EXAMPLE_PORT)


if __name__ == "__main__":
    main()

A real-life config file would be this:

{
  "mqtt": {
    "host": "10.21.1.77",
    "port": 1883,
    "topic": "EXTENSE/Lab/XYZ/move/#",
    "topic_ack": "EXTENSE/Lab/XYZ/move/ack"
  },
  "signal": {
    "save": true,
    "length": 60
  },
  "sensor": {
    "num": 5,
    "trigger": 4
  },
  "logging": {
    "level": 10,
    "filename": "log.txt",
    "console": true,
    "format": "%(asctime)s %(levelname)s: %(message)s"
  }
}

PS: How would you go about writing a test for this, efficiently?

Addendum: Is it feasible and sensible to do some magic on the variable name which is all upper case, split them by underscore, and create the dict automatically?

\$\endgroup\$
9
  • 1
    \$\begingroup\$ Can you include an example JSON file? \$\endgroup\$
    – Reinderien
    Jun 19 at 12:12
  • \$\begingroup\$ certainly, @Reinderien :-) \$\endgroup\$ Jun 19 at 12:45
  • \$\begingroup\$ @Reinderien i updated the config file, perhaps this illustrates the use a little better. \$\endgroup\$ Jun 21 at 17:52
  • 1
    \$\begingroup\$ I don't know if it's just me, but I just store configurations in .py files and import the variables from the files, and I used .__name__, repr() and json.dumps() and quotes and some code to print these configurations to python files. I know it's dirty but it is very effective. \$\endgroup\$ Jun 23 at 13:04
  • 1
    \$\begingroup\$ If you really want to know, I will post an answer tomorrow, but not today. \$\endgroup\$ Jun 23 at 13:53
3
\$\begingroup\$

read_config does not do what's on the tin: it attempts to read the config, conditionally writes out a default config, and sets globals. Those obligations should be separated.

Also, your example is a mystery: is it a section? Currently it offers no value and a flat dictionary would be simpler to manipulate.

GLOBAL_CONFIG_EXAMPLE_PORT on its own is not a useful global. Consider instead moving the entire default configuration dictionary to a global constant.

Something like:

import json
from typing import Any, Dict

CONFIG_FILE = 'config.json'

DEFAULT_CONFIG = {
    'port': 3,
}


def read_config() -> Dict[str, Any]:
    with open(CONFIG_FILE) as f:
        return json.load(f)


def write_config(config: Dict[str, Any]) -> None:
    with open(CONFIG_FILE, 'w') as f:
        json.dump(config, f)


def load_or_default_config() -> None:
    try:
        config = read_config()
    except FileNotFoundError:
        config = DEFAULT_CONFIG
        write_config(config)

    globals().update(config)
\$\endgroup\$
5
  • \$\begingroup\$ thanks for telling me about globals()! I like your approach a lot, and it's a lot more elegant than mine. However, it lacks the list of global variables at the beginning of the program which I thought to be a part of a good python coding style, and replaces it with a dict. Do you think that's a step forward? \$\endgroup\$ Jun 21 at 18:42
  • \$\begingroup\$ How would your approach handle multi-level global constants, like in the real-life config example above? @reinderien \$\endgroup\$ Jun 21 at 18:55
  • \$\begingroup\$ Spin up a @dataclass for each section so that the section variables have types; have a global class instance for each section \$\endgroup\$
    – Reinderien
    Jun 21 at 19:01
  • \$\begingroup\$ and how would you combine those dataclasses into one? could you do the example in the (real life) config file and update your code, perhaps? \$\endgroup\$ Jun 21 at 19:07
  • \$\begingroup\$ i mean: in my example you would have four global classes, because there are four sections. how do you combine those four classes into one dict to write into a config file, and how do you read them back into the right class, if there is a config file that you parsed and need to split up into the correct classes? \$\endgroup\$ Jun 21 at 19:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.