2
\$\begingroup\$

In a current project of I have multiple modules that need to read from and write to a config.json file, where the settings are saved as nested dictionaries. I chose json here, because I want the config file to be human-readable.

Simplified example config.json:

{
  "server_data": {
    "email_address": "something@somewhere.com",
    "server_address": "imap.server.com",
    "server_port": 993
  },
  "handle_emails": {
    "output_directory": "C:\\Users\\riskypenguin\\some_directory",
    "mark_read": false,
    "write_to_file": true,
    "folders": ["inbox"]
  },
  "output": {
    "output_directory": "C:\\Users\\riskypenguin\\some_other_directory",
    "print_timeinfo": true,
    "print_form_counts": true
  }
}

My config_handler.py provides standardised access to the file via get_setting and set_setting. For different use cases throughout the project I need to be able to access the configuration settings at varying depths. For example: Some boolean settings are read and written directly, but the dict at server_data will be read and processed as a whole. Concurrency is not relevant for this project, so there are no collisions regarding read and write access.

config_handler.py

import os
import json
from functools import reduce

from definitions import CONFIG_DIR, CONFIG_FILE, DEFAULT_CONFIG_FILE

__all__ = ["get_setting", "set_setting"]


# Helper functions

def _save_config_data(content):
    if not os.path.exists(CONFIG_DIR):
        os.makedirs(CONFIG_DIR)

    with open(CONFIG_FILE, 'w') as f:
        json.dump(obj=content, fp=f, indent=2)


def _load_config_data():
    file = CONFIG_FILE if os.path.exists(CONFIG_FILE) else DEFAULT_CONFIG_FILE

    with open(file, 'r') as f:
        return json.load(f)


_config_data = _load_config_data()


# Exported functions

# Old version of get_setting, currently not in use
def get_setting_old(*args, default=None):
    current_value = _config_data

    for key in args:
        if isinstance(current_value, dict):
            current_value = current_value.get(key, default)
        else:
            return default

    return current_value


def get_setting(*config_keys, default=None):
    drill_down = lambda x, y: dict.get(x, y, default) if isinstance(x, dict) else default
    return reduce(drill_down, config_keys, _config_data)


def set_setting(*config_keys, value):
    current_value = _config_data

    for key in config_keys[:-1]:
        if isinstance(current_value, dict):
            current_value = current_value.get(key)
        else:
            return False

    last_key = config_keys[-1]
    if isinstance(current_value, dict):
        current_value[last_key] = value
    else:
        return False

    _save_config_data(_config_data)

    return True

As this is my first shot at handling configurations in a central way like this I am grateful for all feedback. I'm particularly interested in feedback about:

  1. General approach
  2. Comparison between the haskell-like approach of get_setting vs. the simpler and more readable approach of get_setting_old
  3. set_setting seems rather unelegant to me. As the types of a setting can be either mutable or immutable I don't see a better solution though. I've thought about wrapper classes for primitive types, but that feels rather unpythonic and creates some unnecessary overhead when reading from and before writing to json. Maybe I'm missing something here.
  4. I've been thinking about creating a simple class for a path of config_keys, to be able to pass a ConfigKeyPath object instead of a bunch of separated string arguments. I'm not sure which of the two approaches is better.
\$\endgroup\$
2
  • \$\begingroup\$ Why are you only allowed to get items from dictionaries? Why are lists on anything else that implements __getitem__ not allowed? \$\endgroup\$
    – Peilonrayz
    Apr 11 at 22:52
  • \$\begingroup\$ The short answer is that I hadn't thought of that up until now. I guess my current approach is: Anything that is not a dict marks the lowest level of a setting. Any further handling needs to be done by the code that retrieves the setting. Refactoring get_setting it to use __getitem__ instead of dict.get should be easy enough though. \$\endgroup\$ Apr 11 at 23:21
1
\$\begingroup\$
  1. Rather than os and os.path use pathlib.Path.

    def _save_config_data(content):
        CONFIG.parent.mkdir(parents=True, exist_ok=True)
        with open(CONFIG, 'w') as f:
            json.dump(obj=content, fp=f, indent=2)
    
    
    def _load_config_data():
        file = CONFIG if CONFIG.exists() else DEFAULT_CONFIG
        with open(file, 'r') as f:
            return json.load(f)
    
  2. Your code doesn't follow idiomatic Python. Python uses errors for control flow so returning True/False in set_setting is not idiomatic.

  3. You should make a walk function which you call from both get_setting and set_setting.

  4. Reduce is generally not a good solution. I would say in the specific example of your code, get_setting_old has far superior readability to get_setting.

import os
import json
from functools import reduce

from definitions import CONFIG, DEFAULT_CONFIG

__all__ = [
    "get_setting",
    "set_setting",
]


# Helper functions

def _save_config_data(content):
    CONFIG.parent.mkdir(parents=True, exist_ok=True)
    with open(CONFIG, 'w') as f:
        json.dump(obj=content, fp=f, indent=2)


def _load_config_data():
    file = CONFIG if CONFIG.exists() else DEFAULT_CONFIG
    with open(file, 'r') as f:
        return json.load(f)


def _walk(obj, path):
    for segment in path:
        obj = obj[segment]
    return obj


def get_setting(*args, default=_SENTINEL):
    try:
        return _walk(_CONFIG_DATA, args)
    except LookupError:
        if default is _SENTINEL:
            raise LookupError(f"cannot walk path; {args}") from None
        else:
            return default


def set_setting(value, *args):
    *args, segment = args
    try:
        node = _walk(_CONFIG_DATA, args)
        node[segment] = value
    except LookupError:
        raise LookupError(f"cannot set path; {args}") from None


_SENTINEL = object()
_CONFIG_DATA = _load_config_data()
  1. I think using a class would be better as then you can define __getitem__ and __setitem__ for a potentially nicer interface.
class Walker:
    def __init__(self, data):
        self._data = data
        
    def __getitem__(self, args):
        if not isinstance(args, tuple):
            args = (args,)
        return self.get(*args)

    def __setitem__(self, args, value):
        if not isinstance(args, tuple):
            args = (args,)
        self.set(value, *args)

    def _walk(self, obj, path):
        for segment in path:
            obj = obj[segment]
        return obj

    def get(self, *args, default=_SENTINEL):
        try:
            return _walk(_CONFIG_DATA, args)
        except LookupError:
            if default is _SENTINEL:
                raise LookupError(f"cannot walk path; {args}") from None
            else:
                return default

    def set(self, value, *args):
        *args, segment = args
        try:
            node = _walk(_CONFIG_DATA, args)
            node[segment] = value
        except LookupError:
            raise LookupError(f"cannot set path; {args}") from None
\$\endgroup\$
2
  • \$\begingroup\$ I guess I was a bit too caught up in last semesters functional programming lecture when opting for reduce. Although I enjoyed its conciseness, I do agree it harms readability. Thank you for the comprehensive suggestions. I will take a closer look at pathlib as well as your suggestions regarding the general approach. \$\endgroup\$ Apr 12 at 14:25
  • 1
    \$\begingroup\$ @riskypenguin wrt pathlib the very last section of the docs will likely be very helpful. Yeah different languages have different idioms. A Haskell programmer would look favourably on code which uses reduce. A C# programmer would use the tryGet pattern here. A good course will expose you to many common patterns, but not all patterns are good in every language. \$\endgroup\$
    – Peilonrayz
    Apr 12 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.