Extract value from config.json (Python)

Question

Usually my project directory structure look like:

project_name/
 lib/
  utils.py
 logs/
 main.py
 requirements.txt
 config.json

My config.json file looks like:

{
    "api": {
        "api_key": "abcdefg",
        "api_secret": "hijklmn"
    },
    "trading": {
        "invest_amount": 50,
        "mode": "futures",
        "cumulative_position": false
    }
}

In utils.py, I have a class Config

import typing
import inspect
import os
import json
from typing import Any


class Config(object):
    _CONFIG_FILE: typing.Optional[str] = None
    _CONFIG: typing.Optional[dict] = None

    def __init__(self):
        config_file = Config.get_config_path(frame=inspect.stack()[1])
        with open(config_file, 'r') as f:
            Config._CONFIG = json.load(f)

    @staticmethod
    def get_config_path(frame: inspect.FrameInfo, config_file_name: str = "config.json") -> str:
        caller_file_name = frame[0].f_code.co_filename
        caller_folder_name = os.path.dirname(caller_file_name)
        config_file_name = os.path.join(caller_folder_name, config_file_name)
        return config_file_name

    @staticmethod
    def get_config_value(param: str) -> Any:
        for key in Config._CONFIG.keys():
            value = Config._CONFIG.get(key).get(param, None)
            if value is not None:
                return value
        raise ValueError(f"{param} doesn't exist")

And I will use it in main.py usually:

from lib.utils import Config


def main(trading_mode, invest_amount, cumulative_position):
    print(trading_mode, invest_amount, cumulative_position)


if __name__ == "__main__":
    config = Config()
    main(trading_mode=config.get_config_value(param="mode"),
         invest_amount=config.get_config_value(param="invest_amount"),
         cumulative_position=config.get_config_value(param="cumulative_position"))

Is there any code that can improve or rewrite in class Config, for the purpose of efficiency, readability or clear logic, etc?

Answer 1

The good parts

You code is very readable. You stick to PEP 8 for the most part (except the annoying line length limitation) and use reasonable naming.

Managing configuration

Your first mistake, imho, is that you make the config part of the project. Having the config in the project's repo, especially if it contains things like api_secret (or password etc.) is a common source for secret leakage via repo hosting services such as Git{Hub,Lab}.

Load config explicitly.

Currently, one of the first things your script main.py does is to read data from the file system. Though this is not immediately visible to the reader, since it is hidden in Config.__init__().

You can write a simple function like

from json import load
from pathlib import Path
from typing import Any

...

def load_config(filename: Path | str) -> dict[str, Any]:
    """Load configuration from a JSON file."""

    with open(filename, 'rb') as file:
        return load(file)

to load configuration from a JSON file by explicitly specifying it's path and use the returned dict.

If you have JSON, use JSON

Python's json library already does all the work to parse JSON objects into associative arrays aka. dicts. Use them directly to access the desired data:

from lib.utils import Config


def main(trading_mode, invest_amount, cumulative_position):
    print(trading_mode, invest_amount, cumulative_position)


if __name__ == "__main__":
    config = load_config('/path/to/my/config.json')
    trading_cfg = config.get('trading')
    main(
        trading_mode=trading_cfg.get('mode'),
        invest_amount=trading_cfg.get('invest_amount'),
        cumulative_position=trading_cfg.get('cumulative_position')
    )

All the introspection automagic might feel comfortable at first, but it hides away a lot of potential side-effects. Furthermore, it won't be needed anyways if you separate your configuration from your project source code.