1
\$\begingroup\$

Usually my project directory structure look like:

project_name/
 lib/
  utils.py
 logs/
 main.py
 requirements.txt
 config.json

My config.json file looks like:

{
    "api": {
        "api_key": "abcdefg",
        "api_secret": "hijklmn"
    },
    "trading": {
        "invest_amount": 50,
        "mode": "futures",
        "cumulative_position": false
    }
}

In utils.py, I have a class Config

import typing
import inspect
import os
import json
from typing import Any


class Config(object):
    _CONFIG_FILE: typing.Optional[str] = None
    _CONFIG: typing.Optional[dict] = None

    def __init__(self):
        config_file = Config.get_config_path(frame=inspect.stack()[1])
        with open(config_file, 'r') as f:
            Config._CONFIG = json.load(f)

    @staticmethod
    def get_config_path(frame: inspect.FrameInfo, config_file_name: str = "config.json") -> str:
        caller_file_name = frame[0].f_code.co_filename
        caller_folder_name = os.path.dirname(caller_file_name)
        config_file_name = os.path.join(caller_folder_name, config_file_name)
        return config_file_name

    @staticmethod
    def get_config_value(param: str) -> Any:
        for key in Config._CONFIG.keys():
            value = Config._CONFIG.get(key).get(param, None)
            if value is not None:
                return value
        raise ValueError(f"{param} doesn't exist")

And I will use it in main.py usually:

from lib.utils import Config


def main(trading_mode, invest_amount, cumulative_position):
    print(trading_mode, invest_amount, cumulative_position)


if __name__ == "__main__":
    config = Config()
    main(trading_mode=config.get_config_value(param="mode"),
         invest_amount=config.get_config_value(param="invest_amount"),
         cumulative_position=config.get_config_value(param="cumulative_position"))

Is there any code that can improve or rewrite in class Config, for the purpose of efficiency, readability or clear logic, etc?

\$\endgroup\$
9
  • 2
    \$\begingroup\$ I think this is over-engineered. Just load a config file from an explicit path and use the dict returned by json.load(). \$\endgroup\$ Oct 26, 2022 at 11:16
  • \$\begingroup\$ Do you mean using with open("config.json", "r")? How should I store the desired values and input these into main()? Should I constructure a function like config.get_config_value for this? \$\endgroup\$
    – tung
    Oct 26, 2022 at 11:42
  • \$\begingroup\$ How is assigning a dict to a variable config any different from assigning an instance of Config to such a variable? And given your config is a dict, you can just use cofig.get(key) instead of config.get_config_value(key). \$\endgroup\$ Oct 26, 2022 at 11:44
  • \$\begingroup\$ Can you post the correction on how should I edit the class Config or using config.get(key)? It's a bit confusing for me to understand the words, better in answer the question instead of comment, it would be helpful, many thanks! \$\endgroup\$
    – tung
    Oct 26, 2022 at 11:55
  • \$\begingroup\$ seems very complex for a simple read function? \$\endgroup\$
    – Barb
    Oct 29, 2022 at 10:10

1 Answer 1

1
\$\begingroup\$

The good parts

You code is very readable. You stick to PEP 8 for the most part (except the annoying line length limitation) and use reasonable naming.

Managing configuration

Your first mistake, imho, is that you make the config part of the project. Having the config in the project's repo, especially if it contains things like api_secret (or password etc.) is a common source for secret leakage via repo hosting services such as Git{Hub,Lab}.

Load config explicitly.

Currently, one of the first things your script main.py does is to read data from the file system. Though this is not immediately visible to the reader, since it is hidden in Config.__init__().

You can write a simple function like

from json import load
from pathlib import Path
from typing import Any

...

def load_config(filename: Path | str) -> dict[str, Any]:
    """Load configuration from a JSON file."""

    with open(filename, 'rb') as file:
        return load(file)

to load configuration from a JSON file by explicitly specifying it's path and use the returned dict.

If you have JSON, use JSON

Python's json library already does all the work to parse JSON objects into associative arrays aka. dicts. Use them directly to access the desired data:


from lib.utils import Config


def main(trading_mode, invest_amount, cumulative_position):
    print(trading_mode, invest_amount, cumulative_position)


if __name__ == "__main__":
    config = load_config('/path/to/my/config.json')
    trading_cfg = config.get('trading')
    main(
        trading_mode=trading_cfg.get('mode'),
        invest_amount=trading_cfg.get('invest_amount'),
        cumulative_position=trading_cfg.get('cumulative_position')
    )

All the introspection automagic might feel comfortable at first, but it hides away a lot of potential side-effects. Furthermore, it won't be needed anyways if you separate your configuration from your project source code.

\$\endgroup\$
3
  • \$\begingroup\$ I usually add config.json into .ignore when using github/gitlab, is there any better alternative for that? If using the load_config function you suggested, will it be a bit slower or redundant to read the config file path everytime I call the function, e.g. I need to extract ten values in config.json then I need to use with open(file_name, "rb") ten times. \$\endgroup\$
    – tung
    Oct 26, 2022 at 14:09
  • \$\begingroup\$ @tung have a read about dotenv files \$\endgroup\$ Oct 27, 2022 at 6:33
  • \$\begingroup\$ but I am using config.json instead of .env @GrajdeanuAlex \$\endgroup\$
    – tung
    Oct 27, 2022 at 9:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.