I'm writing a simple script which will search a source directory for certain types of files and move them to a destination directory. I'm using it as an opportunity to practice writing some "defensive" code which nicely reports errors to the user, rather than spitting out a stack trace.
As a start, the program should read a config file to determine the source and destination directories. There are many things that could go wrong here: the config file could be missing, have bad permissions, be invalid JSON, omit required fields, or have values of the wrong type. I'd like to anticipate these errors and print a helpful error message to the user while logging a more detailed traceback to a file.
I don't write much user-facing code in my day-to-day work, so I'm not familiar with Python idioms for error reporting. I have a vague idea that I'd like to keep the frontend (which consists of doing the error checking and reporting) separate from the backend. Below is my stab at it:
import collections
import json
import logging
import pathlib
import sys
# Logging #####################################################################
class StreamFormatter(logging.Formatter):
def format(self, record):
"""Squash exception information."""
if record.levelno == logging.ERROR:
return "Error: " + record.message
else:
return record.message
logger = logging.getLogger(__name__)
# create a log file handler with detailed tracebacks
file_handler = logging.FileHandler("debug.log")
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - "
"%(message)s")
file_handler.setFormatter(formatter)
# create a stream handler with no exception info, just prints messages
stream_handler = logging.StreamHandler()
stream_handler.setFormatter(StreamFormatter())
logger.addHandler(file_handler)
logger.addHandler(stream_handler)
###############################################################################
def main():
config = _logged_load_config("config.json")
_logged_validate_config(config)
destinations = _logged_find_destinations(config["dest"])
print(config)
# Config Validation ###########################################################
Check = collections.namedtuple("Check", ["check", "message"])
def is_directory(x):
try:
return pathlib.Path(x).is_dir()
except TypeError:
return False
directory_check = Check(is_directory, "Must be a an existing directory.")
REQUIRED_CONFIG = {
"dest": [directory_check],
"source": [directory_check]
}
# Logged Helper Functions #####################################################
def _logged_load_config(path):
try:
f = open(path)
except IOError:
logger.exception("Cannot open config file.")
sys.exit(1)
try:
config = json.load(f)
except ValueError:
logger.exception("JSON is malformed.")
sys.exit(1)
finally:
f.close()
return config
def _logged_validate_config(config):
missing = REQUIRED_CONFIG.keys() - config.keys()
if missing:
logger.error("Missing the following required keys: {}".format(missing))
sys.exit(1)
is_valid, key, rule = validate_config(config)
if not is_valid:
logger.error("The key '{}' is not valid: {}".format(key, rule.message))
sys.exit(1)
def _logged_find_destinations(dest):
try:
return find_destination_dirs(dest)
except IOError:
logger.exception("Couldn't read destination directory.")
sys.exit(1)
# Library Functions ###########################################################
def validate_config(config):
for key, rules in REQUIRED_CONFIG.items():
for rule in rules:
if not rule.check(config[key]):
return (False, key, rule)
else:
return (True, None, None)
def find_destination_dirs(path):
path = pathlib.Path(path)
return [x for x in path.iterdir() if x.is_dir()]
if __name__ == "__main__":
main()
In this code, I've made a serious effort to keep main()
as clean as possible. I want to be able to open this script a year from now and get a good idea of what is going on by glancing at the contents of main
. That means moving all of the clutter of logging and error checking to _logging_*
functions with self-describing names. I actually quite like this layout. Is this something that's considered good practice?
Another thing I've tried to do is to factor out functionality that I might want to reuse into separate "pure" functions which do no logging, and do not cause the process to exit on error. For example, I might want to use validate_config
in another module, and I definitely do not want it calling sys.exit
if the config isn't valid. Instead, I put the logging and error-checking in _logged_validate_config
, which should be "private" to the module. Granted, I don't have many "backend" functions here -- if I did, I'd toss them in a module.
I've got a module-level logger, even though it's only used by the _logging_*
functions. Should I instead create a logger
instance in the body of main()
and pass it to the utility functions as an argument?
Lastly: When I want to see examples of good, idiomatic library code, I usually look through Python's standard library source. What are some open source projects that I might look to in order to find good examples of user-facing code? That is, code that performs input validation, error handling, logging, etc...
I want to be able to open this script a year from now and get a good idea of what is going on
. Welcome to Code Review! \$\endgroup\$