Python logging handler

Question

I'm working on a logger that has a name of the module that called the logger (when I create an instance of a logger in my program, I call LoggingHandler(__name__)) to send all the messages, including info and debug, to the log file, and print the messages specified by max_level to console (so, by default, it will not print info and debug messages to console, but will still write them into file).

The problem came when I was managing levels. If I set level in basicConfig to "WARNING", then it will not print info and debug to file, even though I've set fh.setLevel(logging.DEBUG). It just won't go to levels lower than the one specified in basicConfig. Okay, I could just go ahead and specify filename in basicConfig to make it output to file, but I want a RotatingFileHandler to take care of it (because I require its rollover functionality). So, I've set level in basicConfig to "NOTSET", the lowest one possible. Things go better now except one problem. The output to console doubles. It prints

[2016-08-29 10:58:20,976] __main__: logging_handler.py[LINE:51]# WARNING   hello
[2016-08-29 10:58:20,976] __main__: logging_handler.py[LINE:51]# WARNING   hello
[2016-08-29 10:58:20,977] __main__: logging_handler.py[LINE:48]# ERROR     hola
[2016-08-29 10:58:20,977] __main__: logging_handler.py[LINE:48]# ERROR     hola
[2016-08-29 10:58:20,977] __main__: logging_handler.py[LINE:54]# INFO      info message
[2016-08-29 10:58:20,977] __main__: logging_handler.py[LINE:57]# DEBUG     debug message

So, the global logger does the output and the StreamHandler does. I need to prevent the global logger from outputting anything. So I redirect its output to a "dummy" class Devnull. Now the code works exactly as I need, but it feels like such approach is what they call "bodging". So, I'd like to know if there's a better way to write this code.

#!/usr/bin/python3 -u
# -*- coding: utf-8 -*-
import logging
from logging.handlers import RotatingFileHandler
from os import path, makedirs

LOGS_DIR = "logs"
DEBUG_FILE_NAME = 'debug.log'
MAX_LOG_SIZE = 10*1024*1024
BACKUP_FILES_AMOUNT = 3
LOG_FORMAT = u'[%(asctime)s] %(name)s: %(filename)s[LINE:%(lineno)d]# %(levelname)-8s  %(message)s'


class Devnull(object):
    def write(self, *_, **__): pass


class LoggingHandler:
    def __init__(self, logger_name, max_level="WARNING"):
        makedirs(LOGS_DIR, exist_ok=True)

        logging.basicConfig(format=LOG_FORMAT,
                            level="NOTSET",
                            stream=Devnull(),
                            )

        self.main_logger = logging.getLogger(logger_name)

        # create file handler which logs even debug messages
        fh = RotatingFileHandler(path.join(LOGS_DIR, DEBUG_FILE_NAME),
                                 maxBytes=MAX_LOG_SIZE, backupCount=BACKUP_FILES_AMOUNT)
        fh.setLevel(logging.DEBUG)
        # create console handler with a higher log level
        ch = logging.StreamHandler()
        ch.setLevel(max_level)
        # create formatter and add it to the handlers
        fmter = logging.Formatter(LOG_FORMAT)
        fh.setFormatter(fmter)
        ch.setFormatter(fmter)
        # add the handlers to the logger
        self.main_logger.addHandler(fh)
        self.main_logger.addHandler(ch)


    def error(self, message):
        self.main_logger.error(message)

    def warning(self, message):
        self.main_logger.warning(message)

    def info(self, message):
        self.main_logger.info(message)

    def debug(self, message):
        self.main_logger.debug(message)

if __name__ == '__main__':
    # Tests
    log = LoggingHandler(__name__)
    log.warning("hello")
    log.error("hola")
    log.info("info message")
    log.debug("debug message")

Answer 1

You simply need to understand what's going on when you write:

    logging.basicConfig(format=LOG_FORMAT,
                        level="NOTSET",
                        stream=Devnull(),
                        )

This creates a root logger with level NOTSET and a default handler which won't log anything thanks to Devnull. I think this also has the added benefit of settings the level of all future loggers to NOTSET.

But you don't want a root logger! You're defining your own logger with your own handlers and so on. So basicConfig() is doing more harm than good here. Remove it!

Now there's one remaining issue: if you do this, you won't see debug logs, because the default log level is WARNING. So you need to add self.main_logger.setLevel(logging.DEBUG): the logger will accept all log levels, and each handler will decide whether or not to process the log record depending on the log level of the record and its own log level. (In other words, you're back to the original code you used which worked very well.)