I'm working on a Python script and I was searching for a method to redirect
stderr of a subprocess to the logging module, the subprocess is createdusing the
The difficulty I faced is that with subprocess I can redirect
stderr only using a file descriptor. I did not find any other method, but if there is one please let me know!
To solve this problem I wrote the following code which basically creates a pipe and uses a thread to read from the pipe and generate a log message using the python logging method:
import subprocess import logging import os import threading class LoggerWrapper(threading.Thread): """ Read text message from a pipe and redirect them to a logger (see python's logger module), the object itself is able to supply a file descriptor to be used for writing fdWrite ==> fdRead ==> pipeReader """ def __init__(self, logger, level): """ Setup the object with a logger and a loglevel and start the thread """ # Initialize the superclass threading.Thread.__init__(self) # Make the thread a Daemon Thread (program will exit when only daemon # threads are alive) self.daemon = True # Set the logger object where messages will be redirected self.logger = logger # Set the log level self.level = level # Create the pipe and store read and write file descriptors self.fdRead, self.fdWrite = os.pipe() # Create a file-like wrapper around the read file descriptor # of the pipe, this has been done to simplify read operations self.pipeReader = os.fdopen(self.fdRead) # Start the thread self.start() # end __init__ def fileno(self): """ Return the write file descriptor of the pipe """ return self.fdWrite # end fileno def run(self): """ This is the method executed by the thread, it simply read from the pipe (using a file-like wrapper) and write the text to log. NB the trailing newline character of the string read from the pipe is removed """ # Endless loop, the method will exit this loop only # when the pipe is close that is when a call to # self.pipeReader.readline() returns an empty string while True: # Read a line of text from the pipe messageFromPipe = self.pipeReader.readline() # If the line read is empty the pipe has been # closed, do a cleanup and exit # WARNING: I don't know if this method is correct, # further study needed if len(messageFromPipe) == 0: self.pipeReader.close() os.close(self.fdRead) return # end if # Remove the trailing newline character frm the string # before sending it to the logger if messageFromPipe[-1] == os.linesep: messageToLog = messageFromPipe[:-1] else: messageToLog = messageFromPipe # end if # Send the text to the logger self._write(messageToLog) # end while print 'Redirection thread terminated' # end run def _write(self, message): """ Utility method to send the message to the logger with the correct loglevel """ self.logger.log(self.level, message) # end write # end class LoggerWrapper # # # # # # # # # # # # # # # Code to test the class # # # # # # # # # # # # # # # logging.basicConfig(filename='command.log',level=logging.INFO) logWrap = LoggerWrapper( logging, logging.INFO) subprocess.call(['cat', 'file_full_of_text.txt'], stdout = logWrap, stderr = logWrap) print 'Script terminated'
This is my solution to a problem. I would now like to see your comments and enhancement proposal. I would also like to know if there is already a similar object in the Python library since I found nothing to accomplish this task!
PS: googling around I found many answer to the question "How can I log subprocesses output in python?" but almost all of the answers suggest to directly redirect the output to a file in a way similar to this:
sobprocess.call( ['ls'] stdout = open( 'logfile.log', 'w') )
this is not an option for me since I need to use the formatting and loglevel facilities of the logging module ....also I suppose that having a file open in write mode bu two different entities is not permitted / not a sane thing to do :P