I'm working on a Python script and I was searching for a method to redirect stdout
and stderr
of a subprocess to the logging module. The subprocess is created using the subprocess.call()
method.
The difficulty I faced is that with subprocess I can redirect stdout
and stderr
only using a file descriptor. I did not find any other method, but if there is one please let me know!
To solve this problem I wrote the following code which basically creates a pipe and uses a thread to read from the pipe and generates a log message using the Python logging method:
import subprocess
import logging
import os
import threading
class LoggerWrapper(threading.Thread):
"""
Read text message from a pipe and redirect them
to a logger (see python's logger module),
the object itself is able to supply a file
descriptor to be used for writing
fdWrite ==> fdRead ==> pipeReader
"""
def __init__(self, logger, level):
"""
Setup the object with a logger and a loglevel
and start the thread
"""
# Initialize the superclass
threading.Thread.__init__(self)
# Make the thread a Daemon Thread (program will exit when only daemon
# threads are alive)
self.daemon = True
# Set the logger object where messages will be redirected
self.logger = logger
# Set the log level
self.level = level
# Create the pipe and store read and write file descriptors
self.fdRead, self.fdWrite = os.pipe()
# Create a file-like wrapper around the read file descriptor
# of the pipe, this has been done to simplify read operations
self.pipeReader = os.fdopen(self.fdRead)
# Start the thread
self.start()
# end __init__
def fileno(self):
"""
Return the write file descriptor of the pipe
"""
return self.fdWrite
# end fileno
def run(self):
"""
This is the method executed by the thread, it
simply read from the pipe (using a file-like
wrapper) and write the text to log.
NB the trailing newline character of the string
read from the pipe is removed
"""
# Endless loop, the method will exit this loop only
# when the pipe is close that is when a call to
# self.pipeReader.readline() returns an empty string
while True:
# Read a line of text from the pipe
messageFromPipe = self.pipeReader.readline()
# If the line read is empty the pipe has been
# closed, do a cleanup and exit
# WARNING: I don't know if this method is correct,
# further study needed
if len(messageFromPipe) == 0:
self.pipeReader.close()
os.close(self.fdRead)
return
# end if
# Remove the trailing newline character frm the string
# before sending it to the logger
if messageFromPipe[-1] == os.linesep:
messageToLog = messageFromPipe[:-1]
else:
messageToLog = messageFromPipe
# end if
# Send the text to the logger
self._write(messageToLog)
# end while
print 'Redirection thread terminated'
# end run
def _write(self, message):
"""
Utility method to send the message
to the logger with the correct loglevel
"""
self.logger.log(self.level, message)
# end write
# end class LoggerWrapper
# # # # # # # # # # # # # #
# Code to test the class #
# # # # # # # # # # # # # #
logging.basicConfig(filename='command.log',level=logging.INFO)
logWrap = LoggerWrapper( logging, logging.INFO)
subprocess.call(['cat', 'file_full_of_text.txt'], stdout = logWrap, stderr = logWrap)
print 'Script terminated'
For logging subprocesses' output, Google suggests to directly redirect the output to a file in a way similar to this:
sobprocess.call( ['ls'] stdout = open( 'logfile.log', 'w') )
This is not an option for me since I need to use the formatting and loglevel
facilities of the logging module. I also suppose that having a file open in write mode but two different entities is not permitted / not a sane thing to do.
I would now like to see your comments and enhancement proposal. I would also like to know if there is already a similar object in the Python library since I found nothing to accomplish this task!
super()
to call superclass methods. So instead ofthreading.Thread.__init__(self)
, writesuper(LoggerWrapper, self).__init__()
. \$\endgroup\$self.pipeReader.readline()
keeps returning newlines. \$\endgroup\$