I've recently been writing an application which does a lot of work with the output of commands. To call the commands I've been using the subprocess module, and originally, was using
subprocess.PIPEs for the
stderr, and then using the communicate method once the process was complete to retrieve the output to work with it, log it, etc. Unfortunately, I had a problem with deadlocks when a particular command generated a long output stream.
For this reason, I've written an
OutputStream class, an instance of which can be provided as the
stderr params of
subprocess.Popen. In the background, it writes to a
cStringIO buffer which I can retrieve the value from later, and it seems to work well. However, this is the first time I've worked with anything like this and I'm concerned that I might be doing something potentially unsafe. I'd appreciate any feedback anybody has.
from cStringIO import StringIO import os import threading class OutputStream(threading.Thread): def __init__(self): super(OutputStream, self).__init__() self.done = False self.buffer = StringIO() self.read, self.write = os.pipe() self.reader = os.fdopen(self.read) self.start() def fileno(self): return self.write def run(self): while not self.done: self.buffer.write(self.reader.readline()) self.reader.close() def close(self): self.done = True os.close(self.write)
I would then use it something like this:
import subprocess import time stream = OutputStream() proc = subprocess.Popen(['ls', '-l'], stdout=stream, stderr=subprocess.STDOUT) while proc.poll() is None: time.sleep(0.05) stream.close() output = stream.buffer.getvalue() print output
Edit: When running the test code above, Python doesn't exit once the command output has been printed. Such behaviour isn't exhibited by my application in which this class is being used.