# Is this a safe/correct way to make a python LogHandler asynchronous?

I'm using some slow-ish emit() methods in Python (2.7) logging (email, http POST, etc.) and having them done synchronously in the calling thread is delaying web requests. I put together this function to take an existing handler and make it asynchronous, but I'm new to python and want to make sure my assumptions about object scope are correct. Here's the code:

def patchAsyncEmit(handler):
base_emit = handler.emit
queue = Queue.Queue()
def loop():
while True:
record = queue.get(True) # blocks
try :
base_emit(record)
except: # not much you can do when your logger is broken
print sys.exc_info()
def asyncEmit(record):
queue.put(record)
handler.emit = asyncEmit
return handler


That let's me create an asynchronous LogHandler using code like:

handler = patchAsyncEmit(logging.handlers.HttpHandler(...))


I've confirmed the messages are sent, and not in the calling thread, but I want to make sure I'm getting only one queue and only one thread per Handler, and that I'm not at risk of those somehow going out of scope.

Am I off base or did I get that okay?

-

It would seem to me that writing a class would be better:

class AsyncHandler(threading.Thread):
def __init__(self, handler):
self._handler = handler
self._queue = Queue.Queue()

self.daemon = True
self.start()

def run(self):
while True:
record = self._queue.get(True)
self._handler.emit(record)

def emit(self, record):
self._queue.put(record)


Patching objects can be useful, but its usually clearer to create another object like this.

-
A decorator like that (the pattern not the @pythonthing) is the route I wanted to go, but the LogHandler creation, at least when using the dictionary schema docs.python.org/library/… ), doesn't allow for passing a handler into a handler's constructor -- only literals. So I couldn't think of a way to make that work using dict config (which I recognize I didn't mention in the question), but if there is a way I'd welcome it. –  Ry4an Sep 23 '11 at 16:09
@Ry4an, ah. Actually, I believe you can pass objects using the cfg:// syntax. –  Winston Ewert Sep 23 '11 at 16:36
oh, wow, I hadn't previously seen that at all. –  Ry4an Sep 23 '11 at 16:49
A LogHandler has a few other important methods (configuration, etc.) would all of those have to be written into the AsyncHander to delegate correctly to the _handler? Is there an easy way to pull in all of _handler's method except the one we want to override? –  Ry4an Sep 27 '11 at 4:53
@Ry4an, in that case I'd probably want to subclass the handler of interest. –  Winston Ewert Sep 27 '11 at 12:47