I'm using some slow-ish emit() methods in Python (2.7) logging (email, http POST, etc.) and having them done synchronously in the calling thread is delaying web requests. I put together this function to take an existing handler and make it asynchronous, but I'm new to python and want to make sure my assumptions about object scope are correct. Here's the code:
def patchAsyncEmit(handler): base_emit = handler.emit queue = Queue.Queue() def loop(): while True: record = queue.get(True) # blocks try : base_emit(record) except: # not much you can do when your logger is broken print sys.exc_info() thread = threading.Thread(target=loop) thread.daemon = True thread.start() def asyncEmit(record): queue.put(record) handler.emit = asyncEmit return handler
That let's me create an asynchronous LogHandler using code like:
handler = patchAsyncEmit(logging.handlers.HttpHandler(...))
I've confirmed the messages are sent, and not in the calling thread, but I want to make sure I'm getting only one queue and only one thread per Handler, and that I'm not at risk of those somehow going out of scope.
Am I off base or did I get that okay?