Views Counter made in Python, Gevent and MongoDB

Question

I've created a Views Counter in Python, Gevent and MongoDB (Flask is also included in the full stack as you can see from the context issue in the code).

My gut still tells me that it can be still somehow improved though.

What the code does is initializing a dict "buffer" (just a shelve persisted in memory [no writeback]), define a function where Mongo's bulk_op is initialized, a list comprehension that iterates over the view_count buffer and sets individual find-updates for them with their respective key values that is then ended by an execute with write concern disabled.

Then there's the run_buffer_op function that runs a loop infinitely, which checks if the buffer has over 5000 items, and in that case, flushes its content to the database (by executing the former function), otherwise it just waits 15 minutes before flushing.

This function is finally run (or better, spawned) by Gevent.

Do you see some possible further improvements?

'''
Views counter buffer
====================
'''
# Initialize vc buffer (ObjectId + Views_count pair). We don't need **writeback** as it should just be persisted in memory
view_count=shelve.open('view_count', writeback=False)

def flush_to_db():
  # Apparently you have to set the context to make the db bulk operation work, otherwise it'll return a *Working outside of context* error :(
  with app.app_context():
    bulk = mongo.db.test.initialize_unordered_bulk_op()
    # What this list comprehension does is iterating over the *view_count* buffer
    # and set individual find-updates for them with their respective key value.
    # Like: bulk.find({'_id': '7rhf3d32dh23jd78988ej8'}).update({'$set':{'count':2}})
    #       bulk.find({'_id': '7rhf3d32dh23dg48988ej8'}).update({'$set':{'count':10}})
    #       bulk.find({'_id': '7rhf3d32th23dg48988ek9'}).update({'$set':{'count':7}})
    #       ...

    [bulk.find({ '_id': k }).update({'$set':{'count': v }}) for k,v in view_count.iteritems()]

    # Execute the bulk update with no **write concern**. We can afford to lose some views count if that ever happens in change of better performance.
    bulk.execute(write_concern=None)

def run_buffer_op():
  # Run the loop infinitely
  while True:
    # If buffer has more than 5000 items flush it now to the db
    if len(view_count) > 5000:
      flush_to_db()
    # Else just wait 15 minutes
    gevent.sleep(900)
    flush_to_db()

# Spawn the loop
gevent.spawn(run_buffer_op)

Answer 1

Instead of using a list comprehension, I would simply use a for-loop. It will do the exact same thing and save having to allocate a temporary list.

Also, in run_buffer_op, you first check the number of views. If that value isn't >5000 (almost a DBZ joke) then you sleep for 15min. Once it wakes up, you IMMEDIATELY flush_to_db no matter what. Based on your comments, it seems you only want to flush_to_db if there are >5000 views. Thus, the flush_to_db call immediately after the gevent.sleep(900) seems a little redundant.