E-commerce product price tracker

Question

I building a very simple price tracker web app. I am using MongoDB with pymongo. The user will enter the URL of the product he wishes to track and the desired amount, when the price goes below this amount, he should get an alert. It doesn't matter how many products he is tracking, I will send the alert when price changes in one product. I will sending generalized alert like 'Some of the product/s available at cheaper price' or something like that.

There are three collections in the database, one to store product details, one for user details and one to map products to the users who are tracking it.

A typical document in products collection will be:

{
  _id: ObjectId("53a2bfcfa7603606c2765342"),
  name: "Nexus 7 from Google (7-Inch, 16 GB, Black)",
  url: "http://rads.stackoverflow.com/amzn/click/B00DVFLJDS",
  base_price: 222,
  current_price: 213.96,
  img_url: "http://i.imgur.com/something.jpg",
  history: [
    [
      ISODate("2014-06-19T12:08:13.354Z"),
      293
    ],
    [
      ISODate("2014-06-24T14:31:38.216Z"),
      424
    ],
    [
      ISODate("2014-06-24T14:32:06.992Z"),
      424
    ]
  ]
}

in users, the tracked_products contain the Object Id of the product document and the desired price:

{
  _id: ObjectId("539c4adea760360886d7ef02"),
  email_id: "[email protected]",
  channels: {
    'twitter': 'usertwitterid',
    'ios_push_id': '45462722672762576422'
    'android_push_id': '2572652754762474'
    'chrome_push_id': '456456454646464'
  },
  tracked_products: [
    [
        ObjectId("53a2bfcfa7603606c2765342"), 
        200
    ],
    [
        ObjectId("53a2d2ada7603606c2765344"),
        345
    ],
    [
        ObjectId("53a2d294a7603606c2765343"),
        120
    ]
  ]
}

and lastly trackers, here subscribers contain the Object Id of user document who are tracking this product:

{
    _id: ObjectId("77a2bfcfa7603606c2765399"),
    product_id: ObjectId("53a2bfcfa7603606c2765342"),
    subscribers: [
        [
            ObjectId("539c4adea760360886d7ef02"),
            200
        ],
        [
            ObjectId("12a2bfcfa7603606c2765399"),
            234
        ],
        [   
            ObjectId("14a2bfcfa7603606c2765399"),
            345
        ],
}

Following is the code, which checks for the current_price and alerts the user if there is change in the price:

subscribers = set()

for product in product.collection:
    # price changed? if yes, then update
    current_price = get_current_price(product['url'])

    if current_price == product['current_price']:
        # no price change. move on
        continue

    product['history'].append((datetime.datetime.utcnow(), current_price))

    # find the subscribers:
    tracker_document = db.trackers.find_one({'product_id': product['_id']})

    for tracker in tracker_document['subscribers']:
        user_id, desired_price = tracker
        if current_price <= desired_price:
            subscribers.add(user_id)

    product.collection.save(product)

if not subscribers:
    # no price changes in any products!
    sys.exit()
email_queue = list()
twitter_queue = list()
ios_push_queue = list()
droid_push_queue = list()
chrome_push_queue = list()

for user_id in subscribers:
    user_document = db.users.find_one({'_id': user_id})
    email_queue.append(user_document['email_id'])
    twitter_queue.append(user_document['channels']['twitter'])
    ios_push_queue.append(user_document['channels']['ios_push_id'])
    droid_push_queue.append(user_document['channels']['droid_push_id'])
    chrome_push_queue.append(user_document['channels']['chrome_push_id'])

alert_users(email_queue, twitter_queue, ios_push_queue, droid_push_queue, chrome_push_queue)

I am looking for any suggestions, code improvements/readability, performance improvement, schema change etc. Anything!

PS: I am aware this would be lot easier with RDBMS and join operations, but I am sticking with MongoDB as of now.

Answer 1

You should create a function that returns all the subscribers for a product:

def get_subscribers(product, current_price):
    tracker_document = db.trackers.find_one({'product_id': product['_id']})

    subscribers = set()
    for user_id, desired_price in tracker_document['subscribers']:
        if current_price <= desired_price:
            subscribers.add(user_id)

    return subscribers

This could even be condensed even more using a list comprehension:

def get_subscribers(product, current_price):
    tracker_document = db.trackers.find_one({'product_id': product['_id']})

    return set([user_id for user_id, desired_price in tracker_document['subscribers']
                if current_price <= desired_price])

This change makes your code look like this:

subscribers = set()
for product in product.collection:
    current_price = get_current_price(product['url'])
    if current_price == product['current_price']:
        continue

    product['history'].append((datetime.datetime.utcnow(), current_price))
    product.collection.save(product)

    subscribers |= get_subscribers(product, current_price)

---

The next change I would suggest is: instead of using several lists for your queues, wrap them into a single defaultdict. This simplifies this code and would require only minimal changes in your alert_users code.

It may actually be better, depending on your alert_users function, to to alert a single user at a time instead of building queues of the users and calling alert_users with a batch of users.

Here is the batch example:

from collections import defaultdict

queues = defaultdict(list)
for user_id in subscribers:
    user_document = db.users.find_one({'_id': user_id})

    queues['email_id'].append(user_document['email_id'])
    for queue_type in ['twitter', 'ios_push_id', 'droid_push_id', 'chrome_push_id']:
        queues[queue_type].append(user_document['channels'][queue_type])

alert_users(queues)

Here is the user-at-a-time example:

for user_id in subscribers:
    user_document = db.users.find_one({'_id': user_id})

    alert_user(user_document['email_id'], *user_document['channels'])