3
\$\begingroup\$
import praw
import time
import database

# Create a user agent and log into Reddit
UA = 'OZBOZZ v0.1'
r = praw.Reddit(UA)
# Log UA into reddit with my personal user name and password
r.login('', '', disable_warning='True')
subreddit = r.get_subreddit('listentothis') 
current_time = int(time.time())
user = ''
submission_list = []
results = []
count = 0
genres = ["Big Band Funk", "progressive Rock", "rock", "progressive metal"]

def botstuff():
# while condition is true...
while True:
    # Print Working so That I know the Next iteration has begun
    print("Working...")

    # For submission in subreddit /r/listen to this get up to 20 new posts
    for submission in subreddit.get_new(limit=20):
        # Create a variable called Sub_age that is current time minus the submission creation time divided by 60
        # divided by 60  divided by 12
        sub_age = (current_time - submission.created_utc) / 60 / 60 / 24
        # If the submission age is less than one (posted in last 24 hours)...
        if sub_age < 1:
            # Add that submission to the submission list. Each entry contains the title and Link to post
            submission_list.append([submission.title, submission.permalink])

    # Print the submission list, so I know I got something. Debugging
    print(submission_list)

    # For each list entry in submission_list
    for x in submission_list:
        # any of element in the list genre are found in the 1st... element property(?)
        if any(a in x[1] for a in genres):
            # Append the element with matching genre tags in to the results list.
            results.append(x)

    # Print the submission list, so I know I got something. Debugging
    print(results)

    # if the results list is empty
    if not results:
        # Send the user a message  letting them know I havent found something
        r.send_message(user, 'Nothing Yet', 'Hey, I didn\'t find anything')
    # Otherwise...
    else:
        # Create a message variable from the bot with the list of results.
        msg = 'Hey, Check these out.  -oz \n' + "\n" +'[%s]'.strip('[]') % ''.join(map(str, results)).strip('[]')
        # Sent the message to the user.
        r.send_message(user, 'Suggestions', msg)
        print("Message Sent")

    count = count + 1
    print("Sleep cycle " + str(count))
    #wait 60 minutes
    time.sleep(3600)

So this is the code completely working code so far. What this bot accomplishes is scanning new posts on /r/listentothis that are less than 24 hours old and parsing the text of the submission post that matches my list of genres called genres. If a match is found, it appends it to a results list. If the results list is empty it sends me a message stating that it didn't find anything, otherwise, it sends me a message with a suggestsion to check out some results and then prints out the results list.

  1. I'm looking for ways to multithread this bot.
  2. Make this a modular function because the goal is to make it so that a reddit user can send this bot account a message with genres and it will put the uname and genres into a database and do a search for them every hour.
  3. Clean up results output so each result is printed on a new line.
  4. Improve the string search efficiency if possible.
  5. Search Specifically for only the strings providied. So for example, "rock" will not return "Glam rock"
\$\endgroup\$
0

1 Answer 1

6
\$\begingroup\$

1. General issues

r.login ('','', disable_warning=True) #notice True is boolean and not string

if you did disable_warning='False' it would still disable warning. Every non-empty string in python when cast to boolean will return True.

Too many global objects. Reddit API client can stay global r, having current_time as global is horrific. Stuff like submission_list results count genres shouldn't be global either but I can understand that your intent was to get those things from database so this is only temporary. All of those things should be returned by their functions at the moment when they are needed.

You shouldn't use login. It is being depreciated soon. What it means is that if not for the fact that praw devs don't want to break every single bot there is they give you a little time to remove login() from old bots. New bots shouldn't use this at all, because by the time you have it up and working it might be broken again cause login() will no longer be there. Check OAuth2 - it's bit more that single call to login() but still can be done in 2 minutes (and it's one time).

2. Multithreading

So there are multiple issues with multithreading here. First off you don't need it here. Your bot has only one functionality. Grab data, parse data, give output, repeat. Second one is that multiprocessing/multithreading in python is generally not too common unless there is actual need for that due to massive overhead, that usually makes things slower in the long run. So it would only be feasible if you want your program to extremely long stuff in parallel.

Lastly reddit bot is extremely bad example to try to multithread. The reason are reddit API restrictions. It will not respond to request more often than 0.5 sec. This is how developers have set it and also praw behaves by it. If you make 20 requests you will have to wait 10 seconds no matter how many threads try to do this. This is why it is always done in sequences with those.

3. Modularity

First off your program should have more functions, just for the sake of readability. Like

def get_submission_list():
    submission_list = []
    for submission in sub.....
    ...
    return submission_list

def send_messsages(results):
   .....

def main_loop():
    while True:
        check_inbox_for_new_bot_subscribers()
        s_list = get_submission_list()
        subscribed_users = get_subscribed_users_and_genres_from_db()
        results = prepare_results(subscribed_users,s_list)
        send_messages(results)

submission_list is kinda obvious, subscribed_users could be a list of tuples or a dictionary with 'username':'genres,separated,by comma' or w/e you feel comfortable working with. results in this case would be (username,[list of links to send to this user])

For the sake of checking multiple users it would be beneficial to also reverse username-genres dictionary, so that we also have fast lookup dictionary of 'genre':[list of usernames subscribed to this genre]'. This would be fun to implement, but benefit is obvious since we have now \$\mathcal{O}(n\cdot{}m)\$ time instead of \$\mathcal{O}(n^2\cdot{}m^2)\$.

Simple database model for this would be:

class BotSubscriber(Model):
    redditor_id = CharField()
    genres = CharField()

You obviously could save other stuff like time of subscription, how many songs you recommended to this user and w/e you feel like.

4. Clean out output

I believe '\n'.join(map(str,results)).strip('[]') or some variation of that would work as you want?

5. String search efficiency and better matching

It would be beneficial to parse submission.title before searching. Band - song [genre] (optional year) is pretty much standard format. Move Band - Song and save it as title and [genre] as genre (stripping starting braces as well).

On top of that you definitely use genre.lower() as well as save submitted user genres as lowercase. No need to be case sensitive in this scenario.

Once you have your genre saved as genre you can do any([1 for a in user.genres if a==genre.lower()]) or use reversed dictionary idea that I've posted in 3.

\$\endgroup\$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.