6
\$\begingroup\$

This downloads content from YouTube either as whole playlists or single videos. I find it very useful but would like to know if it's something close to a program that others could maintain. I also wanted to show it off a bit and get some feedback as I've been learning on my own from the net and would be helpful to know if I'm on a right track.

It works through the shell (Windows for now) and doesn't have a GUI. Because I made a custom command line interface it's very user-friendly so anyone can use it and has a few simple commands which are explained by typing "help". It also downloads the videos using an opensource command line script/program called youtube-dl.

enter image description here

Unfinished project outline

import HTMLParser
import urllib2
import os

help_str = '''
* YouTube Offline can be used without these commands.

  For single video downloads enter a valid YouTube username.
  The users playlists will be shown if any in order by numbers
  starting from 1. Select a playlist or video item by simply
  entering the items number. When a video number is selected
  the download will begin automatically.

  Or add multiple items(playlists or videos) to the job queue
  with these commands and use the start command to begin downloads.


List of commands:

    add <item>          Add playlist or videoitem to job queue.
                        eg. add 12
                        adds the 12th item in the list to job queue

    start               Executes tasks in job queue if any.

    clear / clear all   Clears last job from job queue or clears all jobs.

    cancel          Navigate backwards through prompts.

    up                  Scroll up / previous page

    <enter>         Scroll down / next page

...hit <enter> to exit help...
'''

youtube_header = ['','',
' __  __         ______     __         ____  ________',
' \ \/ /__  __ _/_  __/_ __/ /  ___   / __ \/ _/ _/ (_)__  ___ ',
'  \  / _ \/ // // / / // / _ \/ -_) / /_/ / _/ _/ / / _ \/ -_)',
'  /_/\___/\_,_//_/  \_,_/_.__/\__/  \____/_//_//_/_/_//_/\__/',
                '','','']

screen = {'head':[''] * 3,
          'body':[''] * 11,
          'bar':[''],
          'status':[''],}

data = {}
job_q = []

def comm_input(comm=None, comm_data=None):
    'executes commands and returns False, or if no command returns comm'
    global job_q
    if comm == None:
        comm = raw_input('>> ')
    comm = comm.lower().strip()
    if comm == 'help':
        os.system('cls')
        print help_str
        raw_input()
        print_screen('status')
        return False
    if comm_data:
        user = comm_data[0]
    # add <item>
    if comm[:3] == 'add':
        i = comm[3::].strip()
        try:
            i = int(i)
            if len(comm_data) == 2:
                # add video to job_q
                playlist = data[comm_data[0]][comm_data[1]]
                if 0 < i <= len(playlist[2]):
                    i -= 1
                    p_title = playlist[0]
                    y_id    = playlist[2][i][2]
                    v_title = playlist[2][i][0]
                    path = 'videos/'+clean(user)+'/'+clean(p_title)+'/'+clean(v_title)+'.flv'
                    job_q.append((y_id, path))
                    prin_screen('status')
                    return False
                else:
                    # Invalid range
                    print_screen('status', ['Invalid option. Valid options are from 1 to ' + str(len(playlist[2]))])
                    return False
            else:
                # add playlist to job_q
                if 0 < i <= len(data[user]):
                    i -= 1
                    p_title = data[user][i][0]
                    if not data[user][i][3]:
                        get_all_vids(data[user][i])
                    for vid in data[user][i][4]:
                        v_title = vid[0]
                        y_id = vid[1]
                        path = 'videos/'+clean(user)+'/'+clean(p_title)+'/'+clean(v_title)+'.flv'
                        job_q.append((y_id, path))
                    print_screen('status')
                    return False
                else:
                    # Invalid range
                    print_screen('status', ['Invalid option. Valid options are from 1 to ' + str(len(data[user]))])
                    return False
        except ValueError:
            print_screen('status', ['Invalid command for add <item>; type help'])
            return False
    # clear
    elif comm == 'clear':
        job_q.pop()
        return False
    # clear all
    elif comm == 'clear all':
        job_q = []
        return False
    # start
    elif comm == 'start':
        start_work(job_q)
        job_q = []
        return False
    # not a command
    return comm

def download(youtube_id, path):
    os.system('python youtube-dl.py -f 18 -icw -o "' + path + '" http://www.youtube.com/watch?v=' + youtube_id)

def start_work(job_q):
    for job in job_q:
        download(job[0], job[1])

def get_source(url):
    try:
        source = urllib2.urlopen(url)
        return source.read()
    except:
        return False

def unescape(string):
    h = HTMLParser.HTMLParser()
    return h.unescape(string)

def clean(title):
    'replaces invalid filename chars with valid chars'
    invalid_dir_chr = ['\\', '/', ':', '*', '?', '<', '>', '|']
    title = unescape(title)
    title = urllib2.unquote(title)
    title = title.replace('"', "'")
    title = title.replace('?', '.')
    for c in invalid_dir_chr:
        title = title.replace(c, '-')
    return title

def print_screen(key=None, lines=None, disp=True):
    'lines --> [str, ... ]'
    global screen
    if lines:
        for i in range(len(screen[key])):
            try:
                if lines[i] != None:
                    screen[key][i] = lines[i]
            except IndexError:
                screen[key][i] = ''
    elif key:
        screen[key] = ['' for x in screen[key]]
    if disp:
        lines = ['head', 'body', 'bar', 'status']
        os.system('cls')
        for line in lines:
            for subline in screen[line]:
                print subline
        if len(job_q) == 0:
            print ''
        else:
            print 'jobs pending: ' + str(len(job_q))

def display_list(data, page):
    'data --> list | page --> int, actual page not index num'
    block = [line[0] for line in data]
    start = (10 * (page - 1))
    end = start + 10
    if len(data) < 10:
        pad = '%01d'
    elif len(data) < 100:
        pad = '%02d'
    else:
        pad = '%03d'
    block = [(pad % (start+1+i)) + '.' + block[start+i] for i in range(len(block[start:end]))]
    print_screen('body', block)

def browse(user, playlist_i=None):
    'pages through data_list 10 lines at a time'
    data_list = data[user]
    if not data_list:
        return False
    if playlist_i != None:
        print_screen('head', [None, data_list[playlist_i][0]])
        data_list = data_list[playlist_i][5]
    if len(data_list) <= 10:
        pages = 1
    else:
        pages = len(data_list) / 10
        if len(data_list) % 10:
            pages += 1
    page = 1
    while True: # while paging
        # Display
        display_list(data_list, page)
        # page number bar display
        page_bar = [str(x) for x in range(1,pages+1)]
        page_bar = [x if int(x) == page or int(x) == page+1 else ' '+x for x in page_bar]
        page_bar[page-1] = '['+str(page)+']'
        page_bar = ''.join(page_bar)          
        print_screen('bar', [page_bar])
        while True: # while commands being executed, don't page
            # comm_data parameter allows the option of either explicitly
            # executing commands or following the prompt.
            if playlist_i != None:
                comm = comm_input(comm_data=(user, playlist_i))
            else:
                comm = comm_input(comm_data=(user,))
            if comm == False: # A command was executed at comm_input()
                print_screen()
                continue
            elif comm == '': # <enter>, next page
                print_screen('status', disp=False)
                if page == pages:
                    page = 1
                else:
                    page += 1
                break
            elif comm == 'up': # prev page
                print_screen('status', disp=False)
                if page == 1:
                    page = pages
                else:
                    page -= 1
                break
            elif comm.strip().lower() == 'cancel': # prev level
                print_screen('status', disp=False)
                print_screen('head', [None, ''])
                print_screen('bar')
                return
            else: # possible selection
                try:
                    comm = int(comm.strip())
                    if 0 < comm <= len(data_list):
                        if playlist_i != None:
                            comm_input('add ' + str(comm), (user, playlist_i))
                            break
                        else:
                            if not data_list[comm-1][6]:
                                get_all_vids(data_list[comm-1])
                            browse(user, comm-1)
                            break
                    else:
                        print_screen('status', ['Invalid option. Valid options are from 1 to ' + str(len(data_list))])
                        continue
                except ValueError:
                    print_screen('status', ['Invalid command'])
                    continue                
                break

def get_all_vids(playlist):
    url = playlist[1]
    playlist_data = playlist[2]
    print_screen('status', ['...fetching playlist data...'])
    source = get_source(url)
    while True:
        # link
        source = source[source.find('<li class="playlist-video-item')::]
        # youtube id
        a = source.find('data-video-ids')
        a = source.find('"', a)
        b = source.find('"', a + 1)
        if a == -1 or b == -1:
            break
        yid = source[a+1:b]
        source = source[b::]
        # video title
        source = source[source.find("title video-title")::]
        a = source.find('>')
        b = source.find('<')
        if a == -1 or b == -1:
            break
        title = source[a+1:b]
        source = source[b::]
        playlist_data.append([title, yid])
    print_screen('status', disp=None)

def get_playlists(source):
    'returns --> [<playlist data>] or False'
    playlist = []
    while True:
        # URL
        source = source[source.find("yt-uix-tile-link")::]
        a = source.find("href")
        a = source.find('"', a)
        b = source.find('"', a+1)
        if a == -1 or b == -1:
            break
        url = "http://www.youtube.com" + source[a+1:b]
        source = source[b::]
        # playlist title
        a = source.find(">")
        b = source.find("<")
        if a == -1 or b == -1:
            break
        title = source[a+1:b].strip()
        source = source[b::]
        if not title or not url:
            break
        title = unescape(title)
        playlist.append([title, url, []])
    return playlist

def get_all_playlists(user):
    'returns --> [[<playlist data>], ... ] from online'
    playlist = []
    page = 0
    while True:
        page += 1
        print_screen('status', ['...fetching page ' + str(page) + '...'])
        source = get_source('http://www.youtube.com/user/' + user + '/videos?sort=dd&view=1&page=' + str(page))
        p = get_playlists(source)
        if p:
            playlist += p
        else:
            print_screen('status')
            return playlist

def set_username(user):
    'returns --> user or False if not exist or False if no playlists'
    global data
    if user == False:
        return False
    print_screen('head', ['Username: ' + user])
    user = user.strip().lower()
    if not user:
        print_screen('status')
        return False
    if user in data:
        if data[user]:
            return user
        else:
            print_screen('status', ['no playlists found for user: ' + user])
            return False
    else:
        print_screen('status', ['...checking online for username: ' + user + '...'])
        source = get_source('http://www.youtube.com/user/' + user)
        if source:
            data[user] = get_all_playlists(user)
            if data[user]:
                print_screen('status', False)
                return user
            else:
                print_screen('status', ['no playlists found for user: ' + user], False)
                return False
        else:
            print_screen('status', [user + ' does not exist'], False)
            return False

def prompt():
    user = False
    while user == False:
        print_screen('head', ['Username: ', ''])
        print_screen('body', youtube_header)
        user = set_username(comm_input())
    browse(user)

while True:
    prompt()
\$\endgroup\$
2
\$\begingroup\$

One suggestion I would make is to reduce the length of your functions, and reduce the amount of nesting / indentation. This will make your code easier to follow, and easier to maintain.

As one example, you've got a function to respond to user commands. I would probably approach this by defining a new function to handle each command. You can then describe a mapping from command to function, and call the appropriate function.

e.g.

# Define some functions to handle various user commands.
def start():
    # do something
    pass

def add_item(item):
    # do something with the item
    pass

def show_help():
    # display help message
    pass

# Map from command to the associated function.
COMMANDS = { 'help': show_help,
             'add': add_item,
             'start': start,
           } # etc

# Given user input, split it into a command and its arguments, then call
# the appropriate function.
def handle_command(user_input):
    command = user_input.split()[0]
    arguments = user_input.split()[1:]

    try:
        COMMANDS[command](*arguments)
    except KeyError:
        # command not recognised, so show help.
        show_help()
\$\endgroup\$
  • 1
    \$\begingroup\$ nit-pick: you need to define those function before the dict. \$\endgroup\$ – Winston Ewert Aug 7 '12 at 4:22
  • \$\begingroup\$ Thanks John Fouhy. I appreciate you taking the time. I'll spend the rest of today working on the comm_input function and definitively like the idea of mapping commands with a dict. \$\endgroup\$ – Russ Aug 7 '12 at 7:19
  • \$\begingroup\$ Ahh, too much time recently on DotNet... \$\endgroup\$ – John Fouhy Aug 7 '12 at 21:57
  • \$\begingroup\$ Didn't have much time at work yesterday so not yet finished with the code cleanup. \$\endgroup\$ – Russ Aug 8 '12 at 19:09
  • \$\begingroup\$ ...However I did add some new functionality to this one as I needed to use it while I work on a better implementation using OO. Here is the source for the improvement. You can set the quality of vids or playlists and throttle the download speed. Here's the source make sure you put youtube-dl in the python folder to use it: youtube-dl source \$\endgroup\$ – Russ Aug 8 '12 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.