Inefficient Python script eating up memory (I believe)

Question

I created a simple Python script to log track listens in iTunes and it seems that it's pretty inefficient. I'm primarily a front-end developer, so this is clearly not my area of expertise. I know loops can cause problems in any language, so I assume that's the issue here.

If I run this script for a while (say, a couple of hours or more) the fan on my computer starts working overtime and won't stop until I kill the script.

Any idea what might be making this happen? Suggestions on how to optimize? Appreciate any help I can get.

#!/usr/bin/python
# -*- coding: utf-8 -*-

import wx, sqlite3, datetime, threading
from appscript import *
it = app('iTunes')
from AppKit import *



# Create the DB
Tapedeck = sqlite3.connect('Tapedeck.db', check_same_thread = False)
TapedeckQuery = Tapedeck.cursor()
Tapedeck.execute('''CREATE TABLE IF NOT EXISTS plays (id INTEGER PRIMARY KEY, track_id TEXT, track TEXT, artist TEXT, album TEXT, genres TEXT, play_count INTEGER, rating INTEGER, skip_count INTEGER, year_released INTEGER, date_played DATE)''')
Tapedeck.commit()


def getSec(s):
    l = s.split(':')
    return int(l[0]) * 60 + int(l[1])


def addTrack(track_id, track, artist, album, genres, play_count, skip_count, year_released, date_played, rating):
    #print '        --\n        Position: {}\n      Length: {}'.format(it.player_position(),getSec(it.current_track.time()))
    TapedeckQuery.execute('''INSERT INTO plays (track_id, track, artist, album, genres, play_count, skip_count, year_released, date_played, rating) VALUES (?,?,?,?,?,?,?,?,?,?)''',
    (track_id, track, artist, album, genres, play_count, skip_count, year_released, date_played, rating))
    Tapedeck.commit()


loop_int = 1.0
last_track = '0'

def listen():

    global loop_int, last_track
    threading.Timer(loop_int, listen).start()
    if it.player_state() == k.playing:
        # check to see if track was restarted
        if it.player_position() < getSec(it.current_track.time())/2 and last_track == it.current_track.persistent_ID():
            self.last_track = '0'
        # has the track played beyond the halfway mark?
        if it.player_position() >= getSec(it.current_track.time())/2 and last_track != it.current_track.persistent_ID():
            today = datetime.datetime.today()
            now = '{}-{}-{} {}:{}:{}'.format(today.year, '%02d' % today.month, '%02d' % today.day, '%02d' % today.hour, '%02d' % today.minute, '%02d' % today.second)
            print '\nArtist:        {}\nTrack:      {}\nAlbum:      {}\nGenres:     {}\nDatetime:   {}'.format(it.current_track.artist().encode('ascii','ignore'), it.current_track.name().encode('ascii','ignore'), it.current_track.album().encode('ascii','ignore'), it.current_track.genre().encode('ascii','ignore'), now)
            last_track = it.current_track.persistent_ID()
            addTrack(it.current_track.persistent_ID(), it.current_track.name(), it.current_track.artist(), it.current_track.album(), it.current_track.genre(), it.current_track.played_count(), it.current_track.skipped_count(), it.current_track.year(), now, it.current_track.rating())

listen()

Answer 1

#!/usr/bin/python
# -*- coding: utf-8 -*-

import wx, sqlite3, datetime, threading
from appscript import *

import * is frowned upon as it makes it difficult to trace where names came from.

it = app('iTunes')

I recommend avoiding abbreviated names like it its hard to figure out what it means later.

from AppKit import *



# Create the DB
Tapedeck = sqlite3.connect('Tapedeck.db', check_same_thread = False)
TapedeckQuery = Tapedeck.cursor()
Tapedeck.execute('''CREATE TABLE IF NOT EXISTS plays (id INTEGER PRIMARY KEY, track_id TEXT, track TEXT, artist TEXT, album TEXT, genres TEXT, play_count INTEGER, rating INTEGER, skip_count INTEGER, year_released INTEGER, date_played DATE)''')
Tapedeck.commit()

Generally, performing actions at the module level is frowned upon. You should do it in a main function.

def getSec(s):

Python style guide recommends lowercase_with_underscores for function names. Also, don't needlessly abbreviate.

    l = s.split(':')
    return int(l[0]) * 60 + int(l[1])

I'd have done:

minutes, seconds = s.split(':')
return int(minutes)*60 + int(seconds)

I think its easier to see what's going on

def addTrack(track_id, track, artist, album, genres, play_count, skip_count, year_released, date_played, rating):
    #print '        --\n        Position: {}\n      Length: {}'.format(it.player_position(),getSec(it.current_track.time()))
    TapedeckQuery.execute('''INSERT INTO plays (track_id, track, artist, album, genres, play_count, skip_count, year_released, date_played, rating) VALUES (?,?,?,?,?,?,?,?,?,?)''',
    (track_id, track, artist, album, genres, play_count, skip_count, year_released, date_played, rating))
    Tapedeck.commit()


loop_int = 1.0
last_track = '0'

def listen():

    global loop_int, last_track

Avoid global variables. You should generally only be using globals as constants.

    threading.Timer(loop_int, listen).start()

Python isn't Javascript. Instead of playing with threads, use time.Sleep(1) to sleep for a second between each attempt. Put everything in a loop. It'll be easier to follow.

    if it.player_state() == k.playing:
        # check to see if track was restarted
        if it.player_position() < getSec(it.current_track.time())/2 and last_track == it.current_track.persistent_ID():
            self.last_track = '0'

self?

        # has the track played beyond the halfway mark?
        if it.player_position() >= getSec(it.current_track.time())/2 and last_track != it.current_track.persistent_ID():
            today = datetime.datetime.today()
            now = '{}-{}-{} {}:{}:{}'.format(today.year, '%02d' % today.month, '%02d' % today.day, '%02d' % today.hour, '%02d' % today.minute, '%02d' % today.second)

The datetime has a formatting function which may be better suited to that

            print '\nArtist:        {}\nTrack:      {}\nAlbum:      {}\nGenres:     {}\nDatetime:   {}'.format(it.current_track.artist().encode('ascii','ignore'), it.current_track.name().encode('ascii','ignore'), it.current_track.album().encode('ascii','ignore'), it.current_track.genre().encode('ascii','ignore'), now)
            last_track = it.current_track.persistent_ID()
            addTrack(it.current_track.persistent_ID(), it.current_track.name(), it.current_track.artist(), it.current_track.album(), it.current_track.genre(), it.current_track.played_count(), it.current_track.skipped_count(), it.current_track.year(), now, it.current_track.rating())

listen()

As for your actual problem. I don't know this appscript/AppKit you are using. But I'll speculate that under some circumstances iTunes stops responding. Maybe a dialog gets popped or something. As a result, each thread gets stuck waiting for iTunes. But because you set the timer as your first action, a new thread keeps getting started which also gets stuck. Eventually, your poor scripts piles many many threads and that explains the hang.

If you use the sleep method I suggested above, that should prevent the script from running many threads.

Answer 2

You could run ps or top to find out how much memory, CPU your script consumes (psutil allows to do it from Python).

If app('iTunes') supports it you could replace your polling interface with a callback on event e.g., a track change.

If not then you could remove threading.Timer() call and replace it with this code:

import time

while True:
    next_time = time.time() + loop_int
    listen()
    delay = next_time - time.time()
    if delay > 0:
       assert delay <= loop_int # break loudly on clock adjustments
       time.sleep(delay)

This code runs listen() function every loop_int seconds (approximately) or if listen() function takes us longer than loop_int seconds then it runs the next listen() call immediately after the previous one.