I am working on a portfolio project that deals with creating a Netflix-style recommendation engine for movies. This code is currently ran locally but I will need to upload it into Jupyter Notebooks and implement it with Amazon SageMaker. The .tsv files are imported from IMDb Datasets.

The code below is working code created by myself. Is there any room for improvement from an efficiency/cleanliness aspect?

import pandas as pd 
import numpy as np

basics = pd.read_table('basics.tsv')
origin = pd.read_table('akas.tsv')
ratings = pd.read_table('ratings.tsv')

#setting 'basics' columns
basics.drop(basics.columns[[3,4,6,7]], axis=1, inplace=True)
basics.columns = ['tconst','type','title','year','genre']

#removing all non-movies from index
cond1 = basics['type'].isin(['movie','tvMovie'])

#setting new 'basics' variable
basics2 = basics[cond1]

#setting 'origin' columns
origin.drop(origin.columns[[1,2,5,6,7]], axis = 1, inplace = True)
origin.columns = ['tconst','region','language']

#removing non-english movies from index
cond2 = origin['region'].isin(['US','GB'])
cond3 = origin['language'] == 'en'

#setting new 'origin' variable
origin2 = origin[cond2 | cond3]

#setting 'ratings' columns
ratings.columns = ['tconst','rating','votecount']

#converting strings to integers
ratings.rating = pd.to_numeric(ratings.rating, errors='coerce').astype(np.int64)
ratings.votecount = pd.to_numeric(ratings.votecount, errors='coerce').astype(np.int64)

#setting new 'ratings' variable
ratings2 = ratings[(ratings['rating'] >= 5) & (ratings['votecount'] >= 50)]

#finalizing movie recommendation list
rcmd = basics2.merge(origin2,on='tconst').merge(ratings2,on='tconst')
rcmd.drop_duplicates(subset = 'tconst', keep ="first", inplace = True) 

Thanks in advance.

  • \$\begingroup\$ Do your tsv files not have a heading row? \$\endgroup\$
    – Reinderien
    Commented Nov 2, 2020 at 21:32
  • \$\begingroup\$ Oh, they do. So I'm not sure why you're setting your col names explicitly. \$\endgroup\$
    – Reinderien
    Commented Nov 2, 2020 at 21:37

1 Answer 1


The usual

Use a PEP8 linter; move global code into functions; use type hints; use a __main__ guard.

Column use

basics.drop(basics.columns[[3,4,6,7]], axis=1, inplace=True)
basics.columns = ['tconst','type','title','year','genre']

really shouldn't be necessary. Use the usecols kwarg of read_table instead; the col names are a loose guess from the IMDB documentation:

basics = pd.read_table(
        'tconst', 'titleType', 'primaryTitle', 'startYear', 'genres',
  • \$\begingroup\$ I set the columns for organization and aesthetic purposes. I did not like the default values provided, no other reason in particular. \$\endgroup\$
    – cloud_user
    Commented Nov 2, 2020 at 22:00
  • \$\begingroup\$ Aesthetics is not a convincing argument to rename columns. You're hindering your code's ability to correspond with the IMDB documentation. \$\endgroup\$
    – Reinderien
    Commented Nov 2, 2020 at 22:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.