0
\$\begingroup\$

My attempt at multiple linear regression.

I am trying to make a qualified guess about a user's rating of a movie, through machine learning. I am new to this, so my judgement isn't the best. And I am also getting a result but I have no idea if it is the result I want.

The code is setting up a finite selection of genres, which then are one-hotencoded and standardized. This will become the X for fitting the model and the same with previous average values, which will become the Y in the fit.

After the fit, I am emulating a new movie choice by the user x_user = ["Action", "Thriller",]. This is applied to the prediction and the result is being printed.

I need this to be reviewed so I know if I've done it correctly or not. And also if there are improvements to be made for this code.

# create main dataframe df and set first column "avg"
df = pd.DataFrame()
col1 = np.random.uniform(low=1.0, high=5.0, size=(100,))
df = pd.DataFrame(col1, columns=['avg']).round(2)

# create secondary and disposable dataframe df_tmp to set up genres for df
df_tmp = pd.DataFrame()

# list of genres
genres = [
    "Action",
    "Adventure",
    "Biography",
    "Comedy",
    "Crime",
    "Erotica",
    "Fantasy",
    "Historical fiction",
    "Horror",
    "Mystery",
    "Romance",
    "Satire",
    "Scifi",
    "Speculative",
    "Thriller",
    "Western",
]

# create three genre columns with values chosen at random so one avg can be decided by multiple genres
df_tmp['Genres1'] = np.random.choice(genres, df.shape[0])
df_tmp['Genres2'] = np.random.choice(genres, df.shape[0])
df_tmp['Genres3'] = np.random.choice(genres, df.shape[0])

# concatinate the collection of genres into Total column
df_tmp['Total'] = df_tmp['Genres1'] + "," + df_tmp['Genres2'] + "," + df_tmp['Genres3']

# remove duplicates from genre collection
data = []
for i in df_tmp.Total:
    tmp = i.split(",")
    tmp2=set(tmp)
    tmp3 = ",".join(tmp2)
    data.append(tmp3)

# add the genre set to df
df['Genres'] = pd.DataFrame(data)

# convert categorical data to one-hotencode data 
df = pd.concat(
        [df.drop('Genres', 1), df['Genres'].str.get_dummies(sep=",")], 1)

# standardize lambda expression
standarize = lambda q: (q-q.mean())/q.std() 

# standardize the dataset
df_st = standarize(df)

from sklearn import linear_model

# divide df with avg as dependent variable and genres as independent variables
Y = df.avg
X = df.drop('avg', 1)

# do a linear regression fit
clf = linear_model.LinearRegression()
clf.fit(X, Y)

# print values
print(f'Intercept: {clf.intercept_}')
print(f'Coefficients: {clf.coef_}')

# the genres of the movie chosen by the user
x_user = ["Action", "Thriller",]

# create the array to be utilized for the prediction. A bool list of genres matching at indext because the prediction method does not accept strings or an array shorter or longer than X 
X_pred = []
cat = X.columns
for i in cat:
    if i in x_user:
        X_pred.append(1)
    else:
        X_pred.append(0)

# get avg prediction for a new genre combination
print(f'Predicted avg rating: {clf.predict([X_pred])}')
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.