# Predict new ratings for each user based on their pearson correlation with other users

I am new to R and programming. I have a set of ratings for 45000 users and 40 odd movies. I need to predict new ratings for each user based on their pearson correlation with other users. I also need to store the set of similar users for each user-movie combination.The code that I have managed to write is this

# Matrix of users and ratings
x <- matrix(rnorm(1:100), nrow = 10 , ncol = 30)
df = list()

# correlation matrix
cor_mat <- cor(x)

# similarity limits
upper = 1
lower = 0.4

# empty matrix to store predicted values
final_x = matrix(NA,nrow = 10,ncol = 30)

for (i in 1:ncol(x)){

for( j in 1:nrow(x)){

sim_user = which(cor_mat[i,] >= lower & cor_mat[i,] < upper)

final_x[i,j] = t(x[sim_user,j]) %*%
cor_mat[sim_user,j]/sum(cor_mat[sim_user,j])

df[[length(df)+1]] = cbind.data.frame(i,j,sim_user,cor_mat[sim_user,j])

}
}


Questions:

1. I am looping over each element of the matrix which works fine but seems pretty novice to me. Can something better be done?
2. I have heard of the foreach package but read that it adds value only when a single operation takes a long time to execute which is not the case here. Will it still provide me good performance?
• Welcome to Code Review. In the future, please mention your cross-post. – 200_success Jun 27 '16 at 19:49
• Why is x square in your example? More generally, is it m-by-n where m is the number of movies and n is the number of users? Note that cor(x) will be n-by-n so the first loop (i) should be iterating over 1:ncol(x), not 1:nrow(x). So your code will likely break with more realistic data. – flodel Jun 28 '16 at 0:36
• Is this, or is this not, broken code? – Malachi Jun 30 '16 at 12:23
• I am not sure what you mean by broken code – RUser Jun 30 '16 at 12:54
• Maybe you have not read my comment? I'll also point out that df is not initialized so it won't run as-is. Please make sure you provide a reproducible example; which means you should be able to run without error from a fresh session. Then try to make x a non-square matrix to reveal the problem I was describing in my comment. Then please edit your question with the corrected code. – flodel Jul 1 '16 at 0:23

You have incorrect indexing in the loop. Better version with mapply and correct indexing:

df <- mapply(function(i, j) {
sim_user = which(cor_mat[i,] >= lower & cor_mat[i,] < upper)
final_x[j, i] = t(x[j, sim_user]) %*% (cor_mat[j, sim_user]/sum(cor_mat[j, sim_user]))
cbind(i, j, sim_user, cor_mat[j, sim_user])
}, 1:ncol(x), 1:nrow(x))


This version already 44 times faster than the loop version:

    test replications elapsed relative user.self sys.self user.child sys.child
1   loop          100    2.21     44.2      2.21        0         NA        NA
2 mapply          100    0.05      1.0      0.05        0         NA        NA