3
\$\begingroup\$

I am new to R and programming. I have a set of ratings for 45000 users and 40 odd movies. I need to predict new ratings for each user based on their pearson correlation with other users. I also need to store the set of similar users for each user-movie combination.The code that I have managed to write is this

# Matrix of users and ratings
x <- matrix(rnorm(1:100), nrow = 10 , ncol = 30)
df = list()


# correlation matrix
cor_mat <- cor(x)

# similarity limits
upper = 1
lower = 0.4

# empty matrix to store predicted values
final_x = matrix(NA,nrow = 10,ncol = 30)

for (i in 1:ncol(x)){

for( j in 1:nrow(x)){

sim_user = which(cor_mat[i,] >= lower & cor_mat[i,] < upper)

final_x[i,j] = t(x[sim_user,j]) %*%  
      cor_mat[sim_user,j]/sum(cor_mat[sim_user,j])

df[[length(df)+1]] = cbind.data.frame(i,j,sim_user,cor_mat[sim_user,j])


 }
}

Questions:

  1. I am looping over each element of the matrix which works fine but seems pretty novice to me. Can something better be done?
  2. I have heard of the foreach package but read that it adds value only when a single operation takes a long time to execute which is not the case here. Will it still provide me good performance?
\$\endgroup\$
  • 2
    \$\begingroup\$ Welcome to Code Review. In the future, please mention your cross-post. \$\endgroup\$ – 200_success Jun 27 '16 at 19:49
  • \$\begingroup\$ Why is x square in your example? More generally, is it m-by-n where m is the number of movies and n is the number of users? Note that cor(x) will be n-by-n so the first loop (i) should be iterating over 1:ncol(x), not 1:nrow(x). So your code will likely break with more realistic data. \$\endgroup\$ – flodel Jun 28 '16 at 0:36
  • \$\begingroup\$ Is this, or is this not, broken code? \$\endgroup\$ – Malachi Jun 30 '16 at 12:23
  • \$\begingroup\$ I am not sure what you mean by broken code \$\endgroup\$ – RUser Jun 30 '16 at 12:54
  • \$\begingroup\$ Maybe you have not read my comment? I'll also point out that df is not initialized so it won't run as-is. Please make sure you provide a reproducible example; which means you should be able to run without error from a fresh session. Then try to make x a non-square matrix to reveal the problem I was describing in my comment. Then please edit your question with the corrected code. \$\endgroup\$ – flodel Jul 1 '16 at 0:23
3
\$\begingroup\$

You have incorrect indexing in the loop. Better version with mapply and correct indexing:

df <- mapply(function(i, j) {
  sim_user = which(cor_mat[i,] >= lower & cor_mat[i,] < upper)
  final_x[j, i] = t(x[j, sim_user]) %*% (cor_mat[j, sim_user]/sum(cor_mat[j, sim_user]))
  cbind(i, j, sim_user, cor_mat[j, sim_user])
}, 1:ncol(x), 1:nrow(x))

This version already 44 times faster than the loop version:

    test replications elapsed relative user.self sys.self user.child sys.child
1   loop          100    2.21     44.2      2.21        0         NA        NA
2 mapply          100    0.05      1.0      0.05        0         NA        NA
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.