Predict new ratings for each user based on their pearson correlation with other users

Question

I am new to R and programming. I have a set of ratings for 45000 users and 40 odd movies. I need to predict new ratings for each user based on their pearson correlation with other users. I also need to store the set of similar users for each user-movie combination.The code that I have managed to write is this

# Matrix of users and ratings
x <- matrix(rnorm(1:100), nrow = 10 , ncol = 30)
df = list()


# correlation matrix
cor_mat <- cor(x)

# similarity limits
upper = 1
lower = 0.4

# empty matrix to store predicted values
final_x = matrix(NA,nrow = 10,ncol = 30)

for (i in 1:ncol(x)){

for( j in 1:nrow(x)){

sim_user = which(cor_mat[i,] >= lower & cor_mat[i,] < upper)

final_x[i,j] = t(x[sim_user,j]) %*%  
      cor_mat[sim_user,j]/sum(cor_mat[sim_user,j])

df[[length(df)+1]] = cbind.data.frame(i,j,sim_user,cor_mat[sim_user,j])


 }
}

Questions:

1. I am looping over each element of the matrix which works fine but seems pretty novice to me. Can something better be done?
2. I have heard of the foreach package but read that it adds value only when a single operation takes a long time to execute which is not the case here. Will it still provide me good performance?

Answer 1

You have incorrect indexing in the loop. Better version with mapply and correct indexing:

df <- mapply(function(i, j) {
  sim_user = which(cor_mat[i,] >= lower & cor_mat[i,] < upper)
  final_x[j, i] = t(x[j, sim_user]) %*% (cor_mat[j, sim_user]/sum(cor_mat[j, sim_user]))
  cbind(i, j, sim_user, cor_mat[j, sim_user])
}, 1:ncol(x), 1:nrow(x))

This version already 44 times faster than the loop version:

    test replications elapsed relative user.self sys.self user.child sys.child
1   loop          100    2.21     44.2      2.21        0         NA        NA
2 mapply          100    0.05      1.0      0.05        0         NA        NA