# Penalized logistic ridge regression

This code is for the penalized logistic regression that we should to choose $\lambda$ in regularization to control the trade off. I use the first function to estimate the $\hat{\beta}$ and the main code is to choose the best $\lambda$ by using cross validation leave one out method.

This code takes a really long time. Can anyone help to make it faster?

NRfit.Ridge <- function(y,X,start,lambda,n.iter=30,tol=1e-4,verbose=TRUE) {
## used X rather than Z just because it's more standard notation
n.pred <- ncol(X)
II=matrix(0,nrow=ncol(X),ncol=ncol(X))
diag(II)<-1
B <-  matrix(NA,ncol=n.iter,nrow=n.pred)
B[,1] <- start
for (i in 2:n.iter) {
if (verbose) cat(i,"\n")
p <- plogis(X %*% B[,i-1])
v <- diag(c(p*(1-p)))
score=t(X)%*%(y-p)-B[,i-1]*lambda
increm=solve(t(X)%*%v%*%X+lambda*II)
B[,i]=B[,i-1]-incream%*%score
if (all(abs(B[,i]-B[,i-1]) < tol))
return(B)
}
print(B)
}

####################################################################################
############################## The Main Code #######################################

PE=c()
lambda=c(0.005,0.001,0.1,0.5,0.2)

for(j in 1:length(lambda)){
print(lambda[j])
tem.PE=c()
for(i in 1:2014){
cv.y=y[-i]
cv.x=x[-i,]
tem.lambda=lambda[j]
fit=NRfit.Ridge(y,X,start=0,lambda=tem.lambda)
fitted=(exp(X%*%fit[,30]))/(1+exp(X%*%fit[,30]))
f.hat=fitted
#   print(f.hat)
tem.PE[i]=mean(cv.y-f.hat)
rm(cv.y,cv.x,fit,f.hat,tem.lambda)
}
PE[j]=mean(tem.PE)
rm(tem.PE)
}

• First things first, you ought to indent your code! – flodel Oct 14 '15 at 0:07
• move the fit=NRfit.Ridge(...) call outside the for(i in 1:2014) loop since apparently it does not depend on i... Or should it be using cv.x and cv.y? – flodel Oct 14 '15 at 0:15
• we use the fitted value in loop i which is calculated by fit=NRfit.Ridge(...) – user3478697 Oct 14 '15 at 1:58
• So you see what I mean, right? Move the four lines starting with tem.lambda=lambda[j] right above the for(i in 1:2014) loop. – flodel Oct 15 '15 at 1:31
• ... and a few more spaces (around operators, after commas, ) as well as blank lines to group lines that belong together aren't that expensive, either. – cbeleites unhappy with SX Nov 8 '15 at 21:43