1
\$\begingroup\$

This code is for the penalized logistic regression that we should to choose \$\lambda\$ in regularization to control the trade off. I use the first function to estimate the \$\hat{\beta}\$ and the main code is to choose the best \$\lambda\$ by using cross validation leave one out method.

This code takes a really long time. Can anyone help to make it faster?

NRfit.Ridge <- function(y,X,start,lambda,n.iter=30,tol=1e-4,verbose=TRUE) {
## used X rather than Z just because it's more standard notation
n.pred <- ncol(X)
II=matrix(0,nrow=ncol(X),ncol=ncol(X))
diag(II)<-1
B <-  matrix(NA,ncol=n.iter,nrow=n.pred)
B[,1] <- start
for (i in 2:n.iter) {
if (verbose) cat(i,"\n")
p <- plogis(X %*% B[,i-1])
v <- diag(c(p*(1-p)))
score=t(X)%*%(y-p)-B[,i-1]*lambda
increm=solve(t(X)%*%v%*%X+lambda*II)
B[,i]=B[,i-1]-incream%*%score
if (all(abs(B[,i]-B[,i-1]) < tol)) 
  return(B)
 }
 print(B)
 }

 ####################################################################################
 ############################## The Main Code #######################################


PE=c()
lambda=c(0.005,0.001,0.1,0.5,0.2)

for(j in 1:length(lambda)){
print(lambda[j])
tem.PE=c()
for(i in 1:2014){
cv.y=y[-i]
cv.x=x[-i,]
tem.lambda=lambda[j]
fit=NRfit.Ridge(y,X,start=0,lambda=tem.lambda)
fitted=(exp(X%*%fit[,30]))/(1+exp(X%*%fit[,30]))
f.hat=fitted
 #   print(f.hat)
tem.PE[i]=mean(cv.y-f.hat)
rm(cv.y,cv.x,fit,f.hat,tem.lambda)
 }
PE[j]=mean(tem.PE)
rm(tem.PE)
}
\$\endgroup\$
  • \$\begingroup\$ First things first, you ought to indent your code! \$\endgroup\$ – flodel Oct 14 '15 at 0:07
  • \$\begingroup\$ move the fit=NRfit.Ridge(...) call outside the for(i in 1:2014) loop since apparently it does not depend on i... Or should it be using cv.x and cv.y? \$\endgroup\$ – flodel Oct 14 '15 at 0:15
  • \$\begingroup\$ we use the fitted value in loop i which is calculated by fit=NRfit.Ridge(...) \$\endgroup\$ – user3478697 Oct 14 '15 at 1:58
  • \$\begingroup\$ So you see what I mean, right? Move the four lines starting with tem.lambda=lambda[j] right above the for(i in 1:2014) loop. \$\endgroup\$ – flodel Oct 15 '15 at 1:31
  • \$\begingroup\$ ... and a few more spaces (around operators, after commas, ) as well as blank lines to group lines that belong together aren't that expensive, either. \$\endgroup\$ – cbeleites Nov 8 '15 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.