1
\$\begingroup\$

I have a workbook with data sampled. i have multiple samples based on date range. one sample starts from january, where other can start from february and so on. I need to build forecasting function using auto.arima for 15 samples of data. instead of writing auto.arima 15 times, i tried using a loop to execute the code and get results from 15 models in the end for comparison. It takes lot of time to execute this through loops Vs if i do this manually one after another. Is there any way to reduce the processing time? I know that for loop tends to slower, but is there any way to replace for loop in this case?

library("readxl")
library("lubridate")
library("forecast")
library("parallel")
library("doParallel")
library("foreach")

##Use multicore processing
cl <- makeCluster(detectCores() - 1)
registerDoParallel(cl, cores = detectCores() - 1)

  for (i in 1:15) {
    #read excel file and format date column
    da<-paste("El3M","-",i,sep = "")
    a<-read_excel("All Models.xlsm", sheet = da,col_names=TRUE)
    a$Date<-as.Date(a$Date,"%Y-%m-%d")

    #add seasonality dummy variable and remove intercept column    
    abc <- cbind(Weekday=model.matrix(~as.factor(a$DOW)),
                 Day=model.matrix(~as.factor(a$DOM)),a[,7,drop=FALSE])
    abc<-data.frame(abc)
    abc<-data.frame(abc[,c(-1,-8)])

    #partition data into train and test
    abc2 <- subset(abc,abc$Ya==1)
    abc2 <- abc2[,-37]
    abc3 <- subset(abc,abc$Ya==2)
    abc3 <- abc3[,-37]

    #train and insample MAPE
    a1<-subset(a,a$Ya==1)
    a2<-subset(a,a$Ya==2)  

    ###ARIMA###
    val.ts<-ts(a1$Actual,start=c(year(a1$Date[1]),month(a1$Date[1]),day(a1$Date[1])),freq=365)
    ARIMAfit2 <- auto.arima(val.ts,xreg=abc2,stepwise=FALSE,approx=FALSE)

    #test and forecasted MAPE
    azz<-forecast(ARIMAfit2,h=90,xreg=abc3)
    azz<-azz$mean
    azz <- data.frame(azz)
    azz$Actual <- a2$Actual
    azz$MAPE <- abs(azz$azz-azz$Actual)/azz$Actual

    #print error results from model
    print(paste("ARIMA Sample ",da, "MAPE hold Total =",mean(azz$MAPE[1:90],na.rm = TRUE),sep = ""))

}
\$\endgroup\$
  • \$\begingroup\$ All things equal, without parallelization, 15 iterations should take 15 times the time it takes to run one iteration. (Unless you do something really wrong like growing an object, which you're not doing here.) Are you sure you're not missing something? Maybe some iterations take longer than others because the data is larger? \$\endgroup\$ – flodel Jul 26 '16 at 23:21
  • \$\begingroup\$ I'd recommend you look into profiling your code. See the example at the bottom of ?summaryRprof for help. In particular, I'd be curious to see how much time is spent in read_excel. \$\endgroup\$ – flodel Jul 26 '16 at 23:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.