# Merging predicted values into measured values

I created the following function below to merge the real values with the predicted values (when real are absent) in a new column in data.frame. The function actually works, but I would like to optimize it, because with the dataset that I work with, the function takes about two hours to run.

The function seems to be slowed down by the loop.

p <-
function(object, newdata = NULL, type = c("link", "response", "terms"),
rse.fit = FALSE, dispersion = NULL, terms = NULL,
na.action = na.pass, ...)
{
{
pred <- predict (object,newdata)

}

vetor1 <- (newdata$ALT) # Creates a column vector from the actual heights of the data.frame vetor1[is.na(vetor1)] <- 0 # Replaces the NA's present in the vector created above the numeric value 0 vetor2 <- c(pred) # Creates a vector from the predicted data for(i in 1:length(vetor1)){ # The loop is executed until all values vector1 pass the following condition if(vetor1[i]==0.00){ # If a value of the first vector has the value 0, ie, if it is absent vetor1[i]=vetor2[i] # Then the predicted value will replace the missing value newdata$ALTMISTA <- vetor1  # The vector1, already possessing the actual values and the predicted values merged into the same vector goes                   on to become a new column in data.frame, this column is called a ALTMISTA
}
}
return (newdata)
}


## 1 Answer

I am assuming that the call to predict is not what is taking most of the time, so I'm not including that in my analysis.

One thing that is slowing down your code is that you are assigning vetor1 to newdata$ALTMISTA every iteration of the loop (well, every iteration that is 0). That could be pulled out of the loop since it only needs to be done once. vetor1 <- (newdata$ALT)         # Creates a column vector from the actual heights of the data.frame
vetor1[is.na(vetor1)] <- 0      # Replaces the NA's present in the vector created above the numeric value 0
vetor2 <- c(pred)           # Creates a vector from the predicted data
for(i in 1:length(vetor1)){     # The loop is executed until all values vector1 pass the following condition
if(vetor1[i]==0.00){      # If a value of the first vector has the value 0, ie, if it is absent
vetor1[i]=vetor2[i]     # Then the predicted value will replace the missing value
}
}
newdata$ALTMISTA <- vetor1 # The vector1, already possessing the actual values and the predicted values merged into the same vector goes on to become a new column in data.frame, this column is called a ALTMISTA return (newdata)  Second, you replace NA with 0, and then test against 0. That means you are replacing both NA's and any 0's in the data. If you only want to replace NA's, just replace those without recoding them. vetor1 <- (newdata$ALT)         # Creates a column vector from the actual heights of the data.frame
vetor2 <- c(pred)           # Creates a vector from the predicted data
for(i in 1:length(vetor1)){     # The loop is executed until all values vector1 pass the following condition
if(is.na(vetor1[i])){      # If a value of the first vector is NA, ie, if it is absent
vetor1[i]=vetor2[i]     # Then the predicted value will replace the missing value
}
}
newdata$ALTMISTA <- vetor1 # The vector1, already possessing the actual values and the predicted values merged into the same vector goes on to become a new column in data.frame, this column is called a ALTMISTA return (newdata)  Now the explicit loop can be replaced by vectorized functions, in this case ifelse vetor1 <- (newdata$ALT)         # Creates a column vector from the actual heights of the data.frame
vetor2 <- c(pred)           # Creates a vector from the predicted data
vetor1 <- ifelse(is.na(vetor1), vetor2, vetor1) # replace elements of vetor1 that are NA with corresponding elements of vetor2
newdata$ALTMISTA <- vetor1 # The vector1, already possessing the actual values and the predicted values merged into the same vector goes on to become a new column in data.frame, this column is called a ALTMISTA return (newdata)  Finally you can eliminate some single use intermediate variables. This may not appreciably speed up the function, but it does make for cleaner code. vetor1 <- (newdata$ALT)
newdata\$ALTMISTA <- ifelse(is.na(vetor1), pred, vetor1)
return (newdata)


As you did not provide example data, I was not able to benchmark these alternatives (nor, for that matter, even run them).

• Thanks! Running only with the first suggested change, reduced from 3 hours to 15 seconds (which I found rather odd for the size of the data set, but it's all right). =D – Marco Machado Nov 20 '14 at 15:12