Conditionally creating new variables for data manipulation

Question

My problem is related to data manipulation: on the basis of the existing variables in a given data set, I would like to create new ones. Bonus points for performance as well as code readability.

Let's consider a reproducible example for the famous "iris" data set. My goal is to create a new variable, which can be described by the following set of conditions:

1. if species="setosa" or species="virginica" and sepal.length < 5 then new variable = "Species1",
2. if (species="setosa" or species="virginica") and 5<= sepal.length < 10 then new variable = "Species2",
3. if (species="setosa" or species="virginica") and sepal.length >= 10 then new variable = "Species3",
4. otherwise, new variable = species (in this case "versicolor").

In order to achieve this, I have written the following conditional instructions:

iris$Speciec2 <- ifelse(iris$Species %in% c("setosa", "virginica"),
                       ifelse(iris$Sepal.Length < 5, "Species1",
                              ifelse(iris$Sepal.Length >= 5 & iris$Sepal.Length < 7,
                                     "Species2", "Species3")), as.character(iris$Species))

and it's really hard to read. I wonder if there are any better solutions.

Answer 1

You can make a couple easy changes to improve readability. First, use with or transform so you don't have to carry a lot of iris$. Second, reorganize your logical tests so each nested ifelse appear in the "if-false" part of the previous ifelse, not in the "if-true" part. Hopefully, this will make sense:

iris$Speciec2 <- with(iris,
  ifelse(!Species %in% c("setosa", "virginica"), as.character(Species),
  ifelse(Sepal.Length < 5,                       "Species1",
  ifelse(Sepal.Length < 7,                       "Species2",
                                                 "Species3")))
)

I think it reads a lot easier this way, where each line contains a test and the output value if the test is passed. If the test fails, we move onto the next line. And at the end, the last line contains the value if all tests have failed. I also like this layout because it looks a lot like how you would nest Perl's ternary operator:

result = condition1 ? value_if_condition1_is_true
       : condition2 ? value_if_condition2_is_true
       : condition3 ? value_if_condition3_is_true
       :              value_if_condition3_is_false

Another solution would be to use the cut or findInterval functions to handle the conditions you have on Sepal.Length. These functions become really handy when you have many cases ("breaks") in mind, although here you only have three. You could do:

new_labels <- cut(iris$Sepal.Length, breaks = c(-Inf, 5, 7, +Inf), 
                  labels = paste0("Species", 1:3), right = FALSE)
iris$Speciec2 <- with(iris, ifelse(Species %in% c("setosa", "virginica"),
                                   as.character(new_labels),
                                   as.character(Species)))

---

Now about performance, since you mentioned it. Know that ifelse and cut are vectorized functions so unless you are handling many millions of rows or doing some high frequency trading, you should be fine :-)

Minor speed improvements might be made by replacing the ifelse with in-place replacements (x[idx] <- value), I let you decide if this is more readable:

iris$Speciec2 <- with(iris, {
  res <- as.character(Species)
  need_change <- Species %in% c("setosa", "virginica")
  res[need_change & Sepal.Length < 5]                     <- "Species1"
  res[need_change & Sepal.Length >= 5 & Sepal.Length < 7] <- "Species2"
  res[need_change & Sepal.Length >= 7]                    <- "Species3"
  res
})

Finally, switching to a data.table (from the R package of the same name) is always recommended when dealing with large size data.frames and when speed is a concern.

Answer 2

If concerned by performance consider data.table.

Here as a quick and dirty implementation:

require(data.table)
dt <- iris
setDT(dt)

setDT(dt)[Species != "versicolor" & Sepal.Length < 5, 
          new.var := "Species1"]
setDT(dt)[Species != "versicolor" & Sepal.Length >=5 & Sepal.Length < 10,  
          new.var := "Species2"]
setDT(dt)[Species != "versicolor" & Sepal.Length >= 10, 
          new.var := "Species3"]
setDT(dt)[is.na(new.var), 
          new.var := "Species"]