Checking the presence of certain variables in a dataframe

Question

I'm writing an Exploratory Data Analysis using R Markdown. First of all, I need to check the "sanity" of the input data set. Among the other check I perform, I need to check if at least one between the variables var_in and var_out are present in the dataset. If at least one is present, the other one can be computed from the first one. Thus, I want to check which of them is missing, and store its name in a character vector. If both are missing, the analysis is impossible and I need to exit, preferably throwing a meaningful error.

I cannot include the actual dataset on which the check is performed, because it's \$10^6\$ rows with confidential data, so I need to use fake data. The code, however, is as similar to the real code as possible.

# fake data
n <- 10^6
input_df <- data.frame(wind_speed = 10*abs(rnorm(n)), wind_direction = runif(n, 0, 2*pi),
                       var_in = NA, var_out = 3)

# nearly real code
missing_variables <- character(0)

var_in_is_missing <- all(is.na(input_df$var_in))
var_out_is_missing <- all(is.na(input_df$var_out))

if (var_in_is_missing & var_out_is_missing) {
  stop("both var_in and var_out are completely missing, so I cannot continue the EDA")
}
if (var_in_is_missing) {
  missing_variables <- c("var_in", missing_variables)
}
if (var_out_is_missing) {
  missing_variables <- c("var_out", missing_variables)
}

Note that according to the principle of early return, I put the stopping test before the other two. The code runs, but it doesn't seem that readable to me:

1. I test both var_in_is_missing and var_out_is_missing twice: this is definitely not going to impact notably the performance of my code, but it still feels useless
2. is it really necessary to use three separate if statements? Isn't there a way to do it in R in a more...compact way, without sacrificing readability?

Answer 1

I would do something like this:

tt <- function() {
  naI <- sapply(input_df[, c('var_in', 'var_out')], function(x) all(is.na(x)))
  if (sum(naI) == 2) {
    stop("both var_in and var_out are completely missing, so I cannot continue the EDA")
  } else {
    missing_variables <- names(naI[naI == TRUE])
  }
  missing_variables
}

I put the code into function for testing. Tests:

input_df <- data.frame(var_in = NA, var_out = 3)
tt()
#> [1] "var_in"
input_df <- data.frame(var_in = NA, var_out = NA)
tt()
#> Error in tt(): both var_in and var_out are completely missing, so I cannot continue the EDA
input_df <- data.frame(var_in = 3, var_out = NA)
tt()
#> [1] "var_out"
input_df <- data.frame(var_in = 3, var_out = 3)
tt()
#> character(0)

P.S. maybe you also want to add some kind of treatment if the one/both columns are missing in data.frame...

Answer 2

Another possibility :

cols <- c('var_in', 'var_out')
missing_variables <- cols[!colSums(!is.na(input_df[cols]))]
if (length(missing_variables) == 2) 
  stop("both var_in and var_out are completely missing, so I cannot continue the EDA")

Let's unwrap the definition of missing_variables :

• !is.na(input_df[cols]) creates a matrix of logicals indicating if elements are NA (FALSE) or not (TRUE),
• colSums(!is.na(input_df[cols])) sum the columns of this matrix, FALSE is coerced to 0 and TRUE to 1 so a column will be full of NA if and only if this sum is O
• In !colSums(!is.na(input_df[cols])), ! changes 0 to TRUE and any other number to FALSE, so we get a vector of logical indicating if the variable is missing (TRUE) or not
• cols[!colSums(!is.na(input_df[cols]))] subsets cols to keep only "missing variables"