Extract value from scala collect case

Question

I have the following code which reads data from a key/value data storage. At some stage I want to go through everything in the storage and check expiry times and remove values - as done in the code below.

My problem here is that dataMapStorage.get(key) is called twice for keys which pass the filter and this can potentially be a heavy operation (I/O).

dataMapStorage.keys
  .filter { key =>
    val value = dataMapStorage.get(key)
    currentTime.getMillis - value.getMillis > timeout
  }
  .foreach { key =>
    val value = dataMapStorage.get(key) // Want to avid this double lookup
    // ... do some work with "value"
    dataMapStorage.remove(key)
  }

I presume the way around it is using collect (I do not want to map through the storage and extract all the values as this might be large), but am stuck here too with the 'get' method:

dataMapStorage.keys
  .collect { 
    case key if {
      val value = dataMapStorage.get(key)
      currentTime.getMillis - value.getMillis > timeout
    } => (key, value) // How can I get the "value"???
  }
  .foreach { pair =>
    val value = pair._2
    // ... do some work with "value"
    dataMapStorage.remove(pair._1)
  }

How can I run this code getting the value from storage only once and not first creating a list of all the values?

Answer 1

Your filter isn't actually producing any benefit if the resulting collection isn't saved for further processing.

dataMapStorage.keys
  .foreach { key =>
    val value = dataMapStorage.get(key)
    if (currentTime.getMillis - value.getMillis > timeout) {
      // ... do some work with "value"
      dataMapStorage.remove(key)
    }
  }

Also, if dataMapStorage is just a key->value Map then couldn't you simplify it?

dataMapStorage.foreach { case (key, value) =>
    if (currentTime.getMillis - value.getMillis > timeout) {
      // ... do some work with "value"
      dataMapStorage.remove(key)
    }
  }

You could use collect() here but since the procedure results in a side-effect I think foreach() is more explanatory.

Of course the real Scala FP approach is to avoid mutable data structures.

val newDataMapStorage = oldDataMapStorage.filter { case (key, value) =>
    if (currentTime.getMillis - value.getMillis > timeout) {
      // ... do some work with "value"
      false
    } else true
  }

Answer 2

You can move // ... do some work with "value" into filter, so that in foreach you just remove the key.