0
\$\begingroup\$

I have the following code which reads data from a key/value data storage. At some stage I want to go through everything in the storage and check expiry times and remove values - as done in the code below.

My problem here is that dataMapStorage.get(key) is called twice for keys which pass the filter and this can potentially be a heavy operation (I/O).

dataMapStorage.keys
  .filter { key =>
    val value = dataMapStorage.get(key)
    currentTime.getMillis - value.getMillis > timeout
  }
  .foreach { key =>
    val value = dataMapStorage.get(key) // Want to avid this double lookup
    // ... do some work with "value"
    dataMapStorage.remove(key)
  }

I presume the way around it is using collect (I do not want to map through the storage and extract all the values as this might be large), but am stuck here too with the 'get' method:

dataMapStorage.keys
  .collect { 
    case key if {
      val value = dataMapStorage.get(key)
      currentTime.getMillis - value.getMillis > timeout
    } => (key, value) // How can I get the "value"???
  }
  .foreach { pair =>
    val value = pair._2
    // ... do some work with "value"
    dataMapStorage.remove(pair._1)
  }

How can I run this code getting the value from storage only once and not first creating a list of all the values?

\$\endgroup\$

2 Answers 2

2
\$\begingroup\$

Your filter isn't actually producing any benefit if the resulting collection isn't saved for further processing.

dataMapStorage.keys
  .foreach { key =>
    val value = dataMapStorage.get(key)
    if (currentTime.getMillis - value.getMillis > timeout) {
      // ... do some work with "value"
      dataMapStorage.remove(key)
    }
  }

Also, if dataMapStorage is just a key->value Map then couldn't you simplify it?

dataMapStorage.foreach { case (key, value) =>
    if (currentTime.getMillis - value.getMillis > timeout) {
      // ... do some work with "value"
      dataMapStorage.remove(key)
    }
  }

You could use collect() here but since the procedure results in a side-effect I think foreach() is more explanatory.

Of course the real Scala FP approach is to avoid mutable data structures.

val newDataMapStorage = oldDataMapStorage.filter { case (key, value) =>
    if (currentTime.getMillis - value.getMillis > timeout) {
      // ... do some work with "value"
      false
    } else true
  }
\$\endgroup\$
1
\$\begingroup\$

You can move // ... do some work with "value" into filter, so that in foreach you just remove the key.

\$\endgroup\$
1
  • \$\begingroup\$ I only want to do the work on the filtered values before removing them. I guess I could add an if there but thought there would be a more scala-way of doing it? \$\endgroup\$
    – LK__
    Aug 24, 2017 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.