Thread safe double buffered cache

Question

I want cache object for one value, which have two expiration limits. One "soft" limit indicating "value is old, but is still valid, but is good time for reload the value". And one "hard" limit indicating "value is old and reload is required".

The cache will be used in web server and will be accessed from several requests from several users, so members of class must be thread safe. The value loading functions will be expensive sometimes, so I want minimize unnecessary/duplicate value loading.

The target is smooth returning cached (and valid) values for clients with minimum waiting times (I need low latency).

Souce code:

/// <summary>
/// Cache with soft and hard expiration times.
/// </summary>
/// <remarks>
/// All static and public instance members of this class are thread-safe.
/// However the <c>ValueLoader</c> and <c>asyncActionRunner</c> must be thread-safe too for full thread safety.
/// The default <c>asyncActionRunner</c> is thread-safe.
/// </remarks>
/// <typeparam name="TValue">Type of cached value.</typeparam>
public sealed class SoftHardCache<TValue> where TValue : class {
    /// <summary>
    /// Default method for asynchronous calling.
    /// </summary>
    /// <param name="action">Action to run in different thread.</param>
    private static void RunAsync(Action action) {
        if (object.ReferenceEquals(action, null)) { throw new ArgumentNullException("action"); }

        ThreadPool.QueueUserWorkItem(delegate { action(); });
    }

    /// <summary>
    /// Lock for writing into <c>value</c>.
    /// </summary>
    private readonly object locker_Value = new object();
    private TValue value;
    /// <summary>
    /// Last loaded value.
    /// Member is thread-safe.
    /// </summary>
    /// <remarks>
    /// If value was not loaded, returns NULL.
    /// </remarks>
    public TValue Value {
        get { return this.value; }
        private set {
            lock (this.locker_Value) {
                this.value = value;
                this.RestartWatch();
            }
        }
    }

    private readonly Func<TValue> valueLoader;
    /// <summary>
    /// Delegate used for reloading value.
    /// Member is thread-safe.
    /// </summary>
    public Func<TValue> ValueLoader {
        get { return this.valueLoader; }
    }

    /// <summary>
    /// The <c>softExpirationTime</c> in milliseconds.
    /// </summary>
    private readonly long softExpirationMilliseconds;
    private readonly TimeSpan softExpirationTime;
    /// <summary>
    /// Expiration time, when value is still valid, but is time for reloading value.
    /// </summary>
    public TimeSpan SoftExpirationTime {
        get { return this.softExpirationTime; }
    }

    /// <summary>
    /// The <c>hardExpirationTime</c> in milliseconds.
    /// </summary>
    private readonly long hardExpirationMilliseconds;
    private readonly TimeSpan hardExpirationTime;
    /// <summary>
    /// Expiration time, when value is not valid and need reload.
    /// </summary>
    public TimeSpan HardExpirationTime {
        get { return this.hardExpirationTime; }
    }

    private readonly object locker_Watch = new object();
    /// <summary>
    /// Stopwatch measuring expiration times.
    /// </summary>
    private readonly Stopwatch watch;
    /// <summary>
    /// Restarts <c>watch</c>.
    /// Member is thread-safe.
    /// </summary>
    private void RestartWatch() {
        lock (this.locker_Watch) {
            this.watch.Restart();
        }
    }
    /// <summary>
    /// Returns elapsed milliseconds from last load.
    /// Member is thread-safe.
    /// </summary>
    /// <returns></returns>
    private long GetElapsedMilliseconds() {
        var result = this.watch.ElapsedMilliseconds;
        return result;
    }

    /// <summary>
    /// Method for asynchronous calling.
    /// </summary>
    private readonly Action<Action> asyncActionRunner;

    public SoftHardCache(Func<TValue> valueLoader, TimeSpan softExpirationTime, TimeSpan hardExpirationTime) : this(RunAsync, valueLoader, softExpirationTime, hardExpirationTime) { }
    public SoftHardCache(Action<Action> asyncActionRunner, Func<TValue> valueLoader, TimeSpan softExpirationTime, TimeSpan hardExpirationTime) {
        if (object.ReferenceEquals(asyncActionRunner, null)) { throw new ArgumentNullException("asyncActionRunner"); }
        if (object.ReferenceEquals(valueLoader, null)) { throw new ArgumentNullException("valueLoader"); }
        if (softExpirationTime <= TimeSpan.Zero) { throw new ArgumentOutOfRangeException("softExprirationTime", "Expiration time must be greater than zero."); }
        if (hardExpirationTime <= TimeSpan.Zero) { throw new ArgumentOutOfRangeException("hardExpirationTime", "Expiration time must be greater than zero."); }
        if (softExpirationTime >= hardExpirationTime) { throw new ArgumentOutOfRangeException("hardExpirationTime", "Hard expiration time must be greater than soft expiration time."); }

        this.watch = new Stopwatch();
        this.asyncActionRunner = asyncActionRunner;
        this.valueLoader = valueLoader;

        this.hardExpirationTime = hardExpirationTime;
        this.hardExpirationMilliseconds = (long)hardExpirationTime.TotalMilliseconds;

        this.softExpirationTime = softExpirationTime;
        this.softExpirationMilliseconds = (long)softExpirationTime.TotalMilliseconds;
    }

    /// <summary>
    /// Returns currently loaded value with infos about expirations.
    /// Member is thread-safe.
    /// </summary>
    /// <param name="softLimitExpired">
    /// Returns TRUE if soft limit was expired.
    /// Otherwise returns FALSE.
    /// 
    /// When soft limit expired, the value is still valid but is good time for reload.
    /// </param>
    /// <param name="hardLimitExpired">
    /// Returns TRUE if hard limit was expired.
    /// Otherwise returns FALSE.
    /// 
    /// When hard limit expired, the value is not valid and need reload.
    /// </param>
    /// <returns>
    /// Last loaded value.
    /// Member is thread-safe.
    /// </returns>
    public TValue PeekValue(out bool softLimitExpired, out bool hardLimitExpired) {
        var elapsedMilliseconds01 = this.watch.ElapsedMilliseconds;
        var result = this.value;
        var elapsedMilliseconds02 = this.watch.ElapsedMilliseconds;

        if (elapsedMilliseconds01 > elapsedMilliseconds02) {
            // value was changed between geting elapsedMilliseconds01 and geting result
            softLimitExpired = true;
            hardLimitExpired = true;
        }

        hardLimitExpired = (elapsedMilliseconds02 >= this.hardExpirationMilliseconds);
        if (hardLimitExpired) {
            softLimitExpired = true;
        }
        else {
            softLimitExpired = (elapsedMilliseconds02 > this.softExpirationMilliseconds);
        }

        return result;
    }
    /// <summary>
    /// Returns valid value.
    /// Member is thread-safe.
    /// </summary>
    /// <remarks>
    /// If value is not loaded, the value will be reloaded.
    /// If hard limit expired, the value will be reloaded.
    /// If soft limit expired, the reload will be run in different thread and the current value will be returned.
    /// </remarks>
    /// <returns></returns>
    public TValue GetValue() {
        bool softLimitExpired, hardLimitExpired;
        var result = this.PeekValue(out softLimitExpired, out hardLimitExpired);
        if (object.ReferenceEquals(result, null)) {
            // value is not loaded
            result = this.ReloadValueSync();
        }
        if (hardLimitExpired) {
            // value is expired
            result = this.ReloadValueSync();
        }
        else if (softLimitExpired) {
            // value is valid, but soft limit is expired, so we want reload value for usage in future
            this.ReloadValueAsync();
        }
        else {
            // value is up-to-date
        }

        return result;
    }
    /// <summary>
    /// Returns value from <c>ValueLoader</c>.
    /// The value will be NOT assigned into <c>Value</c> property.
    /// Member is thread-safe.
    /// </summary>
    /// <returns></returns>
    public TValue LoadValue() {
        var result = this.valueLoader();
        return result;
    }
    /// <summary>
    /// The <c>ReloadValue()</c> method will call in different thread.
    /// Member is thread-safe.
    /// </summary>
    public void ReloadValueAsync() {
        this.ReloadValueAsync(null, null);
    }

    /// <summary>
    /// The <c>ReloadValue()</c> method will call in different thread.
    /// Member is thread-safe.
    /// </summary>
    /// <param name="context">Context object passed into <c>callback</c>.</param>
    /// <param name="callback">Callback called from different thread, when realod will be done (even if exception will be thrown).</param>
    public void ReloadValueAsync(object context, ResultCallback callback) {
        this.asyncActionRunner(() => {
            TValue result = null;
            Exception thrownException = null;
            try { result = this.ReloadValue(); }
            catch (Exception exc) {
                thrownException = exc;
            }

            if (null != callback) {
                callback(context, result, thrownException);
            }
        });
    }

    /// <summary>
    /// Lock for reloading value.
    /// </summary>
    private readonly object locker_Reloading = new object();
    /// <summary>
    /// Flag: last reload failed.
    /// </summary>
    private bool lastReloadFailed = false;
    /// <summary>
    /// Will load value from <c>LoadValue</c> and the returned value will be assigned into <c>Value</c> property.
    /// If is value currently reloading (in different thread), this call will wait until realoading will finished.
    /// After that:
    /// 1) If previous reloading failed, the new reloading will be processing.
    /// 2) If previous reloading succeeded, the previously reloaded value will be returned. Because of minimalizig unnecesary reloading.
    /// 
    /// Member is thread-safe.
    /// </summary>
    /// <returns></returns>
    private TValue ReloadValueSync() {
        bool lockTaken = false;
        try {
            // try take the lock for value reload
            Monitor.TryEnter(this.locker_Reloading, ref lockTaken);
            if (lockTaken) {
                // reloading is not running in different thread, we can reload value
                return this.ReloadValue();
            }
        }
        finally{
            if (lockTaken) {
                // lock was taken, we must release lock
                Monitor.Exit(this.locker_Reloading);
            }
        }

        // reloading was running in different thread, we will wait for finish
        lock (this.locker_Reloading) {
            // reloading in different thread was finished, we can continue
            if (this.lastReloadFailed) {
                // last reloading failed, we must realod value again
                return this.ReloadValue();
            }
            else {
                // last reloading succeeded, we will use already loaded value
                return this.value;
            }
        }
    }

    /// <summary>
    /// Reloads value from <c>ValueLoader</c> and assigns the result into <c>Value</c> property.
    /// Member is NOT thread-safe.
    /// </summary>
    /// <returns>
    /// Returns reloaded value.
    /// </returns>
    private TValue ReloadValue() {
        TValue result;
        try {
            result = this.LoadValue();
            this.Value = result;
            this.lastReloadFailed = false;
        }
        catch {
            this.lastReloadFailed = true;
            throw;
        }

        return result;
    }
}

Answer 1

Just a few minor quibbles.

You don't need to create a special object locker_Watch for locking. You can just lock the Stopwatch watch itself. But I question why you're locking it at all; what race condition are you guarding against?

Don't bother with the milliseconds. Just keep everything as a TimeSpan. Use watch.Elapsed to get a Timespan out.

In PeekValue this code:

   if (elapsedMilliseconds01 > elapsedMilliseconds02) {
       // mezitim se hodnota zmenila
       softLimitExpired = true;
       hardLimitExpired = true;
   }

is dead. You set hardLimitExpired to true and then immediately reset it on the next statement, then reset softLimitExpired one statement later. The compiler will remove it. But I don't know what your intention was. (I don't speak Polish)

What method do you have to ensure that the value is not read while it's in the process of being asynchronously reloaded? I'd think that both GetValue and ReloadValueAsync (or the closure it passes) should lock.