Throttling commands

Question

I have a scenario where my BaseRepository gets a lot of commands in a short time period. At other moments, this doesn't happen. What I want to achieve is to throttle these commands. I.e., when a command must be executed, wait for a short period (0.2s) for any other calls. If none are made, execute the command (go to the database). If others are made, reset the timer. This way, after all commands have been added, we can execute all the commands over one connection.

This is how I implemented it, and I was wondering for any comments:

public BaseRepository()
{
    _throttledActions = new List<Action<Entities>>();
    _timer = new Timer(ExecuteThrottledActions);
}

protected void Execute(Action<Entities> action)
{
    lock (_locker)
    {
        _throttledActions.add(action);
    }

    _timer.Change(TimeSpan.FromSeconds(0.2), TimeSpan.FromMilliseconds(-1));
}

private void ExecuteThrottledActions(object state)
{
    var throttledActionsCopy = new List<Action<Entities>>();
    lock (_locker)
    {
        throttledActionsCopy.AddRange(_throttledActions);
        _throttledActions.Clear();
    }

    using (var scope...)
    {
        using (var entities = new Entities())
        {
            foreach (var throttledAction in throttledActionsCopy)
            {
                throttledAction(entities);
            }
        }
    }
}

This allows me to do the following in a class that inherits from the BaseRepository:

public void Save(Customer customer)
{
    Executed(e => e.Customers.AddObject(customer);
}

And if lots of calls are made to this Save method, I'll save them up and execute them under one using statement and so under one connection.

In short, what I do is:

• add the Action to a list
• start a timer
• if another Action is added fast enough, add this Action too, and reset the timer
• after 0.2 seconds, I'll loop over the actions and execute them

I'm copying the Actions to another list (with locking) because I don't want other calls to interfere with the collection that I'm looping over (and clearing after I've executed everything).

Any comments?

Answer 1

There is one big issue: if you keep getting requests then it is possible to starve the executor.

You can fix that by either testing the size of the list and if it becomes too large then don't reset the timer,

protected void Execute(Action<Entities> action)
{
    bool resetTimer;
    lock (_locker)
    {
        _throttledActions.add(action);
        reset = _throttledActions.size() < MaxSize;
    }

    if(resetTimer)
        _timer.Change(TimeSpan.FromSeconds(0.2), TimeSpan.FromMilliseconds(-1));
}

Or you can use a second slower timer that doesn't get reset and just runs every so often to purge the list anyway regardless of how long actions have been waiting and how many may still follow.