1
\$\begingroup\$

I'm implementing a queue to update leaderboards when score data changes in the background. Updating the leaderboard is a relatively expensive operation, so if we say "Update leaderboard 5" and then immediately say "Update leaderboard 5" again before the first one is finished, we should just update leaderboard 5 one time.

To do this I'm using TPL dataflow to create an ActionBlock which takes an id to update, then calls the subscriptions delegate which does the actual work. I'm using a BufferBlock to post to, which checks a HashSet to see if the leaderboard is already pending update, and if so posts the data to the ActionBlock, otherwise posts it to a NullTarget. When a leaderboard is updated, I remove the id from dupeMap so it can be processed again if data changes.

The goal is to minimize the duplicates, if there's a few edge cases where the same leaderboard might be updated twice, it's not a huge deal. Here's my code, I'd appreciate any comments or feedback since I'm not a complete TPL-Dataflow master (this is the first time I've used anything besides ActionBlock<T>.

public class LeaderboardQueue : ILeaderboardQueue
{
    private Func<int, Task> subscriptions;
    private readonly ActionBlock<int> block;
    private readonly BufferBlock<int> filter;
    private readonly HashSet<int> dupeMap = new(); // Ensure the same leaderboard isn't being recalc'ed twice in the queue, which would be a waste of time

    public LeaderboardQueue()
    {
        block = new ActionBlock<int>(async activityId =>
            {
                dupeMap.Remove(activityId);
                await subscriptions(activityId);
            },
            new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 3 });
        
        filter = new BufferBlock<int>();
        filter.LinkTo(block, dupeMap.Add);
        filter.LinkTo(DataflowBlock.NullTarget<int>());
    }
    
    public async Task MarkLeaderboardDirty(int activityId)
    {
        await filter.SendAsync(activityId);
    }

    public void Subscribe(Func<int, Task> fnProcess)
    {
        subscriptions += fnProcess;
    }

    public async Task Shutdown()
    {
        filter.Complete();
        await filter.Completion;
        
        block.Complete();
        await block.Completion;
    }
}    

I also have a question on the Shutdown method. Am I correct to Complete() both blocks, or do I just need to complete one of them? Thanks!

\$\endgroup\$
9
  • \$\begingroup\$ Why do you name a BufferBlock as filter? What does it filter? \$\endgroup\$ Commented Oct 7, 2022 at 12:56
  • 1
    \$\begingroup\$ Am I correct to Complete() both blocks, or do I just need to complete one of them Since they are linked it is enough to call Complete on the first block and await the Completion of the last block. \$\endgroup\$ Commented Oct 7, 2022 at 13:00
  • 1
    \$\begingroup\$ I've played a little bit with your code. I've stated earlier that completion propagates automatically if they are linked. I was wrong that's not the default you have to set it in the LinkTo call: new DataflowLinkOptions { PropagateCompletion = true } \$\endgroup\$ Commented Oct 10, 2022 at 13:39
  • 1
    \$\begingroup\$ Because of the dupeMap.Remove(activityId); call your queue implementation has a race condition. If the processing of activityId#1 completed sooner than the second call of MarkLeaderboardDirty with activityId#1 then you process it twice. If the second call happened sooner than the processing then it won't be re-processed. \$\endgroup\$ Commented Oct 10, 2022 at 13:50
  • 1
    \$\begingroup\$ Okay. Otherwise it seems okay for me. \$\endgroup\$ Commented Oct 10, 2022 at 18:34

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.