2
\$\begingroup\$

In our application we watch file changes using .NET FileSystemWatcher. After each file change we process the new content asynchronously. In order to make everything flexible and smooth, we used TPL BufferBlock<T>

After deployment and some real tests we realized, that watcher might be triggered too much times on the same file. And while we process only one file simultaneously, we anyway re-work the same content.

So, basically, we want the following:

  1. Have easy-to-use collection that works similarly to queue and provides a possibility to wait for new items without any while (true) loops.
  2. Thread-safety.
  3. Items inside should be unique.

Since we are not able to extend BufferBlock (it is sealed) and we don't have access to the full items collection inside BufferBlock, we ended with the following solution:

using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;

public class UniqueBufferBlock
{
    private readonly BufferBlock<string> _buffer = new BufferBlock<string>();
    private readonly HashSet<string> _hashSet = new HashSet<string>();
    private readonly object _locker = new object();

    public void Post(string value)
    {
        lock (_locker)
        {
            if (_hashSet.Add(value))
            {
                _buffer.Post(value);
            }
        }
    }

    public string Recieve()
    {
        lock (_locker)
        {
            var value = _buffer.Receive();
            _hashSet.Remove(value);
            return value;
        }
    }

    public Task<bool> OutputAvailable(CancellationToken cancelToken)
    {
        return _buffer.OutputAvailableAsync(cancelToken);
    }
}

Typical usage:

fileWatcher.FileChanged += (_, fileName) => uniqueBufferBlock.Post(fileName);

// ...

while (await uniqueBufferBlock.OutputAvailable(cancelToken))
{
    var fileName = uniqueBufferBlock.Receive();
    await ProcessFile(fileName, cancelToken);
}

What drawbacks could we face with it and what can be improved?

\$\endgroup\$

1 Answer 1

4
\$\begingroup\$

If you're aiming for KISS at the level of maintenance, this looks like a reasonable solution, although there are some minor details which IMO could be improved.

public class UniqueBufferBlock
{
    private readonly BufferBlock<string> _buffer = new BufferBlock<string>();
    private readonly HashSet<string> _hashSet = new HashSet<string>();

Why not genericise the class?

    public string Recieve()

Fix that spelling. Incorrectly spelt names are one of the banes of the maintenance programmer.

    public Task<bool> OutputAvailable(CancellationToken cancelToken)

Follow conventions: this should have a name ending Async, like the method it wraps.


There are a couple of other directions which might be worth considering. One is to go the full hog and implement IPropagatorBlock. The other is to slim down the dependencies by removing _buffer and implementing OutputAvailableAsync with TaskCompletionSource. Both would make the implementation of UniqueBufferBlock more fiddly, and it's reasonable to decide not to go in either direction.


PS on a point of English language pedantry: distinct is a better choice of word than unique. Perhaps deduplicated would be better still in this context.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.