I often query a database to get a batch of items to procees. I do this as long as the query returns some items. I use this pattern quite a lot so I thought I create a small helper so that I don't have to implement this logic again and again.

It's a small class that executes the query until there is nothing more left:

public static class Unfold
{
    public static async Task ForEachAsync<T>
    (
        Func<CancellationToken, Task<IList<T>>> query, 
        Func<IList<T>, CancellationToken, Task> body, 
        CancellationToken cancellationToken
    )
    {
        while (true)
        {
            var result = await query(cancellationToken);
            if (result.Any())
            {
                await body(result, cancellationToken);
            }
            else
            {
                break;
            }
        }
    }
}

The reason why I implemented it exactly this way is:

  • all my queries are async
  • they must always return IList<T> (if they return a collection of course)
  • I always process a batch at a time that I then mark as processed

Example

The typical use-case is like this:

  • get a batch of items from a repository
  • process this batch
  • repeat until the batch is empty
async Task Main()
{
    var numbers = new NumberRepository();
    await Unfold.ForEachAsync
    (
        query: async token => await numbers.GetNumbersAsync(token), 
        body: ProcessBatch, 
        CancellationToken.None
    );
}

A test repository:

public class NumberRepository
{
    private readonly IList<IList<int>> _numbers = new[] { new[] { 1, 2 }, new[] { 3, 4 }, new[] { 5 }, new int[0] };
    private int _batchIndex;

    public Task<IList<int>> GetNumbersAsync(CancellationToken cancellationToken) => Task.FromResult(_numbers[_batchIndex++]);
}

and the processing method:

private Task ProcessBatch<T>(T item, CancellationToken cancellationToken)
{
    item.Dump();
    return Task.CompletedTask;
}

What do you say? Is this a good or a bad solution? Is there anything missing (but null-checks)?

  • Oh, I see some has donvoted it... how so? – t3chb0t yesterday
up vote 3 down vote accepted

Sorry, but you definitely need a null check here:

if (result.Any())

Else there is not much to comment.

About the usage:

I don't understand, why you create a lambda for the query argument:

 async Task Main()
{
    var numbers = new NumberRepository();
    await Unfold.ForEachAsync
    (
        query: async token => await numbers.GetNumbersAsync(token), 
        body: ProcessBatch, 
        CancellationToken.None
    );
}

Why not just:

async Task Main()
{
  var numbers = new NumberRepository();
  await Unfold.ForEachAsync
  (
      query: numbers.GetNumbersAsync,
      body: ProcessBatch,
      CancellationToken.None
  );
}

numbers.GetNumbersAsync is awaitable already?

  • 1
    Right, and even to avoid the null check at all I definitely need to put the [NotNull] attribute there as I always guaratee that collections are never null; Unfortunatelly it does not work well with Task<T> :-( In this example using a lambda is not necessary, true, but in my application I have some EF queries there so I wanted to simulate it as real as possible ;-) – t3chb0t Dec 6 at 12:36
  • 1
    My first try was with observables but I failed to generate them ;-] – t3chb0t Dec 6 at 12:38
  • @t3chb0t: Ah, yes, I remember those things :-). But this one is simple and easy to use. – Henrik Hansen Dec 6 at 12:44
  • 1
    Oh, cool, I didn't know that but one can use the [ItemNotNull] on Task<T>, see - if you're using them too. – t3chb0t Dec 6 at 12:47

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.