Paginate IEnumerable<T> sequence

Question

I used the following to split sequence into batches:

source
    .Select((item, index) => new {Item = item, Index = index})
    .ToLookup(x => x.Index / batchSize, x => x.Item);

but it requires full materialization of the sequence - memory complexity could be high.

Are there anything ready to be used? I think there is nothing in MoreLinq for that.

Here is a custom solution I am trying now:

public static class Batch
{
    public static IEnumerable<T[]> Split<T>(this IEnumerable<T> source, int batchSize)
    {
        var it = source.GetEnumerator();
        for (var b = split(); b.Length > 0; b = split())
            yield return b;

        T[] split() => Enumerable
            .Range(0, batchSize)
            .TakeWhile(i => it.MoveNext())
            .Select(i => it.Current)
            .ToArray();
    }
}

Answer 1

Since you mentioned morelinq in your question. It does have the Batch method which is similar. It's an IEnumerable<IEnumerable<TSource>> instead of IEnumerable<TSource[]> but if you look at the source code it's actually an IEnumerable of an array but you can't count on that as it's an implementation detail.

The only issue I see with your implementation is that source.GetEnumerator() returns back IEnumerator<T> which implements IDisposable and should be wrapped in a using statement.