4
\$\begingroup\$

So I have a method that looks like this:

   public static List<List<List<string>>> Split(List<List<string>> source, int chunksize)
   {
       return source
           .Select((x, i) => new { Index = i, Value = x })
           .GroupBy(x => x.Index / chunksize)
           .Select(x => x.Select(v => v.Value).ToList())
           .ToList();
   }

While this is nice and elegant and readable, unfortunately that groupby is killing performance because of the implicit sort. I need a way to rewrite this to be more performant, while still being reasonably elegant.

\$\endgroup\$
6
\$\begingroup\$

We can make the method more generic:

public static IEnumerable<IEnumerable<T>> Split<T>(IEnumerable<T> source, int chunkSize)
{

Then we can store the current chunk in a List<T>. Using the constructor that takes the initial capacity should be good for performance:

    var chunk = new List<T>(chunkSize);
    foreach (var item in source)
    {
        chunk.Add(item);
        if (chunk.Count == chunkSize)
        {
            yield return chunk;
            chunk.Clear();
        }
    }

Finally we need to return the final (non-empty) chunk

    if (chunk.Count > 0)
    {
        yield return chunk;
    }
}

Depending on your use case, you might want to make a copy of the list and yield return chunk.ToList() instead.

|improve this answer|||||
\$\endgroup\$
  • \$\begingroup\$ This implementation has a bug. You're doing yield return chunk which returns a reference to the list and then you're clearing the list immediately after. Remember, you're returning a reference. \$\endgroup\$ – Vince Panuccio Dec 10 '14 at 2:57
  • \$\begingroup\$ @VincePanuccio it works fine if the chunks are processed as they are generated, e.g. var chunks = Split(new[] { 1, 2, 3, 4, 5 }, 2).Select(chunk => string.Join(",", chunk)); Console.WriteLine(string.Join("|", chunks)); prints 1,2|3,4|5 as expected. If that's not the case, then using ToList as I mentioned will solve the problem. \$\endgroup\$ – mjolka Dec 10 '14 at 3:09
  • 2
    \$\begingroup\$ The problem with that approach is the consumer needs to know that. Always returning a copy will avoid any possibility of a bug. \$\endgroup\$ – Vince Panuccio Dec 10 '14 at 11:34
  • \$\begingroup\$ I am not particularly good with Generic methods, how would I invoke this method when passing in a List<List<string>>? My attempts thus far compile, but give Null reference exceptions when run. \$\endgroup\$ – Alexander Ryan Baggett Dec 11 '14 at 0:20
  • \$\begingroup\$ @AlexanderRyanBaggett you don't need to do anything special to call it with a List<List<string>>. There's an example you can play with here, output is at the bottom of the page. \$\endgroup\$ – mjolka Dec 11 '14 at 0:28
1
\$\begingroup\$

From your question, it looks as though you have a list of lists which you need flattened and chunked.

If you have some time up your sleeve, it would be beneficial to make this method generic so it can handle any type and perhaps even turn it in to an extension method so it's available more easily.

void Main()
{
    var list1 = new List<string>(new[] { "1", "2" ,"3" });
    var list2 = new List<string>(new[] { "4", "5" ,"6" });
    var list3 = new List<string>(new[] { "7", "8" ,"9", "10" });

    List<List<string>> source = new List<List<string>>(new[] { list1, list2, list3 });

    Split(source, 3).Dump();
}

public static IEnumerable<IEnumerable<string>> Split(List<List<string>> source, int chunksize)
{
    var chunk = new List<string>(chunksize);

    foreach (var element in source.SelectMany (s => s))
    {
        if (chunksize == chunk.Count) {
            yield return chunk;
            chunk = new List<string>(chunksize);
        }
        chunk.Add(element);
    }

    yield return chunk;
}
|improve this answer|||||
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.