Split large strings into parts by last index of character and by max length

Question

This is my method to split strings into parts by last index of character and by max length, the rationale for this method is because I have to send, in a payload via POST, a list of strings that cannot have more than 40 characters each, suffice to say that I can't cut words in half otherwise it would be much easier.

public IEnumerable<string> SplitStringByLastIndexOfAndMaxLength(string stringToSplit, int splitMaxLength, char delimiter)
{           
    while (!string.IsNullOrEmpty(stringToSplit))
    {
        if (stringToSplit.Length <= splitMaxLength)
        {
            yield return stringToSplit;
            break;
        }
        var splitIndex = stringToSplit.LastIndexOf(delimiter, splitMaxLength);

        var removeSpace = 1;

        if (splitIndex < 0)
        {
            // if a word is larger than splitMaxLength we still split it
            splitIndex = splitMaxLength;
            // taking care to not remove a non space character
            removeSpace = 0;
        }
        var partialString = stringToSplit[..splitIndex];
        stringToSplit = stringToSplit[(splitIndex + removeSpace)..];

        yield return partialString;
    }
}

Usage, example with 40 characters max and split by space:

var address = "...Are included in two of the substrings. " +
              "If you want to exclude the characters, you can " +
              "add the periods... ...Are included in of the substrings. " +
              "If you want to exclude the period characters, you can " +
              "add the period... Some more text";

var addresses = SplitStringByLastIndexOfAndMaxLength(address, 40, ' ');

foreach (var address in addresses)
{
    Console.WriteLine(address + " -> " + address.Length);
}

Output:

...Are included in two of the -> 29
substrings. If you want to exclude the -> 38
characters, you can add the periods... -> 38
...Are included in of the substrings. If -> 40
you want to exclude the period -> 30
characters, you can add the period... -> 37
Some more text -> 14

Can you think of improvements to be made?

(This could be made into an extension method but for the time being it's enough.)

Answer 1

since your method is for requests payloads, you need to use memory-friendly approaches. I don't know how often the method would be called, and how much requests at average your application would handle in daily bases. In all cases, it would be a good decision to assume the worst case scenarios, so handling them from the start would prevent from future high memory allocations.

Also, I encourage you to use VS profiler or you could use BenchmarkDotNet in which would help give you a first-glance overview on your code diagnostic measurements. Use test samples to mimic the actual requests payloads, and also try to give at least three real test scenarios, (one with least , one with average, and one with high API consumptions) just to diagnose and measure the overall performance (with memory allocation) of this addition.

With this, you would know which approach would be better for your case.

This is much better in logic and memory handling. The only missing part is a handling for splitMaxLength where it might be <= 0 or >= stringToSplit.Length.

For readability part, SplitStringByLastIndexOfAndMaxLength I think if we use SplitByLastIndexOf it would be enough, since the method arguments would tell what type of input it's taking, and what it does returns. The only part is hidden, is the use of LastIndexOf.

I have modified your version to improve readability and code follow-up. and added some comments to explain the changes that I've made.

public static IEnumerable<string> SplitByLastIndexOf(string source, int blockSize, char delimiter)
{
    if (string.IsNullOrWhiteSpace(source)) yield break;

    // in case of invalid block size.
    if (blockSize <= 0 || blockSize >= source.Length)
    {
        blockSize = source.Length;
    }
    
    // reuse it rather than initializing it on each iteration
    var removeSpace = 1;
   
    // reuse it rather than initializing it on each iteration
    var delimiterIndex = -1;
    
    while (source.Length > blockSize)
    {
        delimiterIndex = source.LastIndexOf(delimiter, blockSize);
        
        if(delimiterIndex == -1)
        {
            // if a word is larger than blockSize we still split it
            delimiterIndex = blockSize;
            // taking care to not remove a non space character
            removeSpace = 0;
        }

        yield return source[..delimiterIndex];

        source = source[(delimiterIndex + removeSpace)..];

        // reset
        delimiterIndex = -1;
        removeSpace = 1;
    }

    // return any remaining text. 
    if (source.Length > 0)
    {
        yield return source;
    }
}