The span for output is somewhat inflexible:
- it requires the caller to anticipate the number of results, and
- it requires contiguous storage, restricting the implementation.
It's probably better to make the function return an Input Range, enabling more flexible calling (e.g. I can then create a linked list of the results without overhead by using std::ranges::copy(tokenize(…), std::back_inserter(my_list));, or I could use a View to filter the results).
An alternative that's more like the current approach would be to accept an Output Range - possibly an infinite range such as a stream output range or a collection inserter.
Without that change, I think it would be more useful to operate like std::snprintf() by returning the number of tokens that would have been produced if the output range was large enough. That allows callers to call again with a resized buffer if they need to.
I prefer the version with the range-based for, but that's probably irrelevant given that changes to the interface could well make that look completely different.
A minor observation: this assignment is pointless, since found_tokens_count is going out of scope anyway:
return found_tokens_count = std::numeric_limits<size_t>::max( );
That can be replaced with simply
return std::numeric_limits<size_t>::max();