# Splitting a string into words or double-quoted substrings

For various reasons, I'm parsing a string, this code will explain what I'm after:

string baseString = "This is a \"Very Long Test\"";

string[] strings = baseString.Split(' ');

List<String> stringList = new List<string>();
string temp = String.Empty;
foreach (var s in strings)
{
if (!String.IsNullOrWhiteSpace(temp))
{
if (s.EndsWith("\""))
{
string item = temp + " " + s;
temp = string.Empty;
}
temp = temp + " " + s;
}
else if (s.StartsWith("\""))
{
temp = s;
}
else
{
}

}

stringList.ForEach(Console.WriteLine);


The output should be:

This
is
a
Very Long Test


Basically, given a string, it will split it on spaces, unless its grouped into speech marks, the same way the command line does it.

Any better way to do this code?

• It's usually a bad idea to concatenate strings in a loop. If performance matters to you, you should probably use StringBuilder instead. – svick Apr 12 '12 at 19:06
• What about something like "This is a \"Very Long\" Test\"" ? Is there a guarantee that the \" 's will come in pairs? other wise the problem is too ill defined for a proper solution – Arjang Apr 13 '12 at 4:37

Seems like a job for a regular expression:

string baseString = "This is a \"Very Long Test\"";
var re = new Regex("(?<=\")[^\"]*(?=\")|[^\" ]+");
var strings = re.Matches(baseString).Cast<Match>().Select(m => m.Value).ToArray();


What the regular expression (?<=")[^"]*(?=")|[^" ]+ does is that it either finds a sequence of zero or more characters that are not " ([^"]*) preceded by a " ((?<=")) and followed by a " ((?=")) or a sequence of one or more character that are not " or a space ([^" ]+).

For the sample input, it gives the same output as your version. The code itself is much simpler, but the regular expression might be hard to understand, especially if you're not used to them.

• A header comment could easily resolve the hard to understand part? Other than that I don't think it's any harder to understand than the original Q. – dreza Apr 12 '12 at 19:42
• Looks cool. Could you elaborate on why you need (?<=) and (?=") as opposed to just (")[^"](")? – Leonid Apr 13 '12 at 2:37
• @Leonid, the difference is that if you did it your way, the quotation marks would be part of the match, so you would need to remove them later, for example by using Select(m => m.Value.Trim('"')). – svick Apr 13 '12 at 6:21
    string baseString = "This is a \"Very Long Test\"... not so long actually, eh?";
string[] aux = baseString.Split('"');
List<string> tokens = new List<string>();
for (int i = 0; i < aux.Length; ++i)
if (i % 2 == 0)
else


Notice that if there is a double quote in the middle of a word it will be split (a"ctuall"y would be a, ctuall, y in the final result). If the last double quote is unmatch it won't split from its position to the end of the string.

The usual approach to this kind of tokenising is to count the number of markers (it shares the same idea as hit scanning a polygon).

Scan the string one character at a time, checking if the character is a space or a quote mark, or any other character. If it's quote mark, toggle your quote flag indicating you are within a delimited string and output the current scanned token if transitioning from 'in delimited string' to 'not in delimited string'. If it's a space and you're 'not in a delimited string', output the currently scanned token. All other characters get added to the currently scanned token.

After the final character has been processed, if output the currently scanned token if there are any remaining characters that have been scanned but not output.

Untested, but the general approach:

StringBuilder currentToken = new StringBuilder();
bool inDelimitedString = false;
List<string> scannedTokens = new List<string>();
foreach (char c in source)
{
switch(c)
{
case '"':
if (inDelimitedString)
{
if (currentToken.Length > 0)
{
currentToken.Clear();
}
}
inDelimitedString = !inDelimitedString;
break;
case ' ':
if (!inDelimitedString)
{
if (currentToken.Length > 0)
{
currentToken.Clear();
}
}
else
{
currentToken.Append(c);
}
break;
default:
currentToken.Append(c);
break;
}
}
if (currentToken.Length > 0)
{