I have a tokenizer, where one of the token types happens to be a C# string type. It takes a TextReader as its input, which is created from a StringReader, and it needs to support the basic stuff C# strings supports, such as "jo dude", "joe \"dudes\"", "this is a backslash \\ token" and of course "this is carriage return \r\nthis is next line"
Currently my method looks like this;
private string ReadSingleLineStringLiteral ()
{
StringBuilder buffer = new StringBuilder ();
int nextChar = _reader.Read ();
while (nextChar != -1) {
buffer.Append ((char)nextChar);
if (nextChar == '"' &&
(buffer.Length == 1 ||
buffer [buffer.Length - 1] != '\\' ||
(buffer.Length - buffer.ToString ().TrimEnd ('\\').Length) % 2 == 0)) {
break;
}
nextChar = _reader.Read ();
}
if (buffer [buffer.Length - 1] != '"')
throw new ArgumentException ("unclosed string literal in hyperlisp file");
return buffer.ToString ().Substring (0, buffer.Length - 1)
.Replace ("\n", "\r\n") // normalizing carriage returns
.Replace ("\r\r\n", "\r\n")
.Replace ("\\\"", "\"")
.Replace ("\\\\", "\\");
}
It works I think, however, I suspect this is far from optimal, among other things, I hate that buffer.ToString ().TrimEnd ('\\').Length) % 2
part of it.
The _reader
is the TextReader
object, that at the entry of my methods are at the first character within the content of my string literal.
Can anyone figure out the optimal solution for parsing a C# type of string literal, and optimize my method?
Please don't tell me to use some library, since I don't want to bring in the overhead of any libraries to perform a task I think should be easily possible to implement with some 10-20 lines of code.
For bonus points, I'd love to also have a solution to my "multiline string literal algo" too. This method should read string the same way C# reads them when given @"some string". Currently it looks like this:
private string ReadMultiLineStringLiteral ()
{
StringBuilder buffer = new StringBuilder ();
int nextChar = _reader.Read ();
while (nextChar != -1) {
buffer.Append ((char)nextChar);
nextChar = _reader.Peek ();
if (nextChar != '"' && (buffer.Length - buffer.ToString ().TrimEnd ('"').Length) % 2 == 1)
break;
nextChar = _reader.Read ();
}
if (buffer.Length == 0 || buffer [buffer.Length - 1] != '"')
throw new ArgumentException ("unclosed multiline string literal in hyperlisp close to end of hyperlisp");
return buffer.ToString ().Substring (0, buffer.Length - 1)
.Replace (@"""""", @"""")
.Replace ("\n", "\r\n") // normalizing carriage returns
.Replace ("\r\r\n", "\r\n");
}
Both methods starts on the first character of the actual string content, e.g. for string @"xyz" the multiline method has the TextReader reader position being at "x", for string "abc" the singleline string literal method has the reader position being at "a".
This is probably unnecessary to say, but obviously Regex is a no-no.
"joe \"dudes\""
. It returns"joe \"
. Code is on ideone. \$\endgroup\$