Extracting complete lines from a data stream

Question

When reading from the Console, it's very easy to get a complete line, you simply call Console.Readline.

When reading from a TCPClient there isn't an equivalent function. You can either read a byte at a time, or read a block (up to a maximum size) in which case an arbitrary amount of data will be returned depending how the network behaved.

In order to simplify line extraction I've written a LineBuffer class. It has an Append method that allows new blocks of data to be added to the buffer. Whenever a complete line is received, the action supplied via the constructor is called.

The LineBuffer class:

using System;
using System.Text;

namespace MudCore.Connection
{
    public class LineBuffer
    {
        private readonly Action<string> _onLineFound;
        private readonly StringBuilder _currentLine;

        public LineBuffer(Action<string> onLineFound)
        {
            _onLineFound = onLineFound;
            _currentLine = new StringBuilder();
        }

        public void Append(string input)
        {
            if (input == null) return;

            while (input.Contains("\n"))
            {
                var indexOfNewLine = input.IndexOf('\n');
                var left = input.Substring(0, indexOfNewLine);
                _currentLine.Append(left);

                var line = _currentLine.Replace("\r","").ToString();

                _currentLine.Clear();
                if (indexOfNewLine != input.Length - 1)
                {
                    input = input.Substring(indexOfNewLine + 1);
                }
                else
                {
                    input = string.Empty;
                }

                _onLineFound.Invoke(line);
            }

            if (!string.IsNullOrEmpty(input))
            {
                _currentLine.Append(input);
            }
        }
    }
}

Some unit tests:

using System;
using System.Collections.Generic;
using System.Collections;

using NUnit.Framework;

using MudCore.Connection;
using MudCoreTests.Helpers;

namespace MudCoreTests.Connection
{
    [TestFixture]
    public class LineBufferTests
    {
        [Test]
        public void AppendingEmptyStringDoesNothing()
        {
            int callCount = 0;
            LineBuffer buffer = new LineBuffer((extractedLine) => { callCount++; });

            buffer.Append("");

            Assert.AreEqual(0, callCount);
        }

        [Test]
        public void AppendingNullStringDoesNothing()
        {
            int callCount = 0;
            LineBuffer buffer = new LineBuffer((extractedLine) => { callCount++; });

            buffer.Append(null);

            Assert.AreEqual(0, callCount);
        }

        [TestCase("\r\n")]
        [TestCase("\n")]
        public void SingleLineIsExtractedMinusEndOfLine(string endOfLine)
        {
            int callCount = 0;
            string foundLine = String.Empty;
            string lineToAppend = "This is a line";

            LineBuffer buffer = new LineBuffer((extractedLine) => { foundLine = extractedLine; callCount++; });

            buffer.Append(lineToAppend + endOfLine);

            Assert.AreEqual(1, callCount);
            Assert.AreEqual(lineToAppend, foundLine);
        }

        [TestCaseSource("ReceivedBufferTestCases")]
        public void MultipleLinesAreIdentifiedFromMultipleAppends(Queue<string> receivedData, Queue<string> expectedLines, string scenarioName)
        {
            var expectedCount = expectedLines.Count;
            var callCount = 0;

            LineBuffer buffer = new LineBuffer((extractedLine) => {
                var expectedLine = expectedLines.Dequeue();
                Assert.AreEqual(expectedLine, extractedLine, $"Expected: '{expectedLine}' but go '{extractedLine}' during scenario {scenarioName}");
                callCount++;
            });

            while (receivedData.Count > 0)
            {
                buffer.Append(receivedData.Dequeue());
            }

            Assert.AreEqual(expectedCount, callCount, $"Incorrect number of lines extracted, expected {expectedCount}, but was {callCount} during scenario {scenarioName}");
        }

        public static IEnumerable ReceivedBufferTestCases
        {
            get {
                yield return new TestCaseData(new Queue<string> { "One\n", "Two\n", "Three\n" },
                                              new Queue<string> { "One", "Two", "Three" },
                                              "Simple Complete Lines");
                yield return new TestCaseData(new Queue<string> { "One\r\n", "Two\r\n", "Three\r\n" },
                                              new Queue<string> { "One", "Two", "Three" },
                                              "Simple Complete Lines with \\r\\n");
                yield return new TestCaseData(new Queue<string> { "On", "e\n", "Two\n", "Three\n" },
                                              new Queue<string> { "One", "Two", "Three" },
                                              "Line split across two buffers");
                yield return new TestCaseData(new Queue<string> { "One\r", "\nT", "wo\n", "Three\n" },
                                              new Queue<string> { "One", "Two", "Three" },
                                              "Line split cr/lf across two buffers");
                yield return new TestCaseData(new Queue<string> { "One\r\nTwo\nThree\n" },
                                              new Queue<string> { "One", "Two", "Three" },
                                              "All data from one buffer");
            }
        }
    }
}

In the unit tests, I've made use of the collection initializer. Since Queue<T> doesn't support this, I've also created an extension method to make the tests easier to write.

using System.Collections.Generic;

namespace MudCoreTests.Helpers
{
    public static class QueueExtensions
    {
        static public void Add<T>(this Queue<T> q, T item)
        {
            q.Enqueue(item);
        }
    }
}

Any feedback is welcome. Is there any built-in functionality that does something similar that I haven't come across yet? Is the code readable? Is the extension method a bad idea?

If you need more context for where the class fits, the project is currently used here.

Answer 1

There are some alternatives that are built in, for example in the simplest form combine a NetworkStream with StreamReader:

using (var netStream = new NetworkStream(tcpClient.Client))
using (var reader = new StreamReader(netStream))
{
    var line = reader.ReadLine();
}

Which is unbuffered, if you want to add buffering in, just use a BufferedStream in the middle:

using (var netStream = new NetworkStream(tcpClient.Client))
using (var bufferStream = new BufferedStream(netStream))
using (var reader = new StreamReader(bufferStream))
{
    var line = reader.ReadLine();
}

These are pretty high performance because they operate at a lower level. Ideally you'd want to ditch the TcpClient and go direct with a Socket for best performance, but TcpClient.Client gives direct access to the underlying socket.