UDP Server Design and Performance

Question

What I'm doing

I am writing a server to work as my games backend. The server is being written in C# using UDP as the protocol, while the game is in C++. The biggest thing I've been focusing on was trying to make the packet messages more OOP than what I've typically seen for game servers. As a result of this, I wonder if performance will suffer due to allocations of the packet message (IDatagram interface represents the UDP datagram).

The environment

Running a client app on a MacBook Pro with an i7 3.8ghz quad-core on OS X with 16gm of ram, I see the CPU max out. I'll show the code below for the client, and I'm sure there's a better way of trying to slam my server. I just threw it together today to see what initial numbers looked like (am I getting 5 round-trips a second or 1,000?). If I run the code on the same machine as the server using a loopback, I hit 100,000 round-trips per second. I expected that to be fast, but I wasn't expecting such a massive drop off when moving it off the surface (below) and running it across the network. I'm seeing 3,634 round-trips a second with the maxed CPU.

The server is running on a Surface Pro 4 with an i7 dual-core on Windows 10 with 8gb of ram. When I run the server it uses roughly 15% of my CPU and very little ram. With this setup, between the two machines, I'm averaging 3,634 round-trips from client, to server, back to client. This has me pretty confident considering I'm hardly using the CPU on the Surface and it's only a dual-core. I do however see a GC happen every 10 seconds roughly, which has me a little concerned that I could do something better with my allocations.

This shows the GC frequency. I know that the diagnostic tools in VS are not accurate representations of the server running in a production environment with a Release build, but I use it to determine worse-case scenario with performance.

The Design

I have a general datagram interface that represents all datagrams, as IDatagram. The client and server then have interfaces that inherit from IDatagram to expose client and server specific properties and methods.

IDatagram

public interface IDatagram
{
    void Serialize(BinaryWriter serializer);
    void Deserialize(BinaryReader deserializer);
    bool IsMessageValid();
}

IClientDatagram

public interface IClientDatagram : IDatagram
{   
    IClientHeader Header { get; set; }
}

IServerDatagram (stubbed for now, I've not fleshed this out yet).

public interface IServerDatagram : IDatagram
{
    // IServerHeader { get; set; }
}

All three interfaces have a header that gets serialized into the datagram for sending between the server (not ready yet) and the client. The IClientDatagram is used to deserialize datagrams sent from the game client to the server, while the IServerDatagram is used to serialize messages into datagram packets for sending to the client. Both interfaces can perform the opposite operation so that I can write utilities that can be used to simulate the client (or server) sending a datagram during testing or troubleshooting.

I have two interfaces that represent headers right now, one is a base-head interface that will be shared by both an IClientHeader and a IServerHeader. The server header isn't written yet, but doesn't impact the server running right now as the testing doesn't depend on the header existing at the moment.

public interface IDatagramHeader : IDatagram
{
    long TransmissionTime { get; }
    byte SequenceNumber { get; }
    bool IsLastInSequence { get; }
    byte Channel { get; }
    DatagramPolicy Policy { get; }
}

public interface IClientHeader : IDatagramHeader
{
    byte AppVersion { get; }
    byte MessageType { get; }
    byte OSPlatform { get; }
    byte OSVersion { get; }
    int ClientId { get; }
}

The header has a property of type DatagramPolicy which is a bitmask I use to determine what requirements the datagram has that the server needs to apply. This tells the server the following things:

• Is the message sequenced? More than one packet will be sent to build a composite message if so.
• Does the server have to send an ACK back to the client?
• Does the server have to wait to start processing until every packet in the sequence has arrived.

The enum is represented like this:

[Flags]
public enum DatagramPolicy : byte
{
    None = 0,
    SequencedMessage = 1,
    AcknowledgementRequired = 2,
    CompletedSequenceRequired = 4,
}

Lastly, to round out the datagram API, I have an attribute that is used to specify what version of my datagram protocol each IDatagram implementation supports, a byte number (might be changed to an int later) that represents the implementation itself and a channel that the datagram belongs to.

[AttributeUsage(validOn: AttributeTargets.Class, Inherited = false)]
public class ProtocolVersionAttribute : Attribute
{
    public ProtocolVersionAttribute(int targetVersion, byte datagramName, DatagramChannels category)
    {
        this.TargetProtocolVersion = targetVersion;
        this.Category = category;
        this.DatagramTypeMap = datagramName;
    }

    public int TargetProtocolVersion { get; }

    public byte DatagramTypeMap { get; }

    public DatagramChannels Category { get; }
}

I use a byte to identify the IDatagram implementation the packet belongs to reduce the number of bytes being sent. Originally I was sending the full Type name, but was able to reduce the number of bytes massively (relative) by just sending a single byte that could be mapped back to the appropriate type server-side and client-side.

Next are a couple implementations. The first being the client header implementation and the second being an implementation if IServerDatagram that is used to send an ACK back to the client.

ClientHeader

[ProtocolVersion(1, DatagramNames.Header, DatagramChannels.Account)]
public class ClientHeader : IClientHeader
{
    public ClientHeader()
    {
    }

    public ClientHeader(
        byte appVersion,
        byte messageType,
        byte osPlatform,
        byte osVersion,
        DateTime transmission,
        DatagramPolicy policy,
        byte sequenceNumber)
    {
        this.AppVersion = appVersion;
        this.MessageType = messageType;
        this.OSPlatform = osPlatform;
        this.OSVersion = osVersion;
        this.TransmissionTime = transmission.Ticks;
        this.Policy = policy;
        this.SequenceNumber = sequenceNumber;
        this.TransmissionTime = transmission.Ticks;
    }

    public int ClientId { get; set; }

    public byte AppVersion { get; private set; }

    public bool IsLastInSequence { get; private set; }

    public byte MessageType { get; private set; }

    public byte OSPlatform { get; private set; }

    public byte OSVersion { get; private set; }

    public long TransmissionTime { get; private set; }

    public byte Channel { get; private set; }

    public DatagramPolicy Policy { get; private set; }

    public byte SequenceNumber { get; private set; }

    public void Serialize(BinaryWriter serializer)
    {
        serializer.Write(this.AppVersion);
        serializer.Write(this.MessageType);
        serializer.Write(this.IsLastInSequence);
        serializer.Write(this.OSPlatform);
        serializer.Write(this.OSVersion);
        serializer.Write(this.TransmissionTime);
        serializer.Write(this.Channel);
        serializer.Write((byte)this.Policy);

        if (this.Policy.HasFlag(DatagramPolicy.SequencedMessage))
        {
            serializer.Write(this.SequenceNumber);
        }
    }

    public void Deserialize(BinaryReader deserializer)
    {
        this.AppVersion = deserializer.ReadByte();
        this.MessageType = deserializer.ReadByte();
        this.IsLastInSequence = deserializer.ReadBoolean();
        this.OSPlatform = deserializer.ReadByte();
        this.OSVersion = deserializer.ReadByte();
        this.TransmissionTime = deserializer.ReadInt64();
        this.Channel = deserializer.ReadByte();
        this.Policy = (DatagramPolicy)deserializer.ReadByte();

        if (this.Policy.HasFlag(DatagramPolicy.SequencedMessage))
        {
            this.SequenceNumber = deserializer.ReadByte();
        }
    }

    public bool IsMessageValid()
    {
        return true;
    }
}

Acknowledge

[ProtocolVersion(1, DatagramNames.Acknowledge, DatagramChannels.Account)]
public class Acknowledge : IServerDatagram
{
    public int MessageIdAcknowledged { get; set; }

    public void Serialize(System.IO.BinaryWriter serializer)
    {
        serializer.Write(this.MessageIdAcknowledged);
    }

    public void Deserialize(System.IO.BinaryReader deserializer)
    {
        this.MessageIdAcknowledged = deserializer.ReadInt32();
    }

    public bool IsMessageValid()
    {
        return this.MessageIdAcknowledged > 0;
    }
}

The following enum and struct is used to prevent "magic numbers" being used in my attribute usage on the datagram implementations.

public enum DatagramChannels
{
    Account = 0,
    Singleplayer = 1,
    GroupPlay = 2,
    Multiplayer = 3,
    Chat = 4,
    RouterManager = 5,
}

public struct DatagramNames
{
    public const byte Header = 0;
    public const byte Ping = 1;
    public const byte Acknowledge = 2;
}

The next last thing to show, prior to the server, is how I resolve the datagrams as they come in from the socket. I provide a factory with every IDatagram implementation available during the startup of the Console app. The factory then looks up what Type matches the byte value given by the header.

DatagramFactory

public sealed class DatagramFactory
{
    private static readonly Dictionary<byte, Type> datagrams = new Dictionary<byte, Type>();

    public DatagramFactory()
    {
        Type datagramInterfaceType = typeof(IDatagram);
        IEnumerable<Type> datagramTypes = AppDomain.CurrentDomain
            .GetAssemblies()
            .SelectMany(assembly => assembly.GetTypes().Where(type => datagramInterfaceType.IsAssignableFrom(type)));

        this.cacheDatagramTypes(datagramTypes);
    }

    public DatagramFactory(IEnumerable<Type> datagramTypes)
    {
        this.cacheDatagramTypes(datagramTypes);
    }

    private void cacheDatagramTypes(IEnumerable<Type> datagramTypes)
    {
        foreach (Type datagram in datagramTypes)
        {
            if (datagram.IsAbstract || !datagram.IsClass)
            {
                continue;
            }

            // Expensive the first time, but this is only ever done once.
            ProtocolVersionAttribute protocolVersion = datagram.GetCustomAttribute<ProtocolVersionAttribute>();
            if (protocolVersion == null)
            {
                continue;
            }
            DatagramFactory.datagrams.Add(protocolVersion.DatagramTypeMap, datagram);
        }
    }

    public IClientDatagram CreateDatagramFromClientHeader(IClientHeader header)
    {
        Type datagramType = null;
        if (!DatagramFactory.datagrams.TryGetValue(header.MessageType, out datagramType))
        {
            return null;
        }

        IClientDatagram datagram = (IClientDatagram)Activator.CreateInstance(datagramType);
        datagram.Header = header;

        return datagram;
    }
}

If I'm getting several thousand packets per second, I can totally see the allocation of these datagrams adding some pressure to the GC. After talking with a friend, I'm leaning towards converting all of the IDatagram implementations in to structs, since they're really immutable anyway. That would help my Garbage Collection issues.

UdpServer

This is the server object itself that is responsible for listening to incoming packets and sending packets.

public class UdpServer
{
    private readonly DatagramFactory datagramFactory;

    private Socket udpServerSocket;
    private IPEndPoint serverEndPoint;
    private bool isRunning;

    public UdpServer(DatagramFactory datagramFactory)
    {
        this.datagramFactory = datagramFactory;
        this.ServerPort = 11000;
        this.PacketBufferSize = 256;
    }

    public int ServerPort { get; set; }

    public int PacketBufferSize { get; set; }

    public bool IsEventMessagingEnabled { get; set; }

    public ServerSpecification ServerPolicy { get; set; }

    public int ClientTimeoutPeriod { get; set; }

    public void Start()
    {
        // Setup our socket for use
        this.udpServerSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
        this.serverEndPoint = new IPEndPoint(IPAddress.Any, ServerPort);

        // Bind and configure the socket so we are always given the client end point packet information, such as their IP, when data is received.
        this.udpServerSocket.Bind(serverEndPoint);
        this.isRunning = true;
        this.ListenForData(this.udpServerSocket);
    }

    public bool IsRunning()
    {
        return this.isRunning;
    }

    public void Shutdown()
    {
        this.isRunning = false;

        this.udpServerSocket.Shutdown(SocketShutdown.Both);
        this.udpServerSocket.Dispose();
    }

    public void SendMessage(IServerDatagram message, EndPoint destination)
    {
        if (!this.isRunning)
        {
            return;
        }

        if (message == null)
        {
            throw new ArgumentNullException();
        }
        if (!message.IsMessageValid())
        {
            // TODO: Determine how to handle invalid messages. Exception throwing would be bad, we don't want to crash the server.
        }

        var memoryStream = new MemoryStream();
        byte[] data = null;
        using (var binaryWriter = new BinaryWriter(memoryStream))
        {
            message.Serialize(binaryWriter);

            // Fetch the serialized bytes from the stream
            data = memoryStream.GetBuffer();

            // Send the datagram packet.
            this.udpServerSocket.SendTo(data, destination);
        }
    }

    private void ListenForData(Socket socket)
    {
        if (!this.isRunning)
        {
            return;
        }

        // The BeginReceiveFrom requires us to give it an endpoint, even though we don't use it.
        var state = new PacketState(socket, PacketBufferSize) { Destination = (EndPoint)new IPEndPoint(IPAddress.Any, ServerPort) };
        byte[] buffer = state.Buffer;
        EndPoint destination = state.Destination;

        socket.BeginReceiveFrom(
            state.Buffer,
            0,
            PacketBufferSize,
            SocketFlags.None,
            ref destination,
            new AsyncCallback(this.ReceiveClientData),
            state);
    }

    private void ReceiveClientData(IAsyncResult result)
    {
        PacketState state = (PacketState)result.AsyncState;
        Socket socket = state.UdpSocket;
        EndPoint endPoint = state.Destination;

        int receivedData = socket.EndReceiveFrom(result, ref endPoint);
        if (receivedData == 0)
        {
            this.ListenForData(socket);
            return;
        }

        // Create a binary reader so we can deserialize the bytes delivered into an IMessage implementation
        using (var reader = new BinaryReader(new MemoryStream(state.Buffer)))
        {
            reader.BaseStream.Seek(0, SeekOrigin.Begin);
            // Read the header in from the buffer first so we know what kind of message and how to route.
            IClientHeader header = new ClientHeader();
            header.Deserialize(reader);
            if (!header.IsMessageValid())
            {
                throw new InvalidDataException("The header being returned was malformed.");
            }

            // Acknowledge that we received the packet if it is required.
            if (header.Policy.HasFlag(DatagramPolicy.AcknowledgementRequired) || 
                this.ServerPolicy.HasFlag(ServerSpecification.RequireAcknowledgement))
            {
                this.SendMessage(new Acknowledge(), endPoint);
            }

            IClientDatagram datagram = this.datagramFactory.CreateDatagramFromClientHeader(header);
            if (datagram == null)
            {
                // TODO: handle null
            }

            datagram.Deserialize(reader);
        }

        this.ListenForData(socket);
    }
}

The server uses this PacketState object to pass around the socket async state during the Begin/End states. It also has a bit-flag enum for letting it have policies set.

[Flags]
public enum ServerSpecification
{
    None = 0,
    RequireAcknowledgement = 1,

}

internal class PacketState
{
    internal PacketState(Socket socket, int bufferSize)
    {
        this.UdpSocket = socket;
        this.Buffer = new byte[bufferSize];
        this.CreatedTime = DateTime.Now;
    }

    public byte[] Buffer { get; set; }

    public EndPoint Destination { get; set; }

    public Socket UdpSocket { get; }

    public bool IsComplete { get; }

    public DateTime CreatedTime { get; }
}

Finally, to start the server I have a Console app that creates the instance and starts it.

    public static void Main(string[] args)
    {
        Console.WriteLine("Starting server");
        Run().Wait();
    }

    private static async Task Run()
    {
        var server = new UdpServer(new DatagramFactory());
        server.ServerPolicy = ServerSpecification.RequireAcknowledgement;
        server.Start();

        ////Using UDP sockets
        var localHostEntry = Dns.GetHostByName("10.0.1.3");
        IPEndPoint ipEndPoint = new IPEndPoint(localHostEntry.AddressList[0], 80);
        Console.WriteLine($"Starting on {ipEndPoint.Address}");

        while (server.IsRunning())
        {
            await Task.Delay(1);
        }
    }

Does anyone see anything that stands out in regards to performance or bad design choices with my UDP setup? Like I said earlier in the post, I think I can make up some performance with converting my IDatagram implementations into structs or using an object pool. I've also got a post on re-using Streams to see if I can save allocations by re-using the same MemoryStream and BinaryRead/BinaryWriter for every packet.

Client

This code is gross, and not meant to be part of the review itself. This is just to provide everyone with a client setup if they want to watch the data movement in and out of the server.

using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Net; using System.Net.Sockets; using iMini.Server.Datagrams; using iMini.Server.Datagrams.Client; using System.Diagnostics;

namespace ConsoleApplication1
{
    internal class PacketState
    {
        internal PacketState(Socket socket, int bufferSize)
        {
            this.UdpSocket = socket;
            this.Buffer = new byte[bufferSize];
            TimeStamp = DateTime.Now;
        }

        public byte[] Buffer { get; set; }

        public EndPoint Destination { get; set; }

        public Socket UdpSocket { get; }

        public DateTime TimeStamp { get; set; }
    }

    class Program
    {
        static byte[] ipAddress = new byte[] { 10, 0, 1, 13 };
        private static Stopwatch watch = new Stopwatch();
        private static int requestsPerSecond = 0;
        public static List<double> travelTime = new List<double>();

        public static void Main(string[] args)
        {
            Console.WriteLine("Starting server");

            Run();
        }

        private static void Run()
        {
            //Using UDP sockets
            var clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
            IPEndPoint ipEndPoint = new IPEndPoint(IPAddress.Any, 11000);
            var localEndPoint = (EndPoint)ipEndPoint;

            var header = new ClientHeader(
                            1,
                            4,
                            2,
                            11,
                            DateTime.Now,
                            (DatagramPolicy.SequencedMessage | DatagramPolicy.AcknowledgementRequired),
                            1);
            var response = new PingResponse { Header = header };

            watch.Start();

            clientSocket.Bind(localEndPoint);
            var state = new PacketState(clientSocket, 256) 
            { 
                Destination = new IPEndPoint(new IPAddress(ipAddress), 11000)
            };
            EndPoint remoteEndPoint = state.Destination;
            clientSocket.BeginReceiveFrom(
                state.Buffer,
                0,
                state.Buffer.Length,
                SocketFlags.None,
                ref remoteEndPoint,
                new AsyncCallback(ReceivedData),
                state);

            while (true)
            {
                var writer = new BinaryWriter(new MemoryStream());
                header.Serialize(writer);
                response.Serialize(writer);

                // Fetch the serialized bytes from the stream
                Stream stream = writer.BaseStream;
                writer.Seek(0, SeekOrigin.Begin);
                byte[] data = new byte[stream.Length];
                stream.Read(data, 0, (int)stream.Length);
                clientSocket.BeginSendTo(
                    data,
                    0,
                    data.Length,
                    SocketFlags.None,
                    remoteEndPoint,
                    new AsyncCallback(OnSend), clientSocket);
            }
        }

        private static void ReceivedData(IAsyncResult result)
        {
            PacketState state = (PacketState)result.AsyncState;
            Socket socket = state.UdpSocket;
            EndPoint remoteEndPoint = state.Destination;

            int receivedData = socket.EndReceiveFrom(result, ref remoteEndPoint);
            if (receivedData == 0)
            {
                var newState = new PacketState(socket, 256) 
                { 
                    Destination = new IPEndPoint(new IPAddress(ipAddress), 11000)
                };

                socket.BeginReceiveFrom(
                    newState.Buffer,
                    0,
                    newState.Buffer.Length,
                    SocketFlags.None,
                    ref remoteEndPoint,
                    new AsyncCallback(ReceivedData),
                    newState);
                return;
            }

            requestsPerSecond++;
            double time = DateTime.Now.Subtract(state.TimeStamp).TotalMilliseconds;
            travelTime.Add(time);
            if (watch.ElapsedMilliseconds > 1000)
            {
                Console.WriteLine(requestsPerSecond);
                Console.WriteLine($"Average round-trip {travelTime.Average()}ms");
                requestsPerSecond = 0;
                watch.Reset();
                watch.Start();
            }

            // Create a binary reader so we can deserialize the bytes delivered into an IMessage implementation
            using (var reader = new BinaryReader(new MemoryStream(state.Buffer)))
            {
                reader.BaseStream.Seek(0, SeekOrigin.Begin);
                // Read the header in from the buffer first so we know what kind of message and how to route.
                IClientHeader header = new ClientHeader();
                header.Deserialize(reader);
                if (!header.IsMessageValid())
                {
                    throw new InvalidDataException("The header being returned was malformed.");
                }
            }

            var s = new PacketState(socket, 256) 
            { 
                Destination = new IPEndPoint(new IPAddress(ipAddress), 11000)
            };

            socket.BeginReceiveFrom(
                s.Buffer,
                0,
                s.Buffer.Length,
                SocketFlags.None,
                ref remoteEndPoint,
                new AsyncCallback(ReceivedData),
                s);
        }

        private static void OnSend(IAsyncResult result)
        {
            Socket socket = (Socket)result.AsyncState;
            socket.EndSendTo(result);
        }
    }
}

Answer 1

1. First things first: you should check out C# naming guidelines and try to follow those.

2. I'm leaning towards converting all of the IDatagram implementations in to structs

   Don't. You should read this page, which explains when you should use struct and when you shouldn't. The relevant part is:

   AVOID defining a struct unless the type has all of the following characteristics:

   • It logically represents a single value, similar to primitive types (int, double, etc.).
   • It has an instance size under 16 bytes.
   • It is immutable.
   • It will not have to be boxed frequently.

   Your datagrams violate rules 1,2 and 3. If you are certain that garbage collection is going to become a bottleneck, you should implement obejct pools for your datagrams. Same goes for byte arrays and memory streams you are going to use: you gonna need a byte array pool. Also as you player base grows, you won't be able to handle all the connections on the single server due to performance and bandwidth issues. Sooner or later you will have to scale your system, so it uses multiple servers. Keep this in mind, when makig design decisions.

3. I'm not sure I understand why do you need different datagram interfaces for server and client. It looks like an overkill to me. The header should already identify where the datagram comes from. By using different interfaces and different baseclasses, you will have troubles implementing datagrams which could be sent by both server and client.

4. bool IsMessageValid(); method should be bool IsValid { get; } property.

5. DatagramNames implies, that this class contains a bunch of names, i.e. strings. But it is not the case. Maybe you should rename it to DatagramCodes or something.

6. DatagramFactory looks like an attemt to reinvent a DI container. :) You should consider using one of the existing frameworks, such as Casle.Windsor, Ninject, etc. It will allow you to register and resolve datagrams with a few lines of code.

7. Using attributes to store things such as protocol version can lead to troubles. Only use it if you are absolutely sure, that all your clients are always going to use up-to-date software and you are not going to need your server to be compatible with older versions of clients. Protocols tend to change, and it is usully a good idea to send a protocol version with a datagram. This extra byte can solve a lot of problems in a long run.

8. while (server.IsRunning())
   {
       await Task.Delay(1);
   }
   

   don't use Task.Delay or Thread.Sleep in a loop with 1ms delay. This loop does no meaningful work whatsoever yet it can easily consume up to one CPU core. You should figure out a way to either put your main thread to work or put it to sleep until you explicitly signal it to wake up (by using WaitHandle, for example).

9. In you server code you use memoryStream.GetBuffer(). This method returns the entire inner array of memory stream, which is larger, then the datagram size. This will lead to larger traffic. To avoid this you should also save the actual datagram size, and use it instead of buffer.Length. Also, if you use default MemoryStream constructor, it will dynamically change the inner array size hitting both performance and memory usage. Ideally you want to use MemoryStream(byte[]) instead.

10. I would recommend against using sockets directly. They use pretty obsolete callback-ish api which was the way to go in older versions of C#, but nowadays it is pretty much replaced by TPL and async programming. Instead you should consider using the highest level of abstraction that is available. In your case its going to be UdpClient class. Which has a much cleaner API and is way easier to use and debug. Check this answer out for simple example.