8
\$\begingroup\$

Since asynchronous operations (like Socket's Begin*-End* pairs and *Async methods) that use IOCP under the hood cause the byte array that you use as buffer to be pinned in the memory.

So if you create a new byte array everytime you send to, or receive from a socket using IOCP, you're likely to encounter OutOfMemoryExceptions in your program.

One workaround for this is to pool the buffers, and a better one is to create one big byte array and use its parts as buffers. I first saw an implementation of this approach here and I wanted to wrote a simpler one to learn and use in my own projects:

public sealed class BufferManager : IDisposable
{
    // Fields
    private readonly byte[] _Block;
    private volatile bool _Disposed;
    private readonly BlockingCollection<int> _FreeSegments;
    private readonly int _SegmentSize;

    // Properties
    public byte[] Block { get { return _Block; } }
    public int BlockSize { get { return _Block.Length; } }
    public int BufferSize { get { return _SegmentSize; } }
    public int FreeBuffers { get { return _FreeSegments.Count; } }

    // Constructors
    public BufferManager(int blockSize, int bufferSize)
    {
        // These are some helper extensions that take minimum and maximum values.
        blockSize.ThrowIfOutOfRange(1, paramName: "blockSize");
        bufferSize.ThrowIfOutOfRange(1, blockSize, "bufferSize");

        // Setting the block size,
        // So all the bytes will be available according to buffer size.
        var mod = blockSize % bufferSize;
        if (mod != 0)
            blockSize = (blockSize - mod) + bufferSize;

        // Determining the first byte's index in each segment.
        var freeSegments = new int[blockSize / bufferSize];
        var freeSegment = 0;
        for (int i = 0; i < freeSegments.Length; i++)
        {
            freeSegments[i] = freeSegment;
            freeSegment += bufferSize;
        }

        // Initializing the block and a collection that holds indices to its segments.
        _Block = new byte[blockSize];
        _FreeSegments = new BlockingCollection<int>
        (
            // BlockingCollection uses ConcurrentQueue as default underlaying...
            // ...collection but since the order is not important, I thought it can...
            // ...be faster with a ConcurrentBag.
            new ConcurrentBag<int>(freeSegments),
            freeSegments.Length
        );
        _SegmentSize = bufferSize;
    }

    // Functions
    public int GetOffset()
    {
        // Gets the index of the first byte of a free byte block.
        // If there is none, it blocks the thread until there is.
        return _FreeSegments.Take();
    }
    public void FreeOffset(int offset)
    {
        // Frees the byte block which starts with offset.
        _FreeSegments.Add(offset);
    }

    public ArraySegment<byte> GetBuffer()
    {
        // Gets a free portion of the byte block.
        // If there is none, it blocks the thread until there is.
        return ArraySegment.From(Block, GetOffset(), BufferSize);

        // ArraySegment.From is an extension method to...
        // ...avoid explicitly specifying generic type parameters.
    }
    public void FreeBuffer(ArraySegment<byte> buffer)
    {
        // Frees the byte block which starts with buffer.Offset.
        FreeOffset(buffer.Offset);
    }

    public void Dispose()
    {
        lock (_FreeSegments)
            if (_Disposed) return;
            else _Disposed = true;

        _FreeSegments.Dispose();
    }
}

It can be used like this:

// Initializes a buffer manager with an underlying byte[120]
// I specify 100 here but constructor increases it so it can have equal 30-sized segments.
var bufferManager = new BufferManager(100, 30);

var buffer = bufferManager.Block;
var offset = bufferManager.GetOffset(); // Blocks if there is none.
var size = bufferManager.BufferSize;
someIOCPUsingObject.Receive(buffer, offset, size);
// I should call bufferManager.FreeOffset(offset) after I'm done with the received data.

// or

var segment = bufferManager.GetBuffer(); // Blocks if there is none.
someIOCPUsingObject.Receive(segment.Array, segment.Offset, segment.Count);
// I should call bufferManager.FreeBuffer(segment) after I'm done with the received data.

I chose to block the thread when there is no free portions left until one is freed instead of dynamically allocating more space since these operations usually don't take long and allocating more space seemed risky. I prefer pooling less-sized byte arrays if I need this behavior to creating a buffer manager like this.

  1. Can you detect any flaws in this code?
  2. Do you have any suggestions?
  3. Do you have any alternatives?
\$\endgroup\$
4
\$\begingroup\$
  1. var mod = blockSize % bufferSize;
    if (mod != 0)
        blockSize = (blockSize - mod) + bufferSize;
    

    Wouldn't it be better if the constructor actually accepted bufferCount instead of blockSize, so that you wouldn't have to have this non-obvious logic? Or maybe just throw an exception if blockSize is not divisible by bufferSize.

  2. I thought it can be faster with a ConcurrentBag.

    Are you sure about that? ConcurrentBag is optimized for the situation when a thread consumes what it produces, I'm not sure that's the situation you're likely to be in (though you could be). If you're changing the default for performance reasons, you should first make sure it actually makes a difference (you're dealing with IO, that's most likely going to dominate by a lot). And if it does make a difference, you should measure that your version is actually faster, not just make guesses.

  3. Consider making GetOffset() and FreeOffset() private. Consumers of your class shouldn't know about its implementation (single block with equally sized buffers). GetBuffer() and FreeBuffer() provide a much better abstraction, and ArraySegment is a struct, so it shouldn't cause a noticeable performance penalty.

  4. FreeBuffer() should check that the buffer it receives is valid: that it refers to the block (and not some other array) and that it has correct length and offset. The same applies to FreeOffset(), if you decide to keep it public.

\$\endgroup\$
  • \$\begingroup\$ 1 - You are so right about buffer count instead of block size, I'll change that. 2- Right again, I just tested it and ConcurrentQueue turned out to be faster. 3 - That was the original idea but since ArraySegment already exposes the buffer array, I saw no harm in exposing them directyly (I still may revert that though, it seems more appropriate). 4 - Agreed, validations are needed, I'll add them. Thank you and +1 for this excellent review. \$\endgroup\$ – Şafak Gür Oct 23 '12 at 0:22
  • \$\begingroup\$ I added the updated version. In FreeBuffer method, do you have a suggestion on checking the offset? \$\endgroup\$ – Şafak Gür Oct 23 '12 at 6:55
  • \$\begingroup\$ That depends on how much performance and code are you willing to sacrifice for being sure about correct usage of your class. I think tracking all used offsets (maybe in a ConcurrentDictionary) and using that in validation makes sense. On the other hand, it might be too much code or performance penalty for little benefit for you. \$\endgroup\$ – svick Oct 23 '12 at 11:31
0
\$\begingroup\$

This code does not compile without errors under Visual Studio 2013

These 3 lines of code are in error:

bufferSize.ThrowIfOutOfRange(1, paramName: "bufferSize");

and

bufferCount.ThrowIfOutOfRange(1, paramName: "bufferCount");

and

return ArraySegment.From(_Block, index, _BufferSize);
\$\endgroup\$
  • 1
    \$\begingroup\$ Hello Kevin and welcome to codereview.stackexchange.com. You can get your code formatted as code by indenting it with four spaces (I've gone ahead and done this for you). Also we expect answers to provide help, in your case it would be nice if you explained how to make the code compile cleanly under VS2013. I hope you will enjoy your time with us. \$\endgroup\$ – Emily L. Jun 20 '14 at 13:19
  • \$\begingroup\$ They are extension methods. I stated what they do in the comments above them but didn't feel the need to include their definitions. So, first two ThrowIfOutOfRange calls mean: "if bufferSize or bufferCount is less than 1, throw an ArgumentOutOfRangeException. And the third one can be replaced with new ArraySegment<byte>(_Block, index, _BufferSize). \$\endgroup\$ – Şafak Gür Jun 24 '14 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.