From a stream, I receive buffers of varying length.
I want to consume chunks of fixed length from this stream.
I build a buffer accumulator to accumulate the chunks in an internal array, then I can extract a buffer of a specific size by concatenating and truncating the chunks.

Here is an example usecase:

const assert = require("node:assert");

const acc = new BufferAccumulator();

acc.push(Buffer.from([0x02, 0x03]));
acc.push(Buffer.from([0x04, 0x05, 0x06]));

assert.equal(acc.getAvailableLength(), 6);

let out = acc.extract(4);
assert.equal(out.length, 4);

acc.push(Buffer.from([0x07, 0x08, 0x09]));

assert.equal(acc.getAvailableLength(), 5);

out = acc.extract(3);
assert.equal(out.length, 3);

assert.equal(acc.getAvailableLength(), 2);

Here is the accumulator:

class BufferAccumulator {
    constructor() {
        this._chunks = [];
        this._totalLength = 0;

    getAvailableLength() {
        return this._totalLength;

    push(chunk) {
        this._totalLength += chunk.length;

    extract(n) {
        if (n > this._totalLength) {
            throw new Error("Cannot extract more than total length");

        let outBuffer = Buffer.alloc(n);
        let outOffset = 0;

        while (outOffset < n) {
            if (this._chunks.length <= 0) {
                throw new Error("Out of chunks, this should never happen");

            const chunk = this._chunks[0];

            const remaining = n - outOffset;

            const toCopy = Math.min(remaining, chunk.length);

            chunk.copy(outBuffer, outOffset, 0, toCopy);

            outOffset += toCopy;

            if (toCopy >= chunk.length) {
            } else {
                this._chunks[0] = chunk.subarray(toCopy);

            this._totalLength -= toCopy;

        return outBuffer;

Do you think my implementation is correct?
I am especially wondering about assigning the result of subarray to the element: when losing the reference to the original buffer, does the result of subarray stay valid?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.