0
\$\begingroup\$

I searched but failed to find a canonical pattern for ES6 promise-based queue. In fact all I am making is a step-wise FIFO buffer to ensure that messages don't collide so a very simple queue structure: like a message, sausage machine...

All it does is create a chain of promises, that resolve, at a fixed interval, to the messages in the order that they were received.

It's important for my application to know when the queue has been cleared and the only way I could think to do this was to add a Promise.all after the last message has gone into the queue (I use a setTimeout to simulate this).

To feed the Promise.all, I'm accumulating an array of references to the chained promises and just for the hell of it, shifting them out of the queue after they resolve.

I puzzled over how to do this without adding the array, like to somehow kick a .then(_ => console.log('done') ahead of each new promise. In other words a queue 'head' that I unplug from the previous last element and plug into the current added promise. But I couldn't figure out how to achieve that.

Is there a better way than this?

'use strict';
var t = 500;

function write (x, enc, cb) {
  process.stdout.write('write: ' + x);
  cb(x-1)
}
function f(x, write) {
  return new Promise(res => {
    setTimeout(_ => write(x, null, res), t)
  })

}

console.log('start');

var q = Promise.resolve(0);
var qs = [q];


[1,2,3,4,5,6,7,8,9,10]
  .map(n =>  {
    return {
      cb: function (_) {
        console.log('\tcb ' + n);
        qs.shift();
        return _
      },
      x: n
    }
  })  // [{ cb, x }...]
  .forEach(
    (o) => {

      q = q.then(() =>
        f(o.x, write)   // return a promise to write
          .then(_ => o.cb(_ + 1)) // followed by a synchronous write
      );
      qs.push(q);

    });

// monitor queue
setTimeout(_ => {
  console.log('register all ' + qs.length);
  Promise.all(qs).then(a => {   // detect queue completed
    console.log('all done');
    a.forEach((_, i) => console.log(_ + '\t' + i))
  });
} , t*5);
\$\endgroup\$
  • 2
    \$\begingroup\$ could you provide some test cases as to how this should work? having issues understanding your code \$\endgroup\$ – Dan Pantry Sep 20 '16 at 16:20
  • \$\begingroup\$ This is obfuscated code. Assembly is easier on the eyes than this. \$\endgroup\$ – Mast Sep 20 '16 at 16:24
  • \$\begingroup\$ @DanPantry here is the issue I'm working around. I'm multiplexing stout and stderr onto the console. \$\endgroup\$ – Cool Blue Sep 20 '16 at 16:34
  • \$\begingroup\$ @Mast if you drop it into node and step through it and if you read through the explanation in my question it's quite obvious how it works. These are mostly, standard patterns and it's not "War and Peace". \$\endgroup\$ – Cool Blue Sep 20 '16 at 16:36
1
\$\begingroup\$

Code in the question is hard to follow but it would appear that you are struggling to do something that's achieved very simply with this pattern, with which detection of "the last resolve in an ES6 promise chain" falls out in the wash. Read and understand that before continuing below.

If I'm correct, then you could write :

function makeAsyncSequence(arr, callback) {
    return arr.reduce((chain, item) => {
        return chain.then(() => callback(item));
    }, Promise.resolve());
}

And you would call :

makeAsyncSequence([1,2,3,4,5], doSomethingAsync).then(finalResult => {
    console.log('complete');
});

This is so simple that, in practice, we would normally choose to write arr.reduce(...) out longhand in our application everywhere it's needed. A special utility function like above is seldom needed.

Now, if you wish to impose a minimum delay that is not already imposed by the callback function :

First a delay function ...

function delay(t) {
    return new Promise(resolve => setTimeout(resolve, t));
};

... and an enhanced version of makeAsyncSequence(), in which callback(item) is aggregated with delay(minimumDelay) :

function makeAsyncSequence(arr, callback, minimumDelay) {
    minimumDelay = +minimumDelay || 0;
    return arr.reduce((chain, item) => {
        return chain.then(() => {
            if (minimumDelay) {
                return Promise.all([callback(item), delay(minimumDelay)]); // the slower of the two async processes determines the timing of the returned promise's success path.
            } else {
                return callback(item); // no minimumDelay required, therefore act as in the simple case.
            }
        });
    }, Promise.resolve());
}

And you would call :

makeAsyncSequence([1,2,3,4,5], doSomethingAsync, 1000).then(finalResult => {
    console.log('complete');
});

demo

Due to the increased complexity we would probably choose to write this as above in our application - ie as pair of special utilities.

\$\endgroup\$
  • \$\begingroup\$ Hi, thanks for the reply. Yep, I'm aware of the reduce pattern, but I don't understand how it helps unless the sequence is known in advance, which it's not. The reason I used a forEach and not reduce in my simulation is to model the messages arriving. The queue expands and contracts as messages arrive at random intervals and are dispatched at fixed intervals. At some point the queue will empty and that's when I need to emit a drain event. I was looking for a pattern that did that purely with the chained promises, without mirroring the state to count up and down to detect drain events. \$\endgroup\$ – Cool Blue Oct 19 '16 at 14:56
  • \$\begingroup\$ Ah, I hadn't appreciated that the sequence is not known in advance. Reading the question again, you kind of hint at that but the code says otherwise - and after all, this is Code Review. Anyway, I think you need to go away and read all about Functional Reactive Programming (FRP). I'm not an expert but there are several people on StachOverflow who are. \$\endgroup\$ – Roamer-1888 Oct 19 '16 at 15:15
  • \$\begingroup\$ Sorry if my question is a bit blur. Thank's for the link... It seems more about streams than buffers... Also, I need state information about the queue and functional doesn't do state. My conclusion was that, unless a thenable knows about its subscribers, it's impossible for it to know if it is the last in a chain. I don't think there is anything provided in the ES6 promise API to track subscribers so, the only way to produce a drain event is to maintain seperate state to mirror the queue state. At least that's' what I concluded. \$\endgroup\$ – Cool Blue Oct 20 '16 at 4:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.