Passing API data chunk per chunk to its socket clients every M seconds

Question

Let's say that I want to query some API which will respond with array of random numbers:

[4, ..., 17, ..., 25]

To keep things simple enough, let's say that array has always 10 elements. For every element in that array, I need to emit via socket to all clients just ONE number every M seconds till end of array. When the end of the array is reached, fetch again.

The API that I want data from is found here (it's about true random numbers generator) and I choose some abstraction for fetching data(node package: node-random available here).

Sadly I can't find an abstraction (package) which support JSON API from the random.org. All of them just supports something that random.org calls "Old API for automated clients" (which only support HTML and text/plain responses), there is actually a new version of the API called "API for automated clients" which supports JSON response. So it would be faster or nothing at all to parse raw JSON instead text/plain response.

config.js

exports.fetch_options = {
    minimum : 0,
    maximum : 40,
    number  : 10,
    columns : 1,
    base    : 10,
    random  : "new"
};

Just some boilerplate instruction for node-random package, where I want to fetch 10 decimals number between [0, 40]. That is all. I used just exports here instead of module.exports because in the future I expect to change this file to support all application configs, like:

exports.mongodb_conn_url = "...";

so fetch_options and config about database should be logically divided.

Main logic (server.js)

var http           = require('http'),
    config         = require('./config'),
    evx            = require('events'),
    events         = new evx.EventEmitter(),
    io             = require('socket.io'),
    randGenerator  = require("node-random"),
    _              = require("underscore"),
    fetch          = true;

// Create dummy server

var server = http.createServer(function(req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Socket server');
});

io = io.listen(server);
server.listen(8081);

// Try to hit API whenever you can (if needed)
setInterval(function() {
    if(fetch) {
        console.log("Fetching new data...");

        events.emit('fetch_data');
        fetch = false;
    }
}, 0);

// Fetching data
events.on('fetch_data', function() {
    randGenerator.integers(config.fetch_options, function(err, data) {
        console.log("Fetched data = ", data);
        events.emit('new_data', data);
    });
});

// Passing data to the client via socket
events.on('new_data', function(data) {
    /*  
        Emit via socket to all clients just ONE number every M seconds, where M is:

        minimum_step    = 0.7
        additional_step = Random number between [0, 2] rounded to .3 decimals
        ---------------------------------------------------------------------
        M = minimum_step + additional_step;
    */

    var step   = 0.7 + Math.round( (Math.random() * 2) * 1000) / 1000,
        delays = _.range(0.7, step * 20, step);

    function scheduler(index) {

        setTimeout(function() {
            io.sockets.emit('new_number', data[index]);

            if(index == config.fetch_options.number - 1) {
                fetch = true;
            }

        }, delays[index] * 600);

    }

    for(var i = 0; i < data.length; i++) { scheduler(i); }

});

console.log("End of code in serverB...");

Now another server which acts as a client of this socket server can connect to this:

var client_io = require('socket.io-client');
client_io = client_io.connect('http://localhost:8081');

client_io.on('new_number', function(number) {
    io.sockets.emit('new_bid', number); // Emit to its own clients (browsers)
    console.log(number);
});

server.listen(8080);

I would appreciate the following:

1. I'm aiming for code correctness, best practices and design pattern usage, especially for the part related to emitting events in node. I see massively that developers inherit from EventEmitter before using any emitting. I don't see any advantage of using that practice.
2. Will it be better to implement ReadableStream here? I don't know to much about streams, but it seems here, in some way, i'm creating something that looks a like stream. So instead of mixing that logic with the server - how to implement a subclass of ReadableSteam and then just to require it.

Answer 1

So you're really looking to do two things:

1. Be an Emitter: emit a random number from a list of random numbers to socket clients every m seconds; and
2. Be a Fetcher: fetch n random numbers and add them to the list so the Emitter can get them.

I'm not sure this is a stream problem - if you were to fetch the numbers one at a time and then emit them, sure, a stream might be the right approach - rather, it is a dequeue, with the Fetcher filling it up and the Emitter draining it.

Given that, I would approach it something like this:

var dequeue = [];

function fetchNumbers() {
    // Check to see if we're getting low.  If now, just hold up.
    if (dequeue.length > 2) return setTimeout(fetchNumbers, 500);

    // Since we need data, use whatever you use to get them:
    awesomeRandomNumberFetchFunction(function (err, numbers) {
        numbers.forEach(function (num) {
            dequeue.unshift(num);
        });

        // Set reasonable timeout - if we're getting 10 and emitting one each 700 ms, 5 seconds is good.
        setTimeout(fetchNumbers, 5000);
    });
}

// Start up the Fetcher:
fetchNumbers();

function emitNumber() {
    var number = dequeue.pop();

    // If we got a number, emit it.
    if (number) io.sockets.emit('new_bid', number);

    // If we got a number, wait a random time; if we didn't, try again right quick instead
    var randomDelay = 700 + Math.round(Math.random() * 200);
    var timeout = number ? randomDelay : 50;

    // Lather, rinse, repeat
    setTimeout(emitNumber, timeout);
}

// Start up the Emitter:
emitNumber();