I have a software that does the following in sequence:
download text files from s3
reads the files
collect the data from files
The program is asynchronous-synchronous - each phase is asynchronous, but it heavily depends on the output of the previous phase, so I'm using Promises for build the chaining.
The problem of this architecture is that is using to much CPU since I'm spawning child processes in the extract and collect phase. So I decide to change to events:
How I'm doing:
I have 3 eventEmitters: downloadFile, extract, collect:
downloadfile.emit("copy", s3file);
downloadfile.on("error", callback);
downloadfile.on("close", (file) => {
extract.emit("file", file);
extract.on("error", callback);
extract.on("close", (file) => {
collect.emit("data", file.data);
collect.on("error", callback);
collect.on("close", (data) => {
data is here... i can use without waiting all the other files.
});
});
});
But this isn't very readable... I prefer the Promise way:
Promise.resolve(options)
.then(downloadFiles)
.then(extract)
.then(collect);
So I was thinking... is it possible to chain the events? I mean, the output of an event will be the input of other, just like the Promises? If so, how could I achieve that?
emit
actually do here? Aredownloadfile
,extract
andcollect
global entities, or per-sequence? \$\endgroup\$ – Joseph Mar 28 '17 at 14:17