Write log file in nodeJS with setInterval

Question

I wrote this code to write the log file every 5 seconds to avoid excessive disk I/O (which can cause HDD temperature to increase or SSD to wear off):

var fs = require('fs');
var logText = '';
function log(){
    fs.appendFile('c:/log.txt', logText, (err) => {
        if (err) throw err;
        logText = '';
    }); 
}
setInterval(log, 5000);

Is this the correct approach? I mean this is async right? Will setInterval interfere with the performance?

Answer 1

I'm assuming that you are changing the logText variable to control what's being logged. If you want to keep this same interface, the only optimization that I can see is to make log() check if logText is not empty before it writes it:

var fs = require('fs');
var logText = '';
function writelog(){
    if (logText) {
        fs.appendFile('c:/log.txt', logText, (err) => {
            if (err) throw err;
            logText = '';
        });
    }
}
setInterval(writelog, 5000);

I would also add a function that adds to the log text:

function log(text){
    logtext += ("Log prefix stuff here" + text)
}

Answer 2

Let the operating system handle write caching for you.

Answer 3

You are writing roughly the same amount of data to disk whether you wait to do it in batches or do it right away. So I don't really know how valid of a concern you really have here. It seems like you are solving a problem that doesn't really need to be solved.

Also what happens when something goes wrong 4 seconds into a 5 second log cycle? Do you lose all the log data for that 4 seconds (as well as your ability to trace the problem)? Do you lose timestamp granularity for the actual items being logged?

This seems like a step backwards in making your application easy to maintain.

If you are this worried about writing log files to disk, then perhaps consider a central log store that you can log to over UDP or similar.

You also must consider that this approach would require more memory utilization within your application to store the continually growing (until logged) log string. Depending on the amount of data being logged, this could potentially be a much more important consideration. Having to add application nodes because your code is not efficient with memory usage is likely MUCH more expensive than incremental cost of replacing hard drives at some theoretically more frequent rate would be.

Bottom line - memory is typically more expensive than storage.