# How to write a simple script to read text from a file and compute the number of sentences, number of unique words and punctuation/symbols? (NodeJS)

I just finished a challenge (noted above) and I was wondering how good is this approach or is there some simpler solution that I am not aware of?

If you have a different solution, try to explain the code for people less experienced than myself rather just than pasting a solution.

Here is an example text file to be read from:

This file has eight words in this sentence.
There are two sentences. In this paragraph.

And two paragraphs in total.
Lorem ispsum.
Lorem ispsum.
Lorem ispsum.
Lorem ispsum.

/**
* Write a simple script to read text from a file and compute (approximately) the number of sentences, number of unique words and punctuation/symbols.
* Run the code with node ./problem-2.js test.txt
*/

// Use Node's builtin api to read a file
const fs = require('fs');

//#1 count the number of words
const wordCount = (string) => string.split(" ").length;

//#2 count the number of unique words
const uniqueWords = txt => new Set(txt.toLowerCase().match(/\w+/g)).size;

//#3 count the number of paragraphs
const paragraphCount = (paragraphString) =>{
//seperate each sentence into a seperate string
paragraphString = paragraphString.split("\n\n");
// console.table(paragraphString) // show as a table for better visual
let paragraphArray = paragraphString.length; // length of paragraph array

let paragraphCount = 0;
let strip_whitespace = /\s+/gi;
while (paragraphArray >=0) {
paragraphArray--;
let tmp = paragraphString[paragraphArray];
tmp = tmp ? tmp .replace(strip_whitespace,"") : tmp;

if( tmp && tmp.length > 1 ) paragraphCount++;
}
return paragraphCount;
}

function countPunctuations(punctuationsStrings) {
let punctuationsCount = 0;
const punctuations = [".", ",", "!", "?"];
// const symbols = ["~","","!","@","#","$","%","^","&","*","(",")","_","+","-","=","{","}","|","[","]","\",":",";","'","<",">","?","/",] for (const ch of punctuationsStrings) { if (punctuations.includes(ch)) punctuationsCount++; } return punctuationsCount; } fs.readFile("test-2.txt", 'utf8', function(err, data) { if (err) throw err; console.log("The text in the file:\n\n", data,"\n") // store results in an object to present the log better let result={ "word count": wordCount(data), "unique words": uniqueWords(data), "paragraph count": paragraphCount(data), "punctuation count": countPunctuations(data), } console.table(result) }); $$$$  ## 1 Answer Word count bug This line here: const wordCount = (string) => string.split(" ").length;  will overcount the number of words if the input text happens to have multiple spaces in a row. For example: const wordCount = (string) => string.split(" ").length; console.log(wordCount( foo bar baz )); I'd use .match instead, and match contiguous non-space characters. (Can't use \w+, since that'd result in they're and similar being counted as 2 words, and since \w only works with English letters - matching non-spaces is easier than figuring out all the possibilities, and will work for any romance language) const wordCount = (string) => (string.match(/\S+/g) || []).length; console.log(wordCount( foo bar baz )); The .match(..) || [] is needed because .match will return null (and not the empty array) if no matches are found. uniqueWords bug As noted above, \w+ won't be accurate given apostrophes and other punctuation marks that may be around or inside single words. (You also need to alternate the .match with [] as described above too) paragraphCount issues You have: paragraphString = paragraphString.split("\n\n");  There are a few problems here: • The paragraphString is initially not the string of a paragraph, but a string containing the entire input text - better to call it something like inputText. • Then, the paragraphString becomes an array - but its name is paragraphString? Easy source of confusion. • You're unnecesarily reassigning the paragraphString variable - it may refer to one thing, or it may refer to another, depending on what point in the program you're examining it. Code is generally easiest to reason about when the contents of a given variable name doesn't change. Occasionally it's necessary (like with a counter), but often it isn't - better to put a new expression into a different variable rather than reassigning the old one, if possible. Similarly: let paragraphArray = paragraphString.length; // length of paragraph array  But the .length is a number, not an array. Giving variables understandable, precise variable names makes code easier to read and helps prevent bugs. I'd either call it paragraphIndex - or, even better, avoid manually iterating over the array entirely and use an array method instead, eg: paragraphArray.forEach((paragraphString) => { // check paragraphString  Empty strings can have .replace called on them just fine, and it makes the code easier to read, so you can simplify the assignment of tmp to: const paragraphString = paragraphStrings[paragraphIndex].replace(strip_whitespace,"");  Or you could simplify the whole function here by filtering the array of paragraphs by those with a trimmed length: const paragraphCount = inputText => inputText .split('\n\n') .filter(paragraphText => paragraphText.trim()) .length;  You could also use .reduce to count up, instead of constructing an intermediate array that gets used only for its length, but the intent of this .reduce is probably a bit harder to recognize at a glance for some: const paragraphCount = inputText => inputText .split('\n\n') .reduce((a, paragraphText) => a + paragraphText.trim() ? 1 : 0, 0)  \n\n might not be good enough, though - can you count on the file encoding using \ns only for newlines? Some files/OSs use \r\n. I'd use /\n\r?\n/ instead. Punctuation count idea Rather than: const punctuations = [".", ",", "!", "?"]; // const symbols = ["~","","!","@","#","$","%","^","&","*","(",")","_","+","-","=","{","}","|","[","]","\",":",";","'","<",">","?","/",]


With a lot of punctuation, the above can be both hard to read and easy to make typos on. If you have more than a few, consider using a string that gets parsed into an array later:

const symbols = \~!@#\$%^.split(''); // and so on


You could even make it multiline or put it in a separate file if you wanted, then parse it as needed.

To reduce the computational complexity of the algorithm, make a Set of the punctuations in advance; Set.has is O(1), but Array.includes is O(n). With large files, this could make a big difference, especially if you wree using the large symbols array instead of the smaller punctuations array.

Constructing a regular expression that alternates between all punctuation characters might be even faster.