Count line of code for a JavaScript project

Question

Here is my bash script that I just wrote to count line of code for JavaScript project.

It will list number of:

• Comment lines
• Blank lines
• All lines

Here is my script:

#!/bin/bash

fileExt="*.js"

allFiles=$(find ./ -name "$fileExt")

commentLines=$((perl -ne 'print "$1\n" while /(^\s+\/\/\s*\w+)/sg' $allFiles;\
perl -0777 -ne 'print "$1\n" while /(\*\*.*?\*\/)/sg' $allFiles) | wc -l)

blankLines=$(grep '^[[:space:]]*//' -r --include $fileExt | wc -l)

allLines=$(echo $allFiles | xargs wc -l | tail -n 1 | cut -d " " -f 2)

echo -e "\nTotal comments line is: $commentLines.\n\
Total blank lines is: $blankLines.\n\
\nTotal all lines is: $allLines."

Let me explain it a little bit:

First, we need to list all files that end with ".js" within the project:

allFiles=$(find ./ -name "$fileExt")

Second, we count all comment lines:

commentLines=$((perl -ne 'print "$1\n" while /(^\s+\/\/\s*\w+)/sg' $allFiles;\
perl -0777 -ne 'print "$1\n" while /(\*\*.*?\*\/)/sg' $allFiles) | wc -l)

There are 2 types of comment lines in JavaScript:

1. Line start with only space and // or only //

   Example:

   //this is a comment line
   

   // this is a comment line // this also is a comment line

   All above are comment lines, and we got 3 lines total here. But the following is not comment lines:

   function foo(params 1) { // return void
   
   }
   

   The line contains // return void is not considered as a comment, so we do not need to count it.

   And for the kind of comment line. I using the regex with perl to print all comment lines that match with regex:

   /(^\s+\/\/\s*\w+)/sg
   

2. Multi-line comment (JSDOC):

   Example:

   /**
    * return something
    * @param {object} obj
    * return void
   */
   

   So we need to count all number of lines that start from the line with /** and end with */, we have 5 lines here.

   I use this regex to match:

   perl -0777 -ne 'print "$1\n" while /(\*\*.*?\*\/)/sg' $allFiles
   

   So the total number of comment lines is the sum of two types comment lines above.

Third

We need to count blank lines.

I use the following command to match:

grep '^[[:space:]]*//' -r --include $fileExt | wc -l

Finally

We need to count all lines:

echo $allFiles | xargs wc -l | tail -n 1 | cut -d " " -f 2

I wonder if my solution is good enough or not.

Answer 1

fileExt="*.js"
allFiles=$(find ./ -name $fileExt)

This is a bug. The wildcard in $fileExt will be expanded by the shell, and cause a syntax error when the current directory has more than one matching file in it:

$ touch a.js b.js
$ fileExt="*.js"
$ find ./ -name $fileExt
find: paths must precede expression: `b.js'
find: possible unquoted pattern after predicate `-name'?

You need to quote the variable. Better yet, use the globstar option to populate an array and eliminate find altogether:

fileExt="js"
shopt -s globstar
declare -a allFiles=( **/*.$fileExt )
grep … "${allFiles[@]}" # to use the array

This regex does not match blank lines:

grep '^[[:space:]]*//' 

Just /* is enough to start a multi-line comment in JavaScript (not /**).

The script reads every file four times. That is slow. Since you're using Perl already, just let it count everything. Then you don't need to capture the file names at all, since they are used only once:

#!/bin/bash
shopt -s globstar
exec perl -nle '
    BEGIN { 
        @ARGV=grep -f, @ARGV or die "no matching files\n";
        $comment = $blank = 0;
    }
    if    ( m{ ^\s*/\* }x .. m{ \*/ }x   or   m{ ^\s*// }x ) { $comment++ }
    elsif ( m{ ^\s*$ }x ) { $blank++ }
    END { print "$comment comment lines; $blank blank lines; $. total lines" }
' **/*.js

At this point we're barely using bash anymore, and the globbing can be moved inside the Perl script, using the File::Find module:

#!/usr/bin/perl -wnl
BEGIN {
  use strict;
  use File::Find;
  my $ext="js";
  my @dirs=( @ARGV ? @ARGV : '.' );
  @ARGV=();
  find(sub { -f and /\.$ext$/o and push @ARGV, $File::Find::name }, @dirs );
  our ($comment, $blank) = (0, 0);
}

if    ( m{ ^\s*/\* }x .. m{ \*/ }x   or   m{ ^\s*// }x ) { $comment++ }
elsif ( m{ ^\s*$ }x ) { $blank++ }

END {
  print "$comment comment lines; $blank blank lines; $. total lines";
}

Answer 2

General

Run shellcheck on this script - almost all variable expansions are unquoted, but need to be quoted. That will also highlight the non-portable echo -e (prefer printf instead) and a dodgy use of $(( where $( ( would be safer.

I recommend setting -u and -e shell options to help you catch more errors.

Flexibility

Instead of requiring users to change to the top directory of the project, we could allow them to specify one or more directories as command-line arguments, and use current directory as a fallback if no arguments are provided:

dirs=("${@:-.}")

Finding files

allFiles will include directories and other non-regular files, if they happen to end in .js. We need to add a file type predicate:

allFiles=$(find "${dirs[@]}" -name "$fileExt" -type f)

Since we're using Bash, it makes sense to take advantage of array variables - though we'll still have problems for filenames containing whitespace. To fix that, we need to read answers to How can I store the “find” command results as an array in Bash?:

allFiles=()
while IFS=  read -r -d ''; do
    allFiles+=("$REPLY")
done < <(find ./ -name "$fileExt" -type f -print0)

It may almost be simpler to set globstar shell option and then remove non-regular files from the glob result.

Counting comment lines

I didn't follow your Perl code, but I have an alternative approach using sed:

• convert all lines from initial /** to final */ to begin with // instead,
• then keep only the lines beginning with optional whitespace then //:

sed -e '\!^[[:blank:]]*/\*\*!,\!\*/!s/.*/\\\\/' \
    -e '\|^[[:blank:]]*//|!d'

(Actually, that's a lot less pretty than I'd hoped!)

Blank lines

Here, we've used the regular expression that matches comment lines. We want '^[[:blank:]]*$' instead, to match lines that contain only (optional) whitespace.

All lines

Again, over-complicated: just cat the files together and then use wc -l.

Printing

I find it easier to visualise output formatting if we simply use a here-document:

cat <<EOF
Total comments lines is: $commentLines.
Total blank lines is: $blankLines.
Total all lines is: $allLines.
EOF
exit

---

Modified code

#!/bin/bash
set -eu

fileExt='*.js'
dirs=("${@:-/usr/lib/nodejs/lodash}")

allFiles=()
while IFS=  read -r -d ''; do
    allFiles+=("$REPLY")
done < <(find "${dirs[@]}" -name "$fileExt" -type f -print0)

commentLines=$(sed -e '\!^[[:blank:]]*/\*\*!,\!\*/!s/.*/\\\\/' \
                   -e '\|^[[:blank:]]*//|!d' \
                   "${allFiles[@]}" | wc -l)
blankLines=$(cat "${allFiles[@]}" | grep -c '^[[:blank:]]*$')
allLines=$(cat "${allFiles[@]}" | wc -l)

cat <<EOF
Total comment lines is: $commentLines.
Total blank lines is: $blankLines.
Total all lines is: $allLines.
EOF

Although this makes three passes over the input files, that might be an acceptable trade-off against the complexity of a single-pass approach here (and is already the approach taken in the original code).

---

Single-pass version using awk

A single-pass version doesn't require us to use an array to store the filenames; we can simply stream the file contents into a suitable counting function. We could implement that counting function in shell, but it's probably easier to write a short awk program. Note that with no arrays, we can make this a POSIX shell program:

#!/bin/sh
set -eu

fileExt='*.js'

find "${@:-.}" -name "$fileExt" -type f -print0 | xargs -0 cat |
    awk 'BEGIN { all = 0; blank = 0; comment = 0;  incomment = 0; }

         { ++all
           if ($0 ~ "/\\*") { incomment = 1 }
           if (incomment) { ++comment; if ($0 ~ "\\*/") incomment = 0; }
           else { blank += ($0 ~ "^[[:blank:]]*$"); comment += ($0 ~ "^[[:blank:]]*//") } }

         END { print "Total comment lines is:", comment
               print "Total blank lines is:", blank
               print "Total all lines is:", all }'