Build process for compiling client JS modules into one script

Question

I am currently writing a rather large client-side JS application which is made of of multiple modules, each in different files (before compiling). I am using Node.js to build the final script based on all the individual modules.

For the script to work properly all modules must be added in the proper order, so they can access any modules they depend on. One way to solve this is by maintaining an array of all the modules in the correct order to be inserted. I find that limiting though, having to add each file manually and in the correct position.

I have come up with a different solution to this, and would love to get some feedback on it.

First I have one wrapper file, which is wraped around all the modules. Looks like this:

(function( ns ){

var module = window[ns] = {

    _modules: window[ns] && window[ns]._modules || {},

    exports: function( name, mod ){
        this._modules[name] = mod;
    },

    require: function( name ){
        var mod = this._modules[name];
        if(!mod) throw new Error(name + ' module not found');
        return mod;
    }
};


MODULES_INSERT


})( '_myCoolApp' );

Where it says "MODULES_INSERT" is where the modules get inserted.

Now inside the individual modules it looks something like this:

// export this module
module.exports('view', View);

function View(){ etc.. }

// import other module
var util = module.require('util');

And finally I have my build script (node.js) which adds all the modules/files, and orders them based on any module.requires they contain in their code, ensuring that all dependencies are loaded before they are required:

for(i in srcFiles){
    file = srcFiles[i];
    var content = fs.readFileSync(file.src);
    content = '(function(){\n' + content;
    content = content + '})();\n';

    // find any module dependencies
    var match, requires = [], regex = /module.require\(['"]([^'"]+)/g;
    while(match = regex.exec(content)){
        requires.push(match[1]);
    }

    file.requires = requires;
    file.content = content;
}

for(i in srcFiles){
    file = srcFiles[i];
    recurse(file);
}

function recurse(file){
    if(!file) return;

    if(file.requires.length > 0) {
        file.requires.forEach(function(req){
            recurse(srcFiles[req]);
        });
    }
    if(!file.added) results.push(file.content);
    file.added = true;
}

code = results.join('');
code = wrapper.replace('MODULES_INSERT', code);

Is this solution way overkill? One downside I see to this, is what happens when two files require each other, which gets loaded first? Is there a more effective solution for this?

Answer 1

Your approach is sound. The optimizer of RequireJS called r.js also scans the code for calls to methods with a special name (define and require) to detect dependencies.

You can see how this detection works in the file transform.js. As you can see, this optimizer uses a JavaScript parser, which is more reliable than using regular expressions: for example, your regular expression may match a call in a comment, which would be skipped using a parser.

There is another way to declare dependencies for RequireJS though, which is described in the Asynchronous Module Definition specification: wrap each module in a call to a function and list the dependencies explicitly in an array.

You may also be interested in the use of a simpler pattern for the declaration of modules, independently of the loader used.

With regards to circular dependencies ("when two files require each other"), there are two ways to handle them:

• detect them and fail, forcing the developer to avoid them in their code
• load one of the two modules with only parts of their dependencies, which requires extra care from developers: they can no longer be sure that all dependencies are available when the module runs, and must check before calling methods and accessing properties on required objects

Both approaches have drawbacks.

Answer 2

I have an experience with requirejs as a dependency management framework, it allows the compiling on the server side with the following snippet

node r.js -o path/to/buildconfig.js

the config

// **r.js** configuration
({
  appDir : "../",
  baseUrl: "js",
  dir    : "../target",

  modules: [
    { name: "app" }
  ],

  optimize   : "uglify",
  optimizeCss: "standard",

  preserveLicenseComments: false,
  findNestedDependencies: true,
  skipModuleInsertion: false,

  uglify: {
    gen_codeOptions   : {},
    do_toplevel       : {},
    ast_squeezeOptions: {},
    ast_lift_variables: {}
  },

  paths              : {
    "jquery"      : "../../lib/jquery",
    // .. and so on

  }
})