Performance tuning on a text file to object conversion

Question

I'm using an API which returns text in the following format:

#start
#p 09060 20131010
#p 09180 AK
#p 01001 19110212982
#end
#start
#p 09060 20131110
#p 09180 AB
#p 01001 12110212982
#end

I'm converting this to a list of objects:

var result = data.match(/#start[\s\S]+?#end/ig).map(function(v){

    var lines = v.split('\n'),
        ret = {};

    $.each(lines, function(_, v2){
        var split = v2.split(' ');
        if(split[1] && split[2]) 
            ret[split[1]] = split[2];
    });

    return ret;
});

My concern is that the API returns quite a lot of data, therefore I would like some feedback regarding on how to improve the performance.

For instance, is there any way to reduce the mapping complexity from O(N²) to O(N)?

Also, please suggest regex improvements :)

Answer 1

If you use regular expressions for parsing, then I would recommend using them for everything. Here's a solution that proceeds line by line, using capturing parentheses to see what the line contained.

function parse(data) {
    var re = /(#start)|(#end)|#p\s+(\S+)\s+(\S+)/ig;
    var results = [], match, obj;
    while (match = re.exec(data)) {
        if (match[1]) {           // #start
            obj = {};

        } else if (match[2]) {    // #end
            results.push(obj);
            obj = null;           // ← Prevent accidental reuse if input is malformed

        } else {                  // #p something something
            obj[match[3]] = match[4];
        }
    }
    return results;
}

Answer 2

I dislike regexes with a passion ;) Especially because sometimes they beat solutions that ought to be faster.

I would counter propose a solution where you keep using indexOf while keeping track where you are in the data. This way you only go thru the data once. I would also name your constants 0 and 1 so that the reader instinctively knows what you are doing. Furthermore, given that your script is horizontally quite short, I would spell out your variables. I am not a big fan of v, v2 , _ etc. Finally, if speed is important, then good old loops will always beat forEach.

function parseResults( data )
{
  var index = -1,
      lastIndex = -1,
      objects = [],
      object,
      line,
      parts,
      KEY = 0,
      VALUE = 1;
  //~ is a short circuit for comparing to -1
  while( ~ (index = data.indexOf('\n',index) ) )
  {
    line = data.substring( lastIndex , index );
    if( line == '#start')
      object = {};
    else if( line == '#end' )
      objects.push( object );
    else 
    {
      parts = line.split(' ');
      if( parts[KEY] && parts[VALUE] )
        object[ parts[KEY] ] = parts[VALUE];
    }
    //+1 because I dont want to do ++ in the while, another +1 to make substring work
    //admittedly not very elegant looking :\
    lastIndex = index + 2;
  }
  return objects;
}

I would be most curious if you run this version and you run the 200_success version which one would be more performing with large sets of data.