Speed up javascript code to convert csv to geojson

Question

I'm using a script that converts a CSV file into GeoJSON in order to display it on a map. You can see it for yourself here. The CSV has about four thousand rows. It's quite slow. I'm wondering what is slowing it down and how I can speed it up.

function latLonColumnsToNumbers(data, latName, lonName) {
  return data.map(function(item) {
    if (item.hasOwnProperty(latName)) {
         item[latName] = parseFloat(item[latName]);
         if (isNaN(item[latName])) { //check if row has blank lat/long, and if so, flag it
            var latFlag = 1;
         } else {
             latFlag = 0;
         }
      }
    if (item.hasOwnProperty(lonName)) {
         item[lonName] = parseFloat(item[lonName]);
         if (isNaN(item[lonName])) {
             var lonFlag = 1;
         } else {
             lonFlag = 0;
         }
    }
    if (latFlag == 1 || lonFlag == 1) { //if row has no lat/long, then assign it to the approximate center of the map
        item[latName] = 42.2; 
        item[lonName] = ~83.6;
        return item;            
     }  
    else {
        return item;
    }
  });

I know that a faster way to do this would be to turn the CSV into a sql database and call it with PHP but that involves using a lot of server-side stuff like NodeJS, Ajax, etc that I am not (yet) familiar with. So I would like to keep using the CSV for now until I learn how to do that.

Answer 1

From a short review;

• Old school for loops still beat map

• Prevent repeated object[string] access, like accesssing several times item[latName], because it is relatively slow

• Is it absolutely necessary to create a new object? I have a suspicion that modifying the object would be faster. (Yes, not great practice, but you are looking for speed, you should try this)

• One more thing, I was hesitant to mention this but, but the plus operator can be faster than parseFloat according to jsperf. So you could just go for

    const newLat = +item[latName];
    const newLong = +item[longName];
  

give it a shot, let us know ;)

This is my counter proposal with in place conversion;

function convertLocationToNumbers(data, latName, longName) {
  const dataLength = data.length;
  for(let i = 0; i < dataLength; i++){
    const item = data[i];
    //Dont check for hasownproperty, just go!
    const newLat = +item[latName];
    const newLong = +item[longName];
    if(isNaN(newLat) || isNaN(newLong ){
      item[latName] = 42.2; 
      item[lonName] = ~83.6;
    }else{
      item[latName] = newLat; 
      item[lonName] = newLong;
    } 
  }
}

Answer 2

The slowest part is probably the parseFloats, but I can't think of a faster way.

One thing that stands out to me is the use of map, but at the same time you are modifying the items, so in the end you have two arrays with the same content. You probably can save some time just keeping the same array by using forEach (or a normal for loop which is probably even faster).

Also using global variables (var) and integers where you could use boolean won't help. I also don't see the need for hasOwnProperty.

What is the point of ~83.6 (which equals -84)?

I'd simplify the whole thing to to:

function latLonColumnsToNumbers(data, latName, lonName) {
    data.forEach(item => {
       item[latName] = parseFloat(item[latName]);
       item[lonName] = parseFloat(item[lonName]);
       
       if (isNaN(item[latName]) || isNaN(item[lonName])) { 
          item[latName] = 42.2; 
          item[lonName] = ~83.6; // or just -84
       } 
    });

    // The return isn't needed either. Just use the original array after calling the function.
    return data;
}