0
\$\begingroup\$

The request below takes very long on slower internet connections and Heroku is throwing a Request Timeout error for many users. The API request is being made by many different users hundreds of times per 30-60 seconds. Here's how it works

  • There is Cars collection in MongoDB that contains an array of csv_files
  • A user makes a POST API request to my Node.js server
    • Their POST request contains a very long CSV string
  • The server first finds the Car the user is posting their new CSV data to and then loops over every file in csv_files

    • If there is an existing file that matches the name of the user's POST request file, then we replace mark that we found an existing CSV file and replace it
    • If we don't find an existing file then we just push the CSV string into the Car csv_files array
  • When we're done, we save our newly updated Car to MongoDB and return a 204 response

Here is the code:

router.post('/cars/:url/upload-csv/', async (req, res) => {
  let csv = req.body.csv_string.split('$');

  let csv_name = csv[0];
  let csv_string = csv_name+csv[1];

  let car = await Car.findOne({ url: req.params.url });


  if (!car)
    return res.status(400).json({ msg: 'No car found. Could not upload CSV' });

  if (car.csv_files.length === 0) {
    car.csv_files.push(csv_string);
  } else {
    let foundExistingCSV = false;
    for (let i = 0; i < car.csv_files.length; i++) {
      if (car.csv_files[i].includes(csv_name)) {
        foundExistingCSV = true;
        car.csv_files[i] = csv_string;
        break;
      }
    }

    if (!foundExistingCSV) {
      car.csv_files.push(csv_string);
    }
  }

  car.markModified('csv_files');
  car.save();

  return res.status(204).json({ msg: 'All good!' });
});
\$\endgroup\$
  • \$\begingroup\$ I believe I solved this issue by using the async/queue package. I return the response right away after queuing the desired uploaded Car CSV file. If there is a better way to do this, please let me know! Thanks! \$\endgroup\$ – szier Oct 17 '17 at 19:16
  • \$\begingroup\$ If you would like your revised solution reviewed, replace the code in the question with your new code (which is OK since no answers have been posted yet). Otherwise, you should post your new idea as an answer to your own question. \$\endgroup\$ – 200_success Oct 18 '17 at 20:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.