2
\$\begingroup\$

What I have is a simple express js server where user sends an image url with a specific width all in the URL query. the code will get the image stream pipe it into a sharp.js module then send it back. here is my code flow

  1. in my radis cache database check if similar image URL with similar width already saved, if so just send the cached image
  2. if not get the image stream using a get request
  3. pipe it into the sharp module resize it + convert it to webp
  4. pipe it to the fs.createWritestream so i can save it as cache
  5. once the cache is saved send back the image using fs.createReadstream(path).pipe(res)

right now my code works, the website I'm using this app for have around 1000 concurrent users and each page has around 20 images

for usage this app uses 300mb ram - 500mb at peak hours also at some very short moments the cpu usage goes as high as 80% my cpu model is Intel Xeon X5650 (24) @ 1.605GHz

i'm looking if I can reduce both ram and cpu usage and make the code faster if possible

My Code

app.get("/image/:url", async function (req, res) {
  const url = getHostName(req.params.url); // here i check if the url is the domain i want
  if (!url) return res.sendStatus(400);

  let { width } = req.query;
  width = parseInt(width);
  width = width > 1000 ? 1000 : width <= 0 ? 300 : width;

  if (!width) return res.sendStatus(400);

  const urlHash = md5(url + "?width=" + width); // I save the hash so later i can check if this image is already processed
  const cached = await radisClient.get(urlHash);
  if (cached) {
    return fs.createReadStream(cached).pipe(res);
  }

  pipeAxios({
    url: url,
    responseType: "stream",
  })
    .then(async (response) => {
      const sharpMinupilation = sharp().resize(width).webp(); // my sharp module
      const randomName = Math.random().toString(16).slice(2); // random name for my cache file name
      const fileName = `./images/${randomName}.webp`;

      response.data
        .pipe(sharpMinupilation)
        .pipe(fs.createWriteStream(fileName))
        .on("close", function () {
          fs.createReadStream(fileName).pipe(res, { end: true });
          client.set(urlHash, fileName); // save the image so for the next request we don't do sharp module step
        });
    })
    .catch((e) => res.send("Not an image"));
});
\$\endgroup\$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.