Throttling execution of a function

Question

I have a Javascript function that calls my backend API using AJAX, let's say, updateDataUsingAjax. This is a resource intensive process, and I do not wish to allow this function to be invoked too many times in a short time span.

Let's say I intend to allow only 3 requests in every 5 seconds. If the function is called more than that, it should throw an error.

Something similar to what happens if you click the flag button too many times on Stack Overflow (or here):

---

My initial approach to solving this problem was the following:

• Keep track of the execution count and the first instance of function execution.

• Execute the function logic if,

  • Execution count is less than the limit (3 requests),

  • Or, if more time has passed than the configured time (5 seconds) - in such a case update the timestamp of the first execution as well.

• Update the execution count if the function is allowed to execute.

This approach was okay, but I realized that there might be a need to implement such a logic for multiple processes, with different limits. So I decided to write a "throttling function generator", that would take the

1. function to be executed,
2. maximum request count,
3. and the timespan for which the throttling should apply.

It should return a function that can be called just like the target function, except that it would throw an error if the request count has exceeded the limit.

---

function throttledFunctionGenerator(func, limit, interval) {
  var count = 1;

  return function(arg) {
    // Run the function only if it hasn't yet exceeded the limit.
    if (count <= limit) {

      // Execute the function.
      // Note: Exception handling is not necessary as setTimeout is used.
      setTimeout(func, 1, arg);

      // If this is the first request, schedule the counter to be reset after specified interval.
      if (count === 1) {
        setTimeout(function(arg) {
          // Reset the counter.
          count = 1;
        }, interval * 1000, arg);
      }

      // Increment the count.
      count += 1;
    } else {
      throw "You may perform this action only " + limit + " times in " + interval + " second(s).";
    }
  };
}

So if I have a function doSomething that should be called at most 3 times in 10 seconds, I could wrap it using,

var doSomethingThrottled = throttledFunctionGenerator(doSomething, 3, 5);

So, when I intend execute the following code, the fourth invocation should throw an error:

doSomethingThrottled(obj1);
doSomethingThrottled(obj2);
doSomethingThrottled(obj3);
doSomethingThrottled(obj4); // Throws an error.

Here's a demo on Codepen where you can see this in action.

var btn = document.getElementById("btn");
var btn2 = document.getElementById("btn2");
var logArea = document.getElementById("logArea");

function log(message) {
  logArea.classList.add("yellow");
  logArea.innerText += message + "\n";

  setTimeout(function() {
    logArea.classList.remove("yellow");
  }, 500);
}

function throttledFunctionGenerator(func, limit, interval) {
  var count = 1;

  return function(arg) {
    // Run the function only if it hasn't yet exceeded the limit.
    if (count <= limit) {

      // Execute the function in next event loop.
      // Note: Exception handling is not necessary as setTimeout is used.
      setTimeout(func, 1, arg);

      // If this is the first request, schedule the counter to be reset after specified interval.
      if (count === 1) {
        setTimeout(function(arg) {
          // Reset the counter.
          count = 1;
        }, interval * 1000, arg);
      }

      // Increment the count.
      count += 1;
    } else {
      throw "You may perform this action only " + limit + " times in " + interval + " second(s).";
    }
  };
}

var thriceEveryFiveSeconds = throttledFunctionGenerator(function(msg) {
  log("[Thrice]: " + msg);
}, 3, 5);
var twiceEveryTenSeconds = throttledFunctionGenerator(function(msg) {
  log("[Twice]: " + msg);
}, 2, 10);

btn.addEventListener("click", function(event) {
  var date = new Date();
  thriceEveryFiveSeconds(`Clicked: ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()}`);
});

btn2.addEventListener("click", function(event) {
  var date = new Date();
  twiceEveryTenSeconds(`Clicked: ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()}`);
});

.yellow {
  background-color: yellow;
}

<button id="btn">Thrice every 5 seconds</button>
<button id="btn2">Twice every 10 seconds</button>
<pre id="logArea"></pre>

---

I am aware of the following limitations:

• setTimeout is not precise.

• The code in its current state only works for functions that take only one argument. But I believe it can be easily extended to work with any number of arguments using the ES6 spread syntax.

Do you see any bugs with the code, or any area of improvement?

Answer 1

Your function controls average rate not rate.

Your function can let more function run than you want in the interval given.

Say you have 3 calls over 5 seconds. The first call starts the timer, then at 4.9seconds just before the time interval expires, you call the function twice. Then just after the time interval expires (@5.1 seconds)you call the function 3 more times. Thus you have called the function 5 time in less than a second. The max rate you can call the function is limit * 2 - 1 per time to run each.

You do maintain an overall average rate but you are not limiting the rate per interval.

The solution is to track the time each function starts, adding the time to a queue. Each time you get another request to run the function you check if any of the previous runs start more then the limit ago. If so remove it from the queue. If the queue has less than the execution limit then you can run the function.

function throttler (func, count, seconds) {
    const queue = [];
    return function(...args) {
        var now = performance.now();
        while (queue.length && queue[0] < now) { queue.shift() }
        if(queue.length < count){
            queue.push(now + seconds * 1000);
            func(...args);
        }else{
            // I would just return a false rather than use error to communicate
            throw new Error("Exe count overrun");
        }
    }
}

Or you can use a promise to manage the function async

function throttler (func, count, seconds) {
    const queue = [];
    return function(...args) {
        return new Promise((runOk, timeout) =>
            var now = performance.now();
            while (queue.length && queue[0] < now) { queue.shift() }
            if (queue.length < count) { 
                queue.push(now + seconds * 1000);
                setTimeout(() => runOK(func(...args)) ,0);
            } else {
                timeout(new Error("Rate to high"));
            }
        });
    }
}


 // used 
 const throttled = throttler (test, 5, 5);

 throttled(1,2,3)
     .then(returnData => {/*do somethin*/});
     .catch(error => {/*handle error*/})

Passing arguments

Parsing arguments is easy using rest parameters and the spread ... operator

For example

function callFunctionWith(...arg) {  // rest 
     theFunctionTakesArgs(...args);  // spread
}
function theFunctionTakesArgs(a, b, c, d, e){
     console.log(a, b, c, d, e);
}

callFunctionWith(1, 2, 3, 4, 5);

Answer 2

The first problem I see is that your rate-limit logic is on the client side. You should never rely entirely on client-side code, they're easily bypassed. Nothing stops me from endlessly curling to that endpoint directly or code that calls the AJAX directly.

Instead, implement that rate-limiting logic on the server-side. When the user reaches some threshold, have the server respond with an HTTP 503 (Service Unavailable) until the queue clears up. Whether you do this globally, per operation or per user, is up to you. Then all the UI does is render accordingly whether the request succeeded or received the 503.

Now if actually have a server-side piece in place and this is just your client-side code, the first thing that intrigues me is the use of a timer. You don't actually need a timer. You just need to remember timestamps of successfully called operations, and count how many of them are in flight in the last few seconds.

const throttle = (fn, count, interval) => {
  const inFlight = []

  return (...args) => {
    const now = Date.now()

    // Purge older timestamps
    for(let i = inFlight.length; i--;){
      if(inFlight[i] > now) continue
      inFlight.splice(i, 1)
    }

    // Check the length of ops in flight
    if (inFlight.length < count){
      // Persist this call's timestamp.
      inFlight.push(now + interval)

      // Return what the function would normally have returned.
      return fn(...args)
    } else {
      throw new Error('Too many in flight')
    }
  }
}