Wrap async method in a way that the method is invoked only once for all calls that occur while the first of them is not resolved

Question

Imagine we have a function that returns a Promise, ie call a GET endpoint of an api and return the response data.

function testFn() {
   return fetch('http://localhost/ep')
}

The response will remain valid for some unknown period of time but may need to be refetched later. Basically anytime somebody needs it, it needs to be fresh.

Now let's say multiple asynchronous functions need to call this method at some point for different purposeses.

If for some reason one function calls this method and lets say is waiting for its completion. And another function calls the method while the first promise not yet resolved I want to avoid calling the function again and instead return a promise to the second caller that will resolve/reject at the same time and with the same data as the first one.

To generalize, the input async method may accept arguments and we need to group the calls by those arguments. To do that I use a Map with keys being JSON stringified array of arguments, but I also allow to override this strategy using the options.keyFn

In the following code the bottleneck function accepts the input async method and returns the wrapped method with the same signature.

export type BottleneckOptions<P extends unknown[]> = {
    keyFn?: (...args: P) => string
}

export function bottleneck<T, P extends unknown[]>(
    neck: (...args: P) => Promise<T>,
    options: BottleneckOptions<P> = {},
): (...args: P) => Promise<T> {
    type QueueItem = {
        resolve: (value: T) => void
        reject: (error: any) => void
    }

    const bottles: Map<string, QueueItem[]> = new Map()

    return (...args: P) => {
        const resolveAll = (result: T) => {
            for (const item of bottle) {
                item.resolve(result)
            }
            clear()
        }

        const rejectAll = (error: any) => {
            for (const item of bottle) {
                item.reject(error)
            }
            clear()
        }

        const clear = () => {
            bottle.length = 0
            bottles.delete(key)
        }

        const key = options.keyFn ? options.keyFn(...args) : JSON.stringify(args)

        const bottle: QueueItem[] = bottles.get(key) || []
        bottles.set(key, bottle)

        const result = new Promise<T>((resolve, reject) => {
            bottle.push({ resolve, reject })
        })

        if (bottle.length === 1) {
            neck(...args).then(resolveAll).catch(rejectAll)
        }

        return result
    }
}

Now an example where we just wrap the standard fetch function.

const neck = bottleneck(fetch)
const result = await Promise.all([
  neck(url1),
  neck(url1),
  neck(url1),
  neck(url2),
  neck(url2),
])
console.log(result) // fetch only called twice (most likely) but all results for url1 (resp. url2) are same

The keyFn of the options allows to change the grouping for example stacking all calls ignoring the first argument and using only the second.

const testFn: <A, T>(a: A, b: string) => Promise<T> = ...
const wrappedFn = bottleneck(testFn, { keyFn: (a, b) => b }

Or to ignore all arguments entirely:

const wrappedFn = bottleneck(testFn, { keyFn: () => '' }

Lastly, let me provide a real use case

There is an api that is authenticated with bearer token.

Whenever the token expires the api responds with 401 and the client is supposed to request a new token via refresh token endpoint and then retry the original request with the new bearer token.

Multiple client app components may call different endpoints of the api in parallel. This means that if they are all invoked with an expired token they will all receive 401 and attempt to refresh the token, invoking multiple calls to the refresh token endpoint.

The bottleneck function would be used to wrap the refresh token function causing all parallel requests that fail with 401 to wait for the same refreshed token from the first invoker until they retry their request.

Answer 1

You don't need to create a new promise for each call. A promise can be reused multiple times, so all you have to do is cache the promise. Something like this:

if (bottles.has(key)) {
    return bottles.get(key)
}

const p = neck(...args).finally(() => {
    bottles.delete(key)
})

bottles.set(key, p)

return p