Converting duplicate JS code into a higher order function

Question

I have a block of code that is re-used and I want to use functional programming to remove this duplication.

My code takes an array of items, splits the items into batches of 500 and then does some kind of work on them.

In the first function it deletes items from a database:

Delete function:

const deleteDocuments = async (documentReferences) => {
   const batchesOf500 = Math.ceil(documentReferences.length / 500);

   for(let batchNumber = 0; batchNumber < batchesOf500; batchNumber += 1) {
      const batch = getBatchWriter();
      const startingIndex = batchNumber * 500;
      const maxIndex = startingIndex + 500;

      for(let index = startingIndex; index < maxIndex; index += 1) {
         if(index < documentReferences.length) {
            const documentPath = documentReferences[index];
            batch.delete(documentPath);
         }
      }

      await batch.commit();
   }
}

The second function is almost identical but instead of deleting from a database, it writes to the database:

Add function:

const writeToCollection = async (dataArray, collectionRef) => {
   const batchesOf500 = Math.ceil(dataArray.length / 500);

   for(let batchNumber = 0; batchNumber < batchesOf500; batchNumber += 1) {
      const batch = getBatchWriter();
      const startingIndex = batchNumber * 500;
      const maxIndex = startingIndex + 500;

      for(let index = startingIndex; index < maxIndex; index += 1) {
            if(index < dataArray.length) {
               const [key, value] = dataArray[index];
               const doc = getDocFromPath(key);
               batch.set(doc, value);
            }
         }
      }

      await batch.commit();
   }
}

These functions are almost identical, so I have write a higher order function to do most of the leg-work.

Higher order function:

const runFunctionInBatchesOf500 = (func, dataArray) => {
   const batchesOf500 = Math.ceil(dataArray.length / 500);

   for(let batchNumber = 0; batchNumber < batchesOf500; batchNumber += 1) {
      const batch = this.firestore.batch();
      const startingIndex = batchNumber * 500;
      const maxIndex = startingIndex + 500;

      for(let index = startingIndex; index < maxIndex; index += 1) {
         const document = dataArray[index];
         func(document, batch);
      }
   }

   await batch.commit();
}

And to it you can create your own functionality to apply to each document and use it like this:

const write = (document, batch) => {
   const doc = getDocFromPath(key);
   batch.set(doc, value);
};

await runFunctionInBatchesOf500(write, dataArray);

This all works but I think I am missing something. Is this an efficient use of higher order functions? What would a more elegant, FP-style solution be?

Answer 1

From a short review;

• Why hardcode the batch length to 500 ?
• Why not have the batch length be a nice constant ?
• You have even hard coded the length in the function name, which is really unfortunate
• batchNumber++ is more canonical than batchNumber += 1
• I would have gone for maxIndex = Math.min(startingIndex + 500, dataArray.length); because now you have a lot of calls to func with undefined as a document value
• await requires runFunctionInBatchesOf500 to be async (it is missing now)
• I would use Array.prototype.slice() to create batches as an array, and then use forEach on each slice/batch
• const doc = getDocFromPath(key); <- where does key come from, an evil global?

I personally would be mildly evil by adjusting the Array prototype so that I can keep chaining, FP style;

Array.prototype.mapSlice = function arrrayMapSlice(n){
  //Just return `this` if we get a non-sensical parameter
  if(isNaN(n) || n <= 0){
    return this;
  }
  let start = 0, out = [];
  while(start < this.length){
    out.push(this.slice(start, start+=n));
  }
  return out;
} 

async function runBatches(list, f, batchSize){
   batchSize = batchSize || 500;
   list.mapSlice(batchSize).forEach(batch => {
     const firestoreBatch = this.firestore.batch();
     batch.forEach(document => f(document, firestoreBatch ));
   });
   await batch.commit();
}