2
\$\begingroup\$

I am looking for a way to transform JSON data into a flat "csv-like" data object. In some way, I am looking to "sqlirize" a mongodb collection. I have already check some json flat libraries in NPM but non of them quite solve my problem. I have solved it in my own way but wanted to know if there is a more efficient way.

I have a collection that presents the data through an API in the following way:

[{
    "data": {
        "name": "John",
        "age": 23,
        "friends": [{
            "name": "Arya",
            "age": 18,
            "gender": "female"
        }, {
            "name": "Sansa",
            "age": 20,
            "gender": "female"
        }, {
            "name": "Bran",
            "age": 17,
            "gender": "male"
        }]
    }
}, {
    "data": {
        "name": "Daenerys",
        "age": 24,
        "friends": [{
            "name": "Grey Worm",
            "age": 20,
            "gender": "male"
        }, {
            "name": "Missandei",
            "age": 17,
            "gender": "female"
        }]
    }
}]

This is the function that I have created to reflat a safe-flattened json (e.i.: everything is flattened except arrays)

const { cloneDeep } = require('lodash')
const flatten = require('flat')

const reflatten = (items) => {
  const reflatted = []

  items.forEach(item => {
    let array = false

    for (const key of Object.keys(item)) {
      if (Array.isArray(item[key])) {
        array = true

        const children = Array(item[key].length).fill().map(() => cloneDeep(item))

        for (let i = 0; i < children.length; i++) {
          const keys = Object.keys(children[i][key][i])

          keys.forEach(k => {
            children[i][`${key}.${k}`] = children[i][key][i][k]
          })
          delete children[i][key]
          reflatted.push(children[i])
        }
        break
      }
    }
    if (!array) {
      reflatted.push(item)
    }
  })

  return reflatted.length === items.length
    ? reflatted
    : reflatten(reflatted)
}

const rows = []

for (const item of items) {
  const flat = [flatten(item)]

  rows.push(...reflatten(flat)]
}

console.log(rows)

The expected (and current) output is the following:

[{
    "data.name": "John",
    "data.age": 23,
    "data.friends.name": "Arya",
    "data.friends.age": 18,
    "data.friends.gender": "female"
}, {
    "data.name": "John",
    "data.age": 23,
    "data.friends.name": "Sansa",
    "data.friends.age": 20,
    "data.friends.gender": "female"
}, {
    "data.name": "John",
    "data.age": 23,
    "data.friends.name": "Bran",
    "data.friends.age": 17,
    "data.friends.gender": "male"
}, {
    "data.name": "Daenerys",
    "data.age": 24,
    "data.friends.name": "Grey Worm",
    "data.friends.age": 20,
    "data.friends.gender": "male"
}, {
    "data.name": "Daenerys",
    "data.age": 24,
    "data.friends.name": "Missandei",
    "data.friends.age": 17,
    "data.friends.gender": "female"
}]

Although I achieved the expected output, I keep wondering if there are other libraries there or if there is a more efficient way of doing it.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.