# “Real world” JSON transformation using Node.js

Background

I transformed a non-trivial JSON response that uses arrays into

• a nested object
• eliminating some of the original properties
• inserting a new summary property that is a unique list of values from the original object

I did this with temporary variables, foreach loops and some conditional logic. My first working code, sample input / output JSON are below.

Question / Concern

While this works, I'm concerned about the "smell" of the code and the maintainability. Is there a way to do this that would be more maintainable (perhaps with Array. ..., map, filter, reduce) ? I'm not looking for a working solution but more than "one liners".

CODE

const raw = require('../tests/data/sample-input.json');

const types = new Set();

function parseLoc(loc) {
// constant properties with loc_id the name of the object
let o = { [loc.loc_id]: { last_update: loc.last_update, type_id: loc.type_id } };
// optional property
if (loc.contents) {
o = { [loc.loc_id]: { ...o[loc.loc_id], contents: loc.contents } };
loc.contents.forEach((item) => {
types.add(item.type_id); // can have nested types
});
}
// optional property
if (loc.plan_id) {
o = { [loc.loc_id]: { ...o[loc.loc_id], plan_id: loc.plan_id } };
}
// summary properties
o = {
[loc.loc_id]: {
...o[loc.loc_id],
},
};
return o;
}

const driver = () => {
const { locs } = raw[Object.keys(raw)[0]];

let out = {};
locs.forEach((loc) => {
const t = parseLoc(loc);
out = { ...out, [Object.keys(t)[0]]: t[Object.keys(t)[0]] };
});
console.log('types', ...types);
out = { ...out, Types: [...types] };
console.log('out', JSON.stringify(out));
};

driver();


Input

   {
"Input": {
"locs": [
{
"contents": [{ "amount": 1, "type_id": 2393 }],
"last_update": "2013-09-22T21:53:51Z",
"obsolete1": 1.52384062551,
"obsolete2": 1.56361060962,
"loc_id": 1011160470678,
"type_id": 2524
},
{
"last_update": "2019-07-29T10:56:27Z",
"obsolete1": 1.60921860432,
"obsolete2": 1.60545414964,
"loc_id": 1028580652821,
"plan_id": 97,
"type_id": 2474
},
{
"contents": [
{ "amount": 560, "type_id": 2393 },
{ "amount": 560, "type_id": 9838 },
{ "amount": 560, "type_id": 2317 },
],
"last_update": "2019-02-28T22:09:51Z",
"obsolete1": 1.55924075537,
"obsolete2": 1.58171860958,
"loc_id": 1029669382563,
"type_id": 2544
}
]
}
}


OUTPUT

{
"1011160470678": {
"last_update": "2013-09-22T21:53:51Z",
"type_id": 2524,
"contents": [{ "amount": 1, "type_id": 2393 }]
},
"1028580652821": {
"last_update": "2019-07-29T10:56:27Z",
"type_id": 2474,
"plan_id": 97
},
"1029669382563": {
"last_update": "2019-02-28T22:09:51Z",
"type_id": 2544,
"contents": [
{ "amount": 560, "type_id": 2393 },
{ "amount": 560, "type_id": 9838 },
{ "amount": 560, "type_id": 2317 }
]
},
"Types": [2524, 2393, 2474, 2544, 9838, 2317]
}

• I noticed that the type_id of 2317 is missing from the output. Also, would you consider data normalization (i.e. undefined and empty array values) in the output? – morbusg Aug 10 '19 at 20:41
• The output doesn't match the output from your code. Please fix it. – Slai Aug 11 '19 at 11:11
• morbusg / slai - Thanks for the review. I edited to ensure the code and output are from the same version. – Timothy Vogel Aug 11 '19 at 20:17

## Maintenance by design

There are many times when I see this type of question regarding maintainability of the code. The line between code maintenance and user friendly becomes blurred. This is most evident when the coder is also the end user, the interface UI is misunderstood by familiarity of use, it is of course the IDE. This may sound somewhat pedantic, but the mind set it helps introduce can go a long way into how you design your code.

## Encapsulate the interface

In this example if we look at the task of maintenance as an end user task, we design the code with all the possible changes encapsulated in a simple structure that isolates the code from the user. (Users can't code so don't let them near it, especially those users than think they can code (I am my most difficult client))

If done well the maintenance is trivial and the meat of the logic safe from the corruption inevitable when the logic must be reacquired in some maybe distant future.

So we can create am object called rules. It is the interface (abstract representation of codes functionality), and as with all good UI it will require some help information in the form of comments (remember the interface is the IDE)

As I have little to no clue what the possible alterations may be I have had to guess, this interface may not be adaptable to your needs. It is an example only

const rules = {
name: "Outputs",     // name of transformed object
typesName: "types",  // name of type ids array eg {Outputs: types: [1,4,23] }
itemsName: "locs",   // name of items object in and out  eg {Outputs: locs: {} }
keep: ["contents", "last_update", "plan_id", "type_id"],  // List of properties
// to keep per item
itemName: "loc_id",  // name of property to use as named item
ids: ["type_id"],    // list of property names containing ids for the array of types
};


The code can then use this encapsulated information as input, doing its thing and spitting out the result. Many changes can be made by just changing some text and never going near the logic.

function transform(data, rules) {
const items = {}, types = new Set();
const addTypes = from => rules.ids.forEach(key =>
const addProps = from => rules.keep.reduce((obj, key) =>
from[key] !== undefined ? (obj[key] = from[key], obj) : obj, {});
const transformLoc = loc => {
const lName = loc[rules.itemName];
if (lName !== undefined) {
}
}
data[rules.itemsName].forEach(transformLoc);
return {
[rules.name]: {
[rules.typesName]: [...types],
[rules.itemsName]: items
}
};
}

const transformed = transform(data.Input, rules);


Even if we ignore this UI/code abstraction, the resulting code is more maintainable as you have moved many of the magic constants out of the code, and have been forced to think of what you are doing in a higher level abstract, that helps any future coders wrap their thinking nog around the logic.

• Thank you for the thoughtful review and reply! Your observation about context (user interface / expertise) is spot on. I will review your suggestions to see how to incorporate them in both this code snippet and elsewhere. – Timothy Vogel Aug 11 '19 at 20:19