2
\$\begingroup\$

Today I spent quite some time on providing a solution for a Stack Overflow answer. It looked quite easy to me, but my solution is complex.

The question can be found here and this is my answer.

If I understood the question correctly it came down to obtaining all values for keyWeWants from a data structure like this:

{
    "agg": {
        "agg1": [
            {
                "keyWeWant": "*-20.0",
                "asdf": 0,
                "asdf": 20,
                "asdf": 14,
                "some_nested_agg": [
                    {
                        "keyWeWant2": 20,
                        "to": 25,
                        "doc_count": 4,
                        "some_nested_agg2": {
                            "count": 7,
                            "min": 2,
                            "max": 5,
                            "keyWeWant3": 2.857142857142857,
                            "sum": 20
                        }
                    },
                    {
                        "keyWeWant2": 25,
                        "to": 30,
                        "doc_count": 10,
                        "some_nested_agg2": {
                            "count": 16,
                            "min": 2,
                            "max": 10,
                            "keyWeWant3": 6.375,
                            "sum": 102
                        }
                    }]
            }]}
}

The parsed structure should look like this:

[
   {
      "keyWeWant" : "*-20",
      "keyWeWant2" : 20,
      "keyWeWant3" : 2.857142857142857
   },
   {
      "keyWeWant" : "*-20",
      "keyWeWant2" : 25,
      "keyWeWant3" : 6.375
   },
   {
   ...
   },
   {
   ...
   }
]

In the question it is requested that the function looks something like this:

function_name(data_map, {
   "keyWeWant" : ['agg', 'agg1'],
   "keyWeWant2" : ['agg', 'agg1', 'some_nested_agg'],
   "keyWeWant" : ['agg', 'agg1', 'some_nested_agg', 'some_nested_agg2']
})

I took the challenge to solve it exactly with the data structure provided in the question. This was probably not the best approach and I don't think my solution is optimal.

Here's my solution (selectively copied from the answer):

I placed this test data in a file called data.json. Then Cheshire JSON library parses the data to a Clojure data structure:

(use '[cheshire.core :as cheshire])

(def my-data (-> "data.json" slurp cheshire/parse-string))

Next the paths to get are defined as follows:

(def my-data-map
  {"keyWeWant"  ["agg", "agg1"],
   "keyWeWant2" ["agg", "agg1", "some_nested_agg"],
   "keyWeWant3" ["agg", "agg1", "some_nested_agg", "some_nested_agg2"]})

It is the data_map of the question without ":", single quotes changed to double quotes and the last "keyWeWant" changed to "keyWeWant3".

find-nested below has the semantics of Clojure's get-in, only then it works on maps with vectors, and returns all values instead of one. When find-nested is given a search vector it finds all values in a nested map where some values can consist of a vector with a list of maps. Every map in the vector is checked.

(defn find-nested
  "Finds all values in a coll consisting of maps and vectors.

  All values are returned in a tree structure:
  i.e, in your problem it returns (20 25) if you call it with
  (find-nested ['agg', 'agg1', 'some_nested_agg', 'keyWeWant2'] 
  my-data).

  Returns nil if not found."
  [ks c]
  (let [k (first ks)]
    (cond (nil? k)    c
          (map? c)    (find-nested (rest ks) (get c k))
          (vector? c) (if-let [e (-> c first (get k))]
                        (if (string? e) e ; do not map over chars in str
                            (map (partial find-nested (rest ks)) e))
                        (find-nested ks (into [] (rest c)))) ; create vec again
          :else       nil)))

find-nested finds the values for a search path:

(find-nested ["agg", "agg1", "some_nested_agg", "keyWeWant2"] my-data) 
; => (20 25)

If all the paths towards the "keyWeWant's are mapped over my-data these are the slices of a tree:

(*-20.0
(20 25)
(2.857142857142857 6.375))

The requested structure for (all end results with paths getting there) can be obtained from this tree in function-name like this:

(defn function-name
  "Transforms data d by finding (nested keys) via data-map m in d and 
  flattening the structure."
  [d m]
  (let [tree               (map #(find-nested (conj (second %) (first %)) d) m)
        leaves             (last tree)
        leaf-indices       (range (count leaves))
        results            (for [index leaf-indices]
                             (map (fn [slice]
                                    (if (string? slice)
                                      slice
                                      (loop [node (nth slice index)]
                                        (if node
                                          node
                                          (recur (nth slice (dec index)))))))
                                  tree))
        results-with-paths (mapv #(zipmap (keys m) %) results)
        json               (cheshire/encode results-with-paths)]
    json))

results uses a loop to step back if a leaf-index is larger than that particular slice. I think it will work out for deeper nested structures as well -if a next slice is always double the size of a previous slice or the same size it should work out -, but I have not tested it.

Calling (function-name my-data my-data-map) leads to a JSON string in your requested format:

[{
     "keyWeWant": "-20.0",
     "keyWeWant2": 20,
     "keyWeWant3": 2.857142857142857 }
 {
     "keyWeWant": "
-20.0",
     "keyWeWant2" 25,
     "keyWeWant3" 6.375 }]

To improve my Clojure code (e.g., succinctness) I would love to be pointed towards better solutions.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.