Today I spent quite some time on providing a solution for a Stack Overflow answer. It looked quite easy to me, but my solution is complex.

The question can be found here and this is my answer.

If I understood the question correctly it came down to obtaining all values for keyWeWants from a data structure like this:

    "agg": {
        "agg1": [
                "keyWeWant": "*-20.0",
                "asdf": 0,
                "asdf": 20,
                "asdf": 14,
                "some_nested_agg": [
                        "keyWeWant2": 20,
                        "to": 25,
                        "doc_count": 4,
                        "some_nested_agg2": {
                            "count": 7,
                            "min": 2,
                            "max": 5,
                            "keyWeWant3": 2.857142857142857,
                            "sum": 20
                        "keyWeWant2": 25,
                        "to": 30,
                        "doc_count": 10,
                        "some_nested_agg2": {
                            "count": 16,
                            "min": 2,
                            "max": 10,
                            "keyWeWant3": 6.375,
                            "sum": 102

The parsed structure should look like this:

      "keyWeWant" : "*-20",
      "keyWeWant2" : 20,
      "keyWeWant3" : 2.857142857142857
      "keyWeWant" : "*-20",
      "keyWeWant2" : 25,
      "keyWeWant3" : 6.375

In the question it is requested that the function looks something like this:

function_name(data_map, {
   "keyWeWant" : ['agg', 'agg1'],
   "keyWeWant2" : ['agg', 'agg1', 'some_nested_agg'],
   "keyWeWant" : ['agg', 'agg1', 'some_nested_agg', 'some_nested_agg2']

I took the challenge to solve it exactly with the data structure provided in the question. This was probably not the best approach and I don't think my solution is optimal.

Here's my solution (selectively copied from the answer):

I placed this test data in a file called data.json. Then Cheshire JSON library parses the data to a Clojure data structure:

(use '[cheshire.core :as cheshire])

(def my-data (-> "data.json" slurp cheshire/parse-string))

Next the paths to get are defined as follows:

(def my-data-map
  {"keyWeWant"  ["agg", "agg1"],
   "keyWeWant2" ["agg", "agg1", "some_nested_agg"],
   "keyWeWant3" ["agg", "agg1", "some_nested_agg", "some_nested_agg2"]})

It is the data_map of the question without ":", single quotes changed to double quotes and the last "keyWeWant" changed to "keyWeWant3".

find-nested below has the semantics of Clojure's get-in, only then it works on maps with vectors, and returns all values instead of one. When find-nested is given a search vector it finds all values in a nested map where some values can consist of a vector with a list of maps. Every map in the vector is checked.

(defn find-nested
  "Finds all values in a coll consisting of maps and vectors.

  All values are returned in a tree structure:
  i.e, in your problem it returns (20 25) if you call it with
  (find-nested ['agg', 'agg1', 'some_nested_agg', 'keyWeWant2'] 

  Returns nil if not found."
  [ks c]
  (let [k (first ks)]
    (cond (nil? k)    c
          (map? c)    (find-nested (rest ks) (get c k))
          (vector? c) (if-let [e (-> c first (get k))]
                        (if (string? e) e ; do not map over chars in str
                            (map (partial find-nested (rest ks)) e))
                        (find-nested ks (into [] (rest c)))) ; create vec again
          :else       nil)))

find-nested finds the values for a search path:

(find-nested ["agg", "agg1", "some_nested_agg", "keyWeWant2"] my-data) 
; => (20 25)

If all the paths towards the "keyWeWant's are mapped over my-data these are the slices of a tree:

(20 25)
(2.857142857142857 6.375))

The requested structure for (all end results with paths getting there) can be obtained from this tree in function-name like this:

(defn function-name
  "Transforms data d by finding (nested keys) via data-map m in d and 
  flattening the structure."
  [d m]
  (let [tree               (map #(find-nested (conj (second %) (first %)) d) m)
        leaves             (last tree)
        leaf-indices       (range (count leaves))
        results            (for [index leaf-indices]
                             (map (fn [slice]
                                    (if (string? slice)
                                      (loop [node (nth slice index)]
                                        (if node
                                          (recur (nth slice (dec index)))))))
        results-with-paths (mapv #(zipmap (keys m) %) results)
        json               (cheshire/encode results-with-paths)]

results uses a loop to step back if a leaf-index is larger than that particular slice. I think it will work out for deeper nested structures as well -if a next slice is always double the size of a previous slice or the same size it should work out -, but I have not tested it.

Calling (function-name my-data my-data-map) leads to a JSON string in your requested format:

     "keyWeWant": "-20.0",
     "keyWeWant2": 20,
     "keyWeWant3": 2.857142857142857 }
     "keyWeWant": "
     "keyWeWant2" 25,
     "keyWeWant3" 6.375 }]

To improve my Clojure code (e.g., succinctness) I would love to be pointed towards better solutions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.