13
\$\begingroup\$

Context

I have a bunch of data points that look roughly like this:

(defn rand-key []
  (into [] (repeatedly 3 #(- (rand-int 19) 9))))

(defn rand-val []
  (rand-nth [:foo :bar :baz :qux]))

(def data (into {} (repeatedly 10 (fn [] [(rand-key) (rand-val)]))))

Obviously the actual coordinates would have a much greater range, the actual values would hold some sort of information, and the actual number of data points would be far greater, but you get the idea.

This data structure is going to be evolving over time (so I'll probably store it in an atom or something like that), and as it evolves, I want to be able to quickly find the bounding box for the points it contains in its current state.

The easiest way to do that for an n-dimensional dataset is to keep n independent sets of its keys, each one sorted in one dimension. Since I can't be bothered to write a proper comparator in Clojure, though, I'm going to replace each of those sets with a sorted map from a coordinate in a particular dimension to the number of data points that have that coordinate in that dimension:

(def keymaps (mapv (fn [dimension]
                     (->> (keys data)
                          (map #(get % dimension))
                          frequencies
                          (into (sorted-map))))
                   (range 3)))

With this, it's trivial to find the bounding box of the dataset:

(def bounds (mapv (juxt (comp key first) (comp key first rseq)) keymaps))

Problem

Now, no matter what I do, I have to update my data and my keymaps together. Maybe, like I suggested above, I have an atom that stores some current state, and in that case that atom would look like this:

(def state (atom {:data data :keymaps keymaps}))

Any time I update state, I can't just use the built-in Clojure functions to update the :data because that would cause the :keymaps to become outdated. I could write my own functions to replace assoc and dissoc that would keep the two in sync, but then I wouldn't be able to make use of all the great higher-level functions (such as merge) that are built on top of the original, polymorphic assoc and dissoc.

Solution

So I decided to take the most painful approach possible: build a custom map type that allows efficient bounding box queries using the scheme detailed above. First things first, some protocols:

(defprotocol Space
  (dimension [this]))

(defprotocol Bounded
  (bounds [this]))

Since I need to store extra data alongside an existing map, I can't just use extend-type on the PersistentHashMap class. No, I need a far more powerful tool: deftype! I'll have one field for the underlying data map and another for the vector of keymaps.

In most cases, I could get away with just implementing the map abstraction directly, but I don't want to tie myself to a particular implementation for the wrapped map. What if later I decide that I want to use a quadtree or an octree for the underlying map, to allow for efficient queries of subspaces? To prevent myself from having to extend more protocols to my bounded map type than I need to, I'll just provide a way to get the wrapped map and query that:

(defprotocol Wrapper
  (wrapped [this]))

And here's the deftype itself. Prepare for boilerplate:

(import (clojure.lang Associative
                      Counted
                      ILookup
                      IPersistentCollection
                      IPersistentMap
                      Seqable))

(declare ->BoundedMap)

(deftype BoundedMap [m keymaps]
  Seqable
  (seq [_] (seq m))

  IPersistentCollection
  (cons [this [k v]] (assoc this k v))
  (empty [_] (->BoundedMap (empty m) (mapv empty keymaps)))
  (equiv [_ x] (= m x))

  ILookup
  (valAt [_ k] (get m k))
  (valAt [_ k not-found] (get m k not-found))

  Associative
  (containsKey [_ k] (contains? m k))
  (entryAt [_ k] (find m k))

  Counted
  (count [_] (count m))

  IPersistentMap
  (assoc [_ k v]
    (->BoundedMap (assoc m k v)
                  (if (contains? m k)
                    keymaps
                    (mapv #(update %1 %2 (fnil inc 0)) keymaps k))))
  (without [_ k]
    (->BoundedMap (dissoc m k)
                  (if (contains? m k)
                    (mapv #(if (< 1 (get %1 %2))
                             (update %1 %2 dec)
                             (dissoc %1 %2))
                          keymaps k)
                    keymaps)))

  Wrapper
  (wrapped [_] m)

  Space
  (dimension [_] (count keymaps))

  Bounded
  (bounds [_]
    (mapv (fn [keymap] (mapv #(key (first (% keymap))) [seq rseq])) keymaps)))

And of course, what data structure would be complete without a nifty little constructor function?

(defn bounded-map [dimension m]
  (->> (keys m)
       (iterate (partial map rest))
       (take dimension)
       (mapv #(into (sorted-map) (frequencies (map first %))))
       (->BoundedMap m)))

Now I can define my atom like this instead of what I had before:

(def state (atom (bounded-map 3 data)))

Pretty much all of the Clojure tools for maps will work properly, I can get bounding boxes at my leisure, and if I ever switch to a more featured map implementation for the data itself, I can always compose those extra functions with wrapped.

Questions

  • Is this the right approach?
  • Is there a way to make = work properly without implementing java.util.Map (yuck)?
  • Can this code be improved in any other way?
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.