Collecting the path (code)walker took

[lvh]

I have a nested data structure. Somewhere in the nested data structure, there are vectors with maps in them. (There are also vectors with not-maps in them, and I don't want those.) Those maps all have some primary key identifier; I am trying to turn the vector into a map, indexed by that identifier.

The vec-of-maps constraint seemed easiest to pour into a walker predicate: (sr/walker #(and (vector? %) (map? (first %)))). However, I need the path to each matched data structure in order to do the indexing correctly. Is there a way to get that out of walker, or do I need a totally different selector?

[nathanmarz]

To collect the series of keys leading to each value, you'll need to make your own path that collects values along the way. For an example with a tree represented using vectors, it would look like:

(def PathTreeValues
  (recursive-path [] p
    (if-path vector?
      [(view #(map-indexed vector %)) ALL (collect-one FIRST) LAST p]
      STAY
      )))

(select PathTreeValues [:a :b [:c]])
;; => [[0 :a] [1 :b] [2 0 :c]]

[lvh]

Thanks!

I came up with:

(def vecs-of-maps-with-path
  (sr/recursive-path
   [] p
   (sr/cond-path
    #(and (vector? %) (map? (first %))) sr/STAY
    map? [(sr/collect-one sr/MAP-KEYS) sr/MAP-VALS p]
    vector? [(sr/view #(map-indexed vector %)) sr/ALL (sr/collect-one sr/FIRST) sr/LAST p]
    sr/STAY sr/STAY)))

(sr/transform
 vecs-of-maps-with-path
 (fn [& args]
   (let [path (butlast args)
         val (last args)
         key (-> val first keys first)] ;; imagine this actually depends on path
     (into {} (map (juxt key identity) val))))
 {:a [{:b 1} {:b 2} {:b 3}]})

(sr/transform
 vecs-of-maps-with-path
 (fn [& args]
   (let [path (spy (butlast args))
         val (last args)
         key (-> val first keys first)] ;; imagine this actually depends on path
     (into {} (map (juxt key identity) val))))
 {:some {:prefix {:a [{:b 1} {:b 2} {:b 3}]}}})

Is that somewhat idiomatic? Is there some preference that might lead to increased composability? (#(and (vector? %) (map? (first %))) is quite specific). I'd wrap it in a function that takes that pred and returns the recursive-path, but I don't know if there's a cleverer thing specter can do.

(also somewhat unrelated: I noticed you use CamelCase for this recursive-path. is there a distinction between, say ALL and PathTreeValues I'm missing?)

Thanks!

[nathanmarz]

I would do it like this:

(def vecs-of-maps-with-path
  (sr/recursive-path [term-pred] p
   (sr/cond-path
    (pred term-pred) sr/STAY
    map? [ALL (sr/collect-one sr/FIRST) sr/LAST p]
    vector? [(sr/view #(map-indexed vector %)) sr/ALL (sr/collect-one sr/FIRST) sr/LAST p]
    sr/STAY sr/STAY)))

Your map case wasn't collecting the key properly, and you can define the recursive path to take in a parameter for the termination predicate. You then use vecs-of-maps-with-path as a function that returns a path.

Also, I'm not sure about your last case. This path will navigate to any non-map/vector value that doesn't match the termination predicate. If you don't want it to do that, just remove the last case.

Normally I put global paths as all-uppercase, and functions that return paths use same conventions as normal Clojure functions. No particular reason why I did camel case for my example.

[lvh]

Thanks, that worked a charm :) Complete example for posterity:

(def path-walker-orig
  (sr/recursive-path
   [term-pred] p
   (sr/cond-path
    (sr/pred term-pred) sr/STAY
    map? [sr/ALL (sr/collect-one sr/FIRST) sr/LAST p]
    vector? [(sr/view #(map-indexed vector %)) sr/ALL (sr/collect-one sr/FIRST) sr/LAST p]
    sr/STAY sr/STAY)))


(sr/transform
 (path-walker-orig #(and (vector? %) (map? (first %))))
 (fn [& args]
   (let [path (spy (butlast args))
         val (last args)
         key (-> val first keys first)] ;; imagine this actually depends on path
     (into {} (map (juxt key identity) val))))
 {:some {:prefix {:a [{:b 1} {:b 2} {:b 3}]}}})

I tried to factor out some of the paths so that they'd be easier to understand for people that are still learning specter (myself included), and got to:

(def indexed
  "A path that visits v and collects k in [[k v], ...]."
  [sr/ALL (sr/collect-one sr/FIRST) sr/LAST])

(def indexed-seq
  "A selector that visits all elements of a seq, and collects their indices."
  (sr/comp-paths (sr/view #(map-indexed vector %)) indexed))

(def path-walker
  (sr/recursive-path
   [term-pred] p
   (sr/cond-path
    (sr/pred term-pred) sr/STAY
    map? (sr/comp-paths indexed p)
    vector? (sr/comp-paths indexed-seq p))))

However, running that gets me:

1. Unhandled java.lang.IllegalArgumentException
   Cannot statically combine sequential when not in nav pos

                 impl.cljc:  700  com.rpl.specter.impl$static_combine/invokeStatic
                 impl.cljc:  693  com.rpl.specter.impl$static_combine/invoke
                 impl.cljc:  712  com.rpl.specter.impl$static_combine$fn__8637/invoke
                  core.clj: 2726  clojure.core/map/fn
              LazySeq.java:   40  clojure.lang.LazySeq/sval
              LazySeq.java:   49  clojure.lang.LazySeq/seq
                   RT.java:  525  clojure.lang.RT/seq
                  core.clj:  137  clojure.core/seq
                  core.clj: 3106  clojure.core/dorun
                  core.clj: 3121  clojure.core/doall
 ...

I searched the issue tracker and I can't tell if this is a bug or something I messed up.

[nathanmarz]

comp-paths shouldn't be used within a navigator definition. You should remove that from path-walker. It's also better to use comp-paths for the definition of indexed, although for this case it doesn't really matter. It would matter in cases where you wanted to return indexed from a function somewhere and use it so that you don't pay the cost of compilation over and over.

[lvh]

Gotcha, thanks. For future reference for anyone who finds this ticket: comp-paths isn't what you want, (conj somepath p) also doesn't work -- what you're looking for is [thatpath p], e.g.:

(def INDEXED
  "A path that visits v and collects k in [[k v], ...].

  This is useful if you want to collect a path to something, see [[path-walker]]."
  [sr/ALL (sr/collect-one sr/FIRST) sr/LAST])

(def INDEXED-SEQ
  "A selector that visits all elements of a seq, and collects their indices.

  This is useful if you want to collect a path to something, see [[path-walker]]."
  [(sr/view #(map-indexed vector %)) INDEXED])

(def path-walker
  (sr/recursive-path
   [term-pred] p
   (sr/cond-path
    (sr/pred term-pred) sr/STAY
    map? [INDEXED p]
    vector? [INDEXED-SEQ p])))

I'm guessing if I want to understand why that is, I'm going to go read the source for cond-path :)