Skip to content

Polymorphic Descriptions

Jump to bottom
zalky edited this page Apr 27, 2023 · 5 revisions

EQL, GraphQL, and their variants provide a level of query polymorphism through a feature called union queries. Reflet introduces a new variant of union queries called descriptions to address some of the challenges with union queries, as well as realize additional benefits. Reflet's descriptions are:

  1. More expressive, extensible, and re-usable
  2. More suitable for complex data
  3. More data-driven
  4. Easier to integrate with remote ontology, schema, or knowledge representation layers

This document covers the following:

If you are not already familiar with declarative pull queries, you might consider reading the EQL spec or the Reflet [[Graph Queries]] document first.

If you are already familiar with union queries and their pain points, you could skip right to the section on Polymorphic Descriptions.

What Are Union Queries?

Often either the shape of your graph data or the taxonomies that describe your entities are heterogeneous. A very simple example might be a timeline of events:

[{:system/uuid #uuid "A"
  :kr/type     :kr.type/conference
  :kr/name     "Clojure Conj 2023"
  :kr/about    "A conference for Clojure developers, companies and enthusiasts."
  :kr/date     "04/27/2023"}
 {:system/uuid    #uuid "B"
  :kr/type        :kr.type/lecture
  :kr/name        "The Unanswered Question"
  :kr/description "A series of lectures given by Leonard Bernstein in the fall of 1973."
  :kr/institution "Harvard University"
  :kr/date        "09/01/1973"}
 {:system/uuid #uuid "C"
  :kr/type     :kr.type/performance
  :kr/name     "Quirky"
  :kr/location "Brixton"
  :kr/artist   "Autechre"
  :kr/date     "16/06/1994"}]

Each entity in this list has a different type and set of attributes. A naive pull query that works for each of these entities could take the union of all possible attributes from each entity, maybe something like:

[:kr/name
 :kr/about
 :kr/date
 :kr/description
 :kr/institution
 :kr/date
 :kr/location
 :kr/artist]

But this would not scale in the general case. A polymorphic solution would be better. Enter union queries:

{:kr.type/conference  [:kr/name
                       :kr/about
                       :kr/date]
 :kr.type/lecture     [:kr/name
                       :kr/description
                       :kr/institution
                       :kr/date]
 :kr.type/performance [:kr/name
                       :kr/location
                       :kr/artist
                       :kr/date]}

As the query parsing engine traverses events in the graph, it can use this spec to choose the right attribute set for each event based on the event type. Its usage in an actual query would look something like:

(f/reg-pull ::timeline-component-query
  (fn [ref]
    [[:cmp/uuid
      :cmp/other-state
      {:cmp/events {:kr.type/conference  [:kr/name
                                          :kr/about
                                          :kr/date]
                    :kr.type/lecture     [:kr/name
                                          :kr/description
                                          :kr/institution
                                          :kr/date]
                    :kr.type/performance [:kr/name
                                          :kr/location
                                          :kr/artist
                                          :kr/date]}}]
     ref]))

Here, the :cmp/events attribute is being used to store a heterogeneous list of events that need to be rendered on screen in a timeline component. The union query is used as the sub-query in the :cmp/events join.

If the data being queried is more nested, union queries can themselves use joins:

{:kr.type/conference [:kr/name
                      :kr/about
                      {:kr/time {:kr.type/interval [:kr.time/start :kr.time/end]
                                 :kr.type/date     [:kr.time/date]}}]}

However, as the shape of your data and taxonomies become more complex:

[{:system/uuid #uuid "A"
  :kr/type     :kr.type/conference
  :kr/name     "Clojure Conj 2023"
  :kr/about    "A conference for Clojure developers, companies and enthusiasts."
  :kr/time     {:system/uuid   #uuid "D"
                :kr/type       :kr.type/interval
                :kr.time/start "04/27/2023"
                :kr.time/end   "04/28/2023"}}
 {:system/uuid    #uuid "B"
  :kr/type        :kr.type/lecture
  :kr/name        "The Unanswered Question"
  :kr/description "A series of lectures given by Leonard Bernstein in the fall of 1973."
  :kr/institution "Harvard University"
  :kr/time        {:system/uuid   #uuid "E"
                   :kr/type       :kr.type/interval
                   :kr.time/start "09/01/1973"
                   :kr.time/end   "12/01/1973"}}
 {:system/uuid #uuid "C"
  :kr/type     :kr.type/performance
  :kr/name     "Quirky"
  :kr/location "Brixton"
  :kr/artist   {:system/uuid #uuid "F"
                :kr/name     "Autechre"}
  :kr/time     {:system/uuid  #uuid "G"
                :kr/type      :kr.type/date
                :kr.time/date "16/06/1994"}}]

So too do your union query definitions:

{:kr.type/conference  [:kr/name
                       :kr/about
                       {:kr/time {:kr.type/interval [:kr.time/start :kr.time/end]
                                  :kr.type/date     [:kr.time/date]}}]
 :kr.type/lecture     [:kr/name
                       :kr/description
                       :kr/institution
                       {:kr/time {:kr.type/interval [:kr.time/start :kr.time/end]
                                  :kr.type/date     [:kr.time/date]}}]
 :kr.type/performance [:kr/name
                       :kr/location
                       {:kr/artist [:kr/name]}
                       {:kr/time {:kr.type/interval [:kr.time/start :kr.time/end]
                                  :kr.type/date     [:kr.time/date]}}]}

Even in this relatively simple example, you can see how query re-use quickly becomes a problem. Of course the queries could be imperatively constructed using code, but then you lose the benefit of declarative query specs.

Polymorphic Descriptions

Reflet replaces union queries with polymorphic descriptions:

(require '[reflet.core :as f])

(f/reg-desc [::timeline :kr.type/date]
  [:kr.time/date])

(f/reg-desc [::timeline :kr.type/interval]
  [:kr.time/start
   :kr.time/end])

(f/reg-desc [::timeline :kr.type/conference]
  [:kr/name
   :kr/about
   {:kr/time ::timeline}])

(f/reg-desc [::timeline :kr.type/lecture]
  [:kr/name
   :kr/description
   :kr/institution
   {:kr/time ::timeline}])

(f/reg-desc [::timeline :kr.type/performance]
  [:kr/name
   :kr/location
   {kr/artist [:kr/name :kr/socials]}
   {:kr/time ::timeline}])

Now the spec for how a particular type is described is defined outside of the queries in which the spec will participate. Notice that unlike a reg-pull query, there is no entity reference here, because these are not queries in the same sense. They are descriptions of types of things, not a query about a specific thing.

Also notice that each description has two components:

  1. A unique tuple id: consisting of a context and a type

  2. A declarative data spec that conforms to exactly the same grammar as reg-pull: the entire pull syntax is available, but now joins can reference contexts (discussed next)

Let's take a closer look at contexts.

Contexts

Descriptions are defined with respect to a context and type:

              context           type
                 V               V
(f/reg-desc [::timeline :kr.type/conference]
  [:kr/name
   :kr/about
   {:kr/time ::timeline}])

Contexts enable two things:

  1. Descriptions can refer to other descriptions in their joins using a context:

    When the query parsing engine traverses a join like {:kr/time ::timeline} and sees a context keyword, it will use that context to continue describing the nested data. This makes descriptions easily re-usable, while preserving their declarative properties.

  2. A type can have different descriptions in different contexts:

    For example, if you need a new description for a :kr.type/conference entity in an ::info-bar context, you can define one like so:

    (f/reg-desc [::info-bar :kr.type/conference]
  [:kr/name
   :kr/location
   {:kr/organizers ::info-bar}
   {:kr/time ::timeline}])

Finally note that you can define a :default description to use whenever an entity fails to match all other descriptions:

(f/reg-desc :default
  [:kr/name])

Using Descriptions in Queries

The above description definitions will not do much until you start using them in queries. There are two ways to do this.

reflet.core/desc

First, you can directly ask to describe an entity in a specific context using its entity reference:

@(f/desc [::timeline [:system/uuid #uuid "A"]])
=>
{:kr/name  "Clojure Conj 2023"
 :kr/about "A conference for Clojure developers, companies and enthusiasts."
 :kr/time  {:kr.time/start "04/27/2023"
            :kr.time/end   "04/28/2023"}}

The f/desc function returns a regular Re-frame subscription to which all the normal subscription semantics apply. If the entity referred to by [:system/uuid #uuid "A"] has a type, the query result will be polymorphically resolved and returned in denormalized form.

reflet.core/reg-pull

The second way to use a description is to refer to a context in the join of a reg-pull query. Let's update the ::timeline-component-query query we saw earlier to use descriptions:

(f/reg-pull ::timeline-component-query
  (fn [ref]
    [[:cmp/uuid
      :cmp/other-state
      {:cmp/events ::timeline}]  ; <- Union query replaced by description
      ref]))

@(f/sub [::timeline-component-query [:cmp/uuid #uuid "component"]])
=>
{:cmp/uuid        #uuid "component"
 :cmp/other-state "other state"
 :cmp/events
 [{:kr/name  "Clojure Conj 2023"
   :kr/about "A conference for Clojure developers, companies and enthusiasts."
   :kr/time  {:kr.time/start "04/27/2023"
              :kr.time/end   "04/28/2023"}}
  {:kr/name        "The Unanswered Question"
   :kr/description "A series of lectures given by Leonard Bernstein in the fall of 1973."
   :kr/institution "Harvard University"
   :kr/time        {:kr.time/start "09/01/1973"
                    :kr.time/end   "12/01/1973"}}
  {:kr/name     "Quirky"
   :kr/location "Brixton"
   :kr/artist   {:kr/name "Autechre"}
   :kr/time     {:kr.time/date "16/06/1994"}}]}

Type Hierarchies and Polymorphism

Polymorphic descriptions become even more powerful when combined with a taxonomy, also known as a type hierarchy. A type hierarchy is a collection of "is a" relationships between pairs of types:

(def hierarchy
  (-> (make-hierarchy)
      (derive :kr.type/interval :kr.type/time)
      (derive :kr.type/date :kr.type/time)
      (derive :kr.type/lecture :kr.type/event)
      (derive :kr.type/conference :kr.type/event)
      (derive :kr.type/performance :kr.type/event)))

Or expressed in a declarative vectorized form:

(def hierarchy
  [:kr.type/interval      :kr.type/time
   :kr.type/date          :kr.type/time
   [:kr.type/lecture
    :kr.type/conference
    :kr.type/performance] :kr.type/event])

You may have already used hierarchies with Clojure multimethods. Well their role is basically the same here: the hierarchy is used to dispatch polymorphically to the appropriate description of an entity during query resolution.

The above hierarchy states that lectures, conferences, and performances are all types of events. We can then use this fact to simplify our set of descriptions:

(f/reg-desc [::side-bar :kr.type/event]
  ...)

(f/reg-desc [::side-bar :kr.type/performance]
  ...)

Entities of type :kr.type/lecture and :kr.type/conference would dispatch to the first description while :kr.type/performance events would dispatch to the second. Notice how the more specific match for :kr.type/performance is chosen, even though it could match both.

Importantly, contexts can also participate in the type hierarchy, so that polymorphic dispatch resolves against both elements of the tuple. This allows you to derive one context from another and makes descriptions highly extensible.

Let's say you want to derive a new ::bottom-bar context from an existing ::side-bar context, such that the behaviour of ::side-bar context does not change:

(def hierarchy
  [::bottom-bar ::side-bar])

(f/reg-desc [::bottom-bar :kr.type/person]
  ...)

Above we declare that ::bottom-bar is a ::side-bar context. This means that @(f/desc [::bottom-bar ref]) could resolve to descriptions in either the ::bottom-bar or ::side-bar contexts. In contrast, @(f/desc [::side-bar ref]) could only resolve to ::side-bar descriptions. This gives you a lot of flexibility when combining types and contexts together to produce polymorphic behaviour.

Resolving Dispatch Ambiguities

One thing to be aware of is that it is possible to construct type hierarchies that produce ambiguous dispatch results in certain scenarios. For example:

;; (::a isa ::b) AND (::a isa ::c)

(def hierarchy
  [::a [::b ::c]])

(f/reg-desc [::context ::b]
  ...)

(f/reg-desc [::context ::c]
  ...)

;; Assuming an entity of type ::a
@(f/desc [::context ref-a])
=>
Execution error (Error) at (<cljs repl>:1).
Multiple dispatch entries for: [::context ::a] -> [::context ::b] and [::context ::c], and neither is preferred

Or:

;; (::a isa ::b) AND (::context-1 isa ::context-2)

(def hierarchy
  [::a ::b
   ::context-1 ::context-2])

(f/reg-desc [::context-2 ::a]
  ...)

(f/reg-desc [::context-1 ::b]
  ...)

;; Assuming an entity of type ::a
@(f/desc [::context-1 ref-a])
=>
Execution error (Error) at (<cljs repl>:1).
Multiple dispatch entries for: [::context-1 ::a] -> [::context-2 ::a] and [::context-1 ::b], and neither is preferred

In these scenarios, you can explicitly resolve the conflict by defining some preference pairs for one tuple over the other:

(require '[reflet.poly :as p])

(def prefers
  [[::context ::b] [::context ::c]
   [::context-2 ::a] [::context-1 ::b]])

The above preference table says that [::context ::b] should be preferred over [::context ::c], and that [::context-2 ::a] should be preferred over [::context-1 ::b]. This is analogous to using clojure.core/prefer-method for multimethods.

Configuration: Everything As Data

There are four things you can define when it comes to descriptions:

  1. Descriptions
  2. Type hierarchies
  3. Prefers tables
  4. Type attributes (discussed in the next section)

Of these four, only descriptions can be defined as code using f/reg-desc. Everything else is defined as data in the Reflet db. In fact, even our descriptions can optionally be defined as data.

The motivation behind this is that just like our domain data, our descriptions and our taxonomies can sometimes reside in a remote store, maybe an ontology, schema, or knowledge representation layer. To that end, here are the attributes where the data is stored in the Reflet db:

  1. Descriptions: :reflet.db/descriptions
  2. Type hierarchy: :reflet.db/hierarchy
  3. Prefers table: :reflet.db/prefers
  4. Type attributes: :reflet.db/type-attrs

To configure this data, Reflet provides a built-in ::f/config-desc event:

(f/disp [::f/config-desc
         {:hierarchy    [[:my.type/developer
                          :my.type/designer
                          :my.type/sys-ops] :my.type/person]
          :prefers      [[:my.context/b :my.type/person] [:my.context/a :my.type/developer]]
          :type-attrs   {:default      :my/type
                         :my.context/b :my.context-specific-attr/type}
          :descriptions {[:my/context :my.type/person] [:my/attr1 :my/attr2]}}])

This event will take the data, convert any vectorized hierarchies or prefers to their natural associative form, and transact them to the db.

As for when and how often to dispatch ::f/config-desc, it's really up to you. Everything about the Reflet implementation is reactive to the configuration in the db, and queries should update efficiently to any changes. But in most cases, you'd want to dispatch it at least once on application boot, and once on every hot-reload during development.

Also keep in mind that ::f/config-desc will assoc in the new data, which is effectively a reset. If you want update or merge behaviour, just define your own version of ::f/config-desc.

Type Attributes

Recall that until now all our examples have assumed the entity's type could be found at the :kr/type attribute. The :type-attrs map seen above allows you to configure this attribute on a per-context basis:

{:default      :my/type
 :my.context/b :my.context-specific-attr/type}

The above spec declares :my/type to be the default type attribute for all entities, except when resolving descriptions for the :my.context/b context, in which case the type will be found at :my.context-specific-attr/type. Context specific type attributes can be very useful.

About Combining Hierarchies

When you are working with hierarchies in vectorized form, you can always concat them together safely: all transitive relations and circular dependencies will be caught when they are converted to associative form. However, if you are working with hierarchies in associative form, you cannot do a naive clojure.core/merge. The Cinch library, a transitive dependency of Reflet, has a function cinch.core/merge-hierarchies that lets you combine associative hierarchies safely.

Hot-Reloading

A typical hot-reloadable development setup might look like the following:

(ns my.app
  (:require [my.app.ui :as ui]
            [re-frame.core :as f*]
            [reagent.dom :as dom]
            [reflet.core :as f]))

(def config-desc
  {:hierarchy  [[:my.type/developer
                 :my.type/designer
                 :my.type/sys-ops] :my.type/person]
   :type-attrs {:default :my/type}})

(defn render!
  "Also called on every hot-reload."
  []
  (f*/clear-subscription-cache!)
  (f/disp-sync [::f/config-desc config-desc])  ; Configures Reflet descriptions
  (some->> "container"
           (.getElementById js/document)
           (dom/render [ui/app])))

(defn init!
  "Called once on boot."
  []
  (f/disp-sync [::f/config])                   ; Configures core Reflet features
  (render!))                                   ; Note render! is also called on boot

A couple of key things to note:

  1. ::f/config-desc is dispatched once on boot, and then once on every hot-reload to ensure that any changes to the data definitions are synced after code changes.

  2. The call must be f/disp-sync, not f/disp, to ensure that ::f/config-desc is fully processed before any subscriptions are evaluated.

You can see a working example of this in the reflet.client

Stale f/reg-desc definitions

During development if you are live-coding and you outright remove an existing f/reg-desc from a namespace, it will actually leave behind a stale definition. In this respect f/reg-desc behaves exactly like Clojure's multimethods, or Re-frame's reg-event-* handler registration functions. This is just a general problem with any kind of stateful registration method: updates work fine, but removal is a challenge.

Luckily, if you use Shadow-cljs to build your Clojurescipt application, Reflet can leverage a nice feature of shadow to solve this problem. In addition to everything else it does, the ::f/config-desc handler will automatically clear stale descriptions every time it is dispatched on a hot-reload. If you've already configured your hot-reload as described in the previous section then there is nothing else you need to do.

If you do not use shadow-cljs, then you just need to take care that when you outright remove a f/reg-desc from a namespace, that you also do a browser refresh.

Remember, changes to an existing f/reg-desc work just fine. Also this is only a consideration in development, when you are hot-reloading code: stale f/reg-desc descriptions are not a problem during production.

Example Client Descriptions

A working set of descriptions and their type hierarchy can be found in the reflet.client.ui.desc.impl example client namespace. To see them in action, start the example client using the instructions found here, and then navigate to the "Descriptions" view.

In this view, you can toggle between two different queries, each one describing the same list of events, but in different contexts. The example data can be found in the reflet.client.boot namespace.

The description config event ::f/config-desc is dispatched in the reflet.client namespace.

Try modifying the code in the example client to experiment with the description features outlined in this document.

Next: Finite State Machines

Home: Home

Reflet

Features

Other

Clone this wiki locally