Datomic Pro for Docker

Setting up Datomic Pro for Docker with Docker Compose.

As of April 27, 2023, Datomic Pro is free!

This is not an official Datomic project or documentation and is not affiliated with Datomic in any way.

TL;DR and Quick Start

Assuming you want to use PostgreSQL as your storage service:

git clone https://github.com/gbaptista/datomic-pro-docker.git

cd datomic-pro-docker

cp compose/datomic-postgresql.yml docker-compose.yml

docker compose up datomic-storage

docker compose run datomic-tools psql \
  -h datomic-storage \
  -U datomic-user \
  -d my-datomic-storage \
  -c 'CREATE TABLE datomic_kvs (
        id TEXT NOT NULL,
        rev INTEGER,
        map TEXT,
        val BYTEA,
        CONSTRAINT pk_id PRIMARY KEY (id)
      );'

# password: unsafe

docker compose up datomic-transactor

docker compose run datomic-tools clojure -M -e "$(cat <<'CLOJURE'
  (require '[datomic.api :as d])
  (d/create-database "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")
  (System/exit 0)
CLOJURE
)"

docker compose run datomic-tools clojure -M:repl

graph RL
    Transactor --- Storage[("Storage")]
    REPL("REPL (Peer)") -.- Storage
    REPL --- Transactor

Loading

(require '[datomic.api :as d])

(def connection (d/connect "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe"))

@(d/transact connection
  [{:db/ident       :book/title
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/one
    :db/doc         "The title of the book."}

    {:db/ident       :book/genre
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/one
    :db/doc         "The genre of the book."}])

@(d/transact connection
  [{:db/id      -1
    :book/title "The Tell-Tale Heart"
    :book/genre "Horror"}])

(def database (d/db connection))

(d/q '[:find ?e ?title ?genre
       :where [?e :book/title ?title]
              [?e :book/genre ?genre]]
     database)

; #{[4611681620380877802 "The Tell-Tale Heart" "Horror"]}

(System/exit 0)

Index

• TL;DR and Quick Start
• Index
• Flavors
  • PostgreSQL as Storage Service
    • Run PostgreSQL as the Storage Service
    • Run the Transactor
    • Create a Database
    • Transact and Query through an Embedded Peer
    • Run a Peer Server
    • Transact and Query through a Peer Server
  • Dev Mode
    • Run the Transactor
    • Create a Database
    • Transact and Query through an Embedded Peer
    • Run a Peer Server
    • Transact and Query through a Peer Server
• Utilities
  • REPL
  • Datomic Console
  • MusicBrainz Sample Database
    • Restoring with PostgreSQL as Storage Service
      • Exploring the Data
    • Restoring with Dev Mode
      • Exploring the Data
• Development
  • Principles
    • First Time to Joy
    • Predictable and Reliable
    • Lightweight Images
  • Updating the README
  • Clojure Dependencies

Flavors

PostgreSQL as Storage Service

Run PostgreSQL as the Storage Service

cp compose/datomic-postgresql.yml docker-compose.yml

docker compose up datomic-storage

Create the table for Datomic databases:

docker compose run datomic-tools psql \
  -h datomic-storage \
  -U datomic-user \
  -d my-datomic-storage \
  -c 'CREATE TABLE datomic_kvs (
        id TEXT NOT NULL,
        rev INTEGER,
        map TEXT,
        val BYTEA,
        CONSTRAINT pk_id PRIMARY KEY (id)
      );'

You will be prompted for a password, which is unsafe.

Run the Transactor

docker compose up datomic-transactor

Create a Database

docker compose run datomic-tools clojure -M -e "$(cat <<'CLOJURE'
  (require '[datomic.api :as d])
  (d/create-database "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")
  (System/exit 0)
CLOJURE
)"

Transact and Query through an Embedded Peer

docker compose run datomic-tools clojure -M:repl

graph RL
    Transactor --- Storage[(Storage)]
    REPL("REPL (Peer)") -.- Storage
    REPL --- Transactor

Loading

(require '[datomic.api :as d])

(def connection (d/connect "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe"))

@(d/transact connection
  [{:db/ident       :book/title
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/one
    :db/doc         "The title of the book."}

   {:db/ident       :book/genre
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/one
    :db/doc         "The genre of the book."}])

@(d/transact connection
  [{:db/id      -1
    :book/title "The Tell-Tale Heart"
    :book/genre "Horror"}])

(def database (d/db connection))

(d/q '[:find ?e ?title ?genre
       :where [?e :book/title ?title]
              [?e :book/genre ?genre]]
     database)

; #{[4611681620380877802 "The Tell-Tale Heart" "Horror"]}

(System/exit 0)

Run a Peer Server

docker compose up datomic-peer-server

Transact and Query through a Peer Server

docker compose run datomic-tools clojure -M:repl

graph RL
    Transactor --- Storage
    PeerServer --- Transactor
    PeerServer -.- Storage
    REPL(REPL) --- PeerServer[Peer Server]

Loading

(require '[datomic.client.api :as d])

(def client
  (d/client {:server-type :peer-server
             :endpoint    "datomic-peer-server:8998"
             :secret      "unsafe-secret"
             :access-key  "unsafe-key"
             :validate-hostnames false}))

(def connection (d/connect client {:db-name "my-datomic-database"}))

(d/transact connection
  {:tx-data [{:db/ident       :book/title
              :db/valueType   :db.type/string
              :db/cardinality :db.cardinality/one
              :db/doc         "The title of the book."}

             {:db/ident       :book/genre
              :db/valueType   :db.type/string
              :db/cardinality :db.cardinality/one
              :db/doc         "The genre of the book."}]})

(d/transact connection
  {:tx-data [{:db/id      -1
              :book/title "The Tell-Tale Heart"
              :book/genre "Horror"}]})

(def database (d/db connection))

(d/q '[:find ?e ?title ?genre
       :where [?e :book/title ?title]
              [?e :book/genre ?genre]]
     database)

; #{[4611681620380877802 "The Tell-Tale Heart" "Horror"]}

(System/exit 0)

Dev Mode

Run the Transactor

cp compose/datomic-dev-mode.yml docker-compose.yml

docker compose up datomic-transactor

Create a Database

docker compose run datomic-tools clojure -M -e "$(cat <<'CLOJURE'
  (require '[datomic.api :as d])
  (d/create-database "datomic:dev://datomic-transactor:4334/my-datomic-database/?password=unsafe")
  (System/exit 0)
CLOJURE
)"

Transact and Query through an Embedded Peer

docker compose run datomic-tools clojure -M:repl

graph RL
    Transactor --- Storage[("Memory")]
    REPL("REPL (Peer)") -.- Transactor
    REPL --- Transactor

Loading

(require '[datomic.api :as d])

(def connection (d/connect "datomic:dev://datomic-transactor:4334/my-datomic-database/?password=unsafe"))

@(d/transact connection
  [{:db/ident       :book/title
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/one
    :db/doc         "The title of the book."}

   {:db/ident       :book/genre
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/one
    :db/doc         "The genre of the book."}])

@(d/transact connection
  [{:db/id      -1
    :book/title "The Tell-Tale Heart"
    :book/genre "Horror"}])

(def database (d/db connection))

(d/q '[:find ?e ?title ?genre
       :where [?e :book/title ?title]
              [?e :book/genre ?genre]]
     database)

; #{[4611681620380877802 "The Tell-Tale Heart" "Horror"]}

(System/exit 0)

Run a Peer Server

docker compose up datomic-peer-server

Transact and Query through a Peer Server

docker compose run datomic-tools clojure -M:repl

graph RL
    Transactor --- Storage[("Memory")]
    PeerServer(Peer Server) --- Transactor
    PeerServer -.- Transactor
    REPL --- PeerServer
    REPL -.- PeerServer

Loading

(require '[datomic.client.api :as d])

(def client
  (d/client {:server-type :peer-server
             :endpoint    "datomic-peer-server:8998"
             :secret      "unsafe-secret"
             :access-key  "unsafe-key"
             :validate-hostnames false}))

(def connection (d/connect client {:db-name "my-datomic-database"}))

(d/transact connection
  {:tx-data [{:db/ident       :book/title
              :db/valueType   :db.type/string
              :db/cardinality :db.cardinality/one
              :db/doc         "The title of the book."}

             {:db/ident       :book/genre
              :db/valueType   :db.type/string
              :db/cardinality :db.cardinality/one
              :db/doc         "The genre of the book."}]})

(d/transact connection
  {:tx-data [{:db/id      -1
              :book/title "The Tell-Tale Heart"
              :book/genre "Horror"}]})

(def database (d/db connection))

(d/q '[:find ?e ?title ?genre
       :where [?e :book/title ?title]
              [?e :book/genre ?genre]]
     database)

; #{[4611681620380877802 "The Tell-Tale Heart" "Horror"]}

(System/exit 0)

Utilities

REPL

Starting a REPL:

docker compose run datomic-tools clojure -M:repl

To exit the REPL:

(System/exit 0)

Datomic Console

To run Datomic Console:

docker compose up datomic-console

http://localhost:8080/browse

MusicBrainz Sample Database

The MusicBrainz Sample Database, sourced from the open music encyclopedia MusicBrainz, is useful for exploring Datomic with a pre-populated database.

A backup is already included in the datomic-tools image; to restore it:

Restoring with PostgreSQL as Storage Service

docker compose run datomic-tools \
  sh -c "/usr/download-mbrainz.sh && \
  ./bin/datomic restore-db \
  file:/usr/mbrainz-1968-1973 \
  'datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe'"

If you are receiving:

:restore/collision The name 'my-datomic-database' is already in use

You need to delete the current database so you can restore a new one:

docker compose run datomic-tools clojure -M -e "$(cat <<'CLOJURE'
  (require '[datomic.api :as d])
  (d/delete-database "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe")
  (System/exit 0)
CLOJURE
)"

Exploring the Data

docker compose run datomic-tools clojure -M:repl

(require '[datomic.api :as d])

(def connection (d/connect "datomic:sql://my-datomic-database?jdbc:postgresql://datomic-storage:5432/my-datomic-storage?user=datomic-user&password=unsafe"))

(def database (d/db connection))

(d/q '[:find ?id ?type ?gender
         :in $ ?name
       :where
         [?e :artist/name ?name]
         [?e :artist/gid ?id]
         [?e :artist/type ?teid]
         [?teid :db/ident ?type]
         [?e :artist/gender ?geid]
         [?geid :db/ident ?gender]]
     database
     "Jimi Hendrix")

(System/exit 0)

Restoring with Dev Mode

docker compose run datomic-tools \
  sh -c "/usr/download-mbrainz.sh && \
  ./bin/datomic restore-db \
  file:/usr/mbrainz-1968-1973 \
  'datomic:dev://datomic-transactor:4334/my-datomic-database?password=unsafe'"

If you are receiving:

:restore/collision The name 'my-datomic-database' is already in use

You need to delete the current database so you can restore a new one:

docker compose run datomic-tools clojure -M -e "$(cat <<'CLOJURE'
  (require '[datomic.api :as d])
  (d/delete-database "datomic:dev://datomic-transactor:4334/my-datomic-database?password=unsafe")
  (System/exit 0)
CLOJURE
)"

Exploring the Data

docker compose run datomic-tools clojure -M:repl

(require '[datomic.api :as d])

(def connection (d/connect "datomic:dev://datomic-transactor:4334/my-datomic-database?password=unsafe"))

(def database (d/db connection))

(d/q '[:find ?id ?type ?gender
         :in $ ?name
       :where
         [?e :artist/name ?name]
         [?e :artist/gid ?id]
         [?e :artist/type ?teid]
         [?teid :db/ident ?type]
         [?e :artist/gender ?geid]
         [?geid :db/ident ?gender]]
     database
     "Jimi Hendrix")

(System/exit 0)

Development

Principles

First Time to Joy

From cloning the repository to running 'hello world,' the goal is to minimize setup and boot time. Download delays, long build or boot times, and flakiness are obstacles.

Predictable and Reliable

Assets must state version numbers and be verified with MD5 checksums to ensure integrity and security.

Lightweight Images

Minimize Docker image size with lightweight base images and multi-stage builds.

Updating the README

Install Babashka:

curl -s https://raw.githubusercontent.com/babashka/babashka/master/install | sudo bash

Update the template.md file and then:

bb tasks/generate-readme.clj

Trick for automatically updating the README.md when template.md changes:

sudo pacman -S inotify-tools # Arch / Manjaro
sudo apt-get install inotify-tools # Debian / Ubuntu / Raspberry Pi OS
sudo dnf install inotify-tools # Fedora / CentOS / RHEL

while inotifywait -e modify template.md; do bb tasks/generate-readme.clj; done

Trick for Markdown Live Preview:

pip install -U markdown_live_preview

mlp README.md -p 8076

Clojure Dependencies

• org.clojure/clojure
• com.datomic/peer
• com.datomic/client-pro
• org.postgresql/postgresql
• org.slf4j/slf4j-simple
• com.bhauman/rebel-readline