Notes on deploying Marlowe Runtime

[bwbush]

This discussion is for notes and discussion related to deploying Marlowe Runtime. This will be a source for creating documentation for deploying Marlowe Runtime.

[bwbush]

Chain Seek

Running the Marlowe chain seek daemon.

1. Install PostgreSQL 13.
2. createdb followed by name of the database to use.
3. nix develop in top folder of repository.
4. cd marlowe-sync.
5. Edit sqitch.conf so that it has the username and database to use.
6. sqitch deploy.
7. cabal run exe:chainseekd -- --socket-path ../../node.socket --database-uri postgresql://myusername@0.0.0.0/preview_chainseek --genesis-config-file ../../cardano-node/configuration/cardano/mainnet-byron-genesis.json --genesis-config-file-hash 5f20df933584822601f9e3f8c024eb5eb252fe8cefb24d1317dc3d432e940ebb`

[bwbush]

Running on mainnet requires adjustment to the write-ahead log size in Postgres. I'll post a diff for postgresql.conf.

[bwbush]

Here are the PostgreSQL parameter changes (relative to default) that I'm using for mainnet:

122c122
< shared_buffers = 128MB                        # min 128kB
---
> shared_buffers = 2GB                  # min 128kB
124c124
< #huge_pages = try                     # on, off, or try
---
> huge_pages = try                      # on, off, or try
126,127c126,127
< #temp_buffers = 8MB                   # min 800kB
< #max_prepared_transactions = 0                # zero disables the feature
---
> temp_buffers = 2GB                    # min 800kB
> max_prepared_transactions = 256               # zero disables the feature
229c229
< max_wal_size = 1GB
---
> max_wal_size = 4GB
723c723
< #max_pred_locks_per_transaction = 64  # min 10
---
> max_pred_locks_per_transaction = 256  # min 10

[bwbush]

Here is my configuration for preview:

$ cat configuration.env 
export MARLOWE_ROOT="$(dirname -- "$(readlink -f -- "${BASH_SOURCE[0]}")")"

export CARDANO_NODE_SOCKET_PATH="$MARLOWE_ROOT/node.socket"
export CARDANO_TESTNET_MAGIC=2

MAGIC=(--testnet-magic 2)

CARDANO_NODE_PORT=23001

CHAINSEEKD_PORT=23715
CHAINSEEKD_QUERY_PORT=23716
CHAINSEEKD_JOB_PORT=23720

MARLOWE_HISTORY_COMMAND_PORT=23717
MARLOWE_HISTORY_QUERY_PORT=23718
MARLOWE_HISTORY_SYNC_PORT=23719

MARLOWE_TX_COMMAND_PORT=23723

MARLOWE_DISCOVERY_QUERY_PORT=23721
MARLOWE_DISCOVERY_SYNC_PORT=23722

Set variables:

. configuration.env

Run node:

build-node/bin/cardano-node run --config config/node.json --topology config/topology.json --database-path node.db --socket-path node.socket --port $CARDANO_NODE_PORT

Set chain seek database:

$ createdb mychainseek

$ vi sqitch.conf

$ sqitch deploy

Run chain seek:

chainseekd --genesis-config-file-hash 72593f260b66f26bef4fc50b38a8f24d3d3633ad2e854eaf73039eb9402706f1 --genesis-config-file ../../config/byron-genesis.json --socket-path ../../node.socket --database-uri postgresql://localhost/preview_chainseek ${MAGIC[@]} --port-number $CHAINSEEKD_PORT --query-port-number $CHAINSEEKD_QUERY_PORT --job-port-number $CHAINSEEKD_JOB_PORT

Run contract history:

marlowe-history --chain-seek-port-number $CHAINSEEKD_PORT --chain-seek-query-port-number $CHAINSEEKD_QUERY_PORT --command-port $MARLOWE_HISTORY_COMMAND_PORT --query-port $MARLOWE_HISTORY_QUERY_PORT --sync-port $MARLOWE_HISTORY_SYNC_PORT

Run transaction creation:

marlowe-tx --chain-seek-port-number $CHAINSEEKD_PORT --chain-seek-query-port-number $CHAINSEEKD_QUERY_PORT --chain-seek-command-port-number $CHAINSEEKD_JOB_PORT --command-port $MARLOWE_TX_COMMAND_PORT --history-sync-port $MARLOWE_HISTORY_SYNC_PORT

Run contract discovery:

marlowe-discovery --chain-seek-port-number $CHAINSEEKD_PORT --chain-seek-query-port-number $CHAINSEEKD_QUERY_PORT --query-port $MARLOWE_DISCOVERY_QUERY_PORT --sync-port $MARLOWE_DISCOVERY_SYNC_PORT

Use Marlowe Runtime CLI:

marlowe --history-command-port $MARLOWE_HISTORY_COMMAND_PORT --history-query-port $MARLOWE_HISTORY_QUERY_PORT --history-sync-port $MARLOWE_HISTORY_SYNC_PORT --tx-command-port $MARLOWE_TX_COMMAND_PORT ls

[bwbush]

All of the MARLOWE_ above should be MARLOWE_RT_ instead, for compatibility with the default environment variables used by the marlowe command.

[bwbush]

Here is a tutorial for using Marlowe Runtime history, following, and discovery.

[palas]

Here is a diagram with how the pieces fit together and default ports:

ports.dot

digraph {
   rankdir=LR
   ranksep="2.0"
   ordering=out
   node [shape=record]; 
   subgraph cluster_cardano_node {
     label = "cardano-node";
     subgraph cluster_cardano_node_inp {
       label = "Server" ; 
       cn_unix_socket [ label= "cardano_node: UNIX socket" ];
     }
     subgraph cluster_cardano_node_out {
       label = "Client";
     }
   }
 
   subgraph cluster_chain_seek {
     label = "chain-sync"
     subgraph cluster_chain_seek_inp {
       label = "Server"; 
       cs_port_number [ label= "port-number: 3715" ];
       cs_query_port_number [ label= "query-port-number: 3716" ];
       cs_job_port_number [ label= "job-port-number: 3720" ];
     }
     subgraph cluster_chain_seek_out {
       label = "Client";
       cs_cn_unix_socket [ label= "cardano_node: UNIX socket"];
     }
    }
 
    subgraph cluster_marlowe_history {
     label = "marlowe-history";
     subgraph cluster_marlowe_history_inp {
       label = "Server" ; 
       mh_command_port [ label= "command-port: 3717" ];
       mh_query_port [ label= "query-port: 3718" ];
       mh_sync_port [ label= "sync-port: 3719" ];
     }
     subgraph cluster_marlowe_history_out {
       label = "Client";
       mh_cs_port_number [ label= "chain-seek-port-number: 3715" ];
       mh_cs_query_server [ label= "chain-seek-query-port-number: 3716" ];
     }
    }
 
    subgraph cluster_marlowe_tx {
     label = "marlowe-tx";
     subgraph cluster_marlowe_tx_inp {
       label = "Server" ; 
       mt_command_port [ label= "command-port: 3723" ];
     }
     subgraph cluster_marlowe_tx_out {
       label = "Client";
       mt_cs_port_number [ label= "chain-seek-port-number: 3715" ];
       mt_cs_query_server [ label= "chain-seek-query-port-number: 3716" ];
       mt_cs_command_port [ label= "chain-seek-command-port-number: 3720" ];
       mt_mh_history_sync_port [ label= "history-sync-port: 3719" ];
     }
    }
 
    subgraph cluster_marlowe_discovery {
     label = "marlowe-discovery";
     subgraph cluster_marlowe_discovery_inp {
       label = "Server" ; 
       md_query_port [ label= "query-port: 3721" ];
       md_sync_server [ label= "sync-port: 3722" ];
     }
     subgraph cluster_marlowe_discovery_out {
       label = "Client";
       md_cs_port_number [ label= "chain-seek-port-number: 3715" ];
       md_cs_query_server [ label= "chain-seek-query-port-number: 3716" ];
     }
    }
 
    subgraph cluster_marlowe {
     label = "marlowe (runtime cli)";
     subgraph cluster_marlowe_inp {
       label = "Server" ; 
     }
     subgraph cluster_marlowe_out {
       label = "Client";
       m_history_command [ label= "history-command: 3717" ];
       m_history_query [ label= "history-query: 3718" ];
       m_history_sync [ label= "history-sync: 3719" ];
       m_discovery_query [ label= "discovery-query: 3721" ];
       m_tx_command [ label= "tx-command: 3723" ];
     }
    }
 
   cs_cn_unix_socket -> cn_unix_socket;
   mh_cs_port_number -> cs_port_number;
   mh_cs_query_server -> cs_query_port_number;
   mt_cs_port_number -> cs_port_number;
   mt_cs_query_server -> cs_query_port_number;
   mt_cs_command_port -> cs_job_port_number;
   mt_mh_history_sync_port -> mh_sync_port;
   md_cs_port_number -> cs_port_number;
   md_cs_query_server -> cs_query_port_number;
   m_history_command -> mh_command_port;
   m_history_query -> mh_query_port;
   m_history_sync -> mh_sync_port;
   m_discovery_query -> md_query_port;
   m_tx_command -> mt_command_port;
}

[bwbush]

Here are comprehensive instructions on setting up the EC2 instance: https://github.com/bwbush/local-cardano-service-configs/blob/marlowe-runtime-ec2/ReadMe.md.

[bwbush]

Here are a couple of important notes for workarounds needed in Marlowe Runtime deployments:

• Marlowe Runtime does not survive epoch boundaries: My current hypothesis is that the root cause of SCP-4586 is that Runtime's sense of time gets scrambled after an epoch boundary and transaction-creation will fail with peculiar Ouroboros errors. The workaround is to restart all services after an epoch boundary.
• Chain seek crashes due to some rollbacks: Especially after catching up with the tip after being restarted, chainseekd will crash, SCP-4530. The workaround is to restart chainseekd after it crashes and hope it doesn't crash again.
• Dependencies on chain seek: It appears that marlowe-history, marlowe-discovery, and marlowe-tx can be restarted in any order, but if chainseekd is restarted, then the downstream services may crash or failure to correctly service requests. The workaround is to restart marlowe-history, marlowe-discovery, and marlowe-tx after chainseekd starts listening on its ports after it has been started.