-
Notifications
You must be signed in to change notification settings - Fork 11.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
indexer: ensure schema always matches what diesel generated (#19337)
## Description Create a script that regenerates the schema and run it to standardise our schema. Also merge `schema/mod.rs` and `schema/pg.rs` into one `schema.rs` -- otherwise we have an opportunity for things to go out of sync because a module is present in the generated schema (`pg.rs`) but not in the module entrypoint (`mod.rs`). Run this script to refresh the schema (some inconsistencies already crept in!) and set it up to run under CI. ## Test plan ``` sui$ ./scripts/generate_indexer_schema.rs` ``` and CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
- Loading branch information
Showing
4 changed files
with
98 additions
and
107 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| #!/bin/bash | ||
| # Copyright (c) Mysten Labs, Inc. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Update sui-indexer's generated src/schema.rs based on the schema after | ||
| # running all its migrations on a clean database. Expects the first argument to | ||
| # be a port to run the temporary database on (defaults to 5433). | ||
|
|
||
| set -x | ||
| set -e | ||
|
|
||
| if ! command -v git &> /dev/null; then | ||
| echo "Please install git: e.g. brew install git" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| for PG in psql initdb postgres pg_isready pg_ctl; do | ||
| if ! command -v $PG &> /dev/null; then | ||
| echo "Could not find $PG. Please install postgres: e.g. brew install postgresql@15" >&2 | ||
| exit 1 | ||
| fi | ||
| done | ||
|
|
||
| if ! command -v diesel &> /dev/null; then | ||
| echo "Please install diesel: e.g. cargo install diesel_cli --features postgres" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| REPO=$(git rev-parse --show-toplevel) | ||
|
|
||
| # Create a temporary directory to store the ephemeral DB. | ||
| TMP=$(mktemp -d) | ||
|
|
||
| # Set-up a trap to clean everything up on EXIT (stop DB, delete temp directory) | ||
| function cleanup { | ||
| pg_ctl stop -D "$TMP" -mfast | ||
| set +x | ||
| echo "Postgres STDOUT:" | ||
| cat "$TMP/db.stdout" | ||
| echo "Postgres STDERR:" | ||
| cat "$TMP/db.stderr" | ||
| set -x | ||
| rm -rf "$TMP" | ||
| } | ||
| trap cleanup EXIT | ||
|
|
||
| # Create a new database in the temporary directory | ||
| initdb -D "$TMP" --user postgres | ||
|
|
||
| # Run the DB in the background, on the port provided and capture its output | ||
| PORT=${1:-5433} | ||
| postgres -D "$TMP" -p "$PORT" -c unix_socket_directories= \ | ||
| > "$TMP/db.stdout" \ | ||
| 2> "$TMP/db.stderr" & | ||
|
|
||
| # Wait for postgres to report as ready | ||
| RETRIES=0 | ||
| while ! pg_isready -p "$PORT" --host "localhost" --username "postgres"; do | ||
| if [ $RETRIES -gt 5 ]; then | ||
| echo "Postgres failed to start" >&2 | ||
| exit 1 | ||
| fi | ||
| sleep 1 | ||
| RETRIES=$((RETRIES + 1)) | ||
| done | ||
|
|
||
| # Run all migrations on the new database | ||
| diesel migration run \ | ||
| --database-url "postgres://postgres:postgrespw@localhost:$PORT" \ | ||
| --migration-dir "$REPO/crates/sui-indexer/migrations/pg" | ||
|
|
||
| # Generate the schema.rs file, excluding partition tables and including the | ||
| # copyright notice. | ||
| diesel print-schema \ | ||
| --database-url "postgres://postgres:postgrespw@localhost:$PORT" \ | ||
| --patch-file "$REPO/crates/sui-indexer/src/schema.patch" \ | ||
| --except-tables "^objects_version_|_partition_" \ | ||
| > "$REPO/crates/sui-indexer/src/schema.rs" |