Skip to content

Add monitoring and some housekeeping #156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from
Draft

Add monitoring and some housekeeping #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
601bb50
Justfile DevEx
ggwpez Aug 7, 2025
26d36d3
fix
ggwpez Aug 7, 2025
01a1263
TODOs for Donal
ggwpez Aug 7, 2025
f0ef07f
md typo
ggwpez Aug 7, 2025
77f19f2
Move npm-build
ggwpez Aug 7, 2025
e64ae62
make work
ggwpez Aug 7, 2025
f9a32c4
Gitignore
ggwpez Aug 7, 2025
af07e92
fix justfile pwd
ggwpez Aug 7, 2025
933b7a7
Add monitoring
seadanda Aug 7, 2025
c259e00
Tidy up a bit
seadanda Aug 7, 2025
ec185cc
Force install doppelganger
seadanda Aug 7, 2025
8e7bc5b
Add clean command
seadanda Aug 7, 2025
9511cab
Merge branch 'main' into donal-monitoring
seadanda Aug 7, 2025
56e2027
Remove leftover frontend install
seadanda Aug 7, 2025
6828443
Merge branch 'main' into donal-monitoring
x3c41a Aug 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ AH_NODE_RPC_PORT=9945
# RPC to use for each chain
POLKADOT_RPC=ws://localhost

# AHM monitor variables
ASSET_HUB_URL=${POLKADOT_RPC}:${RELAY_NODE_RPC_PORT}
RELAY_CHAIN_URL=${POLKADOT_RPC}:${AH_NODE_RPC_PORT}

# Local
CARGO_CMD=cargo
RUNTIMES_BUILD_ARTIFACTS_PATH=${RUNTIMES_PATH}/target/release
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@
path = paseo-runtimes
url = https://github.com/paritytech/paseo-ahm.git
branch = dev-ahm
[submodule "ahm-monitor"]
path = ahm-monitor
url = https://github.com/paritytech/asset-hub-migration-monitor.git
11 changes: 9 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ just setup && \
just ahm paseo || echo "Setup failed"
```

To add monitoring, in a separate terminal run:
```
just monitor
```
and go to [AHM Monitor](https://migration.paritytech.io/?backend_url=http://localhost:3000), where you can enter `localhost:3000` as the backend url.

# Just commands

- `just` to see the list of commands
Expand All @@ -18,7 +24,7 @@ just ahm paseo || echo "Setup failed"
- `just zb [bite|spawn|perform-migration]` to run the Zombie-Bite commands. No args prints the help menu.
- `just e2e-tests` to run the E2E tests
- `just wah-e2e-tests` to run the Westend Asset Hub E2E tests
<!-- TODO @donal: Monitoring here -->
- `just ahm monitor` to run the AHM Monitor and connect to the local RPCs

## AHM Flows (manual steps)

Expand All @@ -27,6 +33,7 @@ The cmd `just ahm <runtime>` use the `orchestrator` as main control flow to coor
### Requirements
In order to use this tool you will need these binaries available in your PATH


- [Doppelganger binaries](https://github.com/paritytech/doppelganger-wrapper): doppelganger, doppelganger-parachain, workers
- [Zombie-bite](https://github.com/pepoviola/zombie-bite)
- [Node.js](https://nodejs.org) (v22 or higher)
Expand Down Expand Up @@ -158,7 +165,7 @@ combine orchestrator logs with post-ahm testing logs. You can find different lev
- [Chopsticks](https://github.com/AcalaNetwork/chopsticks) and PET for e2e functionality tests
- [PAPI](papi.how) + PJS for orchstrating/controlling e2e AHM flow
- Zombie-Bite + Doppelganger for forking off the network and making migration blocks
<!-- TODO @donal: Monitoring here -->
- [AHM Monitor](https://migration.paritytech.io?backend_url=localhost:3000) for progress tracking

## Migration tests

Expand Down
1 change: 1 addition & 0 deletions ahm-monitor
Open in desktop
Submodule ahm-monitor added at f1f358
36 changes: 31 additions & 5 deletions justfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,15 @@ help:
@echo "Docs are in the README and the repo: https://github.com/paritytech/ahm-dryrun\n"
@just --list --unsorted

# Initial setup after you cloned the repo. Run it once.
# Initialize or update the submodules.
init:
git submodule update --init --recursive

# Install all dependencies. Run it once.
setup:
git submodule update --remote --merge
# Install all dependencies. Run it when changing branches or pulling.
setup: init
just install-doppelganger
just install-zombie-bite
just install-monitor

# ------------------------- INSTALLING DEPENDENCIES -------------

Expand All @@ -32,12 +32,21 @@ install-doppelganger:
--bin doppelganger-parachain \
--bin polkadot-execute-worker \
--bin polkadot-prepare-worker \
--locked --root ${DOPPELGANGER_PATH}
--locked --force --root ${DOPPELGANGER_PATH}

# Install the `zombie-bite` binary on your system.
install-zombie-bite:
cargo install --git https://github.com/pepoviola/zombie-bite --bin zombie-bite --locked --force

# Install the AHM Monitor
install-monitor:
mkdir -p ./ahm-monitor/backend/data
cd ahm-monitor/backend \
&& npm install \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

&& instead of #!/usr/bin/env bash + set -e for brevity or why?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

&& really mainly for staying in same dir, just runs each line of a recipe with working directory as the root. I.e. here cd changes directory from the root to ahm-monitor/backend, then the next line is run in the root:

test:
    mkdir tmp
    cd tmp
    touch test

you'd end up with the behaviour of touch ./test, rather than the expected touch ./tmp/test

You can fix it with a shebang so it runs it all in the same instance, but I slightly prefer this as the recipes are then consistent, rather than throwing a shebang into some recipes but not others. don't feel too strongly about it though, happy to change if you think it's better style

&& npm run migrate \
&& npm run push \
&& npm run build

# ------------------------- BUILDING RUNTIMES -------------------

# only run once, per the runtime that you want to test.
Expand All @@ -61,3 +70,20 @@ build runtime:

e2e-tests *TEST:
cd ${PET_PATH} && yarn && yarn test {{ TEST }}

# ------------------------- CLEANING UP -------------------------

# Clean up some generated clutter.
clean:
rm -rf migration-run-*
git checkout HEAD -- .papi/descriptors/{package.json,dist/{index.d.ts,index.js,index.mjs}}
rm -f zombie-bite/doppelganger/{.crates.toml,.crates2.json}

# Clean up everything.
clean-harder: clean
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah man, I would love this :)) the repo still generates so many things that should be git-ignored on each run.

rm -f package-lock.json .package.json.sum
rm -rf logs node_modules dist
rm -rf paseo-runtimes/target
rm -rf runtimes/target
rm -rf polkadot-ecosystem-tests/node_modules
rm -rf ahm-monitor/backend/{node_modules,dist,data}
11 changes: 2 additions & 9 deletions justfiles/ahm.justfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,6 @@ _npm-build:
sha256sum -c .package.json.sum || (npm install && sha256sum package.json > .package.json.sum && echo "✅ npm install" && echo "✅ sha256sum saved")
npm run build

# (Untested) Run the Asset Hub Migration Monitor
# Run the Asset Hub Migration Monitor
monitor:
@echo "Currently not implemented, please check https://github.com/paritytech/asset-hub-migration-monitor"
@echo "You can run the following commands to run the monitor:\n"
@echo "git clone https://github.com/paritytech/asset-hub-migration-monitor && cd asset-hub-migration-monitor"
@echo "export ASSET_HUB_URL="ws://localhost:9945""
@echo "export RELAY_CHAIN_URL="ws://localhost:9944""
@echo "just run-backend"
@echo "open https://migration.paritytech.io/?backend_url=http://localhost:3000"
# TODO @donal: Monitoring here
cd ahm-monitor/backend && yarn run start
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yarn here vs npm above? will it cause issues?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yarn is enforced in the overall package.json, I found a way round it (it's basically just for tests) but forgot to revert this line