Skip to content

fix: isolate benchmark-compare.sh for parallel invocations#2898

Merged
pdrobnjak merged 1 commit intopd/benchmark-profiling-improvementsfrom
pd/benchmark-parallel-isolation
Feb 17, 2026
Merged

fix: isolate benchmark-compare.sh for parallel invocations#2898
pdrobnjak merged 1 commit intopd/benchmark-profiling-improvementsfrom
pd/benchmark-parallel-isolation

Conversation

@pdrobnjak
Copy link
Contributor

Summary

  • Namespace BASE_DIR per run via RUN_ID (defaults to PID): /tmp/sei-bench-${RUN_ID}/
  • Auto-claim port offset slots via atomic mkdir (supports 30 concurrent runs, zero coordination)
  • Replace git checkout with git worktrees for isolated builds (no working tree collisions)
  • Replace ~/go/bin/seid with GOBIN-based builds per label (no binary collisions)
  • Replace ~/.sei staging with mktemp + --home on all seid commands (no init collisions)
  • Pass SEI_HOME_DIR/SEID_BIN env vars to populate_genesis_accounts.py (backward-compatible defaults)
  • Fix pre-existing double lifecycle bug by passing DURATION=0 to child start-phase

Test plan

  • Syntax check: bash -n on both shell scripts, py_compile on Python
  • Two concurrent benchmark-compare.sh runs with DURATION=120 — both completed, separate BASE_DIRs, no port conflicts
  • All 6 profile types captured for all 4 nodes (CPU ~145KB, fgprof ~115KB, heap ~248KB, etc.)
  • TPS data collected (36-37 readings per node)
  • pprof -diff_base produces valid analyzable output from both runs
  • Port slot locks cleaned up after exit
  • Git worktrees cleaned up after exit
  • Backward compatible — no env vars needed for single-instance usage

🤖 Generated with Claude Code

Running 3+ benchmark-compare.sh instances concurrently failed because
5 shared resources were not namespaced: /tmp/sei-bench, ~/.sei,
~/go/bin/seid, git working tree, and network ports.

Changes:
- RUN_ID (default: PID) namespaces BASE_DIR as /tmp/sei-bench-${RUN_ID}/
- RUN_PORT_OFFSET auto-claimed via atomic mkdir slots (supports 30
  concurrent runs)
- Replace git checkout with git worktrees for isolated builds
- Replace ~/go/bin/seid with GOBIN-based builds per label
- Replace ~/.sei staging with mktemp in benchmark.sh
- Add --home to all seid commands during init
- Pass SEI_HOME_DIR/SEID_BIN env vars to populate_genesis_accounts.py
- Remove branch save/restore and ~/go/bin/seid copy
- Fix pre-existing double lifecycle bug (parent + child both managing
  profiling/timing) by passing DURATION=0 to child start-phase

All defaults are backward-compatible — zero behavior change for
existing single-instance callers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link

github-actions bot commented Feb 17, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 17, 2026, 12:16 PM

@pdrobnjak pdrobnjak merged commit 62fcfd8 into pd/benchmark-profiling-improvements Feb 17, 2026
13 of 15 checks passed
@pdrobnjak pdrobnjak deleted the pd/benchmark-parallel-isolation branch February 17, 2026 12:16
@codecov
Copy link

codecov bot commented Feb 17, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.35%. Comparing base (6dd8b0b) to head (4d6146e).
⚠️ Report is 1 commits behind head on pd/benchmark-profiling-improvements.

❗ There is a different number of reports uploaded between BASE (6dd8b0b) and HEAD (4d6146e). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (6dd8b0b) HEAD (4d6146e)
sei-chain 1 0
Additional details and impacted files

Impacted file tree graph

@@                           Coverage Diff                            @@
##           pd/benchmark-profiling-improvements    #2898       +/-   ##
========================================================================
- Coverage                                57.19%   48.35%    -8.84%     
========================================================================
  Files                                     2091      671     -1420     
  Lines                                   171173    50615   -120558     
========================================================================
- Hits                                     97897    24474    -73423     
+ Misses                                   64557    24000    -40557     
+ Partials                                  8719     2141     -6578     
Flag Coverage Δ
sei-chain ?
sei-cosmos 48.16% <ø> (+<0.01%) ⬆️
sei-db 68.72% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1543 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant