Skip to content

refactor(ergo-node): split oversized files (wallet_bridge, boot, snapshot, scala_compat, messaging) - #204

Closed
arkadianet wants to merge 7 commits into
mainfrom
refactor/ergo-node-split
Closed

refactor(ergo-node): split oversized files (wallet_bridge, boot, snapshot, scala_compat, messaging)#204
arkadianet wants to merge 7 commits into
mainfrom
refactor/ergo-node-split

Conversation

@arkadianet

Copy link
Copy Markdown
Owner

Summary

Splits the largest/most tangled files in ergo-node into cohesive submodule directories, following the same methodology used for the ergo-compiler splits (#200, #202): incremental extraction with test verification after each step, red-team review of every commit, one bundled PR.

  • node/wallet_bridge.rs (5355 → 1487 lines): extracted the business-logic layer called by the already-split commands/*.rs handlers into node/wallet_bridge/support/ (8 files: hints_codec, key_derivation, dto, multisig_helpers, tx_build, sign_submit, generate_sign, sweep). run_wallet_writer and its dispatch match stay in wallet_bridge.rs.
    • Includes a real bug fix: derive_next_key_impl duplicated persist_tracked_pubkey's redb-write logic inline instead of calling it, and separately advanced WALLET_DERIVATION_HEAD outside that transaction. Unified into one atomic write_txn — a crash between the two writes previously could wedge derivation permanently (tracked pubkey persisted, head stuck, every future call hitting the dedup check and returning DerivationPathExists).
  • node/boot.rs (1834 lines): split run_inner_with_backend's ~1280-line backend-agnostic boot tail into node/boot/{mod,peers,sync_setup,mining,api_wiring}.rs. Each phase function takes the prior phase's outputs and returns a small struct of the values it produces, with run_inner_with_backend itself as the orchestrating body. api_wiring and mining each split into two functions since the real dependency order interleaves them (scaffold → mining subsystem → bind; subsystem → NodeState → engine spawn). Manually boot-tested against real testnet peers (synced header chain to height 400, clean shutdown).
  • node/snapshot_emit.rs + snapshot.rs: split into node/snapshot_emit/{mod,recent_blocks,bootstrap_panel,mempool_projection,events_projection}.rs (mod.rs keeps publish_snapshot as orchestrator) and snapshot/{mod,build,publisher}.rs (the NodeSnapshot/SnapshotParts DTOs stay whole in mod.rs — splitting a single wide DTO struct across files hurts more than it helps).
  • api_bridge/scala_compat.rs: light touch only — the trait impl (NodeChainQuery for ScalaCompatBridge) is one large block of small independent read-only methods, not worth forcing apart. Only the ~300-line fee-per-byte pool-ranking cluster moved to scala_compat/pool_fee_stats.rs.
  • node/messaging.rs: extracted the three sizable per-batch match arms (CODE_INV, CODE_MODIFIER ~190 lines, CODE_PEERS) into named helpers (handle_inv, handle_modifier_batch, handle_peers_response) following the same &mut NodeState -> Vec<Action> convention as handle_message/run_wallet_writer. Split into node/messaging/{dispatch,manifest,utxo_chunk,popow}.rs.

Every commit was independently verified (cargo test -p ergo-node including all integration tests, cargo clippy -p ergo-node --all-targets --all-features -- -D warnings, cargo fmt --all -- --check) and red-team reviewed by an independent Opus pass diffing against the pre-refactor original — no behavior-changing findings survived across any of the 5 splits. The whole-workspace gate (cargo clippy --workspace --all-targets --all-features -- -D warnings and cargo test --workspace) is also green.

Purely structural — no functional changes intended anywhere except the one explicitly-called-out derive_next_key_impl atomicity fix.

Test plan

  • cargo test -p ergo-node (500 lib tests + full integration suite, all green) after each of the 5 splits
  • cargo clippy -p ergo-node --all-targets --all-features -- -D warnings clean after each split
  • cargo fmt --all -- --check clean after each split
  • cargo clippy --workspace --all-targets --all-features -- -D warnings clean (whole-workspace gate)
  • cargo test --workspace clean (whole-workspace gate, every crate)
  • Manual boot test against real testnet peers after the boot.rs split (header sync to height 400, clean SIGTERM shutdown)
  • Independent Opus red-team review per commit, diffing every moved function against the pre-refactor original

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi

arkadianet and others added 7 commits July 17, 2026 05:42
Extracts the ~3500-line business-logic layer (build/sign/submit/sweep/
multi-sig/key-derivation helpers) that run_wallet_writer's commands/*
handlers call into, out of wallet_bridge.rs into a new
node/wallet_bridge/support/ directory, mirroring the existing commands/
grouping:

- tx_build.rs -- shared burn-aware unsigned-tx builder, boxes/select,
  transactions/build
- sign_submit.rs -- transactions/sign, transactions/send, and the shared
  sign/self-verify/serialize building blocks
- generate_sign.rs -- PaymentSend, TransactionGenerate*, TransactionSign,
  BoxesCollect
- sweep.rs -- retrieve-matured-rewards sweep
- dto.rs -- box/tx status strings, wallet-row projections, pagination
- multisig_helpers.rs -- input/data-input resolution, generateCommitments,
  extractHints
- hints_codec.rs -- TransactionHintsBag <-> TxHintsBagDto JSON converters
- key_derivation.rs -- deriveKey, deriveNextKey, getPrivateKey

wallet_bridge.rs itself now holds only the trait/struct infrastructure
(TxSubmitter, WalletCommand, NodeWalletAdmin, ChainStateAccessor(Impl),
WalletStateHook) and the run_wallet_writer dispatch loop, dropping from
5355 to 1487 lines.

Also fixes a real bug found during the extraction: derive_next_key_impl
duplicated persist_tracked_pubkey's tracked-pubkey-insert +
WALLET_VISIBLE_ADDRESSES-rebuild logic inline instead of calling it. Now
both derive_key_impl and derive_next_key_impl share the one persist path;
derive_next_key_impl advances WALLET_DERIVATION_HEAD in its own follow-up
write transaction (a crash between the two commits just means a retry
recomputes the same head and hits the existing dedup check, not silent
double-tracking).

Doc/logic-preserving move otherwise -- no other behavior changes. Verified:
cargo test -p ergo-node --lib (500 passed, same as baseline), cargo clippy
--all-targets --all-features -D warnings, cargo fmt --all --check.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi
…ersist

The wallet_bridge.rs split's derive_next_key_impl fix (previous commit)
called persist_tracked_pubkey then advanced WALLET_DERIVATION_HEAD in a
separate follow-up write transaction. An independent red-team review
caught that this traded the old single-transaction atomicity for a latent
permanent-wedge failure mode: a crash between the two commits leaves the
pubkey tracked but the head not advanced, and since this function is the
sole reader/writer of that head, every future deriveNextKey call
recomputes the same next path, finds it already tracked, and 409s
forever -- requiring manual DB surgery to recover.

Fixes it properly: persist_tracked_pubkey now takes an optional
new_derivation_head and, when Some, advances WALLET_DERIVATION_HEAD
inside the SAME write transaction as the tracked-pubkey insert +
WALLET_VISIBLE_ADDRESSES rebuild. derive_key_impl passes None (it has no
head to advance); derive_next_key_impl passes Some(new_head), restoring
the original single-commit atomicity while still sharing the persist
path with derive_key_impl.

Verified: cargo test -p ergo-node --lib (500 passed), clippy, fmt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi
Extract run_inner_with_backend's backend-agnostic boot tail into
node/boot/{peers,sync_setup,mining,api_wiring}.rs, each phase taking
the prior phase's outputs and returning a small struct of the new
values it produces. mod.rs keeps run/run_inner/expected_sentinel/
build_reemission_rules plus the orchestrating body that assembles the
phases in order.

api_wiring and mining each split into two functions (build_scaffold/
bind, build_subsystem/spawn_engine) since the mining subsystem must be
built between the scaffold and the API bind in the real dependency
order, and engine-spawn needs the constructed NodeState.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi
…caffold

Flagged by red-team review of the boot.rs split: build_scaffold computed
IdentityInputs::from_config and immediately discarded it — the orchestrator
already recomputes it for actual use in NodeState construction.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi
…odules

node/snapshot_emit.rs -> node/snapshot_emit/{mod,recent_blocks,bootstrap_panel,
mempool_projection,events_projection}.rs: mod.rs keeps publish_snapshot as the
per-tick orchestrator; each field group it collects moves to its own submodule
(tip-cached recent-blocks tail + first-deliverer merge, Mode 2 bootstrap-panel
projection, mempool-transaction DTO list, operator event-feed projection).

snapshot.rs -> snapshot/{mod,build,publisher}.rs: mod.rs keeps the NodeSnapshot
and SnapshotParts DTOs plus their small support types (not split further --
splitting a single wide DTO struct across files hurts more than it helps);
build.rs assembles a NodeSnapshot from a SnapshotParts; publisher.rs owns
SnapshotPublisher's per-tick publish + stall-clock bookkeeping.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi
…ol_fee_stats.rs

Light-touch split per plan: the trait impl block (NodeChainQuery for
ScalaCompatBridge) stays whole -- it's one large block of small,
independent read-only methods, not worth forcing apart. Only the
free-function fee-per-byte pool ranking cluster (PoolFeeEntry,
rank_pool_by_fee_per_byte, bin_for_wait_ms, estimate_wait_ms_from_rank,
their consts, and their tests) moves out to its own submodule; the
three call sites in pool_fee_histogram/pool_recommended_fee/
pool_expected_wait_time_ms are updated to the pool_fee_stats:: path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi
…submodules

Extract the three sizable per-batch match arms out of handle_message into
named helpers -- handle_inv (CODE_INV), handle_modifier_batch (CODE_MODIFIER,
~190 lines), handle_peers_response (CODE_PEERS) -- all following the same
&mut NodeState -> Vec<Action> convention as handle_message itself and the
dispatch pattern already used by run_wallet_writer. handle_message plus the
three new helpers live in messaging/dispatch.rs; the three Mode-2/NiPoPoW
consume-side handlers (handle_inbound_manifest, handle_inbound_utxo_chunk,
handle_inbound_popow_proof) each get their own submodule.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kw7uCwqmiGna8fpg6TvxGi
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 58c2c1e2-9718-464f-8ece-6be5602f072b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/ergo-node-split

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@arkadianet

Copy link
Copy Markdown
Owner Author

Splitting into two smaller PRs to stay under coderabbitai's 25-files-per-PR review limit: #205 (wallet_bridge + boot) and #206 (snapshot + scala_compat + messaging).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant