fix(consensus): parse a size-delimited ErgoTree by structure, not the declared size

[arkadianet]

What

The node parsed a size-delimited (has_size) ErgoTree by consuming exactly the declared size
bytes (get_u32_exact + get_bytes(size)). Scala's deserializeErgoTree is structure-delimited:
it bounds the body by a position limit of MaxPropositionSize (4096, anchored at startPos,
checked position > positionLimit before each read), and on success leaves the reader at the
actual structural body end — ErgoBoxCandidate.parseBody reads creationHeight immediately
after the inline tree parse. The declared size is read non-exact (getUInt().toInt) and used
only for the UnparsedErgoTree byte count on the wrap path
(numBytes = bodyPos - startPos + declaredSize, then a rewind + getBytes(numBytes)).

So the node diverged for any box whose declared size ≠ the actual body length. Honest serializers
always write size == body, so this is adversarial-only, but a real reject-valid / accept-invalid
a crafted block could exploit:

• size past i32::MAX → get_u32_exact rejected; Scala ignores it and parses.
• size > available bytes → get_bytes failed; Scala parses the body.
• size > the body → the node consumed trailing box fields as tree padding and read creationHeight
  from the wrong offset; Scala stops at the body end.
• size < the body → the node truncated the body; Scala parses it in full.

How

• Parse the body on a view of all remaining bytes under the reader's position_limit =
  MaxPropositionSize - (header + size length). The position_limit mirrors Scala's
  position > positionLimit begin-check byte-for-byte (a final read beginning exactly at the limit
  proceeds), and a CheckPositionLimit overrun (InvalidData) routes to the soft-fork wrap — like
  Scala's ReaderPositionLimitExceeded → ValidationException.
• parse_body is structure-delimited, so its consumed length is the true body length. On success
  advance the outer reader by that length; only when wrapping reposition to Scala's numBytes
  boundary (take_unparsed_size_region) and preserve those bytes verbatim.
• The declared size, being non-exact, may be negative while numBytes stays in range — Scala
  still wraps (it does not reject for a negative size) and the reader can rewind before the body;
  the helper reproduces that, erroring only when numBytes is negative or past the buffer end.

Validation

Oracle-validated (sigma-state 6.0.2, run directly during review): 080208d3 / 080508d3 /
080108d3 / 08808080800808d3 (size ==, >, <, overflowed vs a 2-byte body) all PARSE;
08ffffffff0f0204 / 08feffffff0f0204 / 08faffffff0f0204 (negative sizes) wrap with byte
counts 5 / 4 / 0; 08f9ffffff0f0204 (numBytes < 0) hard-fails; and the MaxPropositionSize
boundary (total tree 4096/4097 parse, 4098 wraps) matches.

Reviewed source-level against the Scala reference across three passes — the review (which ran
the JVM oracle on edge cases) caught and fixed two real edge bugs: a negative-size over-rejection
and the position-limit anchor/begin-check boundary. New tests pin the structural advance, the
negative-size numBytes byte counts, and the parse cases. The group-element checkpoint,
byte-preservation (block 1,702,686 → 49-byte UnparsedErgoTree), depth / hard-reject, and
non-SigmaProp-root wrap behaviors are unchanged.

Out of scope (tracked follow-up): the nested SBox-constant size-delimited SKIP
(sigma_value.rs) still advances by the declared size.

Test plan

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace      # 4399 passed

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Enhanced ErgoTree deserialization to correctly handle edge cases and align with reference implementation
  • Improved robustness of parser boundaries and size field handling in tree structure parsing
  • Refined error handling for unparsable tree components

• Tests

  • Added validation tests for declared size handling and parsing behavior

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b196167a-a6c1-4ccd-8716-ad4d4189d3a7

📥 Commits

Reviewing files that changed from the base of the PR and between 7cc7941 and ce6545e.

📒 Files selected for processing (1)

• ergo-ser/src/ergo_tree.rs

---

📝 Walkthrough

Walkthrough

In ergo_tree.rs, the has_size ErgoTree deserialization is reworked to match Scala semantics. A new MAX_PROPOSITION_BYTES constant bounds body parsing positionally. A new helper take_unparsed_size_region computes Scala-equivalent numBytes (including negative declared-size cases) for Expr::Unparsed wrapping. Success and error paths now both use this helper instead of the declared size to advance the reader or capture bytes.

Changes

ErgoTree declared-size deserialization rework

Layer / File(s)	Summary
MAX_PROPOSITION_BYTES constant and take_unparsed_size_region helper ergo-ser/src/ergo_tree.rs	Adds the 4096-byte positional cap constant and take_unparsed_size_region, which computes Scala-equivalent numBytes (including negative declared-size handling) and extracts the verbatim byte slice for Expr::Unparsed wrapping.
has_size branch: non-exact declared-size read and bounded body parsing ergo-ser/src/ergo_tree.rs	Replaces strict get_u32_exact + get_bytes with non-exact get_uint_to_i32, caps the body budget at MAX_PROPOSITION_BYTES, parses via a limited reader view, collects v6-method checkpoint and group-element sideband, and routes version-overflow wrap through take_unparsed_size_region.
Success-path and error-path wrapping using declared-size region ergo-ser/src/ergo_tree.rs	Success path wraps via take_unparsed_size_region when pre-v3/v6-method or non-SSigmaProp conditions are met, and otherwise advances by actual body_consumed. Error-to-wrap path now also uses take_unparsed_size_region instead of previously captured full-tree bytes.
Tests: declared-size independence and negative-size wrapping ergo-ser/src/ergo_tree.rs	declared_size_is_not_the_body_bound_nor_the_success_advance asserts oversized/undersized/equal/overflowed declared sizes parse to SSigmaProp at structural end. negative_declared_size_wraps_with_scala_numbytes asserts negative declared sizes wrap with Scala's numBytes = bodyPos - startPos + declaredSize.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• arkadianet/ergo#117: Modifies the same has_size wrapping error-handling logic in ergo_tree.rs so HardReject escapes rather than being wrapped, directly related to this PR's error-path adjustments.
• arkadianet/ergo#119: Modifies read_ergo_tree_tracking_wrap for pre-v3/v6-method wrapping decisions and declared-size region byte slicing, the same code paths reworked here.
• arkadianet/ergo#122: Modifies read_ergo_tree_tracking_wrap soft-fork wrapping logic and declared-size region byte slicing for Expr::Unparsed, overlapping directly with this PR's changes.

Poem

🐇 Hoppity-hop through bytes I go,
Declared sizes? I ignore their show!
MAX_PROPOSITION_BYTES sets the bound,
numBytes from Scala, correct and sound.
Negative sizes still wrap just right —
This rabbit keeps the parser tight! 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(consensus): parse a size-delimited ErgoTree by structure, not the declared size' directly and precisely summarizes the main change: shifting from parsing by declared size to parsing by actual structure.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/structure-delimited-tree-size

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}