feat: Sortition tracking in signer binary

[jbencin]

Description

Fetch and cache sortition state as needed, and use it to validate incoming BlockProposals

Applicable issues

• fixes Integrate signer sortition state tracking into the signer binary #4880

Additional info (benefits, drawbacks, caveats)

Since this is my first time contributing to the signer binary, there are some things I'm unsure of that reviewers should check:

• This adds some blocking i/o in handle_block_proposal()
• I think it's correct to do these checks before submit_block_for_validation(), since that takes longer
• I only implemented this in signer v0. I didn't touch v1
• Is the public key passed to check_proposal() the correct one?
• I used inspect_err(), which was introduced a few months ago in Rust 1.76. Is this okay, and what's our MSRV policy in general?

Checklist

• Test coverage for new or modified code paths
• Changelog is updated
• Required documentation changes (e.g., docs/rpc/openapi.yaml and rpc-endpoints.md for v2 endpoints, event-dispatcher.md for new events)
• New clarity functions have corresponding PR in clarity-benchmarking repo
• New integration test(s) added to bitcoin-tests.yml

[kantai]

This LGTM -- the existing (and future) signer integration tests should do a good job covering this integration (the rest is covered via the prior SortitionsView tests).

One of the signer::.. integration tests does seem to be failing though, so that needs to be investigated.

[jbencin]

The signer::v0::block_proposal_rejection test is failing because fetch_view() fails to get the sortition view:

INFO [1719421727.355629] [stacks-signer/src/chainstate.rs:358] [signer_runloop] Latest state wasn't a sortition: SortitionInfo { burn_block_hash: 3ca9273766f766477b5ebcdb030742d4b17ae778085d75f6a4d4b99b291f180c, burn_block_height: 231, sortition_id: bdbe3ab31d9768df001ea1aee878b617eece9eed464b44540d6019b2aabb9ecd, parent_sortition_id: 3e241e6177a8bd1de1b999d0d047873f18fbb67822b1f44a7f715f2efb03f8ff, consensus_hash: 219ec7a59ae13212a06b2efa1b18d6b484836a98, was_sortition: false, miner_pk_hash160: None, stacks_parent_ch: None, last_sortition_ch: None, committed_block_hash: None }
WARN [1719421727.355646] [stacks-signer/src/v0/signer.rs:261] [signer_runloop] Cycle #11 Signer #1: Failed to update sortition view: NoSortitionOnChain, signer_sighash: ced4ed4e61eef9b2859ae1053bc5df414e7b3a901c927107ea7c66d1a81a7f28, block_id: d6ac3d6bdcb993b0b3d73961d891d06a118a77e611e8018c63818f900bbf7d1d
WARN [1719421727.355655] [stacks-signer/src/v0/signer.rs:271] [signer_runloop] Cycle #11 Signer #1: Cannot validate block, no sortition view, signer_sighash: ced4ed4e61eef9b2859ae1053bc5df414e7b3a901c927107ea7c66d1a81a7f28, block_id: d6ac3d6bdcb993b0b3d73961d891d06a118a77e611e8018c63818f900bbf7d1d

I am going to refactor this PR a bit, and change how I handle the case when we don't have a sortition view, so it doesn't fail

[jbencin]

I've refactored this so that signer::v0::block_proposal_rejection passes. The basic flow now in handle_block_proposal() is:

• Try to fetch a sortition view if we don't have one
  • If proposal fails check_proposal(), mark BlockInfo struct as invalid
  • If we can't get sortition view or there is some other error, print a warning and continue
• Call submit_block_for_validation() if BlockInfo hasn't been marked as invalid
• Insert BlockInfo into SignerDB

@kantai @jferrant please let me know if this is the desired behavior

[jbencin]

I fixed the failing integration test be removing the call to wait_for_validate_reject_response(), which waits for a BlockValidateResponse from the node. Instead, we just check for a BlockResponse from the signer

[kantai]

This LGTM, but I agree with Jacinta's comment in the test: the check at the end should be in a loop, with a longer timeout (like a minute).

[kantai]

LGTM, but it looks like you'll have to resolve a merge conflict

[blockstack-devops]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.