Multi-miner , Signer block rejection

[saralab]

Miner 2 proposes block #156

• Signer 2 accepts: This wasn’t a confirmed block, needs to see lack of confirmation
• Signer 1 accepts
• Signer 0 rejects, because it saw burn block 134 before seeing the proposal
• 3/8 signing power rejects so the block is not accepted

All miners see burn block #134

• Miner 2 ends its tenure
• Miner 0 starts its tenure

Miner 0 (correctly) builds from block 155 and proposes block #156 to the signers

• Signer 2 rejects the block (because it is reorging the previous block Exception in thread Thread-1 & Caught Exception from bitcoind #156 that it accepted)
• Signer 0 accepts the block
• Signer 1 rejects the block (same reason)
• 5/8 reject the block so it is not accepted

This repeats forever

Discussion:

• Timer needs reset, legit bug
• Fix the Signer to see the lack of confirmation
• In this case they shouldn’t accept the last block (Exception in thread Thread-1 & Caught Exception from bitcoind #156) as canonical until signature threshold is reached
• Subscribe to the mined blocks , to update their internal state.
• Signer could poll the node too
• Will not necessarily add latency as a result , this is needed as a fallback for Signers
• Integration test to split the Signer set to have different visibility of blocks
• Safety check: If a block is seen as rejected, it should be always rejected

Miner doesn’t properly track whether a block rejection already came from a signer

• In begin_sign_v0, gathered_signatures isn’t updated with rejections
• Not the root cause, but needs improvement

[saralab]

@jferrant Will draft a proposal and review with the team.

[jcnelson]

I think we flubbed the logic where signers learn whether or not a block proposal was accepted or rejected by the network. The acceptance process need to look like this I think:

• Signers process and submit accept/reject signatures on a block proposal, but make no decision then on whether or not they treat the block as accepted. A block is instead "tentatively" (or "locally") accepted or rejected, until it becomes "definitively" (or "globally") accepted or rejected. A signer may treat multiple blocks at the same height as tentatively accepted or rejected. I think this is the problem -- deciding whether or not a block is accepted/rejected requires the signer to first observe that a global decision has been made. Specifically:

  • Signers will treat a block as definitively rejected if they see a threshold of rejection signatures (this already happens)
  • Signers will treat a block as definitively accepted if one of the following criteria are met:
    • A threshold of "accept" signatures are seen, in which case, the signer will broadcast the block to its node (this already happens)
    • It sees that its node has received, processed and accepted the block (in which case, no broadcast needs to happen). This doesn't happen right now, and is necessary to ensure that a signer's tentative rejection of a block does not prohibit it from signing blocks that descend from it (i.e. a signer may reject a block, but if the network accepts it, then the signer must treat it as definitively accepted).
  • Signers make no decision on a block's status otherwise -- the block remains in a tentative state until the start of the next tenure, in which case, each tentative block is then treated as definitively rejected. This also doesn't happen right now, because a signer's decisions on block acceptance and rejection are not currently preempted by the network's decision to accept or reject blocks (and they need to be).

• A miner treats the block as definitively accepted/rejected only if one of the following global criteria are true (this already happens):

  • It receives enough signatures from signers to accept (in which case, it broadcasts the block)
  • It sees that the block has been processed and accepted in its chainstate, meaning that it received a signer-broadcasted copy of the block (in which case, it does not broadcast the block)
  • The miner treats the block as definitively rejected only if it receives a threshold of "reject" votes.

As before, a miner will continuously try to build atop one of the highest definitively accepted block, and continue to do so in the face of timeouts and rejections. The miner's p2p and relayer threads work in the background to sync with the signers' nodes to ensure that the miner has the same blocks (and same highest definitively accepted block) as the signers.

[jcnelson]

A block is instead "tentatively" (or "locally") accepted or rejected, until it becomes "definitively" (or "globally") accepted or rejected. A signer may treat multiple blocks at the same height as tentatively accepted or rejected.

This is something that I think needs some more elaboration. In the example above, signer 0 tentatively rejects the miner's first block because it saw burn block 134 before seing the proposal. Signers 1 and 2 tentatively accept, but because they don't receive a threshold of signatures (nevermind signer 0's rejection), their decisions aren't definitive.

Once all miners see burn block 134, the miner retries building the block. Signers 1 and 2 would now also tentatively accept the block. Once signer 0's acceptance is received, all signers definitively accept.

---

Here's an interesting question -- can we ensure that at most one block will be definitively accepted at a given block height? Because, what I've written above is insufficient -- it's possible that multiple tentatively accepted blocks at the same height can become definitively accepted.

We can change this to an "at most one" criteria if we're willing to add an extra round of communication:

• Round 1: what we do right now, but no definitive accept/reject decision gets made. In other words, signers treat every block as tentative accepted or rejected even if they've meet the required signature thresholds. Signers do not process or accept blocks into their chainstates.
• Round 2 (new): The miner chooses which tentatively-accepted block at height N will become definitively accepted. Only once this additional miner signature gets generated will a block be treated as definitively accepted. On receipt of this message (either via stackerdb or through a signed p2p block), a signer will treat the block as definitively accepted and simply delete all state for all other tentative blocks at that height.

In essence, this is a flavor of two-phase commit. We'd need the miner to instruct signers to commit to their validated blocks.

How might we achieve this? I think the answer is straightforward -- we just put the miner's round-2 signature into the block header.

EDIT: This assumes that the miner itself won't equivocate. I'll address that below.

[jcnelson]

What happens if the miner equivocates and signs and broadcasts two or more blocks at the same height in round 2? I think what must happen here at a minimum is signers that witness the equivocation will cease signing blocks from that miner for the rest of its tenure. Then, the next miner picks one of them and builds atop it.

This is straightforward to implement -- because the node will gladly process two Nakamoto blocks at the same height, it's easy to check to see if they were signed by the same miner. The node itself would track whether or not a tenure has two or more blocks signed at the same height, and report it to the signer.

If we're feeling adventurous, we could also slash the offending miner's coinbase. But, the above can ship after Nakamoto, since it's ultimately a signer policy choice to refuse to sign.

EDIT: If we're feeling less adventurous, but still want to make the miner suffer for its equivocation after Nakamoto ships, we can have the signer refuse to sign blocks originating from tenure block-commits coming from that equivocating miner, which forces the miner to re-register a VRF key and re-submit multiple block-commits before they can mine again.

[jferrant]

What happens if the miner equivocates and signs and broadcasts two or more blocks at the same height in round 2? I think what must happen here at a minimum is signers that witness the equivocation will cease signing blocks from that miner for the rest of its tenure. Then, the next miner picks one of them and builds atop it.

This is straightforward to implement -- because the node will gladly process two Nakamoto blocks at the same height, it's easy to check to see if they were signed by the same miner. The node itself would track whether or not a tenure has two or more blocks signed at the same height, and report it to the signer.

One question I have is, should the node even process two Nakamoto blocks at the same height? Is it better for the chainstate to reject it outright and not even process it or is there a valid reason why we would ever want to process two blocks at the same height?

[jcnelson]

One question I have is, should the node even process two Nakamoto blocks at the same height? Is it better for the chainstate to reject it outright and not even process it or is there a valid reason why we would ever want to process two blocks at the same height?

The Nakamoto chainstate DB must tolerate Nakamoto blocks at the same height because they can arise from Bitcoin forks.

[jferrant]

One question I have is, should the node even process two Nakamoto blocks at the same height? Is it better for the chainstate to reject it outright and not even process it or is there a valid reason why we would ever want to process two blocks at the same height?

The Nakamoto chainstate DB must tolerate Nakamoto blocks at the same height because they can arise from Bitcoin forks.

Ah of course. So additional logic would be required on the nodes side to ensure a miner was not punished for simply responding to a bitcoin fork. i.e. the burnchain consensus hash for the proposed block would have to be identical between the two proposed blocks signed by the same miner, yeah? EDIT: but this does make me wonder...in the case of a bitcoin fork...how would signers handle this? i.e. what if they signed a block built on a bitcoin fork. The subsequent stacks block...what should it look like and how should te singers handle it? I am sure we have some handling in place, but I don't think I have ever actually thought it through and am wondering how/if these changes would affect it.

[obycode]

Couldn't round 2 just be the miner mining the next block? It's already implicitly selecting the canonical block by mining the next block and proposing it, isn't it?