Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: panic recovery in PrepareProposal and ProcessProposal Handlers #14381

Merged
merged 4 commits into from
Dec 21, 2022

Conversation

alexanderbez
Copy link
Contributor

Description

Closes: #14375


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@alexanderbez alexanderbez added the backport/v0.47.x PR scheduled for inclusion in the v0.47's next stable release label Dec 21, 2022
@github-actions github-actions bot removed the C:x/bank label Dec 21, 2022
@alexanderbez alexanderbez marked this pull request as ready for review December 21, 2022 16:00
@alexanderbez alexanderbez requested a review from a team as a code owner December 21, 2022 16:00
"hash", fmt.Sprintf("%X", req.Hash),
"panic", err,
)
resp = abci.ResponseProcessProposal{Status: abci.ResponseProcessProposal_REJECT}
Copy link
Member

@julienrbrt julienrbrt Dec 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't the blockchain stop producing blocks if there is a deterministic bug that makes all proposal to be rejected? Does not it has effectively the same effect than a halt? 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean it is still better than stopping the node, but it is still stuck then?

Copy link
Contributor Author

@alexanderbez alexanderbez Dec 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since ProcessProposal is defined by the application and could very well define behavior that depends on the proposal's contents, i.e. txs, then the behavior cannot be guaranteed to be deterministic.

That being said, if a panic is triggered, the proposal is rejected. As such, Tendermint will proceed to the next validator to propose a new proposal in PrepareProposal and the round starts again. At some point a valid proposal will need to be produced and accepted. If no such proposal occurs, then yes, the chain will face a liveness halt, but this will be very clear and investigation will need to occur.

Copy link
Contributor Author

@alexanderbez alexanderbez Dec 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a quick followup in case folks come back to this comment in the future:

  • PrepareProposal MAY be non-deterministic
  • ProcessProposal MAY NOT be non-deterministic, i.e. it must be deterministic

This means if ProcessProposal panics and we reject, all honest validator processes p will prevote nil and the round will proceed again until a valid proposal is proposed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sonarcloud
Copy link

sonarcloud bot commented Dec 21, 2022

[Cosmos SDK] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@alexanderbez alexanderbez merged commit 1152e5b into main Dec 21, 2022
@alexanderbez alexanderbez deleted the bez/14375-recovery branch December 21, 2022 17:32
mergify bot pushed a commit that referenced this pull request Dec 21, 2022
alexanderbez added a commit that referenced this pull request Dec 21, 2022
…backport #14381) (#14383)

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
@alexanderbez alexanderbez mentioned this pull request Dec 22, 2022
19 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/v0.47.x PR scheduled for inclusion in the v0.47's next stable release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add recovery for panic in PrepareProposal and ProcessProposal
3 participants