Enforce ProgressiveList limits defined on STF during unmarshal - #17412
Open
kasey wants to merge 2 commits into
Open
Enforce ProgressiveList limits defined on STF during unmarshal#17412kasey wants to merge 2 commits into
kasey wants to merge 2 commits into
Conversation
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background and rationale
EIP-7688 and EIP-7916 define a replacement for the
Listfamily of ssz collection types supporting progressive merkleization: ProgressiveList, ProgressiveBitlist, ProgressiveByteList. Unlike the base ssz design, progressive collection types do not have a strict maximum size defined in their spec type definition. Previously the definition not only changed how the fields were merkleized, but also moved validation as early in the p2p pipeline as possible, to happen at the unmarshal step. We still have limits we want to enforce (e.g.MAX_ATTESTATIONS_ELECTRA), but those limit checks have migrated from the unmarshal step to a validation function on the STF.When upstream specs moved to this scheme, spectest generators stopped limiting the size of test inputs according to the limits, requiring implementations to accept inputs that were outside the bounds. In prysm's codegen we dropped the validation previously performed at the unmarshal step to accept these fixtures. However this change unnecessarily defers validation that we could perform up front, potentially spreading the responsibility for checking these lengths to various parts of the application in ways that could lead to bugs. It could also open us up to limited griefing from malicious inputs that cause extra memory allocation for values that the decoder can easily determine are invalid.
So I've asked @jtraglia for a compromise: modify specs so that spectest generation is aware of the valid range of values for these types with limits enforced at the STF, so that fixtures can avoid exceeding those ranges. That allows us to pass ssz-static spectests using types where we have restored input size validation to our unmarshal methods. This also gives spectest generators a convenient place to pick a size for values to exceed in fixtures where the STF checks are being tested (the spectest generator can use
> $LIMITfor invalid inputs, and== $LIMITfor valid inputs).The upstream changes are in consensus-specs PR 5564. This PR should update to the version pin including that PR once it merges (and will fail with length validation errors in the meantime).
Acknowledgements
Stack created with GitHub Stacks CLI • Give Feedback 💬