Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter votes from disabled validators in BackedCandidates in process_inherent_data #1863

Merged
merged 43 commits into from
Nov 30, 2023
Merged
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
259cb4b
Filter backing votes in `paras_inherent` - initial work
tdimitrov Oct 10, 2023
d451886
Error handling
tdimitrov Oct 12, 2023
a3bafba
Merge branch 'master' into tsv-disabling-runtime
tdimitrov Oct 16, 2023
75d707a
Merge branch 'tsv-disabling' into tsv-disabling-runtime
tdimitrov Oct 16, 2023
cb0d8fc
Add `fn disabled_validators` to `trait DisabledValidators`. Use the t…
tdimitrov Oct 16, 2023
8a8893e
Extract the implementation of `disabled_validators` from rutnime api …
tdimitrov Oct 17, 2023
7b868b0
Error handling
tdimitrov Oct 17, 2023
a41d21c
Remove `DisabledValidators` from paras_inherent
tdimitrov Oct 17, 2023
34a3bf0
Fix a warning
tdimitrov Oct 17, 2023
88eb018
Merge branch 'tsv-disabling' into tsv-disabling-runtime
tdimitrov Oct 23, 2023
305d5b8
Merge branch 'tsv-disabling' into tsv-disabling-runtime
tdimitrov Oct 24, 2023
e241312
Add setters used for unit tests
tdimitrov Oct 26, 2023
2aa1323
Mock for `DisabledValidators` in Test runtime
tdimitrov Oct 26, 2023
a8efd15
Fix test state and add an optimisation for `filter_backed_statements`
tdimitrov Oct 26, 2023
3567a93
Make disabled validators mock mutable
tdimitrov Oct 26, 2023
04dc32a
Extract common code in `get_test_data`
tdimitrov Oct 26, 2023
cce71ee
Additional cases
tdimitrov Oct 26, 2023
7a19f55
Fixes in `filter_backed_statements`
tdimitrov Oct 26, 2023
3260187
Refactor `filter_backed_statements`
tdimitrov Oct 26, 2023
ff577a5
Merge branch 'tsv-disabling' into tsv-disabling-runtime
ordian Oct 26, 2023
876655d
Apply suggestions from code review
tdimitrov Oct 28, 2023
ed2d430
wip: Filtering backed statements from disabled validators is performe…
tdimitrov Oct 31, 2023
9a8985a
para_id_to_core_idx -> scheduled
tdimitrov Oct 31, 2023
f7544d7
Fix code duplication in tests
tdimitrov Oct 31, 2023
59fc176
Fix a compilation error
tdimitrov Oct 31, 2023
cb2fe60
Fix backing votes filtering - if more than `effective_minimum_backing…
tdimitrov Nov 2, 2023
be85b5d
Ensure inherent data contains no backing votes from disabled validato…
tdimitrov Nov 2, 2023
f472bb1
Comments and small fixes
tdimitrov Nov 2, 2023
53aaffa
Fix disabled statements filtering
tdimitrov Nov 3, 2023
16823f4
Updated comments
tdimitrov Nov 3, 2023
452b202
Merge branch 'tsv-disabling' into tsv-disabling-runtime
tdimitrov Nov 6, 2023
cb60cf0
Apply suggestions from code review
tdimitrov Nov 15, 2023
750f2db
Code review feedback
tdimitrov Nov 15, 2023
7022003
Merge branch 'tsv-disabling' into tsv-disabling-runtime
tdimitrov Nov 16, 2023
8b5e5a8
Apply suggestions from code review: remove duplicated assert
tdimitrov Nov 28, 2023
a50800f
Code review feedback - filter_backed_statements_from_disabled -> filt…
tdimitrov Nov 28, 2023
7a308cd
Update parainherent.md with some details about data sanitization
tdimitrov Nov 28, 2023
7ccb942
Add newline to parainherent.md
tdimitrov Nov 28, 2023
1834e20
Apply suggestions from code review
tdimitrov Nov 28, 2023
610b45d
Apply suggestions from code review
tdimitrov Nov 29, 2023
83adc93
Fixes in parainherent.md
tdimitrov Nov 29, 2023
158557d
Merge branch 'tsv-disabling' into tsv-disabling-runtime
tdimitrov Nov 29, 2023
79f4c40
Merge branch 'tsv-disabling' into tsv-disabling-runtime
tdimitrov Nov 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixes in parainherent.md
  • Loading branch information
tdimitrov committed Nov 29, 2023
commit 83adc9332a298c42e63fd64d910a6c6e70385e40
28 changes: 16 additions & 12 deletions polkadot/roadmap/implementers-guide/src/runtime/parainherent.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,28 +62,32 @@ of a block is rather limited. Nevertheless if we ever wanted to optimize this we
collector that has two implementations, where one clones and stores the data and the other just passes it on.

## Data sanitization
`ParasInherent` with the entry point of `create_inherent` sanitizes the input data, while the `enter` entry point enforces already sanitized input data. If unsanitized data is provided the module generates an error.
`ParasInherent` with the entry point of `create_inherent` sanitizes the input data, while the `enter` entry point
enforces already sanitized input data. If unsanitized data is provided the module generates an error.

Disputes are included in the block with a priority for a security reasons. It's important to include as many dispute votes
onchain as possible so that disputes conclude faster and the offenders are punished. However if there are too many
Disputes are included in the block with a priority for a security reasons. It's important to include as many dispute
votes onchain as possible so that disputes conclude faster and the offenders are punished. However if there are too many
disputes to include in a block the dispute set is trimmed so that it respects max block weight.

Dispute data is first deduplicated and sorted by block number (older first) and dispute location (local then remote).
Concluded and ancient (disputes initiated before the post conclusion acceptance period ) disputes are filtered out.
Votes with invalid signatures and from unknown validators (not found in the active set for the current session) are also
Concluded and ancient (disputes initiated before the post conclusion acceptance period) disputes are filtered out.
Votes with invalid signatures or from unknown validators (not found in the active set for the current session) are also
filtered out.

All dispute statements are included in the order described in the previous paragraph until the available block
weight is exhausted. After the dispute data is included all remaining weight is filled in with candidates and
availability bitfields. Bitfields are included with priority, then candidates containing code updates and finally any
backed candidates. If there is not enough weight for all backed candidates they are trimmed by random selection.
Disputes are processed in three separate functions - `deduplicate_and_sort_dispute_data`, `filter_dispute_data` and
All dispute statements are included in the order described in the previous paragraph until the available block weight is
exhausted. After the dispute data is included all remaining weight is filled in with candidates and availability
bitfields. Bitfields are included with priority, then candidates containing code updates and finally any backed
candidates. If there is not enough weight for all backed candidates they are trimmed by random selection. Disputes are
processed in three separate functions - `deduplicate_and_sort_dispute_data`, `filter_dispute_data` and
`limit_and_sanitize_disputes`.

Availability bitfields are also sanitized by dropping malformed ones, containing disputed cores or bad signatures. Refer
to `sanitize_bitfields` function for implementation details.

Backed candidates sanitization removes malformed ones, candidates which have got concluded invalid disputes against them
or candidates produced by unassigned cores. Furthermore any backing votes from disabled validators for a candidate are
dropped. This is part of the validator disabling strategy. These checks are implemented in `sanitize_backed_candidates`
and `filter_backed_statements_from_disabled_validators`.
dropped. This is part of the validator disabling strategy. After filtering the statements from disabled validators a
backed candidate may end up with votes count less than `minimum_backing_votes` (a parameter from `HostConfiguiration`).
In this case the whole candidate is dropped otherwise it will be rejected by `process_candidates` from pallet inclusion.
All checks related to backed candidates are implemented in `sanitize_backed_candidates` and
`filter_backed_statements_from_disabled_validators`.