feat(starfish): send only useful shards #8801
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 6 Skipped Deployments
|
VorobyevIlya
force-pushed
the
consensus/feat/starfish/send-only-useful-shards
branch
from
68da5d0 to
cbbff27
Compare
VorobyevIlya
force-pushed
the
consensus/feat/starfish/send-only-useful-shards
branch
from
4f48fc4 to
57edd30
Compare
| pub(crate) const MAX_TRANSACTIONS_ACK_DEPTH: Round = 50; | ||
| pub(crate) const MAX_HEADERS_PER_BUNDLE: usize = 150; | ||
| pub(crate) const MAX_SHARDS_PER_BUNDLE: usize = 150; | ||
| const MAX_ROUND_GAP_FOR_USEFUL_SHARDS: usize = 5; |
There was a problem hiding this comment.
Please describe what this constant controls.
|
|
||
| // Remove the references from the set and update cordial knowledge | ||
| for block_ref in &to_take { | ||
| set.remove(block_ref); |
There was a problem hiding this comment.
just to make sure: if the blocks are not removed from the set here then they will be evicted at some later point?
There was a problem hiding this comment.
yes, they are removed in evict_cordial_knowledge function
| } | ||
| } | ||
| shards | ||
| useful_authorities |
There was a problem hiding this comment.
imo useful_shards_authors is better understandable than useful_authorities. wdyt? maybe we could also change it for headers? Especially when a method is called take_useful_shards_for_authority and a param is called useful_authorities - useful_shards_authors would be more descriptive.
There was a problem hiding this comment.
yes, useful_shard_authors is better. Changed it.
| } | ||
|
|
||
| impl SerializedBlockBundleParts { | ||
| pub(crate) fn useful_authorities(&self) -> BTreeSet<AuthorityIndex> { |
There was a problem hiding this comment.
better rename to useful_headers_authors
| /// useful to them (communicated inside their block bundles). | ||
| /// Keyed by the peer’s AuthorityIndex. | ||
| useful_authorities_to_peer: Arc<RwLock<BTreeMap<AuthorityIndex, BTreeSet<AuthorityIndex>>>>, | ||
| /// For each peer, stores the latest round in which a received block bundle |
There was a problem hiding this comment.
here also rename the header-related fields to be in line with the shard ones
| useful_authorities_from_peer_write_guard.insert(peer, useful_authorities); | ||
| } | ||
| { | ||
| let useful_shard_authors = additional_block_headers |
There was a problem hiding this comment.
variable is called useful_shard_authors but we iterate over additional_block_headers? is this correct?
Also, add a comment describing what this block of code does, or update comment for step 13. above.
There was a problem hiding this comment.
Yes, these headers went through filters and will now be added to dag_state without transaction data, so we request their shards.
Added comment
VorobyevIlya
force-pushed
the
consensus/feat/starfish/send-only-useful-shards
branch
from
4796d29 to
ea4b2f5
Compare
Description of change
PR tracks shards of which authors could be sent to other validators.
Shards for encoded transaction data of validator$v$ are considered as useful for validator $u$ if validator $u$ is ready to include an additional block header of validator $v$ after filtering those blocks. In particular, this means that$v$ indirectly.
the potential sequenced transaction data comes from validator
If the shard of the block created by the validator$v$ was useful, then the shards from the blocks created by $v$ will be sent for some number of rounds. In other words, there is some inertia; we don't stop sending shards as soon as one of them is useless.
In practice, there are two prerequisites for the efficiency of this mechanism:
Links to any relevant issues
fixes #8767
How the change has been tested