Feature: round up RaptorCast chunk assignments #2215
Conversation
tklenze
force-pushed
the
tobias/roundupchunks
branch
3 times, most recently
from
759da1c to
7f1468c
Compare
tklenze
force-pushed
the
tobias/roundupchunks
branch
6 times, most recently
from
1257918 to
525394e
Compare
Round up the number of chunks assigned to each validator in the first phase of RaptorCast. This is required to guarantee Byzantine Fault Tolerance.
tklenze
force-pushed
the
tobias/roundupchunks
branch
from
525394e to
bf6268f
Compare
tklenze
requested review from
ariqchowdhury,
michael-yxchen and
omegablitz
as code owners
There was a problem hiding this comment.
Pull Request Overview
This PR implements the round-up functionality for RaptorCast chunk assignments to guarantee Byzantine Fault Tolerance. It ensures that each validator receives at least one chunk by computing stake-weighted obligations and rounding up the assignments.
Key changes:
- Added chunk assignment logic that computes initial chunks per validator based on stake weight
- Introduced rounding up mechanism where each validator gets their obligation (rounded down) plus one additional "rounding chunk"
- Modified the packet computation to account for the rounded-up total
Reviewed Changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| monad-raptorcast/src/udp.rs | Core implementation of chunk assignment algorithm and packet computation with round-up logic |
| monad-raptorcast/src/util.rs | Updated Group struct to handle validator stake maps and added iterator functionality |
| monad-raptorcast/tests/raptorcast_instance.rs | Updated test to use specific redundancy value instead of MAX_REDUNDANCY constant |
| monad-raptorcast/Cargo.toml | Added alloy-primitives dependency for U256 arithmetic |
Comments suppressed due to low confidence (2)
monad-raptorcast/src/udp.rs:1
- This comment contains unclear notation '//TK: ???' which appears to be a TODO or question marker left by a developer. This should either be clarified or removed.
// Copyright (C) 2025 Category Labs, Inc.
monad-raptorcast/src/udp.rs:1
- Typo in comment: 'valdiator' should be 'validator'.
// Copyright (C) 2025 Category Labs, Inc.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| // The obligations are scaled by the basis points (bps) to avoid floating point arithmetic on | ||
| // the large values of stake and total_weight. Only at the very end we convert to a float. | ||
| // In the future, we might want to avoid floats alltogether. | ||
| let unit_bias: f64 = 10_000_000.0; |
There was a problem hiding this comment.
The magic number 10_000_000.0 should be defined as a named constant with documentation explaining its purpose in avoiding floating point precision issues.
| let unit_bias: f64 = 10_000_000.0; | |
| let unit_bias: f64 = STAKE_UNIT_BIAS; |
|
If we have 100 equally staked validators and 100 chunks. and say redundancy 3, does it set total n_chunks to 400 chunks ? |
| *addr, | ||
| start_idx * segment_size as usize..end_idx * segment_size as usize, | ||
| )); | ||
| trace_string.push_str(&format!("{:?}: {} chunks, ", node_id, num_chunks)); |
There was a problem hiding this comment.
we probably do not want to leave have the tracing code run conditionlessly. how about put a trace statement here instead?
| // The obligations are scaled by the basis points (bps) to avoid floating point arithmetic on | ||
| // the large values of stake and total_weight. Only at the very end we convert to a float. | ||
| // In the future, we might want to avoid floats alltogether. | ||
| let unit_bias: f64 = 10_000_000.0; |
There was a problem hiding this comment.
is this bias necessary? stake / total_stake is a fractional number between 0 and 1. In IEEE floating point representation, there is roughly same number of representable real numbers between 0 and 1, versus between 1 and infinity. (source) so there should be enough precision if stake / total_stake is precise enough.
4 participants
Round up the number of chunks assigned to each validator in the first phase of validator-to-validator RaptorCast.
This change is required to guarantee Byzantine Fault Tolerance. It implements the publicly documented behavior of RaptorCast, ensuring documentation and implementation are aligned.
This feature only affects the leader when sending out a block proposal using RaptorCast. The leader operates as follows:
num_packets) without rounding up. This is essentially the payload size divided by the payload per chunk times the redundancy factor.num_packets * stake / total_stake. This value is a float.ic[i]) and a rounding chunk (each validator receives exactly one). While this distinction is not currently important, it may become relevant in the future.ic[i]plus the number of validatorsn(since each receives one rounding chunk).As a safeguard, we check that the rounded-up total
rounded_up_num_packetsfalls between the originalnum_packetsandnum_packets + n. This should always be true; if not, we issue a warning. In casenum_packetsis larger, we use that value instead.Other minor features
Performance impact
The solution of rounding up requires sending up to
nmore chunks than necessary -- each one of which will get re-broadcasted by a validator. Each validator sends and receives~nmore chunks than required. For small messages, the overhead is very large. As such, it is only a temporary solution; a more permanent solution is planned.We simulated the overhead created by rounding up. The following graph shows the additional bandwidth consumed at validators due to rounding up, for various values of
num_packetsand numbers of validatorsn. 4740 chunks represents a full block, whereas 474 chunks present a block that is only 10% full. (Note that the graph says "current RaptorCast", but this is actually the solution proposed in this PR.)For the leader's block proposal message, the additional overhead is likely not prohibitive for small-medium network sizes (e.g.,
n = 200).Required actions before merge
encoded_symbol_capacityin the decoder. Right now, this capacity is calculated asmax(app_message_len / symbol_len, SOURCE_SYMBOLS_MIN) * MAX_REDUNDANCY. Due to rounding up, for small messages, more chunks than that could be created, and they would not be added to the decoder state. I think this might not be an issue, because (i) those chunks are not required for decoding and (ii) we still broadcast those chunks if they are assigned to us (self_hash == parsed_message.recipient_hash), even if we don't end up storing them.Closes https://github.com/category-labs/category-internal/issues/898 (but a follow-up issue is needed, to address the inefficiency of rounding up).