RaptorCast: Modular packet assembly #2377

xinyuan-dev · 2025-09-23T12:17:48Z

Design doc: https://www.notion.so/Modular-packet-assembly-28075b0ba8408067b8f9ecf9092d56d4

Benchmark result:

Test Case	Time (mean)	Throughput (mean)
Raptorcast 128K / old	546.91 µs	228.56 MiB/s
Raptorcast 128K / new	551.27 µs	226.75 MiB/s
Raptorcast 2M / old	9.0575 ms	220.81 MiB/s
Raptorcast 2M / new	8.9950 ms	222.34 MiB/s
Broadcast 128K / old	52.624 ms	237.53 MiB/s
Broadcast 128K / new	50.306 ms	248.48 MiB/s

Full Benchmark Logs

Raptorcast 128K/udp::build_messages
                        time:   [541.17 µs 546.91 µs 555.23 µs]
                        thrpt:  [225.13 MiB/s 228.56 MiB/s 230.98 MiB/s]
                 change:
                        time:   [+0.8834% +2.1325% +4.2982%] (p = 0.00 < 0.05)
                        thrpt:  [-4.1210% -2.0880% -0.8757%]
                        Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe
Raptorcast 128K/packet::build_messages
                        time:   [549.25 µs 551.27 µs 554.04 µs]
                        thrpt:  [225.62 MiB/s 226.75 MiB/s 227.58 MiB/s]
                 change:
                        time:   [+0.9739% +1.6640% +2.7696%] (p = 0.00 < 0.05)
                        thrpt:  [-2.6949% -1.6368% -0.9645%]
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe

Raptorcast 2M/udp::build_messages
                        time:   [8.9445 ms 9.0575 ms 9.1865 ms]
                        thrpt:  [217.71 MiB/s 220.81 MiB/s 223.60 MiB/s]
                 change:
                        time:   [-0.9076% +0.9565% +2.8338%] (p = 0.33 > 0.05)
                        thrpt:  [-2.7557% -0.9474% +0.9159%]
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe
Raptorcast 2M/packet::build_messages
                        time:   [8.9289 ms 8.9950 ms 9.0688 ms]
                        thrpt:  [220.54 MiB/s 222.34 MiB/s 223.99 MiB/s]
                 change:
                        time:   [-2.3773% -0.6885% +0.9216%] (p = 0.43 > 0.05)
                        thrpt:  [-0.9132% +0.6932% +2.4352%]
                        No change in performance detected.
Found 15 outliers among 100 measurements (15.00%)
  7 (7.00%) high mild
  8 (8.00%) high severe

Benchmarking Broadcast 128K/udp::build_messages: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.3s, or reduce sample count to 90.
Broadcast 128K/udp::build_messages
                        time:   [52.438 ms 52.624 ms 52.810 ms]
                        thrpt:  [236.70 MiB/s 237.53 MiB/s 238.38 MiB/s]
                 change:
                        time:   [-2.7506% -1.1749% -0.1138%] (p = 0.07 > 0.05)
                        thrpt:  [+0.1139% +1.1888% +2.8284%]
                        No change in performance detected.
Benchmarking Broadcast 128K/packet::build_messages: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, or reduce sample count to 90.
Broadcast 128K/packet::build_messages
                        time:   [50.093 ms 50.306 ms 50.522 ms]
                        thrpt:  [247.42 MiB/s 248.48 MiB/s 249.54 MiB/s]
                 change:
                        time:   [-0.1627% +0.6978% +1.4534%] (p = 0.09 > 0.05)
                        thrpt:  [-1.4326% -0.6929% +0.1629%]
                        No change in performance detected.

How it's tested:

existing features are tested to be compatible with the old udp::build_message function, which itself is extensively tested in udp.rs
benchmarking metrics is attached above to show no performance regression
new features are checked with unit tests
- round-robin packet assembly (implements monad-raptorcast: switch to round robin ordering #2318)
- stake-based with rounding (implements Feature: round up RaptorCast chunk assignments #2215)

Copilot

Pull Request Overview

This PR implements a modular packet assembly system for RaptorCast, introducing structured chunk assignment and UDP message serialization. The implementation adds support for different broadcast types (broadcast, stake-based, and unicast) with configurable redundancy and packet layouts.

Key changes:

Introduces a modular chunk assignment system with three assigner types
Implements packet layout management with merkle tree support and cryptographic signatures
Adds UDP message serialization with grouping by destination and priority

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File	Description
monad-raptorcast/src/packet/assigner.rs	Implements chunk assignment strategies for broadcast, stake-based, and unicast distribution
monad-raptorcast/src/packet.rs	Core packet assembly logic with cryptographic signing, merkle tree generation, and UDP serialization
monad-raptorcast/src/lib.rs	Adds packet module to library exports

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

monad-raptorcast/src/packet/assigner.rs

monad-raptorcast/src/packet.rs

Copilot

Pull Request Overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

monad-raptorcast/src/packet/assembler.rs

monad-raptorcast/benches/encode_bench.rs

xinyuan-dev · 2025-10-10T07:47:09Z

would it be possible to provide iterator-like interface to consume chunks?

Implemented! The last assemble function now accepts a callback function or a container (via collector: &mut impl Collector<UdpMessage>) that will be called as soon as a merkle batch is signed and assembled. This interface should be equivalent to an iterator.

Note that the streaming mode is only validly defined for certain "serialization modes". For example, the round-robin mode would be unavailable for the streaming because it would be impossible to implement round-robin on small batches.

monad-raptorcast/src/packet/mod.rs

dshulyak · 2025-10-13T08:01:15Z

Note that the streaming mode is only validly defined for certain "serialization modes". For example, the round-robin mode would be unavailable for the streaming because it would be impossible to implement round-robin on small batches.

thats unfortunate, i had a hunch that it should be possible by caching signature and outputting first chunk early, but i might be completely wrong. i am also reviewing a change in general, will approve later in a day

monad-raptorcast/src/packet/mod.rs

monad-raptorcast/src/packet/serializer.rs

dshulyak · 2025-10-13T13:32:27Z

monad-raptorcast/src/packet/serializer.rs

+            priority: Option<u8>,
+        }
+
+        let mut messages: HashMap<GroupKey, BytesMut> = HashMap::new();


original implementation was deterministic or it was using hash map as well like this? what it was using for order if it was deterministic?

original implementation was deterministic because it relies on the chunks assigned to be in a specific order so that each recipients' chunks are always adjacent. the new framework allows more versatile chunk assignment algorithms by lifting this restraint. therefore the order of recipients outputted from the round-robin serializer would not be the same as the original order. but the round-robin property is inaffected, and for each recipient, their received packets are still in the correct order.

i just wanted to be sure that we don't break some implicit order assumption, that wasn't well documented.
like for example ordering them by stake-weight

dshulyak · 2025-10-13T13:35:31Z

monad-raptorcast/src/packet/serializer.rs

+        }
+
+        // Each recipient get their own queue of chunks.
+        let mut queues: Vec<VecDeque<Chunk<PT>>> = buckets.into_values().collect();


it converts to vec from hashmap for order? would it make a difference if it was stored in btree and then iteration below is on btree

yes I used vec mainly for the ability of going through it over repeatedly while deleting recipients with empty queues. retain_mut is perfect for this purpose. it should be able to do the same on btreemap with its retain method. however, i expect using btreemap to allocating more than hashmap/vec approach due to its lack of with_capacity constructor, also i expect hashmap/vec to perform better generally due to cache efficiency. that said, those are my speculations and i can benchmark both versions.

Copilot AI review requested due to automatic review settings September 23, 2025 12:17

xinyuan-dev requested review from ariqchowdhury, michael-yxchen and omegablitz as code owners September 23, 2025 12:17

xinyuan-dev marked this pull request as draft September 23, 2025 12:19

Copilot AI reviewed Sep 23, 2025

View reviewed changes

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch 15 times, most recently from 72d6cb8 to 797534c Compare October 1, 2025 13:28

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch 3 times, most recently from 43922a1 to eab84f5 Compare October 1, 2025 15:56

xinyuan-dev requested a review from Copilot October 1, 2025 16:06

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch from eab84f5 to 38da74e Compare October 1, 2025 16:06

Copilot AI reviewed Oct 1, 2025

View reviewed changes

monad-raptorcast/src/packet/assembler.rs Outdated Show resolved Hide resolved

monad-raptorcast/benches/encode_bench.rs Outdated Show resolved Hide resolved

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch 3 times, most recently from 04142a1 to df65d60 Compare October 1, 2025 16:55

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch 6 times, most recently from 5f86e57 to 4d8d0af Compare October 9, 2025 15:28

omegablitz reviewed Oct 10, 2025

View reviewed changes

monad-raptorcast/src/packet/mod.rs Show resolved Hide resolved

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch from 4d8d0af to f3f2ae6 Compare October 13, 2025 13:14

dshulyak reviewed Oct 13, 2025

View reviewed changes

monad-raptorcast/src/packet/mod.rs Outdated Show resolved Hide resolved

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch from f3f2ae6 to b37ec60 Compare October 13, 2025 13:37

dshulyak reviewed Oct 13, 2025

View reviewed changes

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch 3 times, most recently from 9e7916b to b9bca99 Compare October 14, 2025 11:17

dshulyak previously approved these changes Oct 14, 2025

View reviewed changes

xinyuan-dev dismissed dshulyak’s stale review via 8bbe567 October 14, 2025 12:29

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch 4 times, most recently from 4dfd671 to a242d42 Compare October 15, 2025 05:43

dshulyak approved these changes Oct 15, 2025

View reviewed changes

omegablitz approved these changes Oct 16, 2025

View reviewed changes

modular raptorcast packet assembly

a08b279

xinyuan-dev force-pushed the xinyuan/modularize-chunk-assignment branch from a242d42 to a08b279 Compare October 16, 2025 03:21

xinyuan-dev added this pull request to the merge queue Oct 16, 2025

Merged via the queue into master with commit e4277e6 Oct 16, 2025
2 of 4 checks passed

xinyuan-dev deleted the xinyuan/modularize-chunk-assignment branch October 16, 2025 04:08

Uh oh!

RaptorCast: Modular packet assembly #2377

RaptorCast: Modular packet assembly #2377

Uh oh!

Conversation

xinyuan-dev commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

xinyuan-dev commented Oct 10, 2025

Uh oh!

Uh oh!

dshulyak commented Oct 13, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dshulyak Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

xinyuan-dev Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

dshulyak Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

dshulyak Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xinyuan-dev Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xinyuan-dev commented Sep 23, 2025 •

edited

Loading

dshulyak Oct 13, 2025 •

edited

Loading