-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
🚀 Feature Request
Use Gossipsub's new PublishBatch method to broadcast things that can be batched, such as DataColumnSidecars.
Description
Without this, network writes tend to prioritize sending data for one column to all peers before sending data for later columns (e.g for two columns and 4 peers per column it would send A,A,A,A,B,B,B,B*). With batch publishing we can change the write order to round robin across columns (e.g. A,B,A,B,A,B,A,B).
In cases where the process is sending at a rate over the network limit, this approach allows at least some copies of every column to propagate through the network early. In early simulations with bandwidth limits of 50mbps for the publisher, this improved network dissemination by ~20-30%.
Describe the solution you'd like
I have a draft PR that updates the relevant call sites. There are no tests for it, help appreciated for how to test this in Prysm.
Describe alternatives you've considered
An alternative is to no change. With partial messages on data columns, this isn't needed as the data we send out initially is fairly small, and the order we send out cells is somewhat random (as it depends on how our peers respond). That said, partial messages is a backwards compatible change and this may still benefit the node if peers do not support partial messages.
- This isn't exactly correct as the writes may be interleaved in slightly different ways due to goroutine scheduling, but it tends to this pattern.