-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
Description
Summary
Update scheduler-to-worker dataset routing logic to support multiple data node targets per fetch.
- Allow a
Fetchrequest to include multiple peer IDs (data nodes) with a random selection strategy. - Adjust the scheduler to return a list of data nodes for each slice.
- Ensure fallback behavior is graceful: if one data node fails, retries will hit another peer.
- Reuse and extend the existing
Reference::Peersvariant andSelectionStrategy::Random.
Background
Currently, a worker fetches a slice from only one data node, which is a single point of failure. This change enables fault-tolerant fetching and prepares the system for distributed data delivery at scale.
Reactions are currently unavailable