Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,13 @@ This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

### Added

- **ADR-0066 per-class ECMP caps.** New per-table `[[fib_tables]].maximum_paths_ebgp`
and `maximum_paths_ibgp` (FRR's `maximum-paths` / `maximum-paths ibgp`) cap eBGP
and iBGP equal-cost groups independently. They override the table's overall
`maximum_paths` for their class and fall back to it (then `1`) when unset, so
existing configs are unchanged. The equal-cost group is homogeneous, so the
best route's class selects the cap at projection; the RIB gathers siblings at
the widest of the three caps. Validated `>= 1`, capped at 256.
- **ADR-0066 multipath-relax.** New global `[global].multipath_relax` (default
`false`) relaxes unicast ECMP grouping from an exact `AS_PATH` match to
`AS_PATH`-*length* equality, so equal-length paths through different ASes
Expand Down
9 changes: 5 additions & 4 deletions docs/COMPARISON.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,10 +177,11 @@ for the EVPN gate ladder.
route — opt-in per table, default `1` (single next-hop). The global
`[global].multipath_relax` knob relaxes the default exact-`AS_PATH` grouping
to `AS_PATH`-length-only (FRR's `bgp bestpath as-path multipath-relax`).
Add-Path multi-path *send* (RFC 7911, route-server mode) and EVPN aliasing
ECMP (ADR-0059 FDB nexthop groups, default-on) also ship. Per-class
`maximum_paths_ebgp`/`ibgp` and weighted/unequal-cost multipath are the
remaining follow-ups.
Per-class caps (`maximum_paths_ebgp` / `maximum_paths_ibgp`, FRR parity) let
eBGP and iBGP groups carry different widths. Add-Path multi-path *send*
(RFC 7911, route-server mode) and EVPN aliasing ECMP (ADR-0059 FDB nexthop
groups, default-on) also ship. Weighted/unequal-cost multipath is the
remaining follow-up.

## Memory (200k prefixes, bgperf2)

Expand Down
3 changes: 3 additions & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1606,6 +1606,9 @@ kernel row.
| `allowed_peer_groups` | string[] | no | `[]` | Optional source peer-group allow-list. Entries must reference existing `[peer_groups.NAME]` blocks |
| `allowed_neighbors` | string[] | no | `[]` | Optional source neighbor-address allow-list. Entries must parse as IPv4 or IPv6 addresses |
| `max_routes` | u32 | no | unset | Optional hard cap. `0` is rejected; exceeding the cap freezes existing owned rows and suppresses growth for that table |
| `maximum_paths` | u32 | no | `1` | Unicast multipath/ECMP: install up to N equal-cost next-hops per prefix as a kernel `RTA_MULTIPATH` route (ADR-0066). `1` (or unset) = single next-hop, today's behavior. Validated `>= 1`, capped at 256 |
| `maximum_paths_ebgp` | u32 | no | unset | Per-class ECMP cap for **eBGP** groups (FRR's `maximum-paths`). Overrides `maximum_paths` for eBGP best routes; falls back to `maximum_paths` then `1`. Validated `>= 1`, capped at 256 |
| `maximum_paths_ibgp` | u32 | no | unset | Per-class ECMP cap for **iBGP** groups (FRR's `maximum-paths ibgp`). Overrides `maximum_paths` for iBGP best routes; falls back to `maximum_paths` then `1`. Validated `>= 1`, capped at 256 |

**Restart required**: `[[fib_tables]]` is resolved at startup and the
runtime actor is spawned once. SIGHUP edits are surfaced by
Expand Down
21 changes: 17 additions & 4 deletions src/config/schema.rs
Original file line number Diff line number Diff line change
Expand Up @@ -984,12 +984,25 @@ pub struct FibTableConfig {
/// Optional maximum number of equal-cost next-hops to install per prefix
/// (unicast multipath / ECMP, ADR-0066). Unset or `1` programs a single
/// next-hop — exactly today's behavior. Higher values install up to this
/// many equal-cost paths as a kernel `RTA_MULTIPATH` route. Applies
/// uniformly to homogeneous eBGP and iBGP equal-cost groups. Distinct from
/// `max_routes`, which caps the number of prefixes (rows), not the
/// next-hops per row. Validated `>= 1`, capped at 256.
/// many equal-cost paths as a kernel `RTA_MULTIPATH` route. Applies to any
/// homogeneous eBGP or iBGP equal-cost group that does not have a per-class
/// override below. Distinct from `max_routes`, which caps the number of
/// prefixes (rows), not the next-hops per row. Validated `>= 1`, capped at
/// 256.
#[serde(default)]
pub maximum_paths: Option<u32>,
/// Per-class ECMP cap for **eBGP** equal-cost groups (FRR's
/// `maximum-paths`). Overrides `maximum_paths` for eBGP best routes; when
/// unset, eBGP groups fall back to `maximum_paths` (then `1`). Validated
/// `>= 1`, capped at 256.
#[serde(default)]
pub maximum_paths_ebgp: Option<u32>,
/// Per-class ECMP cap for **iBGP** equal-cost groups (FRR's
/// `maximum-paths ibgp`). Overrides `maximum_paths` for iBGP best routes;
/// when unset, iBGP groups fall back to `maximum_paths` (then `1`).
/// Validated `>= 1`, capped at 256.
#[serde(default)]
pub maximum_paths_ibgp: Option<u32>,
}

fn default_fib_families() -> Vec<String> {
Expand Down
30 changes: 30 additions & 0 deletions src/config/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5817,6 +5817,14 @@ fn fib_tables_reject_invalid_guardrails() {
"maximum_paths must be greater than zero",
),
(r"maximum_paths = 9999", "exceeds the supported cap"),
(
r"maximum_paths_ebgp = 0",
"maximum_paths_ebgp must be greater than zero",
),
(
r"maximum_paths_ibgp = 9999",
"maximum_paths_ibgp 9999 exceeds the supported cap",
),
];

for (line, expected) in cases {
Expand All @@ -5843,6 +5851,28 @@ metric = 200
}
}

#[test]
fn fib_tables_per_class_maximum_paths_round_trip() {
let toml = format!(
r#"
{}

[[fib_tables]]
name = "edge"
table_id = 1000
metric = 200
maximum_paths = 2
maximum_paths_ebgp = 4
maximum_paths_ibgp = 8
"#,
valid_toml()
);
let t = &parse(&toml).unwrap().fib_tables[0];
assert_eq!(t.maximum_paths, Some(2));
assert_eq!(t.maximum_paths_ebgp, Some(4));
assert_eq!(t.maximum_paths_ibgp, Some(8));
}

#[test]
fn fib_tables_reject_duplicate_allowed_peer_groups() {
let toml = format!(
Expand Down
21 changes: 12 additions & 9 deletions src/config/validation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -717,24 +717,27 @@ fn validate_fib_table_guardrails(
),
});
}
if let Some(maximum_paths) = table.maximum_paths {
if maximum_paths == 0 {
// Same `>= 1` / `<= cap` guardrail for the overall and per-class ECMP caps.
let check_max_paths = |field: &str, value: Option<u32>| -> Result<(), ConfigError> {
let Some(v) = value else { return Ok(()) };
if v == 0 {
return Err(ConfigError::InvalidFibTable {
reason: format!(
"name {:?}: maximum_paths must be greater than zero",
table.name
),
reason: format!("name {:?}: {field} must be greater than zero", table.name),
});
}
if maximum_paths > FIB_MAX_MAXIMUM_PATHS {
if v > FIB_MAX_MAXIMUM_PATHS {
return Err(ConfigError::InvalidFibTable {
reason: format!(
"name {:?}: maximum_paths {maximum_paths} exceeds the supported cap of {FIB_MAX_MAXIMUM_PATHS}",
"name {:?}: {field} {v} exceeds the supported cap of {FIB_MAX_MAXIMUM_PATHS}",
table.name
),
});
}
}
Ok(())
};
check_max_paths("maximum_paths", table.maximum_paths)?;
check_max_paths("maximum_paths_ebgp", table.maximum_paths_ebgp)?;
check_max_paths("maximum_paths_ibgp", table.maximum_paths_ibgp)?;
let mut seen_groups = std::collections::HashSet::new();
for group in &table.allowed_peer_groups {
if !config.peer_groups.contains_key(group) {
Expand Down
93 changes: 87 additions & 6 deletions src/fib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,19 @@ pub(crate) fn project_fib_intent(
project_fib_intent_with_peer_groups(tables, candidates, &BTreeMap::new())
}

/// Per-class ECMP width: the eBGP/iBGP override if set, else the table's overall
/// `maximum_paths`, else 1 (today's single-next-hop behavior). The equal-cost
/// group is homogeneous, so the best route's class selects the cap.
fn per_class_max_paths(table: &FibTableConfig, is_ebgp: bool) -> usize {
if is_ebgp {
table.maximum_paths_ebgp.or(table.maximum_paths)
} else {
table.maximum_paths_ibgp.or(table.maximum_paths)
}
.unwrap_or(1)
.max(1) as usize
}

/// Project configured FIB tables and Loc-RIB install candidates using the
/// current RIB peer-group map for table allow-list checks.
#[must_use]
Expand All @@ -350,10 +363,6 @@ pub(crate) fn project_fib_intent_with_peer_groups(
let mut table_frozen = false;
let mut route_limit_drops = RouteLimitDropCounters::default();
let route_limit_drop_start = intent.drops.len();
// Per-table ECMP width. Unset / 1 == single-next-hop (today). The RIB
// already capped each candidate at the widest table's `maximum_paths`,
// so this re-caps to *this* table's value over the best-first set.
let max_paths = table.maximum_paths.unwrap_or(1).max(1) as usize;
let allowed_neighbors = table
.allowed_neighbors
.iter()
Expand All @@ -372,9 +381,12 @@ pub(crate) fn project_fib_intent_with_peer_groups(
});
continue;
}
// Per-class ECMP width for this candidate (homogeneous group ⇒ the
// best route's class picks the cap). The RIB already gathered
// siblings at the widest of these, so this re-caps the best-first set.
let max_paths = per_class_max_paths(table, route.is_ebgp());
// Keep only equal-cost next-hops from allowed peers whose family
// matches the prefix, best-first, capped at this table's
// `maximum_paths`.
// matches the prefix, best-first, capped at the per-class width.
let eligible = eligible_next_hops(candidate, route.prefix, max_paths, |peer| {
table_allows_peer(table, &allowed_neighbors, peer, peer_groups)
});
Expand Down Expand Up @@ -756,6 +768,8 @@ mod tests {
allowed_neighbors: Vec::new(),
max_routes: None,
maximum_paths: None,
maximum_paths_ebgp: None,
maximum_paths_ibgp: None,
}
}

Expand Down Expand Up @@ -1660,6 +1674,73 @@ mod tests {
assert_eq!(projected.target.next_hops, vec![ip("203.0.113.1")]);
}

#[test]
fn maximum_paths_ebgp_caps_ebgp_group() {
// Per-class eBGP cap applies even with no overall `maximum_paths`.
let mut table = table("edge", 1000, 200, &["ipv4_unicast"]);
table.maximum_paths_ebgp = Some(2);
let candidate = multipath_candidate(
route(v4_prefix(2, 24), ip("203.0.113.1"), RouteOrigin::Ebgp, 0),
&["203.0.113.2", "203.0.113.3"],
);

let intent = project_fib_intent(&[table], &[candidate]);

let projected = intent.routes.values().next().unwrap();
assert_eq!(
projected.target.next_hops.len(),
2,
"eBGP group capped at maximum_paths_ebgp"
);
}

#[test]
fn maximum_paths_ibgp_caps_ibgp_group() {
let mut table = table("edge", 1000, 200, &["ipv4_unicast"]);
table.maximum_paths_ibgp = Some(3);
let candidate = multipath_candidate(
route(v4_prefix(2, 24), ip("203.0.113.1"), RouteOrigin::Ibgp, 0),
&["203.0.113.2", "203.0.113.3", "203.0.113.4"],
);

let intent = project_fib_intent(&[table], &[candidate]);

let projected = intent.routes.values().next().unwrap();
assert_eq!(
projected.target.next_hops.len(),
3,
"iBGP group capped at maximum_paths_ibgp"
);
}

#[test]
fn per_class_overrides_with_maximum_paths_fallback() {
// `maximum_paths` is the shared fallback; the eBGP override wins for the
// eBGP group, while the iBGP group (no override) falls back to it.
let mut table = table("edge", 1000, 200, &["ipv4_unicast"]);
table.maximum_paths = Some(3);
table.maximum_paths_ebgp = Some(2);
let ebgp = multipath_candidate(
route(v4_prefix(2, 24), ip("203.0.113.1"), RouteOrigin::Ebgp, 0),
&["203.0.113.2", "203.0.113.3"],
);
let ibgp = multipath_candidate(
route(v4_prefix(3, 24), ip("203.0.114.1"), RouteOrigin::Ibgp, 0),
&["203.0.114.2", "203.0.114.3", "203.0.114.4"],
);

let intent = project_fib_intent(&[table], &[ebgp, ibgp]);

let mut lens: Vec<usize> = intent
.routes
.values()
.map(|r| r.target.next_hops.len())
.collect();
lens.sort_unstable();
// eBGP capped at 2 (override), iBGP at 3 (fallback to maximum_paths).
assert_eq!(lens, vec![2, 3]);
}

#[test]
fn family_filter_keeps_only_matching_family_next_hops() {
let mut table = table("edge", 1000, 200, &["ipv4_unicast"]);
Expand Down
16 changes: 15 additions & 1 deletion src/fib_runtime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,19 @@ fn max_install_paths(config: &FibRuntimeConfig) -> u32 {
config
.tables
.iter()
.map(|table| table.maximum_paths.unwrap_or(1).max(1))
.map(|table| {
// Widest across the overall and per-class caps, so the RIB gathers
// enough siblings for projection's per-class re-cap to reach any of
// them.
table
.maximum_paths
.into_iter()
.chain(table.maximum_paths_ebgp)
.chain(table.maximum_paths_ibgp)
.max()
.unwrap_or(1)
.max(1)
})
.max()
.unwrap_or(1)
}
Expand Down Expand Up @@ -1783,6 +1795,8 @@ mod tests {
allowed_neighbors: Vec::new(),
max_routes: None,
maximum_paths: None,
maximum_paths_ebgp: None,
maximum_paths_ibgp: None,
}
}

Expand Down