feat(gateway): reconcile tenant VRF egress default routes - #385
Open
privateip wants to merge 1 commit into
Open
Conversation
scotwells
reviewed
Aug 13, 2026
Comment on lines
+330
to
+348
| go func() { | ||
| ticker := time.NewTicker(cfg.EgressRouteInterval) | ||
| defer ticker.Stop() | ||
|
|
||
| if !mgr.GetCache().WaitForCacheSync(ctx) { | ||
| log.Printf("EgressRoute: cache sync failed, skipping initial pass") | ||
| return | ||
| } | ||
| rec.RunOnce(ctx) | ||
|
|
||
| for { | ||
| select { | ||
| case <-ctx.Done(): | ||
| return | ||
| case <-ticker.C: | ||
| rec.RunOnce(ctx) | ||
| } | ||
| } | ||
| }() |
Contributor
There was a problem hiding this comment.
Why's this use a separate ticker process for reconciling things instead of leveraging the normal controller runtime behavior?
Comment on lines
+401
to
+403
| cmd.Flags().DurationP("egress-route-interval", "", | ||
| config.DefaultRouterEgressRouteInterval, | ||
| "Egress default-route reconcile interval (datum-cloud/enhancements#865)") |
Contributor
There was a problem hiding this comment.
The link to an enhancement here is a little odd.
This was referenced Aug 13, 2026
privateip
force-pushed
the
feat/865-egress-phase-c
branch
from
August 14, 2026 01:16
6d5c25c to
33f95b3
Compare
Phase D of #865, per docs/plans/865-edge-gateway-nat66-egress.md
§4.4/§4.5, resolved toward the new-controller candidate.
- internal/egressroute: per-node reconcile logic -- enumerate local
VRFs, resolve VPC, check NetworkEgressPolicy enablement + gateway
assignment, resolve the destination uSID, install/remove the ::/0
SEG6 encap route. Mirrors internal/gc's split of real-logic-package
plus thin-controller-wrapper.
- internal/controller/egressroute_controller.go: ticker-driven
wrapper (mirrors GCReconciler), registered in galactic-router --
runs on every compute node, not just gateway nodes, since that's
where tenant VRFs actually live.
- internal/controller/networkegresspolicy_controller.go: new
NetworkEgressPolicyReconciler, one-time gateway-node assignment
mirroring NetworkRuleReconciler.assignPrimaryNode. Registered in
galactic-gateway alongside NetworkRuleReconciler.
- internal/plumbing/vrf.ResolveVPC: new exported VPC-from-VRF-name
resolver.
- NetworkGatewayReconciler.publishEgressAddresses: extended to also
publish EgressSID, refactored to share a new
publishHostRouteAdvertisement helper across all three self-address
advertisements (SRv6Address, EgressAddress, EgressSID) instead of
triplicating the get-or-create-or-update block.
- config: GALACTIC_ROUTER_EGRESS_ROUTE_INTERVAL, separate from GC's
own interval.
Two inferences the plan's own text doesn't spell out, made necessary
by actually implementing this phase (see network repo's companion PR
and internal/egressroute's package doc comment for the full
reasoning):
- tenant_arg reuses each VPC's own BGPVRFInstance.Spec.VRFID (already
allocated per-node for ingress SRv6 decap, and already unique
per-node -- exactly the isolation property egress needs).
- NetworkGatewayStatus.EgressSID must be published into BGP the same
way SRv6Address is, or no compute node has a kernel route to
encapsulate toward at all.
Bug found and fixed along the way: srv6.RouteEgressDel has been
broken since it was written -- it set an empty SEG6Encap{} on the
delete request, which netlink unconditionally rejects. Already called
in production by internal/runtime/gobgp/monitor.go's BGP
path-withdrawal handler, so every SEG6 route withdrawal in this
codebase was likely failing, unrelated to egress. Fixed and added the
first test coverage RouteEgressDel has ever had.
privateip
force-pushed
the
feat/865-egress-phase-d
branch
from
August 14, 2026 01:16
a336660 to
4a9a0ff
Compare
This was referenced Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase D of #865, per docs/plans/865-edge-gateway-nat66-egress.md
§4.4/§4.5. Resolves #384 (candidate 2: new controller). Stacked on
#383 (Phase C). Depends on datum-cloud/network#15 (now includes the
Phase D CRD field additions too).
internal/egressroute: per-node reconcile logic -- enumerate localVRFs, resolve VPC, check
NetworkEgressPolicyenablement + gatewayassignment, resolve the destination uSID, install/remove the
::/0SEG6 encap route. Mirrors
internal/gc's split of real-logic-packageplus thin-controller-wrapper.
internal/controller/egressroute_controller.go: ticker-drivenwrapper (mirrors
GCReconciler), registered ingalactic-router--runs on every compute node, not just gateway nodes.
internal/controller/networkegresspolicy_controller.go: newNetworkEgressPolicyReconciler, one-time gateway-node assignmentmirroring
NetworkRuleReconciler.assignPrimaryNode. Registered ingalactic-gateway.internal/plumbing/vrf.ResolveVPC: new exported VPC-from-VRF-nameresolver.
NetworkGatewayReconciler.publishEgressAddresses: extended toalso publish
EgressSID; refactored to share a newpublishHostRouteAdvertisementhelper across all threeself-address advertisements instead of triplicating the
get-or-create-or-update block.
GALACTIC_ROUTER_EGRESS_ROUTE_INTERVAL, separate from GC'sown interval.
Two inferences beyond the plan's text
Neither is spelled out in the design plan; both were necessary to
make this phase actually work (full reasoning in
internal/egressroute's package doc comment):tenant_argreuses each VPC's ownBGPVRFInstance.Spec.VRFID(already allocated per-node for ingress SRv6 decap, and already
unique per-node -- exactly the isolation property egress needs).
NetworkGatewayStatus.EgressSIDmust be published into BGP thesame way
SRv6Addressis, or no compute node has a kernel route toencapsulate toward at all.
Bug found and fixed along the way
srv6.RouteEgressDelhas been broken since it was written -- it setan empty
SEG6Encap{}on the delete request, which netlinkunconditionally rejects (
"No Segment in srh"). Already called inproduction by
internal/runtime/gobgp/monitor.go's BGPpath-withdrawal handler, so every SEG6 route withdrawal in this
codebase was likely failing, unrelated to egress. Fixed, with the
first test coverage
RouteEgressDelhas ever had.Testing
internal/egressroute(unit + root-gated e2e provingthe full install/remove lifecycle against a real kernel VRF),
internal/plumbing/vrf,internal/plumbing/srv6(the bug-fixregression test),
internal/controller(both new reconcilers),internal/config.go build ./...,go vet ./...,task lint(0 issues) -- clean.task test:unitunder root: everything this PR touchespasses. The only failure present (
internal/cni/tap) is the samepre-existing, unrelated environment issue confirmed on earlier PRs
in this stack.
Closes #384.
🤖 Generated with Claude Code