Design: edge XDP egress (masquerade) datapath (865 Phase B)

[privateip]

Part of datum-cloud/enhancements#865. Phase B of docs/plans/865-edge-gateway-nat66-egress.md. Implemented in #381.

Purpose

Add a second personality to edgenat.c: masquerade traffic from tenant VPC backend Pods addressed to arbitrary internet destinations, reusing the existing Full-NAT/PAT machinery rather than a parallel subsystem.

Design

Three address families per gateway node

Address	Kind	Meaning
gw_addr	SRv6 uSID, Argument 0 (existing)	Ingress Full-NAT SNAT source / return-branch match. Unchanged.
egress_sid	SRv6 uSID locator	Tenant/VRF identity carried in the Argument bits. Dispatch matches the locator only (top 64 bits — Block+Node-ID); the unmasked Argument is the flow's tenant_arg, extracted per-packet, no map lookup.
masq_addr	Plain, publicly-routable IPv6	The masquerade SNAT source visible to the internet, and the egress return-branch match address.

egress_sid cannot be a single reserved constant like gw_addr's Argument 0: tenant ULA space is not guaranteed unique (independent orgs' RFC 4193 prefixes can collide), so the Argument bits double as a per-flow tenant discriminator, closing that collision risk with a masked-address dispatch match — no added per-packet cost.

Dispatch (edge_nat)

if daddr == gw_addr:                           handle_return(...)        # existing
if (daddr & locator_mask) == egress_sid:        handle_egress_forward(...) # NEW, tenant_arg = daddr's Argument bits
if daddr == masq_addr:                          handle_egress_return(...)  # NEW
if nexthdr not tcp/udp:                         XDP_PASS                   # existing
else:                                            handle_forward(...)        # existing

Four-way branch on three configured address/mask pairs, read once per packet from two maps — no regression to existing ingress performance.

New maps

• egress_config_table (ARRAY, 1 entry): { egress_sid[16], masq_addr[16] }.
• egress_conn_table (LRU_HASH, self-evicting, same convention as conn_table): new egress_conn_key/egress_conn_value structs (not repurposed ingress ones — no "client"/"VIP" concept applies to an egress flow). Forward key: (proto, tenant_arg, backend_addr:backend_port -> dest_addr:dest_port). Reverse key: (proto, dest_addr:dest_port -> masq_addr:masq_port), tenant_arg fixed at 0 (masq_port is already unique per flow by construction, no tenant dimension needed there).

New packet paths

• handle_egress_forward: SRv6-decapsulate, look up egress_conn_table by forward key, allocate on a SYN/UDP miss (same linear-probe/BPF_NOEXIST SNAT-port claim handle_forward already uses), SNAT source to masq_addr:masq_port, transmit the unwrapped inner packet — no outer header pushed. The one genuinely new tail shape in the file.
• handle_egress_return: match on masq_addr (plain address, no nexthdr==41 requirement — an ordinary internet-originated packet), look up the reverse key, DNAT destination to backend_addr:backend_port, push a fresh outer SRv6 header toward backend_usid, transmit.

IPv4 out of scope for Phase 1

edge_nat() rejects non-IPv6 frames outright; egress only ever masquerades toward IPv6-reachable destinations.

Key assumptions flagged for review

1. backend_usid (needed to route the eventual reply back to the originating worker node) is read directly off the wire — the egress-forward packet's own outer SRv6 source address, captured before the outer header is stripped — since Phase B proposes no per-tenant policy table to look it up from. This assumes the kernel's SEG6 encap route (RouteEgressAdd, used by Phase D's route installer) always stamps the originating node's own uSID as the pushed outer source, the same way every other cross-node SRv6 packet in this codebase behaves. Not independently verified against RouteEgressAdd's actual behavior as part of this phase.
2. handle_egress_return drops any non-TCP/UDP protocol addressed to masq_addr (e.g. ICMPv6) rather than passing it to the kernel stack — the same fail-closed convention gw_addr's own return branch already uses for claimed addresses, but a real operational cost for an internet-facing address that gw_addr's internal-only precedent didn't have to weigh (breaks ping/PMTUD to it from this hook's perspective).

Testing

Root-required kernel tests (edgenat_egress_test.go): both branches end-to-end (SNAT/DNAT, checksum, real FIB-resolved L2), non-SYN/no-conn drops, PAT exhaustion, and the tenant-isolation scenario explicitly called out in the plan's §6 — two tenants presenting the same colliding backend_addr:port -> dest_addr:port tuple via distinct tenant_arg values resolve to two independent rows and masq_port allocations.

References

• Design plan: docs/plans/865-edge-gateway-nat66-egress.md §3
• Implemented in feat(gateway): add egress masquerade datapath to edgenat.c #381 (stacked on chore: pin go.datum.net/network to local replace for Phase A egress types #380)