Align NIC offload config on FreeBSD core routers — cr1-nl1 forwards with LRO/TSO enabled on vtnet0

[Svaag]

Context

Split out of #119 (2026-06-01 comments) so it doesn't die with that issue's stale NAT64 premise.

There is a live NIC-offload config divergence between the FreeBSD core routers:

• cr1-nl1: FreeBSD 14.3-RELEASE-p3; active vtnet0 options include RXCSUM,TXCSUM,TSO4,TSO6,LRO,TXCSUM_IPV6.
• cr1-de1: FreeBSD 15.0-RELEASE; /etc/rc.conf explicitly sets ifconfig_vtnet0="... -rxcsum -txcsum -tso4 -tso6 -lro".

A live A/B test on 2026-06-01 (enabling offloads on cr1-de1, retesting the slow OVH/CloudFront flows, reverting) showed no meaningful throughput change — this is not the cause of the CDN path slowness (that's tracked in #351/#138). It's a correctness/consistency cleanup:

• LRO must not be enabled on interfaces that forward packets. It coalesces TCP segments before the stack sees them, which breaks forwarding semantics (resegmentation, reordering, PMTU transparency). cr1-nl1 is a router forwarding overlay/WireGuard traffic on vtnet0 — cr1-de1's explicit disable is the correct configuration, cr1-nl1's default-enabled state is the defect.
• TSO/checksum offload on virtio under Xen/KVM has a history of subtle corruption bugs; the conservative router convention is to disable the lot, as cr1-de1 already does.

Action items

1. Decide the convention: offloads disabled on all forwarding interfaces of FreeBSD core routers (recommend adopting cr1-de1's -rxcsum -txcsum -tso4 -tso6 -lro line as the standard).
2. Apply to cr1-nl1 vtnet0 live (low-risk, revertible with ifconfig), with before/after throughput probes sourced from 2a0c:b641:b50::a so we have data rather than vibes.
3. Check cr1-ch1 (newest FreeBSD node, Securebit/SBIX) against the same convention — it was built after the DE box and may have inherited defaults rather than the explicit disables.
4. Persist in /etc/rc.conf on each host and mirror into the repo (configs/cr1-nl1/, configs/cr1-ch1/ headers/docs) so the convention survives reprovisioning.
5. Note the FreeBSD version skew while in there (nl1 on 14.3, de1 on 15.0) — not this issue's scope, but worth a line in the upgrade planning.

Related

• NAT64 large-transfer throughput is glacial (~75 KB/s) — blocks Docker image pulls on the customer segment (ci-pr) #119 — origin of the finding (2026-06-01 comments: config diff, live A/B test, revert)
• Track CDN egress routing regressions and policy overrides #351 / Peering & transit expansion (Phases 1–5): eliminate AS24961 dependency, NL-anchored multi-homed IPv6 edge #138 — the actual CDN slowness this was ruled out as a cause of
• configs/cr1-nl1/, configs/cr1-de1/ — live router configs in-repo

[hyrule-engineering-loop]

Reliability Governor Decision

• role: staff_sre_autonomous_operations
• decision: allow_approved
• next_loop: engineering
• handoff_contract: github_issue_labels
• source: human
• intent: runbook / tier 0
• capability: tier0.docs-runbooks-tests
• knowledge: current / A0
• reasons: none

{
  "affected_assets": [
    "AS215932/network-operations"
  ],
  "affected_customers": [],
  "affected_services": [],
  "allowed_paths": [
    "docs/",
    "README.md"
  ],
  "authority_text_hash": "6bef87ffbf8399247bd96f3c1b207c534ee164ec2578cdae091c63aad99a5b5c",
  "blast_radius": "documentation only",
  "controlled_loops": [
    "engineering",
    "noc",
    "knowledge"
  ],
  "created_at": "2026-07-03T13:16:16.066952+00:00",
  "denial_reasons": [],
  "expected_paths": [
    "docs/",
    "README.md"
  ],
  "forbidden_paths": [
    "secrets/",
    "**/secrets/",
    ".env",
    ".env."
  ],
  "governor_name": "Reliability Governor",
  "governor_role": "staff_sre_autonomous_operations",
  "handoff_contract": "github_issue_labels",
  "intent_type": "runbook",
  "issue_id": "AS215932/network-operations#355",
  "issue_number": 355,
  "issue_text_hash": "8c2943a0e3d58f93d1cbabae9b146333442ce398385fb43f73f4adbd19838afb",
  "knowledge_authority_level": "A0",
  "knowledge_context_pack_id": "ctx_60b9b6b5cc0fed2ca98f2feab87316f7",
  "knowledge_export_version": "local-20260702212639",
  "knowledge_status": "current",
  "labels_to_add": [
    "loop:approved"
  ],
  "labels_to_remove": [
    "loop:intake",
    "loop:candidate",
    "loop:needs-context",
    "loop:knowledge-gap",
    "loop:needs-human"
  ],
  "lhp": null,
  "matched_capability": "tier0.docs-runbooks-tests",
  "next_loop": "engineering",
  "policy_rules": [
    "Tier 0/1 within capability envelope may be auto-approved"
  ],
  "record_id": "08cfbe977d4e7c934912",
  "repo": "AS215932/network-operations",
  "required_checks": [
    "targeted_tests_or_docs_review"
  ],
  "risk_tier": 0,
  "rollback_plan": "Use the rollback/revert procedure specified in the request.",
  "routing_decision": "allow_approved",
  "schema_version": "reliability-governor.cdr.v1",
  "source": "human",
  "storage_path": "/var/lib/engineering-loop/reliability-governor/as215932-network-operations-355-08cfbe977d4e7c934912.json",
  "verification_method": "Use the issue-specified verification: tests/checks/evidence named in the request."
}

[hyrule-engineering-loop]

Reliability Governor Decision

• role: staff_sre_autonomous_operations
• decision: allow_approved
• next_loop: engineering
• handoff_contract: github_issue_labels
• source: human
• intent: runbook / tier 0
• capability: tier0.docs-runbooks-tests
• knowledge: current / A0
• reasons: none

{
  "affected_assets": [
    "AS215932/network-operations"
  ],
  "affected_customers": [],
  "affected_services": [],
  "allowed_paths": [
    "docs/",
    "README.md"
  ],
  "authority_text_hash": "6bef87ffbf8399247bd96f3c1b207c534ee164ec2578cdae091c63aad99a5b5c",
  "blast_radius": "documentation only",
  "controlled_loops": [
    "engineering",
    "noc",
    "knowledge"
  ],
  "created_at": "2026-07-03T13:30:55.552844+00:00",
  "denial_reasons": [],
  "expected_paths": [
    "docs/",
    "README.md"
  ],
  "forbidden_paths": [
    "secrets/",
    "**/secrets/",
    ".env",
    ".env."
  ],
  "governor_name": "Reliability Governor",
  "governor_role": "staff_sre_autonomous_operations",
  "handoff_contract": "github_issue_labels",
  "intent_type": "runbook",
  "issue_id": "AS215932/network-operations#355",
  "issue_number": 355,
  "issue_text_hash": "8c2943a0e3d58f93d1cbabae9b146333442ce398385fb43f73f4adbd19838afb",
  "knowledge_authority_level": "A0",
  "knowledge_context_pack_id": "ctx_5a387da825d02850883ea73187338103",
  "knowledge_export_version": "local-20260702212639",
  "knowledge_status": "current",
  "labels_to_add": [
    "loop:approved"
  ],
  "labels_to_remove": [
    "loop:intake",
    "loop:candidate",
    "loop:needs-context",
    "loop:knowledge-gap",
    "loop:needs-human"
  ],
  "lhp": null,
  "matched_capability": "tier0.docs-runbooks-tests",
  "next_loop": "engineering",
  "policy_rules": [
    "Tier 0/1 within capability envelope may be auto-approved"
  ],
  "record_id": "13b8f68a9f06cfb1f150",
  "repo": "AS215932/network-operations",
  "required_checks": [
    "targeted_tests_or_docs_review"
  ],
  "risk_tier": 0,
  "rollback_plan": "Use the rollback/revert procedure specified in the request.",
  "routing_decision": "allow_approved",
  "schema_version": "reliability-governor.cdr.v1",
  "source": "human",
  "storage_path": "/var/lib/engineering-loop/reliability-governor/as215932-network-operations-355-13b8f68a9f06cfb1f150.json",
  "verification_method": "Use the issue-specified verification: tests/checks/evidence named in the request."
}