Skip to content

tcp: do not leak SACKed-then-ACKed segments into Outstanding outside recovery - #14101

Open
davidbell217 wants to merge 1 commit into
google:masterfrom
davidbell217:fix-outstanding-sack-leak-outside-recovery
Open

tcp: do not leak SACKed-then-ACKed segments into Outstanding outside recovery#14101
davidbell217 wants to merge 1 commit into
google:masterfrom
davidbell217:fix-outstanding-sack-leak-outside-recovery

Conversation

@davidbell217

Copy link
Copy Markdown

Fixes #14092.

The cumulative-ACK removal loop in handleRcvdSegment skips the Outstanding decrement for previously-SACKed segments, on the premise that SetPipe() already accounted for them. That premise only holds during loss recovery — SetPipe() is a no-op otherwise — so a segment SACKed outside recovery (benign reordering, RACK correctly declaring no loss) and then cumulatively ACKed leaks its packet count into Outstanding permanently. On long-lived bulk transfers over reordering-but-lossless paths the phantoms accumulate until sendData's Outstanding < SndCwnd gate throttles a healthy connection (details and repro in the issue).

The fix: track packets newly SACKed while no recovery is in progress in a sender-local counter (sackedOutsideRecovery), consume it when those segments are cumulatively ACKed, and reset it wherever Outstanding is recomputed or reset wholesale (recovery entry, RTO, full window drain). Recovery-path accounting is unchanged.

Why not something simpler — both obvious one-line shapes were implemented and measured, and each breaks an existing test for a legitimate reason:

  • Decrement unconditionally in the removal loop's SACKed branch (the shape originally suggested in the issue): double-subtracts segments SetPipe() already accounted for — SACKed during recovery, cumulatively ACKed at/after exit, where leaveRecovery has already run before the removal loop. Breaks TestSACKRecovery (post-recovery pacing sends beyond the expected window).
  • Decrement at SACK arrival (in walkSACK): frees congestion-window slots before the cumulative ACK — effectively unbounded limited transmit — and only covers RACK-enabled senders. Breaks TestRACKWithWindowFull (a transmission observed after a zero-window ACK).

The counter shape keeps the conservative cumulative-ACK release point, never touches segments SetPipe() accounted for, and works with RACK on or off.

Verification

  • New regression test TestSACKedThenAckedSegmentLeavesOutstanding (both RACK on and off): fails before the fix with Outstanding = 3, want 2 — exactly one phantom packet after a SACKed-then-cumulatively-ACKed segment — and passes with it.
  • tcp_sack_test.go and tcp_rack_test.go e2e suites pass with the fix.
  • In the reordering testbench from the issue (195 ms RTT, ±15 ms jitter, 3 % reordering, zero loss, netstack-to-netstack): wedged transfers drop from 12/12 to 3-4/12 with this fix alone (the residual is the separate spurious-recovery-response gap, filed separately).

Assisted-by: Claude Code

…recovery

The cumulative-ACK removal loop in handleRcvdSegment skips the
Outstanding decrement for segments that were previously SACKed, on the
premise that SetPipe() already accounted for them. That premise only
holds during loss recovery: SetPipe() returns immediately when recovery
is not active, and nothing else removes those packets. A segment SACKed
outside recovery (benign reordering) and then cumulatively ACKed
therefore leaks its packet count into Outstanding permanently. On a
long-lived connection over a reordering-but-lossless path the phantoms
accumulate until the sender believes it is cwnd-limited while almost
nothing is in flight, throttling bulk transfers (observed: a wedged
sender with Outstanding=86, cwnd=86 and ~5 packets actually in flight,
and a completed transfer running at ~8% of its believed window).

Track packets newly SACKed while no recovery is in progress in a
sender-local counter, and consume it when those segments are
cumulatively ACKed. The counter resets wherever Outstanding is
recomputed or reset wholesale (recovery entry, RTO, full window drain),
so recovery-path accounting is unchanged.

Two simpler shapes were measured and rejected:
- decrementing Outstanding unconditionally in the removal loop's SACKed
  branch double-subtracts segments SetPipe() already accounted for
  (SACKed during recovery, cumulatively ACKed at/after exit) and breaks
  TestSACKRecovery's post-recovery pacing;
- decrementing at SACK arrival (walkSACK) frees congestion window slots
  before the cumulative ACK, effectively unbounded limited transmit,
  and breaks TestRACKWithWindowFull with a transmission after a
  zero-window ACK.

Fixes google#14092

Assisted-by: Claude Code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

netstack: TCP sender permanently leaks SACKed-then-cumulatively-ACKed packets into Outstanding outside recovery, throttling bulk sends under reordering

1 participant