Skip to content

fix(server): managed Codex refreshes after sign-out even when a check was interrupted - #14376

Merged
juliusmarminge merged 1 commit into
t3code/codex-turn-mappingfrom
v2/fix-codexdriver-disconnect-flake
Sep 30, 2026
Merged

juliusmarminge merged 1 commit into
t3code/codex-turn-mappingfrom
v2/fix-codexdriver-disconnect-flake

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

CodexDriver.test.ts > disconnect refreshes a restored managed account while its auth flow is idle timed out at 120 s about one run in four. The cause is a real production bug, not test-state leakage. When sign-out lands while an account check is still resolving, managed Codex never refreshes its snapshot after sign-out, so it keeps showing the signed-out account.

Cause

makeManagedCodexProvider refreshes the snapshot from a listener on the ChatGPT auth controller (idle/succeeded/failed/cancelled, CodexManagedProvider.ts:210). Each account check resolves the runtime inside the controller's withAccess scope, and logout closes every such scope (stopOwnedSessions, ProviderAuthFlow.ts). If a check is still resolving when sign-out starts:

  1. The listener's in-flight snapshot.refresh fails with an interrupt. That's the interrupt of the withAccess child fiber, surfaced through Fiber.await.
  2. Stream.runForEach propagates that failure, so the forked listener fiber ends for good.
  3. Sign-out then publishes idle, but no listener is left to refresh. streamChanges never emits unauthenticated, and the test waits forever.

I confirmed this with temporary tracing on a hung run: sub refresh exit Failure Cause([Interrupt]) → SUBSCRIPTION ENDED right after logout, then nothing. The startup check and the sign-in listener's initial idle check normally finish before logout. Under load one of them is still resolving, and that's the flake. The test is the first in its file, so no earlier test leaks state into it. The session opened before logout (the lead in the brief) isn't involved: the fake factory dies immediately, and its withAccess scope is already closed.

The race code (ProviderAuthFlow.ts, CodexChatGptAuth.ts, makeManagedServerProvider.ts, and this listener) is identical on main, so main has the same latent bug. This PR fixes it only on the V2 branch.

Fix

  • CodexManagedProvider.ts: the listener ignores and logs a failed refresh (Effect.ignoreCause({ log: true })) instead of dying, so the idle that sign-out publishes still refreshes the snapshot.
  • CodexDriver.test.ts: the installation's acquire interrupts the two startup checks, and the test waits for both before it continues. This reproduces the race deterministically: without the fix the test hangs 3/3 times, and with it it passes. No timeout was raised and nothing was skipped.

Verification

  • Reproduced the flake on the V2 base (82f884d): 0/8 failures sequentially on an idle box, and 1/18 whole-file runs under a 6-way concurrent load (Test timed out).
  • Regression test without the production fix: 3/3 runs time out. With the fix: passes.
  • Whole file, 20 runs in a row with the final code (vp test run src/provider/Drivers/CodexDriver.test.ts):
    run 1 rc=0 8s  Tests 14 passed (14)
    run 2 rc=0 8s  Tests 14 passed (14)
    run 3 rc=0 7s  Tests 14 passed (14)
    run 4 rc=0 7s  Tests 14 passed (14)
    run 5 rc=0 6s  Tests 14 passed (14)
    run 6 rc=0 7s  Tests 14 passed (14)
    run 7 rc=0 7s  Tests 14 passed (14)
    run 8 rc=0 7s  Tests 14 passed (14)
    run 9 rc=0 7s  Tests 14 passed (14)
    run 10 rc=0 7s  Tests 14 passed (14)
    run 11 rc=0 7s  Tests 14 passed (14)
    run 12 rc=0 7s  Tests 14 passed (14)
    run 13 rc=0 7s  Tests 14 passed (14)
    run 14 rc=0 7s  Tests 14 passed (14)
    run 15 rc=0 7s  Tests 14 passed (14)
    run 16 rc=0 6s  Tests 14 passed (14)
    run 17 rc=0 8s  Tests 14 passed (14)
    run 18 rc=0 7s  Tests 14 passed (14)
    run 19 rc=0 7s  Tests 14 passed (14)
    run 20 rc=0 6s  Tests 14 passed (14)
    
  • With the fix alone, before the deterministic test change: 20/20 sequential whole-file passes, and 30/30 whole-file passes under a 6-way concurrent load (5 rounds).
  • cd apps/server && vp exec tsc --noEmit -p .: exit 0, no error TS.
  • vp lint on the two touched files: clean. vp fmt: no changes.
  • vp run knip:check: exit 0.
  • Not run: repo-wide test, typecheck, or check suites.

Model: Claude Opus 5.5 (Claude Code)

🤖 Generated with Claude Code


Devin Review

… was interrupted

The managed Codex provider refreshes its snapshot whenever the ChatGPT auth
controller settles (idle, succeeded, failed, cancelled). An account check
runs its runtime resolution inside the controller's withAccess scope, and
sign-out closes those scopes. When sign-out lands while a check is still
resolving, that check's refresh fails with an interrupt, the refresh loop's
fiber dies with it, and the idle state that sign-out publishes next is never
turned into a refresh. The snapshot keeps reporting the signed-out account.

This is what made CodexDriver.test.ts time out about one run in four: the
startup and sign-in listener checks usually finish before logout, but under
load one is still resolving when logout closes its scope.

Ignore (and log) a failed refresh inside the listener so the subscription
keeps running. The test now interrupts both startup checks deterministically,
so it hangs every time without the fix.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XS 0-9 changed lines (additions + deletions). labels Sep 30, 2026
@macroscopeapp

macroscopeapp Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The PR is a small, well-tested fix that keeps the managed Codex authentication snapshot listener alive when sign-out interrupts an in-flight refresh. Because it changes production authentication-state handling, human review is warranted.

You can add or adjust custom eligibility rules. Learn more.

@github-actions

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire — 4.9 KiB — 6.8 KiB ✅
Codex Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Codex Live turn WebSocket wire — 1.1 KiB — 2.0 KiB ✅
Codex Live turn WebSocket decoded — 20.4 KiB — 29.3 KiB ✅
Codex Live turn messages — 1 — 8 ✅
Claude Total thread wire — 4.9 KiB — 6.8 KiB ✅
Claude Thread snapshot wire — 3.7 KiB — 4.9 KiB ✅
Claude Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Claude Live turn WebSocket decoded — 20.8 KiB — 29.3 KiB ✅
Claude Live turn messages — 2 — 8 ✅

Baseline: unavailable · PR result: 712a110 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 106.1 KiB
  • Claude decoded thread snapshot: 106.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@juliusmarminge
juliusmarminge merged commit 8297ccf into t3code/codex-turn-mapping Sep 30, 2026
23 of 24 checks passed
@juliusmarminge
juliusmarminge deleted the v2/fix-codexdriver-disconnect-flake branch September 30, 2026 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS 0-9 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant