[Bug] The restart drain fence refuses new requests for the full 60s while Codex exhausts its retries in about six seconds #6643

Client or integration

Codex CLI

Area

Service lifecycle

Summary

The restart handoff drains for up to MEMORY_DRAIN_RESTART_MS (60s). The intent is to let an in-flight turn finish, but during that whole window every NEW data-plane request is refused, and Codex spends its five sampling retries in roughly six seconds. So the client gives up well before the replacement is listening, and the turn fails even though the restart itself is quick.

Measured on this machine (2026-09-26): with the 60s default, a 42-second fence failed the turn with unexpected status 503 - and that was already after the refusal stopped being a fatal capacity error (see the related issue below). A two-second cap, plus the replacement's roughly one-second bind gap, passed. In other words the fence length, not the refusal itself, is what the client runs out of.

Reproduction

1. ocx start --port 10100.
2. Start a Codex turn.
3. Trigger the drain-and-restart path (dashboard POST /api/system/restart, or ocx service restart) while the turn is in flight.
4. Watch the turn: the client retries against a listener that is refusing for the full fence, exhausts its retries, and ends with unexpected status 503 instead of continuing against the replacement.

Version

2.78.0

Operating system

macOS 27.0.1 (arm64). The fence is shared code, so this should apply on every platform.

Provider and model

Not provider-specific.

Logs or error output

Source location: src/server/management/system-restart.ts - the deadline is set at line 366 (const restartDeadlineMs = now() + MEMORY_DRAIN_RESTART_MS;) and consumed at line 393 (const remainingMs = Math.max(0, restartDeadlineMs - now());), which is then handed to drain(undefined, remainingMs).

Suggested change

Bound the restart fence rather than the drain - keep the drain free to finish an in-flight turn, but stop refusing new requests for longer than the client can retry:

const RESTART_HANDOFF_INFLIGHT_WAIT_MS = 2_000;
const remainingMs = Math.min(
  Math.max(0, restartDeadlineMs - now()),
  RESTART_HANDOFF_INFLIGHT_WAIT_MS,
);

A turn cut by this cap is retried by the client against the replacement that the same handoff starts, which is why two seconds is enough: the gap it has to cover is the replacement's bind, not the full drain.

This is what we run as a local patch. Happy to open a PR if you would rather review it that way.

Related: the code carried by the refusal is the other half of this handoff problem, filed separately as #6642.

Screenshots and supporting files

None.

Redacted configuration

{ "hostname": "127.0.0.1", "port": 10100 }

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.