Skip to content

HTTP SSE response.failed waits for EOF and can be replaced by idle timeout (0.153.4; tested fix) #43140

Description

@jd-nuva

Summary

The HTTP SSE consumer in Codex CLI 0.153.4 does not terminate when it parses a terminal response.failed event. It saves the error and keeps reading until EOF. If the server keeps the connection open, the client waits for its stream idle timeout, then reports idle timeout waiting for SSE instead of the original server error. A subsequent transport error can also overwrite the terminal error.

This is a deterministic, provider-independent client bug, reproduced against the original installed binary using a loopback HTTP server with no model service or credentials. A narrow source fix and four regression tests are included below, together with baseline and patched results.

I investigated this after repeated reconnecting/stalls with an Azure Responses deployment. I have not established that this exact terminal-event/open-socket sequence caused the live Azure reconnects. Real backend failures were also observed outside Codex. Please treat this issue as a confirmed SSE failure-handling defect, not a claim that a client patch repairs Azure capacity.

What Version of Codex CLI Is Running?

  • Original installed executable: codex-cli 0.153.4.
  • Official source tag used for the patch: rust-v0.153.4, commit 3d2ee51ca2d5db578f328aa75e20aa22c0197c9a.
  • The latest stable release returned by GitHub when checked on September 6, 2026 (UTC) was rust-v0.153.4, published September 4, 2026 at 23:25:48 UTC.
  • Desktop version: 26.901.41600, build 7982.
  • The inspected main snapshot at commit 008bbd5884122dc95aaece19ecfe0fc6a59dcf36 also has the store-and-wait behavior in codex-api/src/sse/responses.rs (lines 663-676). The proposed diff is pinned to the release above, not to moving main.

Subscription, Model, Platform, and Terminal

  • Provider: user-configured Azure OpenAI Responses endpoint, API-key authentication; the failing requests are not routed through a ChatGPT subscription.
  • Model/deployment identifier: gpt-6-astra; response headers identified gpt-6-astra-2026-09-03, West US, pay-as-you-go. These are identifiers observed in this deployment, not a claim about public model availability.
  • Reasoning effort: medium; configured context window: 1000000; auto-compact threshold: 272000.
  • OS: macOS 26.6.2, build 25G83; Darwin 25.6.0; Apple Silicon arm64.
  • Original symptom: Codex Desktop. Deterministic reproducer: noninteractive codex exec --json with piped stdio, no terminal emulator required.

Sanitized effective provider settings:

model = "gpt-6-astra"
model_provider = "azure"
model_reasoning_effort = "medium"
model_context_window = 1000000
model_auto_compact_token_limit = 272000

[model_providers.azure]
name = "Azure Responses"
base_url = "https://<resource>.openai.azure.com/openai/v1"
wire_api = "responses"
requires_openai_auth = false
env_http_headers = { "api-key" = "AZURE_OPENAI_API_KEY" }
request_max_retries = 4
stream_max_retries = 10
stream_idle_timeout_ms = 300000

The fixture below overrides retries to zero and the idle timeout to 800 ms in a separate CODEX_HOME. It does not alter any live configuration. The real configuration also explicitly disables remote_compaction_v2. No model, context, output limit, retry, or timeout change is part of the SSE fix.

What Issue Are You Seeing?

The server has already declared the response failed. Waiting for transport closure after that declaration delays reporting/retry and can discard the useful error classification/message/retry delay. With the production timeout above, a hold-open connection can incur a 300-second idle wait; the reproduction uses 800 ms so this behavior can be tested quickly.

Relevant release source: codex-rs/codex-api/src/sse/responses.rs, process_sse_with_treatment.

The old error branch assigns response_error = Some(error.into_api_error()). EOF returns that saved value, but the timeout and transport-error branches return different errors before consulting it.

Steps to Reproduce

  1. Run the Python 3 standard-library script below against an original 0.153.4 CLI. No Azure/OpenAI access, credentials, catalog, or real model inference is needed.
  2. The script starts a loopback HTTP SSE fixture and uses a fresh isolated CODEX_HOME for each case. Both cases send HTTP 200, Content-Type: text/event-stream, and the same terminal failure:
event: response.failed
data: {"type":"response.failed","response":{"id":"resp_local_terminal_repro","status":"failed","error":{"code":"server_error","message":"LOCAL_TERMINAL_FAILURE_MARKER"}}}

  1. First case closes after the event; second case leaves the socket open. Read summary.json and the CLI error events.
python3 codex_sse_terminal_repro.py --codex /absolute/path/to/codex --output ./repro-original
# Repeat against the patched binary with a new output directory:
python3 codex_sse_terminal_repro.py --codex /absolute/path/to/patched/codex --output ./repro-patched
Complete standalone reproducer (Python standard library only)
#!/usr/bin/env python3
"""Exercise an installed Codex against a local SSE failure; never calls Azure."""

import argparse
import asyncio
from datetime import datetime, timezone
import json
import os
from pathlib import Path
import shutil
import time


MARKER = 'LOCAL_TERMINAL_FAILURE_MARKER'


async def run_case(args, name, hold_open):
    root = args.output / name
    root.mkdir(parents=True, exist_ok=False)
    home = root / 'home'
    home.mkdir()
    requests = []
    handlers = set()
    started = time.monotonic()
    started_at_utc = datetime.now(timezone.utc).isoformat()

    async def handler(reader, writer):
        task = asyncio.current_task()
        handlers.add(task)
        try:
            raw_headers = await reader.readuntil(b'\r\n\r\n')
            headers = {}
            for line in raw_headers.decode().split('\r\n')[1:]:
                if ':' in line:
                    key, value = line.split(':', 1)
                    headers[key.lower()] = value.strip()
            size = int(headers.get('content-length', '0'))
            await reader.readexactly(size)
            event = {'type': 'response.failed', 'response': {
                'id': 'resp_local_terminal_repro', 'status': 'failed',
                'error': {'code': 'server_error', 'message': MARKER}}}
            body = ('event: response.failed\ndata: ' + json.dumps(event) + '\n\n').encode()
            writer.write(b'HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nConnection: close\r\n\r\n' + body)
            await writer.drain()
            requests.append({'sent_failure_after_s': round(time.monotonic() - started, 3),
                             'request_body_bytes': size})
            if hold_open:
                # Model a server that sends a terminal event without closing its socket.
                try:
                    await asyncio.wait_for(reader.read(), timeout=8)
                except asyncio.TimeoutError:
                    pass
        finally:
            writer.close()
            await writer.wait_closed()
            handlers.discard(task)

    server = await asyncio.start_server(handler, '127.0.0.1', 0)
    port = server.sockets[0].getsockname()[1]
    config = [
        'model = ' + json.dumps(args.model),
        'model_provider = "local_sse_fixture"',
        'model_reasoning_effort = "medium"',
    ]
    if args.catalog:
        config.append('model_catalog_json = ' + json.dumps(str(args.catalog.resolve())))
    config += [
        '[model_providers.local_sse_fixture]',
        'name = "Local SSE fixture only"',
        f'base_url = "http://127.0.0.1:{port}/v1"',
        'wire_api = "responses"',
        'requires_openai_auth = false',
        'request_max_retries = 0',
        'stream_max_retries = 0',
        f'stream_idle_timeout_ms = {args.idle_ms}',
    ]
    (home / 'config.toml').write_text('\n'.join(config) + '\n')
    process = None
    try:
        process = await asyncio.create_subprocess_exec(
            args.codex, 'exec', '--skip-git-repo-check', '--json', '--sandbox', 'read-only',
            'Reply only LOCAL_SSE_TEST. Do not call tools.',
            cwd=root, env=dict(os.environ, CODEX_HOME=str(home)),
            stdin=asyncio.subprocess.DEVNULL, stdout=asyncio.subprocess.PIPE,
            stderr=asyncio.subprocess.PIPE)
        stdout, stderr = await asyncio.wait_for(process.communicate(), timeout=15)
        elapsed = round(time.monotonic() - started, 3)
        (root / 'stdout.jsonl').write_bytes(stdout)
        (root / 'stderr.log').write_bytes(stderr)
        events = []
        for line in stdout.decode(errors='replace').splitlines():
            try:
                event = json.loads(line)
            except json.JSONDecodeError:
                continue
            if event.get('type') in ('error', 'turn.failed'):
                events.append(event)
        serialized = json.dumps(events)
        receipt = {'case': name, 'started_at_utc': started_at_utc,
                   'idle_timeout_ms': args.idle_ms, 'exit_code': process.returncode,
                   'elapsed_s': elapsed, 'request_count': len(requests),
                   'failure_to_exit_s': round(elapsed - requests[0]['sent_failure_after_s'], 3) if requests else None,
                   'original_error_preserved': MARKER in serialized,
                   'reported_idle_timeout': 'idle timeout' in serialized.lower(),
                   'events': events}
        (root / 'receipt.json').write_text(json.dumps(receipt, indent=2))
        print(json.dumps(receipt), flush=True)
        return receipt
    finally:
        if process is not None and process.returncode is None:
            process.terminate()
            await process.wait()
        server.close()
        await server.wait_closed()
        if handlers:
            for task in list(handlers):
                task.cancel()
            await asyncio.gather(*list(handlers), return_exceptions=True)


async def main(args):
    results = [await run_case(args, 'failed_then_eof', False),
               await run_case(args, 'failed_then_open_socket', True)]
    (args.output / 'summary.json').write_text(json.dumps(results, indent=2))


if __name__ == '__main__':
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument('--codex', default=shutil.which('codex') or '/Applications/ChatGPT.app/Contents/Resources/codex')
    parser.add_argument('--output', type=Path, required=True)
    parser.add_argument('--model', default='gpt-6-astra')
    parser.add_argument('--catalog', type=Path)
    parser.add_argument('--idle-ms', type=int, default=800)
    args = parser.parse_args()
    args.output = args.output.resolve()
    args.output.mkdir(parents=True, exist_ok=True)
    asyncio.run(main(args))

Expected Behavior

Once a valid terminal response.failed has been classified, emit that error and stop consuming the stream immediately, independently of EOF. Preserve prior output/metadata, the existing error type and retry hint. Do not emit a later completion. Leave retry policy to the existing caller.

Actual Results and Verification

All durations below are measured from sending the failed event to the CLI process exiting, not total process startup time. Small millisecond differences are not a performance benchmark.

Executable Server behavior Failure to process exit Original error preserved Idle timeout reported
Original 0.153.4 Close after failure 11 ms Yes No
Original 0.153.4 Keep socket open (800 ms idle limit) 833 ms No Yes
Patched release CLI Close after failure 11 ms Yes No
Patched release CLI Keep socket open (800 ms idle limit) 10 ms Yes No
  • Four new regression tests against original logic: 0 passed, 4 failed.
  • Same four tests after the patch: 4 passed, 0 failed.
  • Entire codex-api --lib unit suite after the patch and formatting: 168 passed, 0 failed.
  • git apply --check plus application to a pristine copy and byte-for-byte comparison: passed.
  • Full optimized CLI build: cargo build --locked --release -p codex-cli --bin codex, isolated Rust 1.95.0, official release profile. Passed in 973.48 seconds.
  • External dependency versions, source URLs and checksums were verified unchanged from the release lockfile. The tag's manifest version was 0.153.4 while path packages in the checked-in lockfile were 0.0.0; only local workspace package versions were refreshed with cargo update --offline -p codex-api. That build-environment adjustment is not part of the proposed patch.

The new tests cover:

  1. Server error with a stream that stays pending indefinitely; terminal error arrives and producer exits without EOF.
  2. Rate-limit error on the same open stream; the parsed two-second retry hint is retained and the producer exits.
  3. A transport error after terminal failure cannot replace the original error.
  4. Preceding partial output is retained; a bogus completion after terminal failure is not consumed.

Exact test commands from codex-rs/, using Rust 1.95.0:

cargo test --locked -p codex-api --lib terminal_failed_response_ -- --nocapture
cargo test --locked -p codex-api --lib

Proposed Fix

Capture whether the parsed event is response.failed before moving it into process_responses_event. If that terminal event yields an error, send its existing classified error and return. Preserve the old fallback for nonterminal parser/event errors.

This changes one production source file. It does not change response.incomplete, transport routing, tool dispatch, tool replay semantics, backoff, retry budgets, output caps, or context settings. It does not undo tool side effects already performed before a failure. The WebSocket path was reviewed separately and already returns on the corresponding error path.

Complete patch, including four regression tests
--- a/codex-rs/codex-api/src/sse/responses.rs
+++ b/codex-rs/codex-api/src/sse/responses.rs
@@ -656,6 +656,7 @@
             return;
         }
 
+        let is_terminal_failure = event.kind() == "response.failed";
         match process_responses_event(event) {
             Ok(Some(event)) => {
                 let is_completed = matches!(event, ResponseEvent::Completed { .. });
@@ -668,7 +669,13 @@
             }
             Ok(None) => {}
             Err(error) => {
-                response_error = Some(error.into_api_error());
+                let error = error.into_api_error();
+                // A failed response is terminal even if the server keeps the socket open.
+                if is_terminal_failure {
+                    let _ = tx_event.send(Err(error)).await;
+                    return;
+                }
+                response_error = Some(error);
             }
         };
     }
@@ -1082,6 +1089,119 @@
             }
             other => panic!("unexpected event: {other:?}"),
         }
+    }
+
+    #[tokio::test]
+    async fn terminal_failed_response_does_not_wait_for_eof() {
+        let event = json!({
+            "type": "response.failed",
+            "response": { "error": { "code": "server_error", "message": "upstream failed" } },
+        });
+        let body = Bytes::from(format!("event: response.failed\ndata: {event}\n\n"));
+        let stream = stream::iter(vec![Ok(body)]).chain(stream::pending());
+        let (tx, mut rx) = mpsc::channel(16);
+        let mut task = tokio::spawn(process_sse(
+            Box::pin(stream),
+            tx,
+            Duration::from_secs(300),
+            None,
+        ));
+
+        let result = timeout(Duration::from_secs(1), rx.recv()).await;
+        if result.is_err() {
+            task.abort();
+        }
+        assert_matches!(
+            result.expect("terminal failure waited for EOF"),
+            Some(Err(ApiError::Retryable { message, delay: None })) if message == "upstream failed"
+        );
+        let finished = timeout(Duration::from_secs(1), &mut task).await;
+        if finished.is_err() {
+            task.abort();
+        }
+        finished
+            .expect("SSE task did not release the stream")
+            .unwrap();
+    }
+
+    #[tokio::test]
+    async fn terminal_failed_response_preserves_rate_limit_without_eof() {
+        let event = json!({
+            "type": "response.failed",
+            "response": { "error": {
+                "code": "rate_limit_exceeded",
+                "message": "Rate limit reached. Please try again in 2s.",
+            } },
+        });
+        let body = Bytes::from(format!("event: response.failed\ndata: {event}\n\n"));
+        let stream = stream::iter(vec![Ok(body)]).chain(stream::pending());
+        let (tx, mut rx) = mpsc::channel(16);
+        let mut task = tokio::spawn(process_sse(
+            Box::pin(stream),
+            tx,
+            Duration::from_secs(300),
+            None,
+        ));
+
+        let result = timeout(Duration::from_secs(1), rx.recv()).await;
+        if result.is_err() {
+            task.abort();
+        }
+        assert_matches!(
+            result.expect("rate limit waited for EOF"),
+            Some(Err(ApiError::RateLimitExceeded { message, delay }))
+                if message == "Rate limit reached. Please try again in 2s."
+                    && delay == Some(Duration::from_secs(2))
+        );
+        let finished = timeout(Duration::from_secs(1), &mut task).await;
+        if finished.is_err() {
+            task.abort();
+        }
+        finished
+            .expect("SSE task did not release the stream")
+            .unwrap();
+    }
+
+    #[tokio::test]
+    async fn terminal_failed_response_is_not_replaced_by_transport_error() {
+        let event = json!({
+            "type": "response.failed",
+            "response": { "error": { "code": "server_error", "message": "upstream failed" } },
+        });
+        let body = Bytes::from(format!("event: response.failed\ndata: {event}\n\n"));
+        let stream = stream::iter(vec![
+            Ok(body),
+            Err(TransportError::Network(
+                "connection reset afterwards".into(),
+            )),
+        ]);
+        let (tx, mut rx) = mpsc::channel(16);
+        process_sse(Box::pin(stream), tx, Duration::from_secs(300), None).await;
+
+        assert_matches!(
+            rx.recv().await,
+            Some(Err(ApiError::Retryable { message, delay: None })) if message == "upstream failed"
+        );
+        assert!(rx.recv().await.is_none());
+    }
+
+    #[tokio::test]
+    async fn terminal_failed_response_preserves_preceding_output_and_stops() {
+        let output = json!({ "type": "response.output_text.delta", "delta": "partial output" });
+        let failed = json!({
+            "type": "response.failed",
+            "response": { "error": { "code": "server_error", "message": "upstream failed" } },
+        });
+        let completed = json!({ "type": "response.completed", "response": { "id": "resp_late" } });
+        let body = format!("data: {output}\n\ndata: {failed}\n\ndata: {completed}\n\n");
+
+        let events = collect_events(&[body.as_bytes()]).await;
+        assert_eq!(events.len(), 2);
+        assert_matches!(&events[0], Ok(ResponseEvent::OutputTextDelta(text)) if text == "partial output");
+        assert_matches!(
+            &events[1],
+            Err(ApiError::Retryable { message, delay: None }) if message == "upstream failed"
+        );
     }
 
     #[tokio::test]

Related Incident Context and Ruled-Out Hypotheses

These observations explain how the defect was found; they are not required for the local reproduction.

Compact Routing Was a Separate Problem

An Azure preview /responses/compact route returned HTTP 500 even for a two-message input, while ordinary Responses calls succeeded. Moving the provider to /openai/v1 made ordinary Responses and /responses/compact succeed; an actual Codex app-server tool -> compact -> continuation test passed. Both requests went to the configured Azure resource, not a hard-coded OpenAI endpoint. That routing correction was retained, but reconnects continued.

There Were Genuine Upstream Failures Too

Live logs contained generic server errors, no healthy upstream, response-body transport errors, and this explicit admission error:

The system is currently experiencing high demand and cannot process your request. Your request exceeds the maximum usage size allowed during peak load. For improved capacity reliability, consider switching to Provisioned Throughput.

Associated HTTP headers indicated 200/SSE and substantial remaining token/request quota; this does not establish Azure's internal admission policy. Several errors happened seconds after response headers, well below the configured idle timeout. Those timings do not support calling every reconnect an idle-timeout event.

A Python HTTP client bypassing Codex also received HTTP 200 followed by response.failed / server_error in 7.616 seconds. Request start: 2026-09-05T23:55:18.182529Z; request ID carried in the error text: 050e8820-ced4-4ebb-bcf4-379b1fd86cfb. Subsequent requests under the same broad conditions succeeded. This establishes at least one failure at the Azure response layer, not the internal backend root cause. Live arrival time of response.failed versus EOF was not captured precisely enough to attribute those incidents to this client bug.

Image Base64 Was Not Ordinary Context Text in the Audited Request

A request built from an isolated copy of the affected task contained 11 properly typed input_image items, with 4,647,490 data-URI characters. No image data URI or long base64-like string was found in ordinary text fields. Server input usage was 75,508 tokens with image placeholders versus 81,020 with the images. Thus the sampled request did not exhibit the suspected multi-million-character base64-as-text regression. This was a diagnostic request built from history, not a byte-for-byte capture of a failed live request.

Other Controls Were Inconclusive, Not Fixes

  • Short requests using text, JSON function tools, plain custom tools and Lark custom tools completed on both Azure v1 and preview routes. Long-history controls and a nonstreaming control also completed.
  • Natural-history request controls completed 4/4; captured-header/plain-header controls completed 3/3. A captured default-feature Azure request was uncompressed.
  • Six parallel-tool-flag probes completed but generated only one custom call each, so they do not prove multi-call correctness.
  • Output-cap comparisons were too small and inconsistent to establish a causal fix. No global output cap or context reduction was applied.
  • The previous provider had a stream retry setting of 5 versus 10 in the Astra configuration, which changes retry-chain length, not why a request fails. Doctor also reported unbounded_connection_retries among enabled features; the SSE patch does not change that policy or claim a finite overall retry bound.
  • The affected task resumed useful output before this source patch was installed. That recovery must not be attributed to the patch.

Prior local GPT-5.5/5.6 investigations had involved preview compact routing, bounded/redacted log output, a version-specific invalid tool-schema HTTP 400 workaround, and some model claims of unavailable tools despite successful calls. Those were checked rather than blindly reapplied. None establishes that today's HTTP 200/SSE failures have the same cause.

Codex Doctor Report

Privacy-reviewed excerpt from the original installed executable, taken before activating the patch. Full output includes private paths and unrelated local configuration, so those fields are omitted. Overall status was fail because this noninteractive process has TERM=dumb; that is not an SSE diagnosis.

{
  "codexVersion": "0.153.4",
  "overallStatus": "fail",
  "checks": {
    "app_server.status": {
      "status": "warning",
      "summary": "background server socket is stale or unreachable"
    },
    "auth.credentials": {
      "status": "ok",
      "summary": "OpenAI auth is not required for the active model provider"
    },
    "config.load": {
      "status": "ok",
      "summary": "config loaded"
    },
    "desktop.app.version": {
      "status": "ok",
      "summary": "the desktop application is installed"
    },
    "desktop.app_server.handshake": {
      "status": "ok",
      "summary": "no desktop app-server handshake was recorded"
    },
    "desktop.security.enforcement": {
      "status": "ok",
      "summary": "the desktop application passed available macos security assessments"
    },
    "git.environment": {
      "status": "ok",
      "summary": "git version 2.43.0"
    },
    "installation": {
      "status": "ok",
      "summary": "installation looks consistent"
    },
    "mcp.config": {
      "status": "ok",
      "summary": "MCP configuration is locally consistent"
    },
    "network.env": {
      "status": "ok",
      "summary": "network-related environment looks readable"
    },
    "network.provider_reachability": {
      "status": "ok",
      "summary": "active provider endpoints are reachable over HTTP"
    },
    "network.websocket_reachability": {
      "status": "ok",
      "summary": "Responses WebSocket is not enabled for the active provider"
    },
    "runtime.provenance": {
      "status": "ok",
      "summary": "running local build on macos-aarch64"
    },
    "runtime.search": {
      "status": "ok",
      "summary": "search is OK (system)"
    },
    "sandbox.helpers": {
      "status": "ok",
      "summary": "sandbox configuration is readable"
    },
    "security.endpoint": {
      "status": "ok",
      "summary": "no supported endpoint protection detected"
    },
    "state.paths": {
      "status": "ok",
      "summary": "state paths and databases are inspectable"
    },
    "state.rollout_db_parity": {
      "status": "warning",
      "summary": "rollout files and state DB thread inventory differ"
    },
    "system.disk": {
      "status": "ok",
      "summary": "sufficient free disk space (100.5 GiB)"
    },
    "system.environment": {
      "status": "ok",
      "summary": "OS language en-US"
    },
    "terminal.env": {
      "status": "fail",
      "summary": "TERM=dumb - colors and cursor control are disabled"
    },
    "terminal.title": {
      "status": "ok",
      "summary": "terminal title configured"
    },
    "updates.status": {
      "status": "ok",
      "summary": "update configuration is locally consistent"
    }
  }
}

The preexisting stale/unreachable persistent-server socket and rollout/database inventory warning are included for completeness. The local reproducer uses a fresh CODEX_HOME and a separate codex exec process, so neither shared Desktop state nor daemon reuse is needed to reproduce this defect.

Local Deployment and Scope

The full patched release CLI is installed in a versioned user-local directory alongside the matching bundled Code Mode helper. The signed Desktop bundle and original binary were not modified; deep/strict app signature verification still passes. A guarded CODEX_CLI_PATH launcher is configured for the next Desktop launch and persisted by a user LaunchAgent. It falls back to the bundled CLI when the Desktop build changes. The fallback branch was tested.

The installed launcher and binary, not just a source-level test harness, passed the loopback reproduction above. They also passed a real Azure app-server tool -> remote compact -> tool test, with the test marker retained, Code Mode host enabled and observed running, gpt-6-astra, medium reasoning and the original 1M context configuration. Test interval: 2026-09-06T01:45:58.912353+00:00 to 2026-09-06T01:46:20.962310+00:00. No error notifications were recorded in that short smoke test. It is not a long-duration or multi-task reliability claim.

The still-running Desktop app was deliberately not terminated mid-investigation. The user must fully quit and reopen it; post-restart Desktop executable selection has not yet been verified. CLI/app-server functionality was verified directly through the same installed launcher. The override is reversible, and no model/provider/retry/context settings were changed by this installation.

Patched executable SHA-256: cf46450d3bde54feedae108eec0f6245b6969d4402b5b5f8a824d7c6fedd19da. This is a locally built, tested workaround, not an official signed OpenAI release or an upstream-accepted fix.

The issue intentionally includes the complete portable reproduction and patch rather than linking to private chat history. API keys, authorization headers, Azure resource hostname, private user paths, raw prompts, image payloads and local task IDs are omitted. Request correlation above is diagnostic metadata, not a credential.

I searched existing issues for response.failed with EOF/idle-timeout and the SSE function name. Related #4161 concerns parsing retry-delay text, not waiting for EOF after an already-classified terminal failure; I did not find a matching duplicate.

Activity

  1. added
    bugSomething isn't working
    CLIIssues related to the Codex CLI
    connectivityIssues involving networking or endpoint connectivity problems (disconnections)
    on Sep 6, 2026
  2. 1320800521 commented on Sep 12, 2026

    @1320800521

    Independent reproduction from XBSTACK on macOS confirms the same terminal-event handling shape on both codex-cli 0.153.4 (ChatGPT.app bundled binary) and the current stable 0.154.0.

    Test setup: local loopback HTTP SSE server, isolated temporary CODEX_HOME, retries disabled, stream_idle_timeout_ms = 800, no model/API credential/external network.

    Results:

    • response.failed followed by immediate EOF: original marker is preserved: stream disconnected before completion: XBSTACK_LOCAL_TERMINAL_FAILURE.
    • The same response.failed while the server keeps the socket open: Codex waits until the idle timeout and surfaces: stream disconnected before completion: idle timeout waiting for SSE. The original marker is no longer present in the user-visible error.
    • I reproduced the same behavior on 0.153.4 and 0.154.0.

    Minimal independent fixture + version matrix:
    https://github.com/xbstack/codex-response-failed-idle-timeout-repro

    Detailed notes and diagnostic boundary:
    https://www.xbstack.com/en/ai/tools-lab/codex-response-failed-idle-timeout-sse/?utm_source=github&utm_medium=referral&utm_campaign=codex_response_failed_idle_timeout&utm_content=issue_followup&ref=github

    I am not treating the proposed source patch in this issue as a released fix yet; I can rerun the fixture against the first release that contains the fix.

  3. SouthWinter commented on Sep 14, 2026

    @SouthWinter

    I have a matching real-world reproduction on the first-party OpenAI provider through Codex Desktop's remote-SSH app-server.

    Environment:

    • Codex CLI/app-server: 0.153.4
    • ChatGPT Pro subscription
    • Remote Linux host connected through Codex Desktop over SSH
    • Reproduced with both gpt-6-astra and gpt-5.6-sol
    • Reported usable context window: 258,400 tokens
    • Remote compaction v2 enabled

    Observed error:

    Error running remote compact task: stream disconnected before completion: idle timeout waiting for SSE

    This is intermittent rather than deterministic. In the same long-lived thread I counted:

    • 11 successful compactions
    • 4 failures with the exact idle-timeout string
    • rollout JSONL size: 43,505,162 bytes
    • 835 paired custom tool calls/outputs

    One gpt-5.6-sol failure followed a reported token count of 157,617 at 2026-09-14T07:30:55Z; the task completed with the error at 07:47:26Z. Around the same period, the app-server log repeatedly reported failed to refresh available models: timeout waiting for child process to exit.

    Switching from Astra to 5.6 did not eliminate the issue. Restarting the remote app-server unloaded the failed in-memory state and allowed the thread to continue, but this was only a recovery step.

    At failure time, the still-running app-server had retained an earlier 120K auto-compact threshold; after restart it loaded the configured 240K threshold. This explains why compaction triggered below 240K, but not why the stream ended with an idle timeout.

    The visible error is consistent with this issue's failure-handling defect: a terminal upstream response.failed may be getting masked by the later SSE idle timeout. I cannot confirm the original terminal event because the local rollout/log retained only the final idle-timeout message. Unlike the Azure setup in the original report, this case uses the first-party OpenAI/ChatGPT route with remote compaction v2 enabled.

    The session shape also resembles #24732: a tool-heavy, long-lived session and a roughly 43.5 MB rollout, although this case has fewer tool calls and failed at a lower visible token count.

    I can provide a sanitized event timeline or a session/feedback ID if maintainers need it.

  4. diegomrsantos commented on Sep 14, 2026

    @diegomrsantos

    Confirmed with Codex Desktop 26.908.40834, bundled CLI 0.154.0-alpha.6.2, and source at b6a5d5bb14d6c3d6a656ca8d2274d6d4259ac57d.

    I put an adaptation of the proposed fix and four regression tests in diegomrsantos/codex#1. All four tests fail against the original behavior and pass with the fix.

    Repeated stalls lasting several minutes disrupt my daily work and sometimes force me to interrupt tasks. This defect could contribute when an affected request uses SSE, but I have not captured this specific failure sequence during a live incident.

    @jif-oai, could you triage this or route it to the current owner? I identified you through the recent changes to this SSE polling loop in #43408.

  5. 1320800521 commented on Sep 20, 2026

    @1320800521

    Thanks — the four regression tests in your adaptation close an important gap between the deterministic loopback reproduction and a candidate source-level fix.

    From the XBSTACK side, the useful release-level verification boundary is still the same pair:

    1. response.failed + immediate EOF should preserve the original terminal error.
    2. response.failed + socket kept open should terminate from the terminal event rather than later surfacing only the SSE idle-timeout error.

    I’ll rerun the independent fixture against the first tagged Codex release that contains the fix (or against a maintainer-nominated commit if needed) and report the before/after result here.

  6. jaysonsantos commented on Sep 22, 2026

    @jaysonsantos

    I have a matching production incident with Codex CLI 0.153.4 against a custom OpenAI-compatible Responses endpoint.

    Observed evidence:

    • One tool-heavy writer session ran for 987.8 seconds and reported 40 observed turns.
    • The session changed local files and ran focused tests.
    • It then emitted stream disconnected before completion: idle timeout waiting for SSE.
    • The SDK exited with code 1 and produced no final result event.
    • A separate model request through the same route completed successfully 4.7 seconds later.
    • The gateway route had request and stream-idle timeouts disabled.
    • No deployment was recorded during the request window.

    I did not capture the original terminal response.failed event. Therefore, I cannot prove that this client defect masked the upstream error in this incident. The timing and final message match the documented failure shape.

    This incident also shows the recovery impact. The writer completed substantial local work, but the generic terminal error prevented automatic continuation. The caller could not distinguish an upstream model failure from a transport or infrastructure failure.

    I can provide a sanitized event sequence if maintainers need more detail.

  7. 1320800521 commented on Sep 23, 2026

    @1320800521

    Thanks — this is useful production evidence, and I would keep one scope boundary explicit.

    Your incident strengthens the operational impact case: substantial tool work completed, the client ended with the generic SSE idle-timeout error, and a separate request through the same route succeeded shortly afterward. That makes a persistent route outage less likely.

    It still does not prove that this exact incident contained a terminal response.failed that Codex then masked. I would separate the evidence into two classes:

    1. Deterministic local reproduction: response.failed arrives, the socket stays open, and Codex waits until the idle timeout, replacing the original terminal error.
    2. Production incident compatible with that shape: final idle-timeout error, but the upstream terminal SSE event was not captured.

    If maintainers want to distinguish the two in production, the most useful next artifact would be a sanitized SSE/app-server event sequence immediately before the timeout. If it shows response.failed was already received and the client still waited, that would directly connect the production incident to this bug. If no terminal event arrived, it may be a separate upstream/transport stall with the same final symptom.

    From the XBSTACK side I’ll keep the release-level regression check unchanged: after a fix lands, both response.failed + EOF and response.failed + held-open socket should terminate with the original failure rather than a later idle-timeout replacement.

  8. mrstepanrf-a11y commented on Sep 26, 2026

    @mrstepanrf-a11y

    Additional production evidence + current-main check from 2026-09-26.

    A Windows Codex Desktop incident (26.924.1866.0, bundled codex-cli 0.158.0-alpha.2) hit remote auto-compaction at 244,963 / 244,800 scope tokens. After two WebSocket compaction failures (IO error: cannot decrypt peer's message) the client fell back to HTTP. /responses returned 200 OK, then ~302.356 seconds later the runtime surfaced stream disconnected before completion: idle timeout waiting for SSE.

    I also inspected current openai/codex main today. In codex-rs/codex-api/src/sse/responses.rs, response.failed still becomes an error from process_responses_event, but process_sse_with_treatment stores that as response_error = Some(...) and continues reading; the timeout branch still returns idle timeout waiting for SSE directly. So the terminal-failure/held-open-socket behavior described in this issue still appears present in current main.

    There is a tested fork patch in diegomrsantos#1 that treats response.failed as terminal and returns immediately, with regression tests. I have not treated that fork as an official released fix.

    For the production incident, I do not have proof that a response.failed event arrived before the timeout, so this is compatible evidence, not causal proof. OpenAI request IDs are available in #45472 for backend correlation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workingconnectivityIssues involving networking or endpoint connectivity problems (disconnections)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions