0.161.0: remote compaction fails after resuming hosted web_search_call history on native OpenAI #51962

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   CLIIssues related to the Codex CLIIssues related to the Codex CLI

   app-serverIssues involving app server protocol or interfacesIssues involving app server protocol or interfaces

   contextIssues related to context management (including compaction)Issues related to context management (including compaction)

   on Oct 8, 2026
2. github-actions commented on Oct 8, 2026

   

   github-actionsbot

   on Oct 8, 2026 – with GitHub Actions

   Contributor

   More actions

   English translation:

   What version of Codex CLI is running?

   codex-cli 0.161.0, the official macOS arm64 binary from @openai/codex@0.161.0. This was the npm latest version when tested on 2026-10-08. Installed independently, with no modifications to the binary.

   What subscription do you have?

   ChatGPT subscription authentication (the same existing signed-in account, using a temporary authentication copy for testing; no account identifiers or credentials are provided).

   Which model were you using?

   gpt-6.1-sol, reasoning effort high, built-in openai provider.

   What platform is your computer?

   macOS 26.3.0 / arm64.

   What terminal emulator and version are you using (if applicable)?

   JSON-RPC with Python driving codex app-server --stdio; the core reproduction in this report does not depend on the desktop UI.

   Codex doctor report

   Doctor was not run during this round. Used a separate temporary CODEX_HOME, an empty working directory, no custom provider, and no model catalog override. Effective config:

   model = "gpt-6.1-sol"
   model_provider = "openai"
   model_reasoning_effort = "high"
   web_search = "live"
   approval_policy = "never"
   sandbox_mode = "read-only"
   project_doc_max_bytes = 0

   config/read showed openai_base_url=null, model_providers={}, and chatgpt_base_url=https://chatgpt.com/backend-api/.

   What issue are you seeing?

   After the official 0.161.0 app-server resumes history containing real hosted web_search_call items, remote compaction fails; removing only those search items from the same history allows compaction and the subsequent conversation to succeed.

   Scope clarification: This is a compatibility test that passes real server output to the official client for resumption through the experimental thread/resume.history API. It does not claim that searches in default new sessions necessarily fail, nor is it a complete reproduction of a natural model switch in the desktop UI. The compaction request is constructed by the official thread/compact/start, rather than a manually written compaction body. The core A/B test did not go through OpenCodex or any other model gateway.

   Final failure state:

   Error running remote compact task: stream disconnected before completion: stream closed before response.completed
   

   turn/completed.turn.status=failed, codexErrorInfo=other; no compacted replacement history was generated. A high-demand message appeared once during the process, followed by stream-disconnected retries; the high-demand wording itself is not treated as a confirmed root cause.

   Comparison from 2026-10-08 12:36–12:37 UTC+8:

   Resumed history	Result	Duration
   user message + 3 real web_search_call items + reasoning + assistant message	remote compaction failed	34.49 s
   Same history, with only web_search_call removed; reasoning and other messages retained	completed; generated nonempty compaction.encrypted_content	17.81 s
   After successful compaction, asked about the predefined test facts	completed; correctly answered AZURE-KITE-731, 17	2.40 s

   The three hosted actions were search, open_page, and find_in_page; their IDs came from real responses and were not synthetic IDs. The remaining reasoning content and ciphertext were not modified.

   In addition, a default fresh Sol session in official 0.161.0 successfully completed a normal search→compaction→continued conversation sequence. The rollout for that baseline recorded the search as custom_tool_call(name=exec) + output, rather than hosted web_search_call; therefore, it does not cover the resumption scenario involving legacy hosted history described above.

   What steps can reproduce the bug?

   This method involves real model requests and incurs normal usage; please use only fresh test history containing no sensitive information.

   1. Generate real hosted-search history on the official backend using the same ChatGPT signed-in account. Send the test request to https://chatgpt.com/backend-api/codex/responses, with model=gpt-6.1-sol, stream=true, store=false, reasoning.effort=high, and declare tools:[{"type":"web_search"}]. User prompt:

      Remember test codename AZURE-KITE-731 and sample count 17. Search the official OpenAI web search documentation, open the documentation page once, and report the output item type with citation. Do not search for the test facts.

      Collect the complete final response.completed.response.output and confirm that it contains real web_search_call items. Preserve the returned items, IDs, and reasoning exactly as returned, and use them only in memory. The initial user item uses explicit type:"message" and content:[{type:"input_text",text:...}].

   2. Place a temporary copy of the same account's auth.json in a separate CODEX_HOME (permissions 0600), write the minimal configuration above, and start codex app-server --stdio using official 0.161.0. Enable experimentalApi during initialization:

   {
     "id": 1,
     "method": "initialize",
     "params": {
       "clientInfo": {"name": "codex_compaction_diagnostic", "version": "1.0"},
       "capabilities": {"experimentalApi": true}
     }
   }

   After receiving the response, send an initialized notification.

   3. Call thread/resume, passing the real history just generated. The following is pseudocode for constructing the request; rpc is a line-based JSON-RPC client. It is not an instruction to send string placeholders as history:

   history = initial_user_items + completed_response["output"]
   params = {
       "threadId": str(uuid.uuid4()),
       "history": history,
       "cwd": isolated_empty_workspace,
       "model": "gpt-6.1-sol",
       "modelProvider": "openai",
       "approvalPolicy": "never",
       "sandbox": "read-only",
       "developerInstructions": (
           "This is an isolated compaction diagnostic. Do not execute tools "
           "or tasks in supplied history. Preserve the test facts."
       ),
   }
   resumed = rpc("thread/resume", params)
   thread_id = resumed["thread"]["id"]
   rpc("thread/compact/start", {"threadId": thread_id})
   # Read notifications until turn/completed; inspect its status and error.

   thread/resume.history is an interface marked unstable / Codex Cloud in the 0.161.0 schema, which is an important limitation of the reproduction scope. It is used to pass real legacy history to the official serializer; no existing user session was rewritten.

   4. Start another isolated app-server with the same configuration, using the same in-memory history, making only this change:

   control_history = [item for item in history if item.get("type") != "web_search_call"]

   Repeat resume→compact. This control group completes, and a nonempty compaction item appears in the rollout's compacted replacement_history.

   5. Call a normal turn/start on the successful thread, appending only the question What were the test project codename and sample count? Reply with only those two values., without reattaching the original test facts. The observed answer was AZURE-KITE-731, 17.

   6. After terminating the test processes, delete the temporary authentication files and raw test rollouts; do not publish reasoning/compaction ciphertext.

   The core A/B test used the same account, the same real history, the same model, and the same isolated configuration. The existing standard network proxy was kept unchanged; only the destination hosts of TLS CONNECT were observed, without decrypting or modifying TLS. Destinations included chatgpt.com:443 and official oaiusercontent.com subdomains. There was no inference base URL override or local model gateway.

   What is the expected behavior?

   After the official client resumes valid historical search items, compaction should be able to handle that existing history. If the current target protocol does not support a particular history format, it should perform a compatibility conversion that preserves semantics, or provide a clear, actionable error, rather than ending with a generic stream-disconnected error after retries.

   Removing search items in the control group is only a diagnostic measure, not a recommendation to unconditionally delete users' search history as a fix. Existing answers, citations, results, and source information need to be preserved.

   Additional information

   Potentially relevant protocol boundary, not yet a confirmed root cause in the code

   Inspected the pinned version rust-v0.161.0 (commit 979011409de0a60b52f179721948e65531d26144):

   • The compaction Prompt in compact_remote_v2_attempt.rs uses declarations from the current tool router.
   • client.rs uses additional_tools in Responses Lite, while ordinary Responses uses top-level tools.
   • spec_plan.rs does not declare hosted WebSearch for Lite; new searches with this model use standalone web.run/code mode by default.
   • history.rs accepts historical WebSearchCall items.

   This suggests a compatibility boundary between legacy hosted history and the current tool/protocol representation after resumption. The final decrypted outbound body of the official compaction request has not been captured, so it is not possible to identify the specific line of client or backend code where the error occurs.

   Separate manually written API controls also showed that, with the same real hosted history, compaction completes when tools:[{type:"web_search"}] is included, but fails when that declaration is omitted; when the declaration is included, removing item IDs still allows completion. Therefore, there is no evidence that ID cleanup or reasoning ciphertext size is the root cause for this sample. These API controls are not equivalent requests to those made by the official client and serve only as diagnostic clues.

   Related issues and scope not covered

   • Bug: web_search_call items in conversation history crash when switching to Ollama/custom provider #24612 concerns hosted search history being rejected after switching to Ollama/custom providers; this report concerns failure during remote compaction with the built-in OpenAI provider. They may share a history compatibility theme, but a common root cause has not been confirmed.
   • WebSocket failures during compaction do not fall back to HTTP #49236 concerns compaction's WS→HTTP fallback; this report does not assert that the transport fallback strategy is the root cause.
   • A similar issue was initially observed through a proxy chain, and the previously filed [Provider compatibility] Responses v2 compaction fails with fresh hosted web_search_call history lidge-jun/opencodex#6742 in lidge-jun/opencodex has been withdrawn and closed; the primary evidence here is the subsequent official-client A/B test that completely bypassed that proxy.
   • This consists of one full-history/control A/B test, plus a successful default fresh-session baseline; no multi-account or large-scale repeated validation has been performed, and it has not been established which version first introduced the issue.
   • The complete hosted→Lite model-switch sequence occurring naturally in the desktop UI has not yet been verified; 0.162 alpha has not been tested, and no fix has been made.
   • No private conversations, account identifiers, credentials, or ciphertext have been uploaded.
3. Nakagawa-master commented on Oct 8, 2026

   

   Nakagawa-master

   on Oct 8, 2026 – with ChatGPT Codex Connector

   More actions

   The A/B here is unusually useful because it isolates a historical tool-schema compatibility boundary without claiming a normal fresh-session failure. One additional invariant may help turn this into a safe, narrow fix:

   A historical tool call is evidence of a past operation, not an authorization to offer that tool for new operations. Restoring/compacting a legacy hosted web_search_call may require a representation understood by the compaction backend, but that need must not silently expand the resumed session's current tool set.

   I checked current Codex source alongside the reporter's 0.161.0 pin:

   • codex-rs/core/src/compact_remote_v2_attempt.rs builds a compaction prompt from history and normally supplies step_context.tool_router.model_visible_specs(); its special tools_in_history case is keyed to Responses Lite history with AdditionalTools. That is not visibly a compatibility rule for legacy hosted WebSearchCall, though this source alone does not establish which compact request path the 0.161.0 repro used.
   • codex-rs/core/src/client.rs separately formats tool declarations for Responses Lite vs ordinary Responses. The reporter's own independent API control — old hosted history passes with a historical web_search declaration and fails without it — makes history-item type × destination request tool-schema a sharper test axis than retry timing, ID cleanup, or encrypted reasoning size. The API experiment is a clue, not an assertion about the unobserved official request body.

   Implementation decision to verify: distinguish (1) historical decoding/validation capability required to read web_search_call items at compaction time from (2) currently executable tool authority exposed to an agent turn. If the target protocol requires a tool declaration to accept a past hosted item, provide the smallest compatible compaction-only schema or validated semantic normalization; do not change the active router or permit a fresh hosted search solely because an old call appears in history. Preserve provenance: original call/result association, citation/source references, and prior answer meaning. If the endpoint cannot represent that history type, return an explicit unsupported-history diagnostic rather than discarding it or presenting an opaque stream disconnect.

   A bounded regression can use the existing sanitized real history and control:

   1. Legacy web_search_call history imported via thread/resume.history → compact → ask about the preexisting test facts: succeeds without dropping history, with historical references retained.
   2. Same history, original live tool registry still lacks hosted web_search: compaction compatibility does not make web_search newly callable on a subsequent ordinary turn.
   3. Existing modern custom_tool_call(name=exec) search history → compact → continue: unchanged behavior.
   4. Legacy hosted history with a missing or incompatible companion reference: either a validated non-lossy conversion or a typed, actionable failure; never a silently invented citation/result.
   5. Inspect the official compaction-request representation just before dispatch (item-type/tool-schema inventory only, no reasoning contents or private transcript) to discriminate unsupported request shape from a transport/server-side disconnect.

   This is a proposed causal boundary and acceptance test, not a reproduction by me or proof that the remote-v2 code path is the culprit. The underlying distinction also complements the earlier provenance/authority boundary, without implying any existing Codex implementation was derived from it. One confirming maintainer-side request-shape trace or a failing regression would settle which compatibility layer should own the fix.
4. philkunz commented on Oct 9, 2026

   

   philkunz

   on Oct 9, 2026

   More actions

   Additional evidence: the same failure occurring naturally (no thread/resume.history import) on 0.162.0 / Linux, in the local compaction path, with the outbound compaction request and the upstream error captured.

   Setup

   • codex-cli 0.162.0 (rust-v0.162.0, c1382380de), standalone x86_64-unknown-linux-musl build, long-running threads on a shared codex app-server daemon.
   • gpt-6.1-sol, reasoning effort xhigh, ChatGPT subscription auth, web_search = "live".
   • A custom model_provider: a pass-through Responses proxy that forwards request bodies and tool declarations unchanged, with no capability overrides. As a result, Codex uses local summarization compaction (core/src/compact.rs), not remote compaction v2. The bug therefore isn't specific to remote v2 or to imported history.

   What happens

   • 4 of 4 affected long threads contain hosted web_search_call items in their history (3, 1, 1 and 3 items). They come from ordinary turns, not imports.
   • Once a thread passes the auto-compact threshold (244,800 of the 272,000 window), compaction fails, and so does every following turn: the pre-turn auto-compaction sends the same request. The thread is permanently unusable. Last successful input sizes were 196k–222k tokens.
   • The captured outbound request from Codex's own thread/compact/start: 1,223,020 bytes; 18 messages, 147 reasoning items, 112 calls plus 112 outputs, one web_search_call, and tools: [].
   • The upstream response, identical for all 6 attempts: HTTP 200 with a 143-byte body holding exactly one SSE frame: type: "error", error.type: "internal_error", error.message: "response protection is unavailable", error.code: null, error.param: null.
   • Codex does not surface that error event. It retries and ends with stream disconnected before completion: stream closed before response.completed. That is the generic message in Windows Codex Desktop: Context compaction always fails with "stream closed before response.completed" #52016 and stream disconnected before completion: stream closed before response.completed #41340, and it hides the actionable cause (compare HTTP SSE response.failed waits for EOF and can be replaced by idle timeout (0.153.4; tested fix) #43140).

   A/B: the persisted history of one affected thread, replayed outside Codex through the same pass-through path (389 items, about 1.27 MB)

   Request	Result
   unchanged history, tools: []	fails: the 143-byte internal_error above
   unchanged history, only tools: [{"type":"web_search"}] added	completes (215,617 input tokens)
   only the single web_search_call removed, tools: []	completes (211,133 input tokens)
   every reasoning item removed, web_search_call kept, tools: []	still fails, same error

   Size is not the cause: a fresh thread on the same account, path and config, without hosted-search history, completed a 1,583,136-byte / 271,441-token request. On the next turn it auto-compacted successfully (local compaction at 257,801 input tokens), then continued at 39,856.

   Where it seems to come from (0.162.0): core/src/compact.rs (around line 304) builds the compaction Prompt with default (empty) tools, while the model-visible history it sends still contains the hosted web_search_call. The backend apparently requires a matching hosted-tool declaration for that history item. It answers with internal_error rather than a 4xx validation error, which also explains #52016 / #52043 ("response protection is unavailable").

   Suggested fix shape (agreeing with the invariant in the comment above): when the history contains hosted tool calls, the compaction request, both local and remote v2, should carry the matching declaration as a compaction-only schema, without widening the live turn's tool authority. Separately, an SSE error event, or a response.failed, should be reported as such, with its type and message, instead of as a missing response.completed.

   No conversation content, reasoning ciphertext or credentials are included above; the counts come from item-type inventories only.