[Bug]: Non-Codex clients on public endpoints get zero prompt-cache reuse on ChatGPT-backed OpenAI models (no synthesized session_id/originator) #6520

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   proxyHTTP proxy, routing, reverse-proxy / management authHTTP proxy, routing, reverse-proxy / management auth

   on Oct 3, 2026
2. github-actions

   closed this as not plannedon Oct 3, 2026
3. github-actions commented on Oct 3, 2026

   

   github-actionsbot

   on Oct 3, 2026 – with GitHub Actions

   Last edited by github-actions

   Contributor

   More actions

   Issue reopened

   The report now contains the information required by the automated check. Thanks for updating it.
4. github-actions

   reopened this on Oct 3, 2026
5. lidge-jun commented on Oct 4, 2026

   

   lidge-jun

   on Oct 4, 2026

   Owner

   More actions

   Partial implementation landed on dev via #6554, commit f85925d.

   For /v1/responses and /v1/messages, a safe caller-provided x-session-id is now promoted when no stronger explicit session identity exists. Authenticated identities are scoped by the admitted principal; loopback caller markers retain their value. Forwarding, precedence, admission, body/abort preservation and isolation have regression coverage, and exact-head hosted CI passed before integration.

   Keeping this issue open for the broader acceptance in the report: this change does not synthesize an originator, does not invent a stable identity for a caller that supplies none, and has not independently reproduced the reported live cache ratio on the affected third-party/plain-curl flows. The original PR's live measurements remain source-author evidence, not a new deployed-runtime result. No installed runtime was changed during integration.
6. lidge-jun commented on Oct 8, 2026

   

   lidge-jun

   on Oct 8, 2026

   Owner

   More actions

   Triage on current dev (a12bb1f).

   Already addressed (#6554, #6588): on /v1/responses and /v1/messages, a safe caller-provided x-session-id is promoted to session_id when no stronger identity exists, and session-id / thread-id aliases are normalized at ChatGPT egress. Explicit session_id and body prompt_cache_key are preserved.

   What remains is a product decision rather than a defect:

   • Callers that send no session marker get no invented identity. OpenCodex does not synthesize originator either; doing so would contradict the documented non-impersonation contract.
   • Chat Completions ingress does not yet apply the x-session-id promotion that Responses has.
   • A shared prefix/cache cohort is intentionally never turned into conversation identity.

   Your A/B changed session_id and originator together, so we do not yet know which one the backend needs for cache reuse. A sanitized four-way comparison on current dev would settle it: neither header, session only, originator only, and both, with the same stable prefix and prompt_cache_key, one pinned account, the exact version, and the raw upstream usage.
7. chano-gpt commented on Oct 10, 2026

   

   chano-gpt

   on Oct 10, 2026

   More actions

   Four-way A/B requested in the triage above, on opencodex 2.81.0 (npm), Linux, /v1/responses, loopback ingress, model gpt-6-astra--fast.

   Setup: synthetic ~9.6k-token instructions prefix with a fresh random nonce per variant (so every variant starts cold), a fixed body prompt_cache_key per variant, stream: true, 4 identical requests 3 s apart. Values are cached_tokens / input_tokens from the upstream response.completed usage. OpenAI pool: 4 Codex accounts, one selected, auto-switch not touched during the run.

   Variant	req 1	req 2	req 3	req 4
   neither header	0/9643	0/9643	0/9643	0/9643
   session_id only	0/9638	9472/9638	9472/9638	9472/9638
   originator only	0/9643	0/9643	0/9643	9472/9643
   both	0/9645	9472/9645	9472/9645	9472/9645

   So session_id alone is sufficient (98.3% reuse from request 2); originator is not needed and does not help on its own. A stable body prompt_cache_key without session_id gets no reuse.

   Real-client impact, from oh-my-pi (omp 18.8.7) configured with a custom provider pointing at opencodex: omp sends a stable per-session prompt_cache_key but only attaches the session_id header when its provider id is literally openai, so through a provider named opencodex every GPT turn in a 5-turn session measured 3.8% cached overall (74,945 of 88,086 cacheable tokens missed). Adding a per-process session_id header on the client side brought in-session requests to 91–98% cached.

   Given the non-impersonation contract, a narrow option that does not invent identity: on loopback ingress, when no session_id/session-id/thread-id/x-session-id is present, promote a safe body prompt_cache_key to session_id (or document that non-Codex clients must send session_id/x-session-id themselves, with this table as the reason).

   Repro script (Python stdlib only)

   import json, time, uuid, urllib.request
   URL = "http://127.0.0.1:10100/v1/responses"
   BASE = "You are a careful assistant. " + " ".join(
       f"Rule {i}: when handling input category {i}, prefer explicit, verifiable steps and cite the relevant section number." for i in range(400))
   
   def send(prefix, pck, headers):
       body = {"model": "gpt-6-astra--fast", "instructions": prefix, "prompt_cache_key": pck,
               "input": [{"role": "user", "content": "Reply with just: ok"}],
               "stream": True, "store": False, "max_output_tokens": 64, "reasoning": {"effort": "low"}}
       h = {"content-type": "application/json", "authorization": "Bearer <loopback-key>", **headers}
       usage = None
       with urllib.request.urlopen(urllib.request.Request(URL, data=json.dumps(body).encode(), headers=h), timeout=120) as r:
           for line in r:
               line = line.decode().strip()
               if line.startswith("data:") and '"response.completed"' in line:
                   usage = json.loads(line[5:])["response"]["usage"]
       return usage["input_tokens"], usage.get("input_tokens_details", {}).get("cached_tokens", 0)
   
   variants = {"neither": lambda s: {}, "session_id only": lambda s: {"session_id": s},
               "originator only": lambda s: {"originator": "codex_cli_rs"},
               "both": lambda s: {"session_id": s, "originator": "codex_cli_rs"}}
   for name, mk in variants.items():
       prefix, pck, sid = f"[{uuid.uuid4().hex}] " + BASE, str(uuid.uuid4()), str(uuid.uuid4())
       res = []
       for _ in range(4):
           res.append(send(prefix, pck, mk(sid))); time.sleep(3)
       print(f"{name:16s}", " ".join(f"{c}/{t}" for t, c in res))
8. lidge-jun

   added a commit that references this issue on Oct 10, 2026

   fix(chat): promote caller x-session-id on Chat Completions ingress (#…

   ...

   This issue will close once commit 0025374 is merged into the 'main' branch.

   0025374
9. lidge-jun commented on Oct 10, 2026

   

   lidge-jun

   on Oct 10, 2026

   Owner

   More actions

   Fixed on dev by #6880 (squash 0025374d7b). A caller-supplied x-session-id on Chat Completions ingress is now promoted to the upstream session identity. Non-Codex clients that send one get prompt-cache reuse on ChatGPT-backed models.
10. lidge-jun

    closed this as completedon Oct 10, 2026