Repository navigation
fix(codex): classify spent subscription windows as quota rejections - #434
Merged
MagicalTux merged 3 commits intoSep 19, 2026
Merged
MagicalTux merged 3 commits into
MagicalTux merged 3 commits into
Conversation
…hrottle A 429 is classified two ways here, and the difference decides everything that follows. A quota rejection is durable: the account cannot serve again until a window resets, so it is held and the request rotates. A rate-limit throttle is momentary: the account is paused, the request waits out retry-after and goes back to the SAME account. That classification read only `anthropic-ratelimit-unified-*-status`. The ChatGPT backend never sets it. It reports how much of each window is gone instead, and a window at its limit is the same fact in the other spelling. So every Codex 429 fell through to the throttle path. With no sibling to fail over to, the proxy slept out retry-after and asked the exhausted subscription again, to be refused again. With one, the single failover hop still carried the request — but the account was left `active` behind a silent pause: nothing recorded that it was out of service or until when, the revalidation probe that keys off `throttledAt` never armed, and the pause taught the concurrency learner that the account had choked at that load, which a spent quota is no evidence of. Read the refusal through `parseCodexQuota` — the parser the quota sweep already uses — rather than off the raw headers. It groups the `x-codex-*` families and classifies each window by its own `window-minutes`, so `codexSpentWindows` sees what the account-wide pair cannot: a subscription states its only 5-hour window inside a NAMED, model-scoped family, and `x-codex-primary/secondary-used-percent` alone would read 42% / 0% while the account was refusing on a spent session window. The labels it returns name what is spent, and the rejection log line now says which window it was. Two annotations come with it: `parseCodexQuota` gets the `@returns` typedef it never had, so a caller can read `modelBuckets` off it, and `rateLimitHeaders` gets the `Record<string, string>` it is now passed as. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ucket A spent named-family weekly bucket was classified as a general rejection, which parks the whole account for every model. Nothing in selection reads the recorded model buckets, so once the hold lapsed the account was picked again, refused again and was parked again. Only the account-wide `5h` and `weekly` windows now hold the account; a rejection where only named-family windows are spent rotates this request to another account, the way the Fable-only rejection does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merged
MagicalTux
added a commit
that referenced
this pull request
Sep 20, 2026
Thirty-six commits since 1.1.20. Several change what a client sees on a failure, so read the first section before upgrading a shared deployment. Behaviour changes #439 a 401 from upstream is no longer relayed to the client: the request fails over like a 403, and an API-key account or an OAuth account with no refresh token leaves rotation (it used to be picked again on every request). With nothing left the client gets the proxy's own error #439 a path under /teamclaude/ that no control route claims — a typo, or the wrong verb — answers 404 locally instead of being forwarded upstream under a fleet credential #429 the synthetic 429's retry-after is the real reset of the windows blocking the request's candidate accounts, not a flat 60s, and the message counts only those candidates; #408 names accounts that need a re-login instead of calling them "at quota" #438 session pins are per conversation (session id plus a digest of the first message), so a session's subagents spread across accounts. `sessions.items[].id` in status is the composite key, load is counted per conversation, and a persisted concurrency cap re-learns #378 a `thread: continue` bound for a per-account third-party upstream is refused with the 400 Anthropic gives, so the client resends the whole conversation; `messageThreads: true` opts a relay out #434 a 429 whose x-codex-* headers show a spent account-wide window holds the account like an Anthropic rejection; a spent model-scoped bucket only moves the request #437 a Codex response head is awaited for five minutes (Anthropic unchanged) #411 idle keep-alive connections are held 120s on both listeners #389 with session distribution on, requests carrying no session id rotate on a cursor of their own instead of all resting on the current account #405 `defaultClientMode` ("mitm" | "base-url") sets what `run` and `env` do without a flag; `--mitm` / `--no-mitm` decide per launch, and in base-URL mode `env` unsets a stale proxy export naming this proxy #439 route `--bucket` is validated; an array `switchThreshold` reads as the default with one line saying so Features #419 an MCP management endpoint at POST /teamclaude/mcp, off unless `proxy.mcp` is "read" or "full"; a named client key is read-only even in full mode, and with no proxy key it serves only this machine #428 per-account `switchThreshold`, a number or a per-bucket table #406 a Claude+Codex pool is drawn as two panes on a wide terminal, each with its own current marker; #392 names the provider in a mixed list; #418 lets the operator arrange the list (`displayOrder`); #376 draws loopback-served accounts last; #435 shows the percentage beside a bar's countdown; #394 shows the running version in the header #430 free Codex rate-limit reset credits in status, the TUI and the dashboard #385 `proxy.terminalOnly` tunnels chatgpt.com so ChatGPT Desktop stays out Fixes #404 a TUI paint can no longer block the proxy (stdout non-blocking, frames dropped while the terminal is behind); #410 a dead terminal no longer takes the proxy with it, and SIGHUP shuts down cleanly #433 token usage is booked from Codex Responses streams #386 #387 #388 the Codex five-hour window is read from the model-scoped family and the usage probe, and extra limits are named from their entries #431 a headerless 429 that follows the request is retried once #432 #439 startup and collaborator log lines reach the TUI's activity pane and log file instead of the covered terminal #415 #403 #439 reload mirrors `priority`, `disabled`, `stripRequestFields` onto the config entry, and a reload during a removal does not re-add it #439 the Host check uses the address actually bound; sx.org calls time out #381 the dashboard polls status before asking for a key #403 session outcome accounting classifies the decoded path; account names in daemon log lines are sanitised Tooling #371 #372 #373 `npm run typecheck` (tsc over the JS sources) in CI, with a strict-mode ratchet: per-file strict diagnostics may not grow past the pre-merge commit (2006 at introduction, 1735 now) #401 docker workflow actions bumped Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
rikbrown
added a commit
to rikbrown/teamclaude
that referenced
this pull request
Oct 7, 2026
…indows Upstream KarpelesLab#434 classifies a spent Codex window through codexSpentWindows, which reads each window by its own `window-minutes` and covers named families. The sidecar's flat reading of the primary/secondary percentages is gone with it, along with its tests; a forwarded `x-codex-*` header reaches the upstream classifier exactly as a direct one does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
forwardRequestdistinguishes:retry-after, and resend to the same account.Quota rejection currently requires
anthropic-ratelimit-unified-5h-status === 'rejected'or its 7d equivalent. The ChatGPT backend does not set these headers. It reports percentage quota inx-codex-*, whichsrc/codex-quota.jsalready parses, so every Codex 429 entered the throttle path even when its subscription was spent.With no sibling, the proxy slept a
retry-afterbelow the inline-absorb cap and resent to the exhausted subscription. With an idle sibling, the request could succeed after the single failover hop (#137), but the account stayedactivebehind a silent pause: the TUI showed it healthy, thethrottledAtrevalidation probe did not arm, andpauseAccounttrained the concurrency learner from a spent quota.The change
codexSpentWindows(headers)now reads the response throughparseCodexQuota, the parser used by the quota sweep, and returns labels for windows at or past their limit.generalRejectedtests whether the returned array has entries.The parser groups
x-codex-*families and uses each window'swindow-minutes, so it sees named model-scoped 5-hour windows that account-wideprimary/secondarycannot. A spent session window can report flat values of 42% / 0% while refusing every request.classifyalso discards unparseable percentages and zero-length windows, which mean "not applicable".updateQuotahas already stored utilisation from these headers. The existing rejection branch then holds the account forretry-afterand rotates. The log now includes the window, for example:Quota rejection (429) on "codex-1" (5h spent) — throttling 45s and switching account.parseCodexQuotaadds its missing@returnstypedef, andrateLimitHeadersgets theRecord<string, string>annotation now passed to it.Hold semantics
The hold remains
retry-after, clamped to[1s, 3600s], as in the Anthropic path. It does not use the window reset:retry-afteris upstream's requested quiet period, while the recorded utilisation over the switch threshold keeps selection off the account for the rest of the window. Tests pin a 45sretry-afterwith a reset six hours out.Verification
npm test— 1871 pass, 0 fail, including six newtest/codex-quota-rejection.test.jscases. They cover account-wide and named-family windows, non-cases, and end-to-endcreateProxyServerrequests to a local Codex-429 upstream.master: the two-account case leaves the accountactiveand paused, and the lone-account cases wait 45s and call the exhausted subscription twice.npm run lint,npm run typecheck, andnode scripts/typecheck-strict.mjsare clean; the ratchet reports 6 fewer strict diagnostics than base.Deliberate choice
A named family's weekly bucket is model-scoped, like Anthropic
7d_oi, which the existing branch treats as a Fable-only rejection without an account hold. This change treats it as general rejection. For now, upstream has refused this request on this account, the hold is bounded byretry-after, andupdateQuotahas recorded the bucket.codexSpentWindowslabels already distinguish the case if a narrower model-scoped rejection is needed later.🤖 Generated with Claude Code