Skip to content

fix(accounts): bring an API-key account back after a 401 instead of benching it for good - #481

Merged
MagicalTux merged 2 commits into
masterfrom
fix/apikey-401-cooldown
Sep 29, 2026
Merged

MagicalTux merged 2 commits into
masterfrom
fix/apikey-401-cooldown

Conversation

@MagicalTux

Copy link
Copy Markdown
Member

Fixes #473.

#439 stopped the 401 loop by taking an account out of rotation on its first 401. For an apikey account nothing ever brought it back: the revalidation probe skips error accounts and nothing else re-checks a key, so a gateway that answered a single 401 while its own upstream was down benched a working fallback for 22 hours.

An API-key 401 is now a cooldown rather than a permanent error:

  • the request still fails over as before (the client never sees the 401);
  • the account is held for 1 min, then 5 min, 15 min and 1 h for every further rejection with no success in between; 401s from requests already in flight during a hold are not counted, so one bad minute does not run straight to the ceiling;
  • any response below 400 clears the hold and the count; a 429 or 5xx says nothing about the key and leaves it;
  • the hold is respected by selection, by the revalidation probe and by the soonest-reset fallback, so a revoked key costs one failed-over request per cooldown and never loops;
  • a reload that brings a different key, or disable/re-enable, lifts it at once.

There is deliberately no escalation to permanent error: the 1 h ceiling is what a revoked key costs, while a fallback silently dead for good is what the report is about. OAuth accounts are unchanged (an account with no refresh token still goes to error).

getStatus exposes credentialRejectedUntil and unavailable: 'credential'; teamclaude status and the dashboard show the reason and the retry time, and the hold counts toward the synthetic 429's retry-after. Documented in docs/accounts.md.

Tests: test/apikey-401-cooldown.test.js (10 tests: first 401 holds and returns after the cooldown, escalation, success reset, in-flight 401s not counted, OAuth unchanged, not selected or probed while held, reload/re-enable lift it, status fields); two existing assertions that expected error for an API key now expect the hold.

🤖 Generated with Claude Code

MagicalTux and others added 2 commits September 29, 2026 10:05
…enching it for good (#473)

#439 stopped the 401 loop of #412 by putting an account in `error` on its
first 401. For an API key nothing ever took it back out: the revalidation
probe skips `error` and nothing else re-checks a key. One 401 from a gateway
whose own upstream was unreachable benched a working fallback account for 22
hours, until it was toggled by hand.

An API-key account is now held out of rotation for a cooldown and retried
after it. The hold lengthens while the key keeps being rejected with no
success in between: 1 min, 5 min, 15 min, then 1 h for every one after that.
Any non-error response from the account resets the sequence. It never
escalates to a permanent `error`, so a key that really is revoked costs one
failed-over request per hour and a gateway that recovers comes back by itself.

While held the account is unavailable to selection, to the revalidation probe
and to the soonest-reset fallback, so the loop stays fixed. 401s from requests
already in flight when the hold was armed do not escalate it. A reload that
brings a different key, or re-enabling the account, lifts the hold.

The hold is its own field beside the entitlement and routing cooldowns rather
than the 429 hold, which any non-429 response clears and the probe reopens
after a minute. Status carries `credentialRejectedUntil` and the reason
`credential`; the renderer and the dashboard explain it.

OAuth accounts are unchanged.

Closes #473

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@MagicalTux
MagicalTux merged commit 576ea54 into master Sep 29, 2026
5 checks passed
@MagicalTux MagicalTux mentioned this pull request Sep 29, 2026
MagicalTux added a commit that referenced this pull request Sep 29, 2026
Thirty-two commits since 1.1.21. Two change routing on an existing
config without an opt-in (#480, #481); the rest is opt-in, additive, or
display.

Behaviour changes
  #481 an API-key account's 401 is a cooldown, not a permanent `error`:
       1 min, then 5, 15 and 60 for every further rejection with no success
       in between; any 2xx/3xx resets it. The request still fails over and
       the client never sees the 401. OAuth accounts are unchanged
  #480 with session distribution on, requests carrying no session id stay
       within the top priority tier, so a fallback gateway no longer answers
       Claude Code's bootstrap and connector calls
  #470 a 200 whose SSE stream reports a provider failure before any output
       (`server_is_overloaded`, `response.failed`) fails over once, like a
       status-shaped failure would
  #465 a reload removes running accounts whose config entry is gone from
       disk, so `teamclaude remove` from another shell takes effect at once
  #460 `import` refuses an account whose token upstream has definitively
       rejected (401/403), even with `--name`; a 5xx or timeout still imports

Rename
  #483 the project is being renamed to TeamRouter (#72). This release accepts
       the new name everywhere the old one is read and changes nothing an
       install has on disk: `teamrouter` runs the same CLI, every
       `TEAMCLAUDE_*` variable is also read as `TEAMROUTER_*` (which wins when
       both are set), every `/teamclaude/…` control route also answers at
       `/teamrouter/…`, and `~/.config/teamrouter.json` is used when it exists

Features
  #441 per-account egress proxy (`accounts[].routing`: http, socks4/4a,
       socks5/5h) for refresh, probes and requests; `login --routing`,
       `teamclaude routing set/show/clear`, a connection check before it is
       relied on, and a short hold when the proxy is unreachable
  #427 `accounts[].allowExtraUsage: true` lets a paid extra-usage account
       serve once every account is past its threshold, instead of a 429
  #466 `accounts[].maxSpend`, a money cap judged against the month-to-date
       extra-usage spend upstream reports; the TUI shows what an account
       has billed
  #436 `autoRedeemResets` spends a free Codex rate-limit reset credit when
       the Codex pool runs dry (off by default)
  #482 `advisorEligibility: "strict" | "prefer"`; when the advisor model
       narrows selection to a subset of the fleet the log says so, and status
       carries the reading (`advisorNarrowing`)
  #478 `stripOverageHeaders` drops another org's per-organization billing
       headers from responses, for a pool spanning several orgs (#476)
  #471 `quota.unified5hSeenAt` / `unified7dSeenAt` in status: when upstream
       last stated each shared window
  #446 client and dimension usage for the last 5h and 24h in status and the
       dashboard, resumed across restarts
  #458 #459 #461 the dashboard sets the switch threshold, enables/disables and
       reprioritizes an account, and has a light theme remembered per browser
  #464 `l` in the TUI signs an account in `error` in again from the dashboard
  #457 status records which Codex limit meters each model
       (`quota.codexModelLimits`)
  #442 `quotaBarPercent` drops the percentage beside a TUI bar's countdown
  #451 `stripRequestFields` takes `content.<block type>` to drop content
       blocks a strict Anthropic-compatible upstream rejects
  #469 `proxy.mcp` schemas declare their item types, the write audit line
       records what happened, and the write queue has a depth (#447–#450)

Fixes
  #477 a refused WebSocket handshake whose headers all drop is relayed as a
       well-formed head instead of a blank line and body bytes
  #474 two members of one ChatGPT workspace are told apart by user id, so a
       second `login --codex` no longer replaces the first
  #469 a Codex Responses stream with no Content-Type is relayed as a stream
       and booked; thread repair on the global upstream; TUI settings and
       status gaps; a hint when a local login would have served
  #463 a Codex row with no session window draws one wide weekly bar

Tests
  #484 #485 #486 the suite asserts behaviour, not the scheduler: wall-clock
       upper bounds are gone, and subprocess tests spawn the server through
       `test-helpers/spawn-server.js`, which verifies the server it reached by
       `server.pid` (new in status) instead of trusting a port

Tooling
  #452 #453 #454 #455 docker workflow actions bumped
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An apikey account behind a gateway is taken out of rotation for good after a single 401, with no cooldown or retry

1 participant