You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug] API-key pool does not rotate on Anthropic "credit balance is too low" (400/billing_error) #6958
An Anthropic API-key provider (authMode: "key") with a multi-key apiKeyPool does not rotate to the next key when the current key's organization runs out of prepaid credit. Anthropic answers that case with:
HTTP 400
{"type":"error","error":{"type":"invalid_request_error","message":"Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits."}}
(billing_error has also been observed for the same condition.)
The response is returned to the client unchanged. Sibling keys in the pool that still have credit are never tried, and the exhausted key is not cooled down, so subsequent requests keep hitting it.
Expected
For key-auth providers with apiKeyPool.length >= 2: treat Anthropic's credit-balance refusal (400 invalid_request_error / billing_error with the "credit balance is too low" message, and 402) as a key-scoped failure — cool that key with a long cooldown and retry on the next key, like the existing 401 path.
In combo failover: classify the same refusal explicitly as hop with provider scope, ahead of the generic 400 / invalid_request_error handling (similar to how isResponseFormatCapabilityRefusal is ordered for invalid_request_error没有触发failover #4903), so a failover combo reliably moves to the next target.
Reusing the whole-message pattern already in isAccountRefusal keeps this narrow; ordinary invalid_request_errors would still stop.
Code references (2.83.0)
src/server/messages-native.ts, src/server/chat-native.ts, src/server/responses/adapter-dispatch.ts: key-pool rotation loops exist only for status === 401 and status === 429 (hasKeyPoolFailover).
src/providers/key-failover.ts: only rotateKeyOn401 / rotateKeyOn429 (and the rotateProviderTransportOn401|429 wrappers).
src/oauth/anthropic-account-refusal.ts → isAccountRefusal already recognises the exact credit-balance message for OAuth accounts and rotates the OAuth pool, but key pools don't use it.
src/combos/failover.ts: hops on 401/402/403/404/408/425/429/5xx; there is no message rule for the credit-balance 400. I have not traced the generic 400 path end to end, so please confirm whether a failover combo currently hops or stops on it.
Use case: a self-hosted hub where background jobs (a memory service) are pinned to anthropic-apikey/… to save subscription quota. With several funded keys in the pool, one org running dry should not take the route down.
Related:#6839 (metered combo targets; doesn't cover rotation within an API-key pool or the 400 classification), #6937 (request-local key round-robin; skips cooldowns but adds no billing cooldown trigger), #1572 (closed; terminal 429/400 re-evaluation).
Reproduction
Found by code reading on 2.83.0, not yet reproduced against a live empty-credit key.
Configure an anthropic key-auth provider (authMode: "key") with apiKeyPool of 2+ keys, where the active key's org has $0 prepaid credit and another key's org has credit.
POST /v1/messages (or /v1/chat/completions) with model: "<provider>/claude-haiku-5-5".
Observe: the 400 credit-balance error is returned; no key-401/key-429-style rotation happens; the next request goes to the same key again.
HTTP 400
{"type":"error","error":{"type":"invalid_request_error","message":"Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits."}}
Linked implementation: #6962 proposes bounded, pre-output Anthropic API-key billing recovery and explicit combo classification. It is still an open PR, not a released fix. Current trusted dev is a0191a1b6f; I have not used a live empty-credit key or made a provider call.
Review needs to keep the actual sent key/endpoint and current selection bound across asynchronous refusal inspection, then charge any replacement to the existing dispatch budget. Ordinary 400s, other adapters/OAuth, malformed or stalled bodies, cancellation, non-replayable output and no eligible replacement must remain terminal as appropriate. The PR's dedicated ownership/refusal helpers address those seams in its design, but passing author fixtures/current CI alone do not waive the required independent boundary review. Keeping the issue open until verified landing; #6937's pacing/selection work is not itself a billing-refusal fix.
Client or integration
Direct HTTP/API client
Area
Authentication and account pool
Summary
An Anthropic API-key provider (
authMode: "key") with a multi-keyapiKeyPooldoes not rotate to the next key when the current key's organization runs out of prepaid credit. Anthropic answers that case with:(
billing_errorhas also been observed for the same condition.)The response is returned to the client unchanged. Sibling keys in the pool that still have credit are never tried, and the exhausted key is not cooled down, so subsequent requests keep hitting it.
Expected
apiKeyPool.length >= 2: treat Anthropic's credit-balance refusal (400invalid_request_error/billing_errorwith the "credit balance is too low" message, and 402) as a key-scoped failure — cool that key with a long cooldown and retry on the next key, like the existing 401 path.hopwith provider scope, ahead of the generic 400 /invalid_request_errorhandling (similar to howisResponseFormatCapabilityRefusalis ordered for invalid_request_error没有触发failover #4903), so afailovercombo reliably moves to the next target.Reusing the whole-message pattern already in
isAccountRefusalkeeps this narrow; ordinaryinvalid_request_errors would still stop.Code references (2.83.0)
src/server/messages-native.ts,src/server/chat-native.ts,src/server/responses/adapter-dispatch.ts: key-pool rotation loops exist only forstatus === 401andstatus === 429(hasKeyPoolFailover).src/providers/key-failover.ts: onlyrotateKeyOn401/rotateKeyOn429(and therotateProviderTransportOn401|429wrappers).src/oauth/anthropic-account-refusal.ts→isAccountRefusalalready recognises the exact credit-balance message for OAuth accounts and rotates the OAuth pool, but key pools don't use it.src/combos/failover.ts: hops on 401/402/403/404/408/425/429/5xx; there is no message rule for the credit-balance 400. I have not traced the generic 400 path end to end, so please confirm whether a failover combo currently hops or stops on it.Use case: a self-hosted hub where background jobs (a memory service) are pinned to
anthropic-apikey/…to save subscription quota. With several funded keys in the pool, one org running dry should not take the route down.Related: #6839 (metered combo targets; doesn't cover rotation within an API-key pool or the 400 classification), #6937 (request-local key round-robin; skips cooldowns but adds no billing cooldown trigger), #1572 (closed; terminal 429/400 re-evaluation).
Reproduction
Found by code reading on 2.83.0, not yet reproduced against a live empty-credit key.
anthropickey-auth provider (authMode: "key") withapiKeyPoolof 2+ keys, where the active key's org has $0 prepaid credit and another key's org has credit.POST /v1/messages(or/v1/chat/completions) withmodel: "<provider>/claude-haiku-5-5".key-401/key-429-style rotation happens; the next request goes to the same key again.Version
2.83.0
Operating system
Ubuntu 24.04 (hub), macOS 26 (clients)
Provider and model
anthropic (authMode: key) / claude-haiku-5-5, claude-sonnet-5-5, claude-opus-5-5
Logs or error output
HTTP 400 {"type":"error","error":{"type":"invalid_request_error","message":"Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits."}}Screenshots and supporting files
No response
Redacted configuration
{ "providers": { "anthropic-apikey": { "adapter": "anthropic", "baseUrl": "https://api.anthropic.com", "authMode": "key", "apiKeyPool": ["<redacted-1>", "<redacted-2>", "<redacted-3>", "<redacted-4>"] } } }Checks