Activity

1. Ingwannu commented on Aug 8, 2026

   

   Ingwannu

   on Aug 8, 2026

   Owner

   More actions

   Thanks — I checked this against the current implementation and the request is valid as a scoped enhancement. apiKeyPool is presently an active-key pool with same-request rotation on 429 (src/providers/key-failover.ts / src/server/responses/core.ts); proactive distribution exists for OAuth account pools and combos, but not for API-key pools. So the N-provider workaround you described is real.\n\nThe safest shape is an opt-in provider-level strategy while preserving today’s failover behavior as the default. A future implementation should keep the selection cursor in runtime state, define restart/concurrency semantics, keep key identity secret-safe in usage attempts, and add same-request recovery without blindly treating every 403 as quota exhaustion (403 can also mean invalid scope or authorization). Quota rotation should use the existing normalized provider error classification or an explicit, tested set of status/body signals.\n\nI am leaving this open with that scope. Acceptance should cover deterministic round-robin/sticky behavior, concurrent requests, restart behavior, 429 plus validated quota-exhaustion responses, all-keys-unavailable behavior, and no key material in logs or management DTOs.
2. Wibias commented on Aug 8, 2026

   

   Wibias

   on Aug 8, 2026

   Contributor

   More actions

   Clarification on #1267: what already exists vs. what is still missing

   The current implementation already covers the core API-key failover behaviour.

   What we already have

   apiKeyPool is already an active-key pool for API-key providers:

   • One key is active at a time via provider.apiKey.
   • If the active key receives an HTTP 429, OpenCodex can rotate to the next available key in the pool.
   • The failed key is placed into cooldown.
   • The replacement key can be retried within the same request.
   • If multiple keys are exhausted/cooling down, the pool skips them and continues to the next available key.
   • The newly selected key becomes the active key.

   So the basic behaviour:

   Key A -> quota/rate-limit 429 -> Key B -> continue request
   

   already exists today.

   That also matches observed behaviour in practice: when one configured API key ran out of usable quota and the upstream returned 429, OpenCodex automatically switched to the next key.

   What #1267 would actually add

   The remaining work is narrower and consists of two separate capabilities.

   1. Broader quota-exhaustion failover

   Today, API-key pool rotation is primarily triggered by 429.

   Some providers report exhausted account quota differently, for example:

   • HTTP 402
   • insufficient_quota
   • provider-specific quota-exhaustion error codes
   • in some cases 403, although 403 must not be treated generically as quota exhaustion because it can also mean invalid permissions, scope, or authentication

   What is still needed is same-request key rotation for recognised quota-exhaustion responses beyond 429, using the existing normalised provider error classification or a small explicit and tested set of status/body signals.

   In other words:

   Key A
     -> recognised quota exhaustion
     -> mark Key A unavailable/cooling down
     -> retry with Key B in the same request
   

   This is an extension of the existing failover system rather than a new key-pool concept.

   2. Optional proactive round-robin distribution

   This part does not exist for API-key pools today.

   Current behaviour is reactive failover:

   Request 1 -> Key A
   Request 2 -> Key A
   Request 3 -> Key A
   ...
   Key A fails -> Key B
   

   The proposed opt-in round-robin strategy would instead allow:

   Request 1 -> Key A
   Request 2 -> Key B
   Request 3 -> Key C
   Request 4 -> Key A
   

   This should remain optional so the existing failover behaviour stays the default.

   A possible configuration shape is:

   {
     "apiKeyPool": [
       { "id": "key-a", "key": "..." },
       { "id": "key-b", "key": "..." }
     ],
     "apiKeyPoolStrategy": "round-robin"
   }

   with the existing behaviour treated as:

   {
     "apiKeyPoolStrategy": "failover"
   }

   Suggested scope

   I would therefore describe #1267 as a scoped enhancement, not as adding API-key rotation from scratch.

   Already implemented

   • API-key pools
   • active-key selection
   • automatic rotation on 429
   • cooldown handling
   • same-request retry after rotation
   • skipping unavailable keys
   • persistence of the newly active key

   Still needed

   • recognised quota-exhaustion failover beyond 429
   • optional provider-level round-robin strategy
   • deterministic cursor/sticky behaviour
   • concurrency-safe key selection
   • clearly defined restart semantics
   • all-keys-unavailable behaviour
   • secret-safe key identity in usage attempts/logging
   • no raw key material in logs or management DTOs

   Acceptance criteria for the remaining work

   1. Existing failover remains the default and does not change behaviour.
   2. 429 rotation continues to work exactly as today.
   3. Recognised quota-exhaustion responses can trigger same-request rotation.
   4. Generic 403 responses are not automatically treated as quota exhaustion.
   5. round-robin can be enabled explicitly per provider.
   6. Round-robin selection is deterministic under sequential requests.
   7. Concurrent requests do not corrupt or double-advance selection state.
   8. Restart behaviour is documented and tested.
   9. All-keys-unavailable behaviour is explicit and tested.
   10. Usage attempts may identify which pool slot was used, but must never expose raw API-key material.

   Bottom line

   The feature request is valid, but the distinction matters:

   We already have reactive API-key failover. What is missing is broader quota-error recognition and optional proactive round-robin distribution across keys within a single provider.

   That framing keeps the implementation focused and avoids rebuilding behaviour that is already present.
3. github-actions commented on Aug 8, 2026

   

   github-actionsbot

   on Aug 8, 2026 – with GitHub Actions

   Last edited by github-actions

   Contributor

   More actions

   _{Automated translation bookkeeping — detected language: English.}
4. Wibias

   changed the title [-]Feature: round-robin load balancing across API keys within a single provider (apiKeyPool)[/-] [+][Feature]: add proactive round-robin and broader quota failover to apiKeyPool[/+] on Aug 9, 2026
5. Wibias

   added

   enhancementNew feature or requestNew feature or request

   providerProvider adapters, OpenAI-compat presets, upstream API quirksProvider adapters, OpenAI-compat presets, upstream API quirks

   account-poolOAuth, credentials, Codex pool, quota, failover, plansOAuth, credentials, Codex pool, quota, failover, plans

   on Aug 9, 2026
6. lidge-jun commented on Aug 18, 2026

   

   lidge-jun

   on Aug 18, 2026

   Owner

   More actions

   Closing as consolidated into #695. Proactive round-robin and quota-aware failover for apiKeyPool is the same policy surface as the generic OAuth account-pool failover tracked there — selection policy, session affinity, and quota-aware rotation should ship as one contract rather than two parallel ones for key pools and account pools. If a key-pool-specific behavior falls out of scope when #695 is designed, it can fork back out into its own issue then.
7. lidge-jun

   closed this as not plannedon Aug 18, 2026