fix: treat "Resource has been exhausted" as temporary capacity#194
Conversation
Problema: Quando o servidor retorna 429 com mensagem "Resource has been exhausted" sem quotaResetTime, o plugin marca todas as contas como rate-limited permanentemente no antigravity-accounts.json, mesmo sendo um problema de capacidade temporária do backend. Solução: - Distinguir entre quota esgotada (persiste) e capacity exhausted (global) - Capacity exhausted usa cooldown em memória por família/modelo (não persiste) - Fluxo de quota real permanece inalterado (429 com quotaResetTime continua) Arquivos: - src/plugin.ts: adiciona capacidade global e helpers
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughRefactors capacity exhaustion from per-account to a global, keyed (family/model) backoff. Adds constants and maps for global cooldown state plus helpers ( Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryDifferentiates between temporary backend capacity exhaustion and account-level quota exhaustion to prevent permanent account blocking. Key Changes:
Issue Resolution: Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant Client
participant Plugin as plugin.ts
participant AccountMgr as AccountManager
participant Server as Antigravity Server
Client->>Plugin: Request with model
loop Account Selection Loop
Plugin->>Plugin: Check global capacity cooldown
alt Capacity cooldown active
Plugin-->>Client: Show toast "Server at capacity"
Plugin->>Plugin: Sleep for cooldown duration
else No cooldown
Plugin->>AccountMgr: Get next available account
AccountMgr-->>Plugin: Return account
Plugin->>Server: Send request
alt HTTP 429 with "Resource has been exhausted" (no quotaResetTime)
Server-->>Plugin: 429 + message
Plugin->>Plugin: Detect SERVICE_CAPACITY_EXHAUSTED
Plugin->>Plugin: recordAndGetCapacityBackoff()
Plugin->>Plugin: markCapacityCooldown(family, model)
Plugin-->>Client: Show toast with backoff time
Plugin->>Plugin: Sleep, then break to outer loop
Note over Plugin: Does NOT persist to disk<br/>Does NOT mark account as rate-limited
else HTTP 429 with quotaResetTime (real quota)
Server-->>Plugin: 429 + quotaResetTime
Plugin->>Plugin: Detect QUOTA_EXHAUSTED/MODEL_CAPACITY_EXHAUSTED
Plugin->>AccountMgr: markRateLimitedWithReason()
Note over Plugin,AccountMgr: Persists to disk
Plugin->>Plugin: Switch to next account
else Success (200)
Server-->>Plugin: Response
Plugin-->>Client: Return response
end
end
end
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
src/plugin.ts
Outdated
| const { delayMs: capacityBackoffMs, attempt } = getCapacityBackoffForKey(family, model); | ||
| markCapacityCooldown(family, model, capacityBackoffMs); | ||
|
|
||
| const backoffFormatted = formatWaitTime(capacityBackoffMs); | ||
| pushDebug(`capacity exhausted on account ${account.index}, backoff=${capacityBackoffMs}ms (failure #${failures + 1})`); | ||
| pushDebug(`capacity exhausted (global) family=${family} model=${model ?? ""} backoff=${capacityBackoffMs}ms (attempt ${attempt})`); | ||
|
|
||
| await showToast( | ||
| `⏳ Server at capacity. Waiting ${backoffFormatted}... (attempt ${failures + 1})`, | ||
| "warning", | ||
| ); | ||
| if (!quietMode) { | ||
| await showToast( | ||
| `⏳ Server at capacity. Waiting ${backoffFormatted}... (attempt ${attempt})`, | ||
| "warning", | ||
| ); | ||
| } | ||
| await sleep(capacityBackoffMs, abortSignal); | ||
| continue; |
There was a problem hiding this comment.
logic: After handling capacity exhaustion with a cooldown, the code continues the endpoint loop (line 1304). However, this doesn't reset the account selection - it retries with the same account and endpoint. Consider whether the outer while(true) loop should be continued instead to trigger fresh account selection after the cooldown expires.
Currently: cooldown → retry same endpoint with same account
Expected: cooldown → restart from account selection (line 926)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/plugin.ts
Line: 1291:1304
Comment:
**logic:** After handling capacity exhaustion with a cooldown, the code continues the endpoint loop (line 1304). However, this doesn't reset the account selection - it retries with the same account and endpoint. Consider whether the outer `while(true)` loop should be continued instead to trigger fresh account selection after the cooldown expires.
Currently: cooldown → retry same endpoint with same account
Expected: cooldown → restart from account selection (line 926)
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.
src/plugin.ts
Outdated
| function getCapacityBackoffForKey(family: ModelFamily, model?: string | null): { delayMs: number; attempt: number } { | ||
| const key = getCapacityKey(family, model); | ||
| const now = Date.now(); | ||
| const previous = capacityFailureStateByKey.get(key); | ||
| const failures = previous && (now - previous.lastAt < CAPACITY_STATE_RESET_MS) | ||
| ? previous.consecutiveFailures | ||
| : 0; | ||
| const delayMs = getCapacityBackoffDelay(failures); | ||
| capacityFailureStateByKey.set(key, { consecutiveFailures: failures + 1, lastAt: now }); | ||
| return { delayMs, attempt: failures + 1 }; | ||
| } |
There was a problem hiding this comment.
style: The function getCapacityBackoffForKey updates state (line 553) even when just called to get the backoff delay. This creates a side effect during what appears to be a query operation.
Consider separating state mutation from state reading, or rename to clarify the side effect (e.g., incrementCapacityBackoffForKey).
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/plugin.ts
Line: 545:555
Comment:
**style:** The function `getCapacityBackoffForKey` updates state (line 553) even when just called to get the backoff delay. This creates a side effect during what appears to be a query operation.
Consider separating state mutation from state reading, or rename to clarify the side effect (e.g., `incrementCapacityBackoffForKey`).
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.
IsraelAraujo70
changed the title
- Logic: capacity exhausted agora força nova seleção de conta (break ao invés de continue) - Style: separar função de leitura de mutação (recordAndGetCapacityBackoff + calculateCapacityBackoffDelay) - Docs: adicionar JSDoc para funções de capacity (5 funções)
Greptile found no issues!From now on, if a review finishes and we haven't found any issues, we will not post anything, but you can confirm that we reviewed your changes in the status check section. This feature can be toggled off in your Code Review Settings by deselecting "Create a status check for each PR". |
|
I just change merge target to dev, please fix conflict |
- Added SERVICE_CAPACITY_EXHAUSTED rate limit reason type - Updated parseRateLimitReason to detect "resource has been exhausted" without quotaResetTime as SERVICE_CAPACITY_EXHAUSTED - Added progressive backoff tiers for service capacity exhaustion (5s, 10s, 20s, 30s, 60s) - Integrated global capacity tracking with upstream/dev's rate limit system - For SERVICE_CAPACITY_EXHAUSTED: uses global in-memory cooldown (not persisted) - For other rate limits: uses upstream/dev's account-level tracking
2 participants
Summary
Fixes an issue where
Resource has been exhausted(HTTP 429 withoutquotaResetTime) permanently marks all accounts as rate-limited on disk, even though it is a temporary backend capacity limitation.Problem
When the Antigravity server returns HTTP 429 with the generic message "Resource has been exhausted" without
quotaResetTime, the plugin treats it as account-level quota exhaustion and persists this state inantigravity-accounts.json. This causes:Log example:
Solution
Differentiate between account-level quota exhaustion (persisted to disk) and global capacity exhaustion (in-memory cooldown, not persisted):
Service capacity (not persisted)
Resource has been exhaustedwithoutquotaResetTimeas capacity-relatedcapacityCooldownByKey)Real quota (unchanged)
quotaResetTimecontinues to be persisted on diskQUOTA_EXHAUSTEDorMODEL_CAPACITY_EXHAUSTEDkeeps the existing flowModified files
src/plugin.ts: 58 insertions, 10 deletionscapacityCooldownByKeyandcapacityFailureStateByKey(global, in-memory)getCapacityKey,getCapacityCooldownRemainingMs,markCapacityCooldown,getCapacityBackoffForKeyquotaResetTime)Tests
bun run test: 558 passed)