Repository navigation
fix(capacity): count uncalibrated Codex plans instead of excluding them - #3198
Conversation
A Business seat upgraded to a Premium Seat started reporting a plan string the capacity weight map does not list, so the account was dropped from its own capacity estimate and the dashboard warned about incomplete coverage. The map knows five plan names. The bundled upstream snapshot alone carries 21, and CodexAccount.plan is an unrestricted string, so sixteen real plans were already being excluded - Premium Seat is only the one a user noticed. src/codex/quota.ts reached this same conclusion about this same field: an allowlist is a list of the plans someone remembered. Weight an uncalibrated plan at the baseline seat instead. Under-counting a large seat is a visibly conservative estimate; excluding it silently overstates the coverage an operator is reading. The calibrated weights are unchanged, and paused, needs-reauth, missing-quota and stale-quota accounts are still excluded. The dashboard string is split to match: unknown plans are no longer described as excluded, because they are not. Closes #3155
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
✅ Deterministic PR hygiene checks passed. |
✅ READY
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 541cdae89e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| {aggregation && aggregation.unknownPlanAccounts > 0 && ( | ||
| <div className="pws-capacity-incomplete"> | ||
| {t("pws.capacity.uncalibratedPlan", { count: aggregation.unknownPlanAccounts })} |
There was a problem hiding this comment.
Do not label excluded unknown-plan accounts as counted
When an uncalibrated account is paused, needs reauthentication, lacks quota, or has stale quota, unknownPlanAccounts is still incremented before the source excludes it from every window; nevertheless, this component tells the operator that every account in that count "is counted at the baseline seat weight." The coverage-only fixture demonstrates the contradiction with includedAccounts: 0 and unknownPlanAccounts: 1. Track included uncalibrated accounts separately or use wording that does not claim all of them contributed.
AGENTS.md reference: gui/AGENTS.md:L9-L10
Useful? React with 👍 / 👎.
| function configuredWeight(plan: unknown): number { | ||
| const normalized = codexPlanKey(plan); | ||
| return normalized && Object.hasOwn(CODEX_CONFIGURED_CAPACITY_WEIGHTS, normalized) | ||
| ? CODEX_CONFIGURED_CAPACITY_WEIGHTS[normalized as keyof typeof CODEX_CONFIGURED_CAPACITY_WEIGHTS] | ||
| : undefined; | ||
| : CODEX_DEFAULT_CAPACITY_WEIGHT; |
There was a problem hiding this comment.
Update the documented unknown-plan behavior
This fallback reverses the published contract in docs-site/src/content/docs/guides/providers.md:35-37, which still says an unknown plan cannot safely contribute and is a reason for incomplete coverage; after this change it contributes at weight 1 and can leave aggregation.incomplete false. Update that section and its translated copies so operators do not interpret the new estimate using the obsolete exclusion semantics.
AGENTS.md reference: AGENTS.md:L343-L344
Useful? React with 👍 / 👎.
#3200) #3198 deliberately stopped excluding uncalibrated Codex plans from the pool capacity estimate: a plan name the weight map does not list now counts at the baseline seat weight, because exclusion silently overstated the coverage an operator was reading. It updated provider-capacity.test.ts and the GUI tests, but tests/provider-quota.test.ts kept one test asserting the old exclusion shape, so it has failed deterministically on every clean dev checkout since — expecting the malformed-plan account to be dropped (weekly 11, excluded 1, incomplete) where the new contract blends it at baseline (weekly 44, both included, complete coverage). A malformed non-string plan such as `{ tier: "pro" }` is not a separate axis: codexPlanKey normalizes it to undefined exactly like an unlisted name (the tolerance 095a9a5 established), and #3198 routes every undefined key to the baseline weight. Re-splitting malformed from unknown here would reintroduce the distinction #3198 just collapsed, so the test moves to the new contract instead; `unknownPlanAccounts` still records the seat, which is the observable the test's cache half already pins. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The review's headline finding was that the new comment stated a wrong mechanism at a wrong layer: on this test's integration path the malformed plan never reaches codexPlanKey — poolAccountDto strips it via codexPlanValue, so the aggregation sees an ABSENT plan — and an unlisted string name normalizes to a defined key that fails the weight-map lookup, not to undefined. The comment now states both routes correctly, cites lidge-jun#3155 (PR lidge-jun#3198) the way the ten sibling references do, and replaces its fourth copy of the policy rationale with a pointer to CODEX_DEFAULT_CAPACITY_WEIGHT's own doc. The assertions grew where the review showed regressions could ship green: - weekly: { includedAccounts, excludedAccounts, incomplete } — the per-window flags are what the dashboard renders per bar, and nothing pinned them. - JSON.stringify(refreshed) must not contain "tier" — with exclusion gone, nothing else proved the malformed value cannot escape into the public shape. - the sibling's anti-leak regex over the aggregation, closing that asymmetry. - the cached half now asserts the identity contract itself (toBe on the aggregation object) instead of re-reading a field of the object checked one line up, with a comment saying why content is only covered once. - the load-bearing mock is documented: a fetched plan_type outranks the persisted plan, so it must stay non-string for every value below. The test is renamed to what it now pins — counted at baseline, kept out of the public shape — since "through cache and aggregation" was false on both nouns. gui/tests/provider-capacity-shell.test.tsx gains the one missing pairing: incomplete:false with unknownPlanAccounts>0, the lidge-jun#3155 reporter's own fully-included pool. Mutation-checked: gating the uncalibrated notice under the incomplete branch fails only the new test; every prior fixture passes, which is exactly the silent-hide the review predicted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
리뷰 · 우선순위 58 / 80이 PR은 Rate Limits 용량 추정에서 Codex 플랜 이름이 가중치 맵에 없으면 계정을 통째로 빼 버리던 버그를 고칩니다. 증상은 #3155로 보고된 Business → Premium Seat 업그레이드였습니다. 대시보드에 Incomplete coverage / unknown plan 경고가 뜨고, 그 계정이 자기 풀 추정에서 사라졌습니다. 원인은 이 변경은 맵에 중요한 안전 경계도 테스트로 고정했습니다. paused / needs-reauth / missing-quota / stale-quota는 플랜이 미보정이어도 계속 제외되고, 보정된 Pro=20 가중치는 그대로이며, 현재 라인 234 - 메인테이너의 판단이 필요한 지점
너의 추천 이 댓글은 grok-bot이 작성했습니다 |
The review's headline finding was that the new comment stated a wrong mechanism at a wrong layer: on this test's integration path the malformed plan never reaches codexPlanKey — poolAccountDto strips it via codexPlanValue, so the aggregation sees an ABSENT plan — and an unlisted string name normalizes to a defined key that fails the weight-map lookup, not to undefined. The comment now states both routes correctly, cites lidge-jun#3155 (PR lidge-jun#3198) the way the ten sibling references do, and replaces its fourth copy of the policy rationale with a pointer to CODEX_DEFAULT_CAPACITY_WEIGHT's own doc. The assertions grew where the review showed regressions could ship green: - weekly: { includedAccounts, excludedAccounts, incomplete } — the per-window flags are what the dashboard renders per bar, and nothing pinned them. - JSON.stringify(refreshed) must not contain "tier" — with exclusion gone, nothing else proved the malformed value cannot escape into the public shape. - the sibling's anti-leak regex over the aggregation, closing that asymmetry. - the cached half now asserts the identity contract itself (toBe on the aggregation object) instead of re-reading a field of the object checked one line up, with a comment saying why content is only covered once. - the load-bearing mock is documented: a fetched plan_type outranks the persisted plan, so it must stay non-string for every value below. The test is renamed to what it now pins — counted at baseline, kept out of the public shape — since "through cache and aggregation" was false on both nouns. gui/tests/provider-capacity-shell.test.tsx gains the one missing pairing: incomplete:false with unknownPlanAccounts>0, the lidge-jun#3155 reporter's own fully-included pool. Mutation-checked: gating the uncalibrated notice under the incomplete branch fails only the new test; every prior fixture passes, which is exactly the silent-hide the review predicted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…comment Carried from #3327. Two gaps the original #3198 tests left open: the uncalibrated-plan notice was never pinned independently of the incomplete-coverage gate, so folding it under that branch would have passed every existing fixture while silently hiding it; and the malformed-plan path was described as if it reached aggregation the same way an unlisted plan name does, when poolAccountDto strips it earlier via codexPlanValue and the aggregate sees an absent plan instead. One assertion is narrowed from the original. over the whole envelope also matches any unrelated field whose name contains that substring -- serviceTier, tierOutcome -- so it would fail on changes with nothing to do with plan leakage. Scoped to the report rows and to the quoted key, which is where the malformed value could actually surface. Carried rather than merged in place: #3327 is a fork PR, and Cross-platform CI never ran on its head. enforce-target was also red there because touching gui/tests/ trips the UI-screenshot gate on a test-only change. Co-authored-by: olddonkey <olddonkeyblog@gmail.com>
…redit redeems a stable identity (#3474) Reimplements #3332 by @full999, which could not be cherry-picked. One line differs from the original and it matters: the PR mapped the vendor table's maxTokens -- an OUTPUT ceiling -- onto maxInputTokens, and because aggregation takes Math.min over member input ceilings, a single Claude member would have dragged a 1M combo down to 128k and the auto-compaction budget from 900k to 128k with it. Also wires the reset-credit operation ledger, which was complete and had zero production callers while the consume endpoint minted a fresh UUID per call. An optional operationId now becomes the redeem_request_id. Opening fails closed, because falling back to a random id is the double-spend the identity prevents; settling fails open, because by then the credit is gone and reporting failure would invite a manual retry. Omitting operationId keeps today's behavior unchanged. Carried from #3327 by @olddonkey as well: two coverage holes from #3198, with one over-broad assertion narrowed. Also lands #3251 by @abhisheksharma2411: the GUI now consumes tierOutcome, which the backend already shipped and nothing displayed. Co-authored-by: full999 <daiki.furutani@walker-s.co.jp> Co-authored-by: olddonkey <olddonkeyblog@gmail.com> Co-authored-by: Abhishek Sharma <abhicse24@gmail.com>
…em (lidge-jun#3198) A Business seat upgraded to a Premium Seat started reporting a plan string the capacity weight map does not list, so the account was dropped from its own capacity estimate and the dashboard warned about incomplete coverage. The map knows five plan names. The bundled upstream snapshot alone carries 21, and CodexAccount.plan is an unrestricted string, so sixteen real plans were already being excluded - Premium Seat is only the one a user noticed. src/codex/quota.ts reached this same conclusion about this same field: an allowlist is a list of the plans someone remembered. Weight an uncalibrated plan at the baseline seat instead. Under-counting a large seat is a visibly conservative estimate; excluding it silently overstates the coverage an operator is reading. The calibrated weights are unchanged, and paused, needs-reauth, missing-quota and stale-quota accounts are still excluded. The dashboard string is split to match: unknown plans are no longer described as excluded, because they are not. Closes lidge-jun#3155 Co-authored-by: jun <jun@lidge.dev>
…contract (lidge-jun#3200) lidge-jun#3198 deliberately stopped excluding uncalibrated Codex plans from the pool capacity estimate: a plan name the weight map does not list now counts at the baseline seat weight, because exclusion silently overstated the coverage an operator was reading. It updated provider-capacity.test.ts and the GUI tests, but tests/provider-quota.test.ts kept one test asserting the old exclusion shape, so it has failed deterministically on every clean dev checkout since — expecting the malformed-plan account to be dropped (weekly 11, excluded 1, incomplete) where the new contract blends it at baseline (weekly 44, both included, complete coverage). A malformed non-string plan such as `{ tier: "pro" }` is not a separate axis: codexPlanKey normalizes it to undefined exactly like an unlisted name (the tolerance 095a9a5 established), and lidge-jun#3198 routes every undefined key to the baseline weight. Re-splitting malformed from unknown here would reintroduce the distinction lidge-jun#3198 just collapsed, so the test moves to the new contract instead; `unknownPlanAccounts` still records the seat, which is the observable the test's cache half already pins. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…em (lidge-jun#3198) A Business seat upgraded to a Premium Seat started reporting a plan string the capacity weight map does not list, so the account was dropped from its own capacity estimate and the dashboard warned about incomplete coverage. The map knows five plan names. The bundled upstream snapshot alone carries 21, and CodexAccount.plan is an unrestricted string, so sixteen real plans were already being excluded - Premium Seat is only the one a user noticed. src/codex/quota.ts reached this same conclusion about this same field: an allowlist is a list of the plans someone remembered. Weight an uncalibrated plan at the baseline seat instead. Under-counting a large seat is a visibly conservative estimate; excluding it silently overstates the coverage an operator is reading. The calibrated weights are unchanged, and paused, needs-reauth, missing-quota and stale-quota accounts are still excluded. The dashboard string is split to match: unknown plans are no longer described as excluded, because they are not. Closes lidge-jun#3155 Co-authored-by: jun <jun@lidge.dev>
…contract (lidge-jun#3200) lidge-jun#3198 deliberately stopped excluding uncalibrated Codex plans from the pool capacity estimate: a plan name the weight map does not list now counts at the baseline seat weight, because exclusion silently overstated the coverage an operator was reading. It updated provider-capacity.test.ts and the GUI tests, but tests/provider-quota.test.ts kept one test asserting the old exclusion shape, so it has failed deterministically on every clean dev checkout since — expecting the malformed-plan account to be dropped (weekly 11, excluded 1, incomplete) where the new contract blends it at baseline (weekly 44, both included, complete coverage). A malformed non-string plan such as `{ tier: "pro" }` is not a separate axis: codexPlanKey normalizes it to undefined exactly like an unlisted name (the tolerance 9accc38 established), and lidge-jun#3198 routes every undefined key to the baseline weight. Re-splitting malformed from unknown here would reintroduce the distinction lidge-jun#3198 just collapsed, so the test moves to the new contract instead; `unknownPlanAccounts` still records the seat, which is the observable the test's cache half already pins. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…redit redeems a stable identity (lidge-jun#3474) Reimplements lidge-jun#3332 by @full999, which could not be cherry-picked. One line differs from the original and it matters: the PR mapped the vendor table's maxTokens -- an OUTPUT ceiling -- onto maxInputTokens, and because aggregation takes Math.min over member input ceilings, a single Claude member would have dragged a 1M combo down to 128k and the auto-compaction budget from 900k to 128k with it. Also wires the reset-credit operation ledger, which was complete and had zero production callers while the consume endpoint minted a fresh UUID per call. An optional operationId now becomes the redeem_request_id. Opening fails closed, because falling back to a random id is the double-spend the identity prevents; settling fails open, because by then the credit is gone and reporting failure would invite a manual retry. Omitting operationId keeps today's behavior unchanged. Carried from lidge-jun#3327 by @olddonkey as well: two coverage holes from lidge-jun#3198, with one over-broad assertion narrowed. Also lands lidge-jun#3251 by @abhisheksharma2411: the GUI now consumes tierOutcome, which the backend already shipped and nothing displayed. Co-authored-by: full999 <daiki.furutani@walker-s.co.jp> Co-authored-by: olddonkey <olddonkeyblog@gmail.com> Co-authored-by: Abhishek Sharma <abhicse24@gmail.com>
Summary
A Business account upgraded to a Premium Seat (introduced 2026-08-25) started showing "Incomplete coverage: 1 account(s) excluded, including 1 unknown plan(s)" under Rate Limits, and disappeared from its own capacity estimate.
configuredWeightreturnedundefinedfor any plan outside a hardcoded five-entry map, and an account with no weight was skipped entirely.The map is far behind the plan vocabulary. The bundled upstream snapshot alone carries 21 distinct plan strings —
business,edu,edu_plus,edu_pro,education,enterprise,enterprise_cbp_automation,enterprise_cbp_usage_based,finserv,free,free_workspace,go,hc,k12,plus,pro,prolite,quorum,sci,self_serve_business_usage_based,team— against the five with weights. So sixteen real plans were already being excluded, includinggo. Premium Seat is just the one that made a user look.That is why this does not add
premiumto the map.src/codex/quota.tsreached the same conclusion about the same field, one file over:The change
An uncalibrated plan is now weighted at the baseline seat (1 — the value
plus,teamandbusinessalready carry) instead of being dropped. Under-counting a large seat is a visibly conservative estimate; excluding it silently overstates coverage the operator is reading as complete.unknownPlanAccountsstill counts these, so the uncertainty stays on screen — it just no longer gates inclusion.Blast radius is bounded:
aggregateCodexPoolCapacityis documented display-only, "It never participates in account selection or routing". This changes a dashboard estimate, not routing.The dashboard string is split to match. It claimed unknown plans were "excluded", which is now false. Exclusion (paused / needs-reauth / missing quota / stale quota) and calibration are separate sentences, across all 9 locales.
Closes #3155.
Verification
Exact head
541cdae89:bun test ./tests/provider-capacity.test.ts— 18 pass, 0 fail.cd gui && bun test tests/provider-capacity.test.ts tests/provider-capacity-credits.test.tsx tests/provider-capacity-shell.test.tsx tests/i18n-locales.test.ts— 31 pass, 0 fail.bun x tsc --noEmit— clean for touched files.Red-green on both halves:
Five existing tests changed, each because the behavior they pinned is the defect. Two are worth calling out:
prototype and unknown plan names never become configured weights— theObject.hasOwnguard is untouched, soconstructorstill cannot resolve toObject.prototype.constructor; it now resolves to the numeric baseline. Verified the aggregate stays finite.all exclusion reasons retain a coverage envelope— itsfuture-planrow is nowpaused, so every row is still excluded for a reason that survives this change.A new test pins the risk directly: the default must never resurrect an account excluded for a real reason. Paused, needs-reauth, missing-quota and stale-quota rows stay excluded regardless of plan.
UI change
No layout change; the wording is what changed. Rendered output from
provider-capacity-shell.test.tsx, which mounts the real component:The strings are the shipped locale values rendered with the dashboard stylesheet. The same two lines are asserted in
provider-capacity-shell.test.tsx, which mounts the real component:One honesty note: this is not a capture of the reporter's own dashboard — reproducing that needs a real Business Premium Seat, which I do not have. It is the notice region rendered from the changed strings, next to the string it replaces.
Checklist
devi18n-localesparity green