test(usage): stop asserting the overlay version against a moving oracle

[lidge-jun]

Summary

tests/api-usage.test.ts fails reproducibly outside CI's dedicated single-file job. It blocked the release preflight, and it reproduces on the already-shipped main (71c57ea64, v2.32.1) — it is not caused by anything in the current release.

(fail) usage route does not cache a summary whose overlay version changed mid-read
Expected: 10   Received: 9

Cause

Two independent problems, both in the test rather than the route.

1. The final assertion used a moving oracle. It compared the cache stamp against userCostOverlayVersion() read after the response:

expect(getUsageSummaryCacheEntry("30d:all")?.overlayVersion).toBe(userCostOverlayVersion());

The server runs a 5-second overlay reconciler that re-reads the config from disk and refreshes the registry. Instrumenting the test showed the counter moving 9 → 10 during the settled request, so the cache was correctly stamped with the version its summary was priced under, and the assertion then compared it against a newer number.

The contract is "the cache is stamped with the version its summary was priced under" — not "the counter never moves again". The test now captures the version before issuing the request and asserts the stamp is at least that.

2. Module-level overlay state leaked across files. preservedDiskOnlyProviders outlives a test file. Every other overlay suite calls resetPreservedDiskOnlyProvidersForTests(); this one did not, so a leftover provider could make a refresh byte-identical and the version would not bump at all.

Why CI never saw it

ci.yml runs api-usage in a dedicated job on a fresh runner with only this file, so neither the cross-file leak nor enough overlay history to make the timing matter ever existed there.

Verification

bun test tests/api-usage.test.ts    # 32 pass, 0 fail — 5 consecutive runs
bun x tsc --noEmit                   # exit 0

The relaxed assertion is not vacuous. I injected a deliberate defect into the route — stamping the cache with userCostOverlayVersion() - 1, exactly the stale-price bug the test guards — and the suite went to 3 fail. Reverting restored 32 pass. No production file is touched by this PR.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. Test-only change; no runtime behavior is modified.

Summary by CodeRabbit

• Bug Fixes

  • Improved API usage test reliability by preventing state from carrying between tests.
  • Updated overlay-version checks to handle timing changes during requests more accurately.

• Chores

  • Updated the package version to 2.32.1-preview.20260825.

[github-actions]

✅ Deterministic PR hygiene checks passed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a075a990-f5ff-4712-9e0c-a0dd84530d8a

📥 Commits

Reviewing files that changed from the base of the PR and between c0c9544 and 121ecbc.

📒 Files selected for processing (2)

• package.json
• tests/api-usage.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

The package version is updated to a preview release. API usage tests now reset preserved disk-only provider state and use a settled overlay-version baseline for the mid-read race assertion.

Changes

Package release metadata

Layer / File(s)	Summary
Preview package version package.json	The package version changes from 2.32.0 to 2.32.1-preview.20260825.

Overlay test stability

Layer / File(s)	Summary
Overlay state isolation and race assertion tests/api-usage.test.ts	The tests import resetPreservedDiskOnlyProvidersForTests, clear preserved provider state before and after each test, and compare the cached overlay version with a captured settled baseline in the race test.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 121ec

This PR adjusts a flaky usage test and related test setup without changing production behavior; the targeted suite and type check pass, so no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: ingwannu, luvs01

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately and concisely describes the main change: updating the usage test to avoid asserting the overlay version against a changing value.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/api-usage-overlay-version-oracle

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}