Repository navigation
fix(store): skip completed usage totals backfill - #665
Merged
Merged
Conversation
Co-authored-by: gajesh <gajesh@eigenlabs.org>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Gajesh2007
had a problem deploying
to
benchmarks
August 22, 2026 07:46 — with
GitHub Actions
Failure
Co-authored-by: gajesh <gajesh@eigenlabs.org>
Co-authored-by: gajesh <gajesh@eigenlabs.org>
Gajesh2007
marked this pull request as ready for review
August 25, 2026 05:41
5 tasks done
Gajesh2007
added a commit
that referenced
this pull request
Aug 25, 2026
Resolve the post-#665 branch update by retaining only the serialized checkpoint DDL fix and its regression coverage. Co-authored-by: Cursor <cursoragent@cursor.com>
Gajesh2007
added a commit
that referenced
this pull request
Aug 31, 2026
…60822 Brings the test-suite refactor up to date with the 22 commits master gained since the last sync (#700-#790: Qwen3-VL and Qwen3.8 serving, first-content deadline admission, MTP-in-the-weights, serial-number privacy, MLX core 0.32.2, usage-checkpoint fixes). Every master-side test added or modified inside files this branch deletes, splits, or rewrites was re-homed into the refactor's successor suites and lanes rather than dropped, including: - PrefillDeadlineProductionConfigTests and the Qwen required-tool-choice tests into the slimmed EngineV2ProductionWiringTests - bridgeDeadlineRefusalReleasesRealStaging into SSDPrefixCacheReservationTests - excludesInferenceControls into WatchdogRecoveryTests - the rewritten #777 MTP config/beta suites into ConfigTests and BetaFeaturesTests (MTPConfigTests stays deleted) - vision/DeepStack/media additions into the split EngineV2Vision* and MediaIngest suites - the #704 writeDeferred provider-writer tests, first-content-budget wire test, #730 serial-privacy subtests, and #752 benchmark-control tests into their branch successors Master-added test files were audited against the lane architecture: the Qwen3.8 production canary trio moved to Live/ (with a #filePath depth fix), SchedulerPrefillDecision's real-weights suite split out to Probes/ behind its per-probe gates, and hermetic additions stay in the default lane. docs/developer/test.md updated to match. Production-code unions of note: registry.go keeps both the branch's PrivateOnly/catalog public-listing gates and master's per-model capability gating; provider_writer.go carries both determinism-seam families; LaunchAgent forwards the union of both env allowlists; postgres_model_registry.go extends nil-canonicalization to master's new required_provider_capabilities column. The branch-side prefill-keepalive tests were dropped with the feature master removed in #704. Two master-branch interaction defects fixed so the merged tree is green: the store contract harness now re-seeds the usage_totals counter row its truncate wipes (the #665 migration guard treats the bare marker as corruption), and the #730 log-report migration test restores the pooled connection's search_path before release. Submodules: mlx-swift and mlx-swift-lm test branches merged with master's 0.32.2-era pins; relocated suites verified present. Verified: go vet + full coordinator/testbed tests green; store contracts green twice consecutively against real Postgres 16; swift build --build-tests clean; default Swift lane 2,264 tests in 248 suites all passing.
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make the
usage_totalsbackfill genuinely one-time and lossless under concurrent writes. Existing deployments adopt the already-populated counter without scanningusage; fresh databases use a durable two-phase cutoff so the historical scan runs without holding a writer-blocking table lock.Linked issue
N/A
Before
flowchart LR subgraph Behavior B1[Coordinator startup] --> B2[Aggregate every usage row] B2 --> B3[Discover usage_totals conflict] B3 --> B4[Bind HTTP listener late] end subgraph Code C1[NewPostgres] --> C2[migrate statement loop] C2 --> C3[INSERT SELECT FROM usage ON CONFLICT] endAfter
flowchart LR subgraph Behavior B1[Coordinator startup] --> B2{Applied marker or existing counter?} B2 -->|Yes| B3[Skip historical usage entirely] B2 -->|No| B4[Briefly fence inserts and persist max usage ID] B4 --> B5[Resume writers against zero counter] B5 --> B6[Aggregate only historical IDs] B6 --> B7[Atomically add baseline and mark complete] B3 --> B8[Continue startup] B7 --> B8 end subgraph Code C1[NewPostgres] --> C2[migrateUsageTotals] C2 --> C3[prepareUsageTotalsMigration] C3 --> C4[usage_totals_backfill_state cutoff] C4 --> C5[finalizeUsageTotalsMigration] C5 --> C6[schema_migrations marker] endTest plan
DATABASE_URL=... go test ./store -run 'Test(MigrateNoUngatedUsageTotalsAggregation|UsageTotalsMigration)' -count=10DATABASE_URL=... go test ./store -count=1go test -count=1 ./...go vet ./storeComponents touched
Protocol / interface changes
Notes for reviewers
The old
ON CONFLICT DO NOTHINGwas not a fast path: PostgreSQL evaluated the aggregate source before checking the destination conflict. The new checkpoint captures a stable ID cutoff while inserts are fenced only for a primary-keyMAX(id), counter insert, and checkpoint insert. The full scan happens after writers resume; post-cutoff writes increment the live counter while finalization atomically adds onlyid <= cutoff. An interrupted process resumes from the checkpoint without double-counting.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.