Repository navigation
perf(test): serialize the five full-seed integration classes - #1003
Merged
Merged
Conversation
DemoDataSeeder's daily-entry loops are CPU-bound on EF change detection (docs/plans/863-integration-collection-split/03-demo-seed-profile.md). Four in-process test classes and SeedCommandTests's three subprocess seeds all run the full seeder, and under xUnit's default parallelism they overlap and contend for the same CPU-bound work, stretching a 16.4-second isolated seed to 120-190 seconds. Putting all five classes in one xUnit collection with no shared fixture serializes them against each other while every other class still runs in parallel. Three load-verified clean runs beat three load-verified clean baseline runs by a non-overlapping median 23.8 seconds (6.9%). Each class keeps its own factory and its own migrated Postgres database; the collection carries no ICollectionFixture, so this changes ordering only. See docs/plans/863-integration-collection-split/04-seed-serialization.md for the full measurement, including a four-class intermediate configuration and the host-contamination controls.
mforce
changed the title
One CI run of this PR's code passed with a VSTest time below all three post-#1002 baselines in 02-shared-postgres-spike.md, consistent with the local win but not a controlled comparison on its own.
The docs-only follow-up commit re-triggered CI, giving a second run of the new test code: 505.7 s VSTest, versus 361.9 s on the first run. The two runs bracket the prior baseline range rather than sitting below it, so CI evidence is inconclusive, not corroborating. Correcting the earlier overstatement.
The prior comment called this a spike with "the hypothesis under test," which reads as a leftover experiment once merged and invites deletion by a reader who assumes the contention it removes is already gone. Restate it as a measured result, and add the rule for which future classes belong here.
This was referenced Oct 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
03-demo-seed-profile.mdidentified the demo seed as CPU-bound on repeated EF change detection, and found that under full-suite contention this cost multiplies: four in-process test classes andSeedCommandTests's three subprocess seeds all run the full seeder, and left to xUnit's default parallelism they overlap and compete for the same CPU-bound work, stretching a 16.4-second isolated seed to 120-190 seconds.Scope
03-demo-seed-profile.md: the original diagnosis (phase, SQL, audit, boot, CPU, and suite-load measurements).04-seed-serialization.md: tests the hypothesis that serializing the five full-seed classes against each other, while every other class still runs in parallel, removes that self-contention.DemoSeedCollection: a bare[CollectionDefinition]with no shared fixture, applied toDemoSeedTests,DemoSeedBootTests,DemoSeedDisabledOwnerTests,DemoSeedAttributionTests, andSeedCommandTests. Each class keeps its own factory and its own migrated Postgres database; this changes execution order only.Verification
SeedCommandTests's smaller additional contribution.02-shared-postgres-spike.md(385.7-483.3 s), so CI evidence is inconclusive on its own; see04-seed-serialization.md's T4 for detail.Refs #863, a broader issue about the integration suite's wall-clock floor. This PR records the profile, tests one hypothesis from it, and ships the change that hypothesis supports; the parent issue stays open for the rest of its scope.