perf(ci): start the serialized integration collection first

[mforce]

What this is

Measurement for #839, plus one small scheduling change. Read the numbers before the title's verb — this does not make the suite faster. It stops the suite sometimes being much slower than it needs to be.

docs/plans/839-integration-wall-clock/measurement.md is the deliverable. The code is two files.

The measurement

Local, Ryzen 5 6600H / 12 logical CPUs / 18.8 GiB, cached images, Docker 29.8.0, .NET SDK 10.0.112.

Run	Config	Wall	Tests
main (18b45dc)	Debug	320s	1,801 passed
this branch	Debug	326s	1,815 passed
this branch	Release	341s	1,815 passed

Full-suite counts differ because the branch is rebased past #820 and #844.

What the ~5.5 minutes is made of (instrumented run, 331.5s wall):

• Container readiness: 156.6 distinct wall-clock seconds, overlapping test execution. 119 Postgres + 7 Redis + 1 Ryuk.
• The shared integration collection: 1,024 of 1,815 tests, one fixture, running serially end to end.
• Three one-shot-process classes dominate: SeedCommandTests 131.14s, ProcessRoleGuardTests 127.26s, OneShotVerbMinimalConfigTests 119.05s. Repeated app process startup, not SQL.

Two premises in the issue did not survive

1. Container reuse is already done. Lever 1 assumed per-class startup was the dominant cost. The shared collection already holds one ICollectionFixture<CluckworkWebApplicationFactory> covering 1,024 tests. The remaining 119 Postgres containers are the specialized factories that genuinely need their own database, migration state, or advisory-lock behaviour.
2. The remaining cost is subprocess, not SQL. The issue lists container reuse and parallelism. The measured cost is repeated one-shot process startup in three classes, which the issue never names.

The change

IntegrationCollectionOrderer puts the shared collection first and delegates every other collection to xUnit's default orderer. Fixture ownership, concurrency limits, serialization, and the race assertions are untouched — StealLossConnectionReleaseTests keeps its dedicated factory, one-slot pool, and timing assertions.

Why it is variance, not speedup: xUnit 2.9.3 documents its default collection order as unstable between runs. When the 95-class serialized collection draws a late slot, its ~300s of serial work becomes a tail nothing else can overlap. Pinning it first removes that schedule. It makes no test cheaper.

The first draft of this PR claimed 25.8% (434.51s → 322.46s). That compared against a single slow run. A second run of the unmodified scheduler came in at 327.06s, and the paired runs above put both versions inside each other's spread. The correction is in the measurement doc, not quietly dropped.

Also in here

• tools/test-timing/ — measure.py captures wall clock, TRX, and Docker events; summarize.py reports per-class cost, container readiness, and fixture phases. Opt-in timing in the base factory behind CLUCKWORK_TEST_TIMING=1.
• Two guard collisions found while building the tooling, both fixed in the tooling, neither guard weakened — worth knowing because they will bite the next person who adds a file under tools/:
  • the #508 tracked-file image-pin guard read a Python dict key as a live image reference (fixed by naming the variable reference);
  • the same guard's bare-postgres: literal detector fired on a timing phase label (fixed by naming the phase container).

Verification

• Solution build: 0 warnings, 0 errors.
• Integration suite on this branch: 1,815/1,815 passed in Debug and Release.
• Domain 491/491, Application 290/290.
• tools/test-timing/measure.py driven end to end on the branch as it now stands: exit 0, 1,815 passed, summary regenerated.
• CI is not measured. Local results do not establish a CI improvement or an optimal worker count on hosted runners.

Not done

No test deletion, no SQLite, no higher parallelism, no product change. Remaining levers, in the order the data now suggests: the three subprocess classes; splitting the shared collection after auditing its shared-state assumptions; CI-side measurement.

Closes #839

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 27265f8e-66b3-421b-8f9f-adc06bbb38e9

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}