Repository navigation
Run tests with nextest so binaries execute in parallel instead of one after another - #883
Closed
WaylandYang wants to merge 1 commit into
Closed
WaylandYang wants to merge 1 commit into
WaylandYang wants to merge 1 commit into
Conversation
… after another Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: WaylandYang <145302500+WaylandYang@users.noreply.github.com>
Contributor
Author
|
Closing without merging: on the 4-vCPU runner the nextest run phase measured 157s versus 86s for cargo test (run 35842332445). Per-test process startup dominates there (server tests average 0.73s each under nextest), and the serialized store group holds one of the four slots for 80s. The local 2× win was a 10-core artefact. The real path to a faster test job is fixing the three store-test isolation issues so the store suite can run in one binary in parallel; that is being done separately. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #879.
Why
cargo test --workspaceruns the 140 test binaries one after another, each parallel only within itself. Most binaries hold one to three tests, so thetestjob spends most of its 86 s run phase waiting for one binary to finish before starting the next. cargo-nextest runs tests across binaries in parallel, one process per test. Locally the run phase drops from 136 s to ~61 s.What
.config/nextest.tomlmakes explicit the two isolation assumptionscargo testprovided implicitly:utopia-storeintegration tests never run concurrently with each other (amax-threads = 1test group). Several assert whole-database state, e.g.a_vector_index_is_built_by_a_jobcounts queued builds, and the timeline tests take row locks that deadlock when other store tests write at the same time. Under plain nextest these failed deterministically or sporadically; serialized, 8/8 runs were clean.a_queued_job_wakes_an_idle_workerruns exclusively (threads-required = "num-cpus"). Its three tests share theutopia_jobsNOTIFY channel and serialize through a process-local staticMutex, which does nothing once each test is its own process; other tests enqueueing jobs would also wake it. Under nextest without the rule 2 of 3 tests failed every time; with it 3/3 passed in every run.mcp_tests::Fixture::cleanretriesremove_dir_allfor up to a second. Dropping the tantivyIndexWriterdoes not wait for its merge threads, which keep writing segment files intosearch/; deleting the directory immediately raced them about one run in five under nextest (Directory not empty). The other fixtures already ignore the error withlet _ =; this one keeps reporting a directory that genuinely cannot be removed.testjob: installs the pinned nextest 0.9.146 prebuilt binary from get.nexte.st (seconds;cargo installwould compile for minutes), skipped when~/.cargo/binfrom thedevcache already has it.Testrunscargo nextest run --workspacefollowed bycargo test --workspace --doc, since nextest does not run doctests (there are none today; the step is there so future ones are not silently skipped).CONTRIBUTING.zh-CN.md: one paragraph saying CI uses nextest and where the rules live;cargo testlocally still works.Workflow-level
permissions: contents: readuntouched.Verification (local, docker compose Postgres,
UTOPIA_TEST_REQUIRE_DB=1)The 5 skipped are the opt-in MySQL, Trino and live-network RSS tests, skipped under
cargo testas well.Expected on CI: the
testjob's run phase from ~86 s to ~40 s. Compilation is unchanged; that is the remaining cost and it needs the store integration tests restructured (separate follow-up, see #879 discussion).🤖 Generated with Claude Code