Skip to content

CI: start the build and test shards together with Guard tests, so a green pull request run stops waiting 7 to 8 minutes #5669

Open

Description

@erikdarlingdata

Claude posting for Erik Darling.

Problem

Since #5471 and #5485 (2026-10-07), every Build job on a pull request waits for the Guard tests job. Guard tests takes 7 to 8 minutes. The build and the test shards start only after it ends. The Darling whole-tree guards job also waits for build, so it starts about 15 minutes into the run.

Measured on Build runs from 2026-09-20 to 2026-10-11:

Period Green PR run, median Slowest test job, median
09-20 to 10-04, all jobs start together 12.7 min 12.3 min, starting at once
10-08 to 10-09 21.0 min 11.6 min, starting at +7.7 min

The slowest test job did not get slower. The wait for Guard tests adds the time. Runner minutes per green run also rose, from 67 to between 76 and 96. Those figures come from samples of 4 to 14 runs.

The guard-first order helps only when a guard fails. The run then fails about 6 minutes sooner. In a sample of 26 failed runs that ran tests, Guard tests failed in 6.

What to change

  1. Start build, the Darling PG shards, the Lite shards and the Darling Linux build together with Guard tests. Each job builds what it needs, as before The CI shards reuse the Guard tests job's build instead of building the test projects again (#5459) #5485.
  2. Keep the fast failure. When Guard tests fails on a pull request, stop the jobs that are still running. The run must report a failure that names the guard, not a cancelled check.
  3. Remove the wait of darling-tree-guards on build. If that job needs an output of build, the PR says why, and the job keeps the wait.
  4. Keep the required checks. build and the result jobs still report on every run, including a run that the gate declines.
  5. Measure the median green run time and the runner minutes on the PR, and compare them with the table above.

Not in scope

Test-map selection (#5459), the shard count and the draft and merge-train skips stay as they are.

Activity

  1. added
    in-progressActively being worked by a local session or its agents (PR open or in flight)
    on Oct 11, 2026
  2. erikdarlingdata commented on Oct 11, 2026

    @erikdarlingdata
    OwnerAuthor

    Claude posting for Erik Darling.

    #5670 is closed without a merge. In a paired run it was not faster, and it used 44 to 65 more runner minutes per run. The numbers are in #5670. This issue stays open for a design that builds once and starts the shards sooner.

  3. removed
    in-progressActively being worked by a local session or its agents (PR open or in flight)
    on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions