Skip to content

Version-bump-only regressions surface only on the Version Packages PR — rehearse pnpm run version on main on a schedule and test the bumped tree #20562

Description

@hotlong

Problem

CI on main only tests the tree at its current version. Some code depends on the version number itself, for example:

  • tests that pin the current minor;
  • protocol-major checks;
  • scaffold templates;
  • docs image tags.

That code is first exercised after pnpm run version, and that runs only on changeset-release/main. So these failures show up at release time, on a PR that nobody can commit to: the version-pr lane rebuilds it with git reset --hard. The release then waits a full cycle:

  1. file the defect;
  2. fix it on main;
  3. merge the fix;
  4. wait for the next scheduled or dispatched refresh of the version PR;
  5. re-run CI.

Precedents of the same class

(Advisory-discovery reds such as #20561 are a different class. The daily OSV scan already covers them, and they are out of scope here.)

Proposal

Add a scheduled job (daily, or on each push to main that adds a changeset) that:

  1. clones main and prepares it with scripts/pm/release-rehearsal-clone.mjs --prepare;
  2. runs pnpm run version there, with no push and no publish;
  3. builds, then runs the test suites against the bumped tree. The whole Test Core matrix is the ideal. If that is too expensive, run at least the version-sensitive packages: create-objectstack, the CLI scaffolds, Scaffold E2E, and the protocol-version sync checks;
  4. on red, opens or updates one finding card. It never touches changeset-release/main.

Acceptance

Open question for maintainers

The cost trade-off: run the full Test Core matrix on the bumped tree, or only a curated subset. A curated subset needs its own gate so that the list does not rot.


Generated by Claude Code

Activity

  1. added theissue type on Sep 29, 2026
  2. objectstack-fleet commented on Sep 29, 2026

    @objectstack-fleet
    Contributor

    Triage: first touch — closed not_planned. A new scheduled gate defaults to no until a maintainer names it

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-09-29T04:05Z. ⛔ Not a claim, ⛔ not a dispatch.

    The class is real, and the precedents are right. #4894, #15255 and #20560 are each a version-sensitive failure first seen on the Version Packages PR, which costs a release cycle. #20566 is the next one, found by reading rather than by the release.

    Why it closes now. The card proposes a new scheduled CI job that opens finding cards: a new patrol, a new gate. The standing rule is 「新增门禁默认否」: a new gate lands only when a maintainer names it. The card's own open question (the full Test Core matrix or a curated subset, and a gate for the subset) is the maintainer's cost call.

    It re-enters pm:queue when a maintainer names it, answering the matrix-or-subset question. Triage then grades it tooling under domain:devx, with the named release surface as its reach. The question has also been raised in the triage seat's chat.

    Meanwhile, the fixes stay per defect. #20560 has landed, and #20566 is graded this round.

  3. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    Contributor

    Recorded: the maintainer keeps this card closed. No daily version-bump rehearsal on main

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-09-30T06:10Z. ⛔ Not a claim, ⛔ not a dispatch.

    This card was closed not_planned on 2026-09-29 under 新增门禁默认否, with the question of reopening it put to the maintainer. The maintainer answered 「同意」 in the live PM chat with the triage seat, 2026-09-30, to triage's recommendation to keep it closed.

    The recommendation rested on this: the two release faults since (#20613, the publish of the wrong head, and #20747, main red on Console Pin Gate) were each root-caused and fixed where they arose: #20613 by its release fix (PR #20625), and #20747 by PR #20743. The path-filter gap that let #20747 land is #20765, which is queued. Neither would have been caught by a version-bump rehearsal. A new version-PR-only failure with a measured cost is the trigger to raise this again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions