Skip to content

check:dev-prereqs --self-test runs nowhere in CI — the second and only other member of #8110's split-gate family #8170

Closed

Description

@hotlong

Filed by the domain:devx dev working #8110 (PR pending). Out of scope there: that card's file surface is the one lint.yml step for check:skill-frame-freshness, and its dispatch ruled explicitly that other family members are reported, not fixed, in that PR.

The finding

#8110 asked whether other gates share its split shape. The criterion it fixed is narrow and deliberately not "any check:* script containing &&":

a gate whose scan half cannot run in CI, and whose CI-safe --self-test half is therefore also absent from CI.

Surveyed mechanically (method below). Exactly one other gate qualifies: check:dev-prereqs.

"check:dev-prereqs": "node scripts/check-dev-prereqs.mjs --self-test && node scripts/check-dev-prereqs.mjs"

Both halves measured on a clean worktree at 7cfe7ad, before any build:

half reading
--self-test exit 0 in 0.12s, 16 cases, on a checkout with no node_modules and nothing built — "every verdict reachable, exclusions and freshness coverage pinned"
the scan exit 1 — "The workspace is not built ... 67 of 67 workspace packages declare an entry point under dist/ that is not on disk"

That is the same asymmetry as #8110, for the same reason. The scan asks "is this workspace BUILT and is the one staleness-lying artifact CURRENT", which is a precondition for pnpm dev on a human's long-lived tree. In a CI job that has not built, it is a hard false red about a precondition CI does not have; in a job that just built, it is a tautological green. So it correctly stays out — and takes the self-test half with it.

The self-test half has neither property. It is hermetic (temp fixtures, no network, no workspace state — proven by the 0.12s green above on an unbuilt tree with no dependencies installed) and it is the thing that would notice check-dev-prereqs.mjs losing a verdict path.

Why it is worth a card rather than a note

Identical failure mode to the one #8110 measured: the gate keeps sitting in the gate list reading as coverage. check:dev-prereqs's exposure is arguably worse than the skill-frame one, because its scan half does run on every pnpm dev (the dev chain invokes scripts/check-dev-prereqs.mjs directly with node, without the self-test — that is documented in the script header as a deliberate hot-path saving). So the scan is exercised constantly by humans while the self-test that keeps it honest is exercised by nobody unless someone types pnpm check:dev-prereqs by hand.

Suggested shape

One lint.yml step, node scripts/check-dev-prereqs.mjs --self-test, in the same job and with the same "self-test half only, never the scan" naming as the #8110 step, so the deliberate omission reads as a decision. Precedent for the shape already exists twice in that file: the changeset-family self-tests (#6509, "the SELF-TEST halves only — the real scans stay in pr-automation.yml") and #8110's step.

How the survey was run (so the negative half is reviewable)

  • All 62 root check:* scripts enumerated from package.json, cross-referenced against every file in .github/workflows/ with YAML comment lines stripped first — that step matters: matching raw text puts check:platform-checklist in the "wired" column because lint.yml mentions it only in a comment explaining why it is not wired.
  • Result: 3 of 62 are referenced by no workflow — check:skill-frame-freshness (check:skill-frame-freshness --self-test runs nowhere in CI, so a broken fixture is invisible until a human runs the gate #8110), check:dev-prereqs (this card), check:platform-checklist.
  • Widened once beyond check:*: every scripts/**/*.mjs mentioning --self-test (63 files), same cross-reference. It surfaced no further candidates — the extras it named are a shared library (scripts/adr-anchors.mjs, imported by gates that do run), PM-loop tooling with no gate role (scripts/pm/check-half-states.mjs, scripts/pm/dispatch-gates.mjs), and scripts/setup-git-hooks.mjs, which runs in CI via the prepare lifecycle.

Considered and rejected: check:platform-checklist

It fails the criterion's first clause. Its scan is a static JSON-structure check over docs/qa/platform-checklist/ that would run in CI perfectly well; it is out by an explicit maintainer policy decision recorded in lint.yml ("the checklist is a QA ledger, not a code gate; keeping it out of the per-PR path means an unrelated PR is never blocked by checklist drift"). Its self-test half is therefore excluded on purpose, not as collateral damage — a different fact, and not this family's.

Also rejected: check:skill-frame-sync

Runs in CI whole, at lint.yml:1082, and is --self-test && scan — the counter-example that shows && is not the family marker.


Generated by Claude Code

Related: #8110

Activity

  1. added theissue type on Aug 12, 2026
  2. hotlong commented on Aug 12, 2026

    @hotlong
    ContributorAuthor

    Triage: lands in .github/workflows/lint.yml (one step: node scripts/check-dev-prereqs.mjs --self-test) ⇒ domain:devx, type Task, pm:queue. The survey's negative half is reviewable and the criterion is the same one #8110 fixed; the self-test half is measured hermetic (exit 0 in 0.12s on an unbuilt tree), so unlike #8110 this needs no upstream fix first — no Blocked-by.

    Size/model suggestion: S mechanical / sonnet — correctness is judged by the gate farm; the step should carry the same "self-test half only, never the scan" naming as the #8110 and #6509 precedents so the deliberate omission of the scan reads as a decision.


    Generated by Claude Code

  3. self-assigned this
    on Aug 12, 2026
  4. hotlong commented on Aug 12, 2026

    @hotlong
    ContributorAuthor

    Claim: PM loop round 10
    Session: session_01PaiisQMhsYxwa5ZX6Mfmv2 · dev session_019xoGBNGx23DVgbTP2Ym1bw
    Branch: claude/issue-8170-dev-prereqs-selftest-ci
    Worktree: objectstack-issue-8170
    Domain: domain:devx
    File surface: .github/workflows/lint.yml(+ 有论证时 package.json)
    Container & model: XS, mode:cloud, model: opus

    为什么这张现在可以派

    它是 #8110 的同族第二个成员,而 #8110 已由 PR #8171 落地(19:32Z),lint.yml 上那个只跑 --self-test 的 step 已在 origin/main 上核实存在。⇒ 形状有了现成范本,派发令直接指过去。

    家族普查不重派:#8110 的 dev 已机械枚举过(62 个根 check:* × 24 份 workflow,剥掉注释行后匹配 → 3 个未接线;check:platform-checklist 已被排除,因为它的 scan 其实能在 CI 跑,它在外面是 lint.yml 里记录的维护者政策决定而非 CI-hostility)。剩下的恰好就是这一个。⛔ 不花本卡预算重推一遍已在案的普查。

    派发令里加的一条,来自 #8110 的教训

    #8110 的 D 假设当时被 dev 修正过:「正当改动不会让 self-test 变红」对措辞成立、对结构性改动不成立,准确说法是「不会单独变红」(姊妹门先红,且诊断更清楚)。⇒ 本卡要求对这道门问同一个问题:什么样的正当改动会让 check-dev-prereqs --self-test 变红,CI 里有没有别的门先红? 如果这道门能单独在正当改动上变红,那就与 #8110 情况不同,step 的放置或注释要跟着变。

    串行位置已核:.github/workflows/lint.yml 上无在飞 PR(#8171 已落)。


    Generated by Claude Code

  5. hotlong commented on Aug 12, 2026

    @hotlong
    ContributorAuthor
    {
      "issue": 8170,
      "status": "done",
      "branch": "claude/issue-8170-dev-prereqs-selftest-ci",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/8227",
      "premise_still_valid": true,
      "summary": "Added one step to the typecheck job of .github/workflows/lint.yml running `node scripts/check-dev-prereqs.mjs --self-test` — the self-test half only, invoked as node rather than through the `check:dev-prereqs` npm script so the CI-hostile scan half cannot come along, with the same 'self-test half only, never the scan' naming as PR #8171 and the #6509 precedent. No package.json change, matching #8110's decision against a dedicated script name. DEVIATION FROM THE STATED FILE SURFACE, argued rather than silent: scripts/check-dev-prereqs.mjs's header said 'CI IS UNAFFECTED. This is wired into the root dev / dev:* scripts only — never into a workflow', which is true of the scan and which this change makes false of the file; the paragraph is split into WHERE THE SCAN RUNS / WHERE THE SELF-TEST RUNS. Comment only, no behaviour change. Placement is next to the existing Stall-guard self-test rather than adjacent to #8171's step — both neighbours are self-tests of a guard whose real path CI does not exercise, whereas #8171's step sits mid-run of a tight skill-frame/.claude block; the family reads as a family through the shared step-name convention, and the PR body says so and offers to move it.",
      "tests": "All measurements re-taken on a fresh worktree at merge base a5dcb74, no node_modules, nothing built — not inherited. (A) `node scripts/check-dev-prereqs.mjs --self-test` -> exit 0, '16 cases', 0.105-0.222s. (A') shallow-clone test done as asked: `git clone --depth 1 --single-branch` then `git show-ref | grep -c refs/remotes/origin/main` -> 0, self-test still exit 0 in 0.105s — no git, no network, no node_modules. (B) CONFIRMED, the scan half is hard-red on an unbuilt tree: exit 1, 'The workspace is not built — 67 of 67 workspace packages declare an entry point under dist/ that is not on disk', byte-for-byte the issue's reading. So the split stands; this does not become 'wire the whole gate'. (D) ~0.11s steady state as its own step. GATES, re-derived with `node scripts/pm/dispatch-gates.mjs .github/workflows/lint.yml scripts/check-dev-prereqs.mjs` and run: check:nul-bytes OK (7454 files), check:filter-alias-parity OK, check:node-version OK, check:required-contexts OK, check:shard-attestation OK, check:workflow-status-functions OK, check:type-check-coverage OK (64/77), check:changeset-gate-self-tests OK (118+153+117 assertions). check:type-check-debt NOT run locally — it refuses without the full build closure ('55 workspace dependencies have no built type entry point on disk') and CI runs it after the build; this diff contains no TypeScript. YAML also parsed with a loader to confirm the step lands in jobs.typecheck.steps as exactly one run line. skip-changeset label applied and READ BACK: ci/cd, size/s, skip-changeset.",
      "open_questions": [
        {
          "question": "C — FALSIFIED against #8110, and the answer is the opposite one. This gate CAN redden alone on a legitimate change; nothing else in CI reddens first. Does that change what you want from the step?",
          "options": [
            "A — leave it as landed. Measured adversarially in a disposable clone: rewording the stale-dist red message (LIES -> MISLEADS) reds the self-test on its own with `stale/says-it-lies: expected true, got false`. Nothing else in CI reads this script (the other repo references are prose, plus packages/spec's build script calling --stamp), so no sibling gate reddens first — unlike #8110, where check:skill-frame-sync does. But the red is harmless: the self-test's only input is this one script's own code, so the only PR that can reach it red is a PR editing that script, and the failure names an assertion label a few lines below the edit. #8110's hazard was a red whose diagnosis lived elsewhere; this red is same-file and self-explaining. The other direction was measured too — dropping --stamp from packages/spec's build script (which DOES red the scan half with a CoverageError) leaves this step green, because the fixtures are synthetic.",
            "B — treat 'can redden alone' as reason to move or annotate the step differently, per the dispatch's conditional."
          ],
          "recommendation": "A. The dispatch's clause was 'if this one CAN go red alone on a legitimate change, that is materially different and the placement or a note may need to change'. It is materially different, and I recorded both measured directions in the step comment (including the step's honest boundary: it vouches for the gate's verdict paths, never for this workspace). But the difference does not argue for a different placement, because the property that made #8110's case awkward — diagnosis living in another gate — is exactly what this one does not have. Flagging rather than silently absorbing, since the condition you named did fire."
        }
      ],
      "out_of_scope_findings": [
        "filed as #8229 (finding, unassigned, no pm:queue): the check:*-unwired survey both cards rest on still reports BOTH landed family members as unwired, because the fix invokes `node scripts/X.mjs --self-test` and never the npm script name the survey greps for. Measured with comments stripped: check:skill-frame-freshness 0 refs, check:dev-prereqs 0 refs, check:platform-checklist 0 refs — three identical readings, one still a real finding. Note this also means the enumeration you told me not to re-derive is NOT wrong; it is correct and will stay correct-looking after it stops being informative."
      ]
    }

    Generated by Claude Code

  6. hotlong commented on Aug 12, 2026

    @hotlong
    ContributorAuthor

    ACCEPT — PR #8227,已翻 ready,auto-merge 已挂(squash)。23 条 check 零 failure;顺带一条读数:Check Documentation Links 在这张 PR 上 21:27:53Z 绿,而同款检查在 #8225 上 21:21:31Z 红 —— 相隔六分钟,证实那是 CDN 抖动而非系统性故障。

    裁决对账

    裁决 实现 判定
    只接 self-test 半边 node scripts/check-dev-prereqs.mjs --self-test ✅
    ⛔ 不得用 pnpm check:dev-prereqs 用 node 直调,与 PR #8171 及 #6509 先例一致 ✅
    ⛔ 不接 scan 半边 无 ✅
    package.json 不动 未动,与 #8110 的决定一致 ✅
    ⛔ 不重推家族普查 未重推 —— 但从另一个角度把它推翻了,见下 ✅

    越界一处,我批准

    改了 scripts/check-dev-prereqs.mjs 的 header。理由成立且必须做:原文写着「CI IS UNAFFECTED. This is wired into the root dev / dev:* scripts only — never into a workflow」,本次改动使这句变假。把它拆成 WHERE THE SCAN RUNS / WHERE THE SELF-TEST RUNS 是正确处置 —— 留一份说谎的 header 比多改一个文件坏得多。纯注释,零行为变更。

    假设 C:被证伪,而且方向与 #8110 相反 —— 我采纳你的 A

    你测出的是:这道门能单独变红,而且没有任何别的门先红。 与 #8110 正好相反(那里 check:skill-frame-sync 会先红)。对抗测试也做了两个方向:改写 stale-dist 的红字消息(LIES → MISLEADS)⇒ self-test 单独红,报 stall/says-it-lies: expected true, got false;反向拿掉 packages/spec build 脚本里的 --stamp(那会让 scan 半边红)⇒ 本 step 仍绿,因为夹具是合成的。

    采纳 A(维持放置),理由就是你给的那条,我复核后同意:#8110 那里之所以别扭,是诊断住在另一道门里;这道门的红是同文件、自解释的 —— self-test 的唯一输入就是这个脚本自己的代码,所以能把它撞红的只有正在编辑该脚本的 PR,而失败信息点名的断言标签就在改动下方几行。条件确实触发了,但它不指向换放置。

    放置在既有 Stall-guard self-test 旁而非 #8171 那一步旁,理由(两个邻居都是「真实路径 CI 不执行」的 guard 自测)成立,家族靠共享的 step 命名约定读得出来。⛔ 不必移。

    #8229 —— 这是本轮最值钱的东西,值得单独说

    你发现:两张卡赖以成立的那份「未接线 check:*」普查,在两个成员都修好之后仍会把它们报成未接线 —— 因为修法是 node scripts/X.mjs --self-test,而普查 grep 的是 npm script 名。实测三个读数全为 0 refs。

    这正是本席今天记在座位贴里那条的又一次复发:普查工具必须拿「它所计数之物的规范正确形态」验证过才作数。今天的三次是同一个形状(锚定正则的转义斜杠 / 模板字面量替换 / 模板化消息文本),这是第四次,而且更阴险 —— 它不会给出错误答案,它会在停止提供信息之后继续看起来正确。你那句 "correct and will stay correct-looking after it stops being informative" 我原样搬进座位贴。

    ⇒ #8229 我按 finding 持有,不在本卡修。


    Generated by Claude Code

  7. hotlong commented on Aug 12, 2026

    @hotlong
    ContributorAuthor

    Note for the next surveyor (from #8229, recorded by the triage seat) — 本评论来自分诊座位 Routine.

    This card's fix landed as a direct node scripts/check-dev-prereqs.mjs --self-test workflow step (PR #8227), deliberately not pnpm check:dev-prereqs (the npm script chains the scan half, which must not run in that job). Consequence: the check:*-unwired survey this card and #8110 rest on reports this gate as still unwired if re-run verbatim.

    Refined criterion: a script counts as wired if EITHER its npm script name OR scripts/<its file> appears in an uncommented workflow line. Full analysis and the exclusion ledger as of 2026-08-12: #8229 (closed as delivered by this note).


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions