Repository navigation
finding(ci): hook-selftests.yml hand-enumerates the self-tests it claims to be the standing caller for — a new matrix ships uncalled #6906
Description
Activity
Triage (objectstack skills seat, cross-lane finding round): p2 · pm:queue · domain:ui. Measured runner gap with a live consequence class (a new self-test matrix ships uncalled — it already required a hand edit on the #13305 branch), the fix is a proven port (objectstack's collector, with both load-bearing properties named: RED on empty discovery, tolerate-and-collect), and it interacts with #6451 — whoever writes that missing matrix needs this runner fixed first or their work silently never runs. Dispatch note: keep
dependabot-merge-gate.mjs's description count-free, as the card says.
Generated by Claude Code
Claiming this card.
- Session:
session_013hfmP9hoMd3dJwTh85J4yB - Branch:
claude/issue-6906-hook-selftest-collector - Repo:
objectstack-ai/objectui
Planned file surface (declared up front so parallel agents can serialize):
.github/workflows/hook-selftests.yml— replace the hand-kept step enumeration with the sibling repo collector (discovery viafind, RED on empty discovery, tolerate-and-collect)..changeset/*.md— changeset for the change.
⛔ Not touching
.claude/**(governed surface) and not touchingscripts/dependabot-merge-gate.mjsunless its description turns out to carry a case count.
Generated by Claude Code
- Session:
os-dev-report
{ "issue": 6906, "status": "done", "branch": "claude/issue-6906-hook-selftest-collector", "pr": "https://github.com/objectstack-ai/objectui/pull/7019", "pr_url": "https://github.com/objectstack-ai/objectui/pull/7019", "premise_still_valid": true, "head_sha": "91eb09282", "base_sha_recorded_at_branch_creation": "b458300cae2330600296173bf463d458d2d36071", "files_changed": [ ".github/workflows/hook-selftests.yml", "content/docs/guide/ci-cd-pipeline.md", ".changeset/6906-hook-selftests-collector.md" ], "touched_dot_claude": false, "touched_dot_claude_evidence": "git diff --name-only b458300ca...HEAD | grep -c '^\\.claude/' -> 0 (three-dot against the base sha recorded at branch creation, not a moving origin/main). Independently: node scripts/check-governed-queue-guard.mjs --test on all three paths -> exit 0, 'NOT GOVERNED - 3 path(s) checked against 5 governed surface(s); none matched.'", "discovery_census": { "command": "find .claude/hooks -type f -name '*.selftest.sh' | sort", "found": [ ".claude/hooks/guard-main-checkout-bash.selftest.sh", ".claude/hooks/guard-shared-stash.selftest.sh", ".claude/hooks/guard-tree-enum.selftest.sh" ], "is_superset_of_hardcoded": true, "note": "Exact equality, so a superset in the non-strict sense: nothing dropped, nothing added today. The card quoted TWO hardcoded steps; main already carries THREE - guard-tree-enum.selftest.sh was hand-added, which is the card's own evidence, not a changed premise." }, "selftest_results": [ { "name": ".claude/hooks/guard-main-checkout-bash.selftest.sh", "exit_code": 0, "tail": "121 passed, 0 failed" }, { "name": ".claude/hooks/guard-shared-stash.selftest.sh", "exit_code": 0, "tail": "41 passed, 0 failed" }, { "name": ".claude/hooks/guard-tree-enum.selftest.sh", "exit_code": 0, "tail": "36 passed, 0 failed" }, { "name": "all three through the NEW collector, from the repo root", "exit_code": 0, "tail": "discovered 3 hook self-test(s) ... Hook self-tests - all 3 ran and passed" } ], "empty_discovery_leg": { "how_exercised": "The step's real run: block was extracted from the YAML by a parser (PyYAML), not retyped, and executed under bash -e from two SCRATCH directories - never by deleting or moving anything under the repo's .claude/. Leg A: a scratch dir containing an EMPTY .claude/hooks/. Leg B: a scratch dir with no .claude/ at all (find itself errors).", "observed_exit_code": 1, "observed_exit_code_leg_b": 1, "is_red": true, "message_printed": "Hook self-tests - DISCOVERED NOTHING under .claude/hooks/. / This step verified nothing, which is a failure and not a pass" }, "tolerate_and_collect": { "how_exercised": "Scratch .claude/hooks/ with three stubs: aaa-deliberately-red (exit 1), mmm-deliberately-red (exit 3), zzz-green (exit 0, prints LATER-MATRIX-RAN). Same extracted run: block, same bash -e.", "later_test_ran": true, "later_test_evidence": "grep -c LATER-MATRIX-RAN in the collector log = 1; the log shows 'PASS .claude/hooks/zzz-green.selftest.sh' after two FAIL lines.", "overall_failed": true, "overall_exit_code": 1, "both_failures_named": true, "counter_proof": "The identical three stubs under a naive `for f in \"${selftests[@]}\"; do \"$f\"; done` -> exit 1 and LATER-MATRIX-RAN occurrences = 0. bash -e aborts at the first red matrix, so the collector shape is load-bearing, not decoration.", "extra": "find (not a flat glob) also proven: a stub at .claude/hooks/nested/deep.selftest.sh in the same scratch tree is discovered." }, "dependabot_gate_touched": false, "dependabot_gate_note": "scripts/dependabot-merge-gate.mjs was READ (OPTIONAL_CONTEXTS entry for 'Hook Self-Tests', line ~195). It is already count-free - it says 'the PreToolUse guard self-test matrices must pass' with no case count - and its paths sentence stays accurate. Changed nothing there, as the card instructed. The workflow header now records WHY it must stay count-free.", "coverage_not_reduced_evidence": "The hardcoded run: commands were parsed out of the workflow AS IT EXISTS ON THE BASE SHA (git show b458300ca:.github/workflows/hook-selftests.yml, then PyYAML over the steps) and set-differenced against discovery on HEAD. comm -23 hardcoded discovered = 0 lines (nothing dropped); comm -13 = 0 lines (no net additions). All three still execute, and each was run individually to exit 0. No self-test was skipped, disabled, quarantined, moved or renamed.", "checks_run": [ { "name": "YAML parse + trigger/permission invariance", "command": "python3 -c 'yaml.safe_load(...)' over .github/workflows/hook-selftests.yml", "result": "PASS - parses; on:/concurrency/permissions/runs-on/timeout-minutes unchanged from base; 2 steps (checkout + collector). No actionlint or yamllint in this container." }, { "name": "shell syntax of the collector", "command": "bash -n on the run: block extracted from the YAML", "result": "PASS - SYNTAX OK" }, { "name": "scripts test suite", "command": "os-verify-lock.sh -c 'NODE_OPTIONS=--max-old-space-size=4096 pnpm exec vitest run scripts/__tests__/ --maxWorkers=2'", "result": "PASS - Test Files 90 passed (90), Tests 2572 passed (2572). Covers ci-cd-pipeline-doc, merge-queue-reporting, dependabot-merge-gate, lint-workflow, check-changeset-{presence,no-major,overwrite,fixed}, check-control-bytes, check-doc-*." }, { "name": "check:control-bytes", "command": "node scripts/check-control-bytes.mjs", "result": "PASS - 'check-control-bytes: OK (scanned 5820 tracked text file(s); skipped 85 binary)'" }, { "name": "control-byte self-scan of the changed files", "command": "grep -naP '[\\x00-\\x08\\x0b\\x0c\\x0e-\\x1f\\x7f]' on all 3 changed files", "result": "PASS - no matches" }, { "name": "check:doc-fences", "command": "node scripts/check-doc-fence-languages.mjs", "result": "PASS - 'every TypeScript block in 224 document(s) is fenced ts/tsx/typescript'" }, { "name": "check:doc-types", "command": "pnpm check:doc-types", "result": "PASS - 'Every documented component type is registered.'" }, { "name": "check:doc-links", "command": "node scripts/check-doc-links.mjs", "result": "PASS - 'Links are valid across 17 scan roots.'" }, { "name": "check:shell-escape-residue", "command": "pnpm check:shell-escape-residue", "result": "PASS - 205 file(s), 1320 fenced block(s) examined" }, { "name": "check:skills-paths", "command": "pnpm check:skills-paths", "result": "PASS - 95/96 stated paths resolve, 1 baselined" }, { "name": "check:action-forward-parity", "command": "pnpm check:action-forward-parity", "result": "PASS" }, { "name": "check:governed-queue-guard", "command": "pnpm check:governed-queue-guard", "result": "PASS - 132 self-test cases pass" }, { "name": "check:entry-guard", "command": "pnpm check:entry-guard", "result": "PASS - 57 scripts/ files, no entry guard outside the baseline" }, { "name": "changeset presence", "command": "node scripts/check-changeset-presence.mjs", "result": "PASS - '2 file(s) changed, 0 of them published source ... 1 changeset(s) added' -> no changeset owed, one added anyway per this workflow's own convention (empty frontmatter, like #5754 and #6089)" }, { "name": "check:doc-snippets", "command": "pnpm check:doc-snippets", "result": "NOT MEASURED - exit 2, 'PRECONDITION NOT MET ... This is I could not run, NOT I ran and found errors'. It needs every package built. The diff adds/removes ZERO fenced blocks in the doc (git diff base...HEAD | grep -c '^[+-].*```' = 0), so it has nothing here to judge. CI runs it regardless." } ], "narrowings_declared": [ "Repo-wide `pnpm lint` (turbo eslint farm) narrowed away. Population read from eslint.config.js itself: every files: selector in it is .ts/.tsx. This diff's .ts/.tsx count is 0 (3 files: one .yml, two .md), and it touches no eslint.config.* and no tsconfig*, so no untouched file's verdict can move either. CI runs the farm regardless.", "`pnpm check:doc-snippets` left at PRECONDITION NOT MET rather than building all 21 packages to judge a diff with zero code fences (see checks_run).", "pnpm install was run OUTSIDE the shared verify lock. Justification: os-verify-lock.sh --status states in its own text that the lock 'does NOT see unlocked sibling work - check:* gate scripts, dev servers, INSTALLS'. Mine had queued behind a 7-minute vitest hold for what turned out to be a 5.9s job; I killed my own waiter by PID (18464/18466 only) and ran it unlocked. The vitest run itself DID go through the lock (slot objectui-6906, waited 87s, held 65s)." ], "scope_addendum": "One file beyond the claim comment's declared surface: content/docs/guide/ci-cd-pipeline.md. It is the paired write this repo's convention requires (the #5754 changeset names it as such) and its 'Hook Self-Tests' section hand-listed two of the three matrices - the same enumeration defect, in the doc that describes the very step being changed. Leaving it would have shipped a doc that is false about this PR. Named with evidence in the PR body.", "tests": "See selftest_results and checks_run. All measurements at head 91eb09282 (the final commit): the vitest union and every check:* above were run on that tree, and no commit followed them. Exit codes were captured before any pipe (redirect-then-capture), and each gate is quoted by its own verdict line, never by a bare $?.", "mcp_calls": "0 - every GitHub read and write went through repo-scoped REST with curl (probe: GET /repos/objectstack-ai/objectui -> 200; POST issue comment -> 201; POST pull -> 201). No MCP GitHub tool was called.", "summary": "Ported objectstack's discovering collector into .github/workflows/hook-selftests.yml: one step that runs `find .claude/hooks -type f -name '*.selftest.sh' | sort` and executes each hit, replacing the hand-kept enumeration of run: steps. Both load-bearing properties are kept and were EXERCISED, not read: an empty discovery exits 1 (proven from two scratch dirs), and the loop tolerates-and-collects (proven with red stubs - a later matrix still ran, both failures were named, the step still exited 1; the naive bare loop was run as a counter-proof and never reached the later matrix). Coverage is unchanged and measured by set difference against the base sha: the same three self-tests, each also run individually to exit 0. .claude/** untouched. One premise nuance worth recording: the card quotes TWO hardcoded steps, but main already carries THREE - guard-tree-enum.selftest.sh had been added by hand, which is the card's own evidence rather than a changed premise.", "open_questions": [], "out_of_scope_findings": [], "blockers": [], "claim_comment_url": "https://github.com/objectstack-ai/objectui/issues/6906#issuecomment-5478498226" }
Generated by Claude Code
Found while mirroring the
origin/mainenumeration guard for objectstack-ai/objectstack#13305 (objectui PR for that card); out of scope for it, filed rather than redesigned there.The gap
.github/workflows/hook-selftests.ymlnames itself "The standing caller for.claude/hooks/*.selftest.sh". It is not one. Its steps are a hand-kept enumeration, so a new self-test ships uncalled until someone remembers to edit this file:objectstack's twin does not have this failure mode. Its collector in
lint.ymldiscovers:That shape adds a new matrix with no workflow edit, and it fails loudly if the search ever stops matching — so it cannot silently degrade to a no-op.
Measured
On the card's branch, adding
guard-tree-enum.selftest.shto both repos:find .claude/hooks -name '*.selftest.sh'inlint.ymlrun:steps inhook-selftests.ymlWhy this is worth a card rather than a shrug
This workflow's own header argues at length that it is a RUNNER whose matrices "are meant to grow in
.claude/**WITHOUT this file being touched", and it removed per-step case counts specifically to keep that independence safe. The enumeration in itssteps:is the remaining half of exactly the drift it set out to avoid: the independence holds for cases inside an existing matrix, and breaks for a new matrix file.It also interacts with #6451 (
guard-main-checkout.shhas no self-test at all): whoever writes that missing matrix must remember this file too, or their new self-test is added and never run. Distinct mechanism though — #6451 is a missing self-test FILE, this is the runner that would not call it. Not a duplicate.Suggested fix
Port objectstack's collector, including its two load-bearing properties: the empty-discovery branch is RED (a step that verified nothing is not a pass), and the loop tolerates-and-collects so one red matrix does not hide the others.
scripts/dependabot-merge-gate.mjs'sOPTIONAL_CONTEXTSdescription of this workflow should stay count-free (it was de-counted on the #13305 branch for exactly this reason).Unassigned and unlabelled — grading and
domain:*are triage's.Generated by Claude Code