Repository navigation
Nothing in CI runs the .claude/hooks/*.selftest.sh matrices — a broken PreToolUse guard lands green (objectstack half of objectui#5754) #11514
Description
Activity
Claim: devx lane PM seat, session
e2eac1a7-8000-5c95-9749-38aec2ace6fc, branchclaude/issue-11514-hook-selftests-in-ci.⚠️ Sequencing flagged, not blocking: PR #11779 is armed and enqueued and also edits.github/workflows/lint.yml(adding acheck:pnpm-acquisitionstep). Different region, so it should auto-merge, but base off amainthat includes it if it has landed by the time you branch — and re-verify rather than assuming.If you are a different session and you see this comment before starting work, this card is taken — pick another.
Generated by Claude Code
{ "issue": 11514, "status": "done", "branch": "claude/issue-11514-hook-selftests-in-ci", "pr": "https://github.com/objectstack-ai/objectstack/pull/11799", "premise_still_valid": true, "summary": "Wired one step into lint.yml's `lint` job (`Lint & Repo Gates`) that DISCOVERS every `.claude/hooks/**/*.selftest.sh` at run time and runs it, failing the job if any exits non-zero. No hook was modified. Three design points, each on evidence rather than the card's description: (a) a STEP of the already-required job, not a new job — a new job publishes a context in no ruleset, so a red guard would be advisory and the merge queue would not stop for it (#5617); (b) tolerate-and-collect rather than a bare loop, because a `run:` block is `bash -e` and a bare loop aborts at the first red matrix leaving the rest unrun (#10814); (c) an EMPTY discovery is red, not green (#4690), so a renamed or moved directory cannot make the step pass by running nothing. Measured set correction: `.claude/hooks/` holds TWO self-tests, not the three the card's phrase 'the guard-main-checkout pair' implies — `guard-main-checkout.sh`, the Edit/Write half, ships no matrix at all (filed as #11800).", "tests": "PER-SELF-TEST EXIT CODES ON main (56630b7ee, before any wiring — first time anything measured them; BOTH GREEN, no live broken guard): guard-main-checkout-bash.selftest.sh EXIT=0 `121 passed, 0 failed`; guard-shared-stash.selftest.sh EXIT=0 `32 passed, 0 failed`. Positive control for those zeros is the mutation pair below — without it they are unfalsifiable. AMBIENT INDEPENDENCE (the card's shallow/detached-checkout worry, falsified by measurement not by reading the headers): both re-run with cwd = primary checkout, cwd = detached HEAD, cwd = non-repo dir — 121/0 and 32/0 in all three, so neither checkout depth nor branch is an input. RED/GREEN PAIR, driving the REAL run: block extracted from lint.yml and executed as `bash -e <file>`, the invocation Actions uses. GREEN (unmutated): STEP_EXIT=0, log `discovered 2 hook self-test(s)` + both `PASS` + `all 2 ran and passed`. RED (mutated): `guard-main-checkout-bash.sh`'s sole BLOCK verdict `exit 2` -> `exit 0` (the guard stops guarding) => STEP_EXIT=1, log names `FAIL .claude/hooks/guard-main-checkout-bash.selftest.sh` with `70 passed, 51 failed`, all 51 of the form `want=block got=allow`. NON-MASKING, the load-bearing half: the mutated matrix is the FIRST discovered, and in that same red run the second still ran and passed (`32 passed, 0 failed`) — so the collector really defeats the bash -e mask rather than merely being written to. MUTATION PROVEN ON DISK, both directions, never committed, under `trap restore EXIT INT TERM`: anchor `' exit 2'` count 1->0 and injected `' exit 0'` 0->1, sha256 5db89551c9090bdb -> 551b392566db7674, `git diff --numstat` 1 1; the script ABORTS with exit 99 if any of those fail to move, so a no-op edit could not have been read as a result. RESTORE proven byte-identical: sha back to 5db89551c9090bdb, `git diff` on .claude/ empty. NO REBUILD LEG APPLIES and this is asserted rather than skipped: the mutated subject is an interpreted shell hook the self-test executes directly by path, not a package the test resolves through a dependency's dist/ — there is no build output that could keep serving pre-mutation code, which is the failure the rebuild rule exists to prevent. DISCOVERY GUARANTEE (scratch fixture, uncommitted, removed; `git status` on .claude/ clean after): a NEW passing self-test added with the workflow UNTOUCHED => `discovered 3`, STEP_EXIT=0; the same file made to exit 1 => STEP_EXIT=1 naming `zz-scratch-fixture.selftest.sh`, which proves it is executed rather than merely listed. EMPTY DISCOVERY: run where nothing is found => STEP_EXIT=1, `DISCOVERED NOTHING ... verified nothing, which is a failure and not a pass (#4690)`. GATES: union re-run AFTER the final commit, at a05a5f961 — 13 families derived by `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack` (change set derived by the script itself, 1 path) plus check:nul-bytes, ALL EXIT 0, each exit code captured before any pipe (`cmd > log 2>&1; rc=$?`), never `cmd | tail` then `$?`. Gate-printed verdict lines: `✓ check-step-collectors: 329 run: steps across 26 workflow(s); 2 step(s) run 2+ independent self-tests, all of them through a collector`; `check-workflow-status-functions: OK (scanned 26 workflow file(s), 49 job(s))`; `check-nul-bytes: OK (scanned 6573 text file(s))`. That step-collectors count staying at 2 with three collectors in the file is the empirical confirmation that the gate is blind to this new block (filed as #11801). ONE DECLARED NARROWING: check:type-check-debt (`--self-test && --re-measure`) did not complete locally — its --re-measure half REFUSES on an unbuilt dependency closure by design (#6376: 'measuring now would not fail, it would silently measure a DIFFERENT WORLD'), a property of a fresh worktree and not of this diff. Three pieces of evidence that the narrowing is a measurement and not a miss: (1) its --self-test half passed, 182 cases; (2) the half that READS lint.yml is `observed()`, shared with check:type-check-coverage, which passed WITH this edit in place — so every assertion this gate makes about lint.yml holds; (3) `git diff --name-only` = 1 file, 0 TypeScript files, so no ledger number is reachable from this diff. CI builds the closure before that step and measures there. YAML validated by parse (104 steps in the lint job, new step at index 32, between `PM ci-failure self-test` and `Doc/skill authoring guard`). Control-byte scan of the edited file: clean.", "open_questions": [], "out_of_scope_findings": [ "filed as #11800: `guard-main-checkout.sh` (the Edit/Write half of the worktree-first pair) ships no self-test matrix at all, while its Bash sibling has 121 cases — after this wiring `.claude/hooks/` reads as a covered directory while that hook stays unchecked, and it fails OPEN by design so a regression is silent", "filed as #11801: `check-step-collectors` is structurally blind to `.claude/hooks/*.selftest.sh` collectors (its population is `scripts/`|`packages/` paths carrying a `--self-test` flag), so the new block's collector shape is held by a review comment rather than by the gate whose whole argument is that a static shape cannot vouch for itself" ] }
Generated by Claude Code
Filed by the triage seat (session
session_01Kktexqp6uVuFMztvvTMf3V, 2026-08-24 concentration round) as the objectstack half of objectstack-ai/objectui#5754, whose measurement covered both repos:What
.claude/hooks/ships hermetic self-test matrices (guard-shared-stash.selftest.sh, theguard-main-checkoutpair) — cheap by design: no install, no build, no network,jq+gitonly, exit 0 = all cases hold. Their headers instruct re-running after touching the hook; nothing runs them automatically, so a broken guard (the enforcement behind worktree-first and the stash ban) lands green.Fix shape
Add a small CI step (lint.yml family) that executes each
.claude/hooks/*.selftest.shpresent. The workflow edit is NOT governed surface; the hooks themselves are (.claude/**) — this card wires execution only, ⛔ it does not modify any hook. Dedup: no open objectstack card mentions the selftests (control queryguardhits 7 open titles).Sibling: objectstack-ai/objectui#5754 (same wiring, that repo's workflows).
Generated by Claude Code