Audit CI/CD: green pipelines certified a phantom watchdog, 7 hooks that hang forever, and a hook whose test encodes its own defect

[BaseInfinity]

Prompted by the 2026-08-04 /doctor run and the Opus 5 / Fable 5 harness audit (#486). Every finding below was verified against the repo, not inferred.

CI is green. It was green through all of the following.

What CI is already doing right (verified, so the audit doesn't re-litigate it)

• All 64 top-level tests/*.sh suites are invoked by .github/workflows/ci.yml. Zero orphans.
• Every test path CI references exists. No phantom test steps.
• ci.yml enumerating each suite as its own step is now documented as the authority (CLAUDE.md, fd57dcd).

So this is not "CI doesn't run the tests." It's that the tests, and CI, cannot see whole classes of defect that shipped anyway.

Class 1 — phantom references in shipped docs (VERIFIED, has caused harm)

CLAUDE_CODE_SDLC_WIZARD.md:2535 tells every consumer:

"The wrapper scripts/codex-review.sh already has a 30-min stall watchdog (STALL_SECONDS=1800) as the real timeout control."

Neither exists. scripts/codex-review.sh is not in the repo, and STALL_SECONDS appears nowhere in scripts/ or hooks/. A grep across shipped docs finds exactly one phantom scripts/*.sh reference — this one.

Not cosmetic: this session found 14 stale codex processes, oldest 3.5 days, precisely because the watchdog everyone was told to rely on does not exist. Both cross-model reviewers independently repeated the claim as fact because it is in our own shipped documentation. One of them called it "the exact WebFetch-class error in a different costume."

Proposed check: every scripts/*.sh, hooks/*.sh, and tests/*.sh path named in shipped docs must exist on disk. Trivially mechanical, would have caught this the day it was written.

Class 2 — hook robustness is untested, and the hooks ship

Seven shipped hooks perform an unbounded read on stdin:

• hooks/sdlc-prompt-check.sh:45
• hooks/codex-gate-check.sh:16
• hooks/tdd-pretool-check.sh:24
• hooks/codex-review-stop-check.sh:23
• hooks/precompact-seam-check.sh:30
• hooks/token-spike-check.sh:24
• .claude/hooks/merge-gate-check.sh:31

The guard is [ ! -t 0 ] — "is stdin not a terminal." A unix socket is not a terminal, so the guard passes and cat blocks until an EOF that never arrives.

Observed, not theoretical: sdlc-prompt-check.sh ran for 10h19m against a 10-second timeout, lsof showing 0u unix. The documented hook timeout did not reap it. Consumers inherit all of this via cli/templates/settings.json.

CI has no adversarial-stdin test for any hook.

Proposed checks: feed each hook stdin from a FIFO that never closes and assert it exits within N seconds; feed malformed JSON, empty input, and a closed pipe. Fix is read -t, not timeout(1) — that does not exist on macOS and has already produced two "61 files, 0 failing" reports that ran zero tests.

Class 3 — assertions that pass while guarding nothing

CI cannot distinguish a passing assertion from a vacuous one.

• Six assertions passed against broken code in a single review (ROADMAP Ship a model-ADOPTION policy, not just a model recommendation — "if it works, don't break it" #490).
• The merge gate shipped in v1.90.0 with 49 assertions that all checked evidence structure and never read the verdict inside. It merged a PR over two explicit NO votes at 97 and 99 confidence. CI was green.
• tests/test-doc-consistency.sh has 5 remaining _check_line_has_and_lacks call sites pinned to hardcoded line numbers (~1441, 1445, 1458, 1460, 1544). Proven false-passable: inserting a decoy line carrying the expected tokens at the pinned position let the real rule be downgraded while the assertion stayed green. One was converted to content-anchoring in fd57dcd; the other five were deferred with reviewer agreement.
• A verification reported PASS on a mutation that never applied — the write was silently blocked and the "mutation" degraded to a no-op. Same class as ROADMAP Make GitHub Issues the source of truth; ROADMAP.md becomes a view, not a store #482's three false results.

Proposed direction: a non-vacuity convention for new guards (a canary asserting the predicate detects a real defect), and converting the 5 line-pinned sites. Not a mutation-testing campaign — that was deliberately killed (ROADMAP #488/#491) and is not being reintroduced.

Class 4 — scoring drifts from what we ship

CLAUDE_CODE_SDLC_WIZARD.md:113 states the E2E rubric has "zero discriminating power." The same rubric scores self_review as a critical criterion.

Both reviewers, independently and from opposite overall verdicts, recommend deleting the same-model self-review layer from skills/sdlc/SKILL.md. If that lands without updating the scorer, CI scores a criterion the skill no longer instructs — and the rubric is already known not to discriminate.

Proposed check: every criterion the E2E scorer scores must be traceable to instruction text that actually ships.

Class 5 — structural mismatches CI has no way to see

hooks/codex-review-stop-check.sh is registered on Stop, which fires every turn. It uses a once-per-session sentinel. Therefore it warns on the first ordinary mid-work turn and is silent on the actual walk-away turn it was built for (#436). It also accepts a stale CERTIFIED status without the commit-SHA freshness check the real gate uses.

tests/test-hooks.sh:3169-3182 asserts exactly this behaviour — first invocation non-empty, second empty. The test encodes the defect as the spec. CI is green on a hook that is structurally incapable of its stated job.

That is the deepest finding here: our tests can certify a component that cannot work.

Scope

Audit, then propose. Do not bundle fixes into this issue — Classes 1 and 2 are cheap and separable; Class 3 is a convention change; Classes 4 and 5 are tied to the #486 deletions.

Explicitly out of scope: reintroducing systematic mutation testing (killed in #488/#491/#482), and splitting skills/sdlc/SKILL.md per model (both reviewers converged on DON'T after cross-examination).

Provenance

/doctor 2026-08-04; #486 harness audit; Fable 5 and Codex (GPT-5.6 Sol, high) advisories plus a cross-examination round in which both changed position. Artifacts in .reviews/: opus5-harness-brief.md, opus5-harness-codex.md, split-crossexam.md, split-crossexam-codex.md.