Guard rules ship with an unmeasured FP rate: wire replay-guard-corpus into CI as a blocking precision gate

[jarvis-drakon]

The actual problem

Every Action Guard rule currently ships without a measured false-positive rate. The corpus replay tool exists (`scripts/replay-guard-corpus.mts`) but is not referenced by any workflow in `.github/workflows/` and not wired into `npm test`. So precision is discovered in production, on operator boxes.

The bill for that, in the last 72 hours alone, on one box:

• secret-egress catastrophic-blocks ordinary OAuth automation: fold reads 'client_secret = …' assignments as exfiltration #175 — secret-egress read program source as a command line; blocked a legitimate Outlook OAuth refresh
• Secret-egress rule blocks all OAuth token refreshes, and matches secret *references* not values #177 — OAuth token endpoints counted as exfil destinations; secret references matched as secret values
• Shell-command rules fire inside folded interpreter source — #160 exposed it on the Claude Code surface, and it blocks npm test #165 — REGRESSION in 4.47.24: shell rules fired on language APIs in folded source, blocking `npm test`
• Action Guard FP: script-content scan matches comments/docstrings — denies the cron repair tool over the word 'kill' in a comment #170 — script-content scan matches comments/docstrings; denies the cron repair tool over the word "kill" in a comment
• Action Guard FP: file-delete gates an agent's own /tmp scratch files — under bypassPermissions cleanup is impossible #180 — an agent's own `/tmp` scratch-file delete is gated, so cleanup is impossible under bypassPermissions
• secret-egress blocks vault-backed credential reads — correct posture denied identically to hardcoded secrets #173 — vault-backed credential reads denied identically to hardcoded secrets (correct posture punished)

Six FPs, one regression, one week. Every one was found by an operator being blocked, not by CI.

Why this is the priority over any individual FP fix

Each fix above is correct and each was shipped fast. That is the trap: we are fixing FPs one at a time, at production speed, and the FP rate is not a number anyone can state. A guard whose precision is unmeasured cannot be tuned — it can only be reacted to. And the operator response to an untrustworthy gate is to route around it, which is strictly worse than no gate.

What to build

1. A committed FP corpus of known-good operations that MUST verdict `allow` or at worst `sensitive`: vault-fetched credentials, standards-compliant OAuth refreshes against known IdP hosts, containerised test runs, an agent deleting its own scratch files, source files containing destructive words in comments/strings, package-manager installs inside a venv/container.
2. A committed TP corpus of operations that MUST `block` (already largely exists as fixtures — consolidate).
3. A CI job that replays both and fails the build on: any TP regression (non-negotiable), or FP rate above a declared threshold. Print the rate on every run so it is a visible number in the PR, not folklore.
4. Every new guard rule PR must add to both corpora. No rule merges without its FP cases.

Acceptance

• `.github/workflows/ci.yml` runs the replay and fails on regression
• The FP rate appears in CI output as a percentage
• Shell-command rules fire inside folded interpreter source — #160 exposed it on the Claude Code surface, and it blocks npm test #165 (the 4.47.24 regression) would have been caught by this gate before publish