Skip to content

Audit AGENTS.md and CODE_REVIEW_EXCEPTIONS.md — the files that govern every Codex review, and one exception whose revisit trigger fired the day after it was written #499

Description

@BaseInfinity

Researched before filing rather than filing a vague "audit it". AGENTS.md is in
better shape than expected
— the finding is not drift in it, so the scope below is
narrower and more specific than the title of a general audit would suggest.

What is already correct (so the audit does not re-litigate it)

Finding 1 — a live exception whose own revisit condition fired 4.5 months ago

AGENTS.md line 83 tells every reviewer:

Read CODE_REVIEW_EXCEPTIONS.md before flagging findings. If your finding matches
a documented exception, skip it.

That file has one entry, last touched 2026-04-13. It waives SHA-pinning of
third-party GitHub Actions across all workflows, justified as:

"this is a meta-documentation repo, not a deployed service"
"Revisit if: The repo starts publishing artifacts, runs in production
environments, or if a referenced action has a security incident."

The trigger has fired. Verified against the npm registry:

Exception written 2026-03-27
First npm publish (agentic-sdlc-wizard@1.15.0) 2026-03-28
Versions published before the exception 0
Current 1.94.0

The repo began publishing artifacts the day after the exception was written, and
has published continuously since. The premise the waiver rests on — "not a deployed
service", no published artifacts — was false within 24 hours and has been false for
every release since.

This matters more than an ordinary stale doc because the file's function is to
suppress reviewer findings. A stale exception does not sit inert; it actively
tells every Codex review to skip a class of supply-chain finding on a package that
thousands of installs pull from npm.

Scope: re-decide the exception on today's facts. Keeping tag-pinning may still be
the right call — the maintenance-churn argument is real — but it has to be re-argued
now that the repo ships an npm package, not inherited from a premise that expired.
Whatever is decided, update the Revisit if: clause so it names a condition that has
not already happened.

Also: nothing tests that a Revisit if: condition is still unmet. That is the
same defect shape as #491 — a documented safety property with nothing checking it.

Finding 2 — AGENTS.md predates the last three releases

Last substantive update: 2026-08-02. Since then v1.92.0, v1.93.0 and v1.94.0
shipped, plus PRs #495 and #497. AGENTS.md mentions none of:

Concept Mentions in AGENTS.md
the merge gate / scripts/merge-pr.sh 0
--user-approved 0
HARD vs clearable denylist tiers 0
the unbounded-stdin hang class 0
the Stop hook removal (v1.92.0) 0

Consequence, concretely: Codex reviewed PR #497 — which touches hooks/, a HARD
merge-evidence path — without AGENTS.md telling it that path carries special weight
in this repo's merge policy. It weighted the finding correctly anyway, but on its own
judgement, not on guidance we provided.

The stdin-hang omission is the costlier one. That class consumed five review rounds
in PR #497 and produced the single highest-severity shipped defect of the last month
(a hook alive at 10h19m against a 10s timeout). A reviewer that knows the class
recognises it in one pass.

Scope: add the merge-evidence tiers and the unbounded-read class to Review Focus
Areas. Drop the Stop hook if referenced anywhere implicitly.

Explicitly out of scope

Merging AGENTS.md into CLAUDE.md, or the CLAUDE.md@AGENTS.md single-import
pattern (see #492, where AutoGPT's CLAUDE.md is 11 bytes). That is a separate
architectural question and should not ride along with a correctness fix.

Refs #491 (documented properties with nothing checking them). Follows PR #497.

Metadata

Metadata

Assignees

No one assigned

    Labels

    impact:repo-onlyOutside package.json files (npm pack --dry-run) — reaches only us

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions