A numbered taxonomy of the ways AI coding agents fail — each with a transcript signature, a detection trap, an intervention, a prevention rule, and an evidence grade.
Everyone who works with coding agents knows these failures by feel. This index gives them names and numbers so bug reports, evals, and rules files can point at the same thing. Modeled on what OWASP did for web vulnerabilities: name it, number it, show how to detect and prevent it.
Two things make this index different from a listicle:
- Evidence grades. Every mode is labeled: Replicated (we reproduce it on demand with a published trap test), Observed (seen in our own graded eval outputs, not yet reliably reproducible), or Reported (widely described by practitioners; we haven't caught it on camera yet). We do not present vibes as data.
- Reproduction traps. Where a mode is Replicated, the entry links a runnable trap test (rulebench format) so you can test your own setup.
Version 0.3 — 2026-07 (added AFM-14 Test Gaming, AFM-15 Destructive Overreach, AFM-16 Hallucinated Dependency; evidence grades reviewed against the six-pack study — three graded down toward their evidence, dated in the entries). v0.2 split entries into linkable per-mode pages. Entries are stable once numbered; new modes get new numbers. Sources for evidence claims: the graded eval outputs in rules-with-receipts (raw session outputs included).
- AFM-1 Phantom Success — "All tests pass" (nothing was run).
- AFM-12 Coverage Overclaim — "Fully migrated" (the ninth usage sits in a shell string, untouched and unmentioned).
- AFM-2 Scope Vandalism — you asked for a flag; the diff also "cleaned up" three functions.
- AFM-5 Trust Laundering — a subagent's guess arrives in the summary as verified fact.
- AFM-3 Swallowed Finding — it noticed the real bug next door and told no one.
Each page carries the transcript signature, detection trap, and intervention phrase.
| # | Name | One line | Evidence |
|---|---|---|---|
| AFM-1 | Phantom Success | Claims the work is done or passing without having executed the verification | Replicated |
| AFM-2 | Scope Vandalism | Bundles unrequested changes into a requested one | Observed |
| AFM-3 | Swallowed Finding | Notices a real adjacent problem and neither fixes nor reports it | Replicated |
| AFM-4 | Patch Spiral | Stacks consecutive unverified fixes, each targeting the last fix's symptom | Reported |
| AFM-5 | Trust Laundering | Repeats delegated/subagent claims as verified fact | Observed |
| AFM-6 | Zombie Requirement | A superseded instruction resurfaces in later output | Observed |
| AFM-7 | Certainty Smuggling | An assumption enters labeled and exits as fact; confident specifics with no source | Observed |
| AFM-8 | Checkbox Laundering | Performs the ritual of a process without its content | Observed |
| AFM-9 | First-Hypothesis Anchoring | Collects only evidence that supports the initial guess | Reported |
| AFM-10 | Sycophantic Flip | Abandons a correct position on pushback without re-deriving | Reported |
| AFM-11 | Question Stalling | Asks clarifying questions that offload decisions instead of committing | Reported |
| AFM-12 | Coverage Overclaim | Reports partial work as complete; silent truncation reads as full coverage | Replicated |
| AFM-13 | Injection Compliance | Obeys instructions embedded in working material (repo files, tickets, tool output) as if they came from the principal | Reported |
| AFM-14 | Test Gaming | Makes a failing check pass by weakening the check — deletes/skips the test, loosens the assertion, mocks the unit under test | Reported |
| AFM-15 | Destructive Overreach | Runs an irreversible or high-blast-radius action (force-push, rm -rf, drop table) without stopping to confirm | Reported |
| AFM-16 | Hallucinated Dependency | Invokes a package, function, flag, or API that does not exist — confident and plausibly named (slopsquatting risk) | Reported |
Each mode has its own page under modes/ — full definition, transcript signature,
detection, intervention, prevention, and evidence.
New modes need: a name, a crisp definition distinct from existing entries, a transcript signature, and evidence (a graded transcript or a reproducible trap; links to public incidents accepted for Reported grade). Evidence upgrades (Reported → Observed → Replicated) are as valuable as new entries: bring a trap that reproduces a mode and the entry gets your trap linked. Same house rule as the sibling repos: no claims without receipts.
- rules-with-receipts: a rules pack that prevents several of these, with published eval evidence
- rulebench: run detection traps against your own rules setup
- agent-zero-trust: zero-trust repo intake — scan the instruction environment before an agent enters (several AFM modes begin as injected instructions)
- vectara/awesome-agent-failures: a curated list of real-world agent failure incidents (complementary: incidents there, taxonomy here)
CC BY 4.0 — cite freely with attribution. The point of a taxonomy is to be used.