Agents that catch and fix their own mistakes: a generate → validate → critique → repair loop with zero runtime dependencies, demonstrated on structured invoice extraction.
📖 Blog post: Self-Correcting Agents: Teaching an Agent to Catch and Fix Its Own Mistakes (markdown)
input ───────► [ GENERATE ] ───► candidate ───► [ VALIDATE ] ──ok──► output
▲ │ fail
│ ▼
repair prompt ◄──── [ CRITIQUE ] ◄── violations
(bounded: max N attempts, then fail loudly with the best attempt)
git clone https://github.com/sugeerth/self-correcting-agents
cd self-correcting-agents
uv run python -m selfcorrect demo --task inv_004 # watch one invoice get caught and repaired
uv run python -m selfcorrect bench --ablation # full benchmark -> bench_out/The default engine is a deterministic fault-injection simulation: a flawed extractor that makes realistic, feedback-addressable mistakes. It runs anywhere, costs nothing, and is fully reproducible (seeded) — so the benchmark measures the mechanics of self-correction, not any particular model's quality.
24-invoice corpus, seed 42, max 3 attempts — simulated engine (see disclaimer above):
| configuration | fully valid | mean attempts |
|---|---|---|
| self-correction OFF | 58.3% | 1.00 |
| self-correction ON (targeted critic) | 95.8% | 1.50 |
| ablation: generic "please fix it" critic | 58.3% | 1.83 |
The ablation is the headline: the same retry budget with untargeted feedback fixes nothing — the lift comes from structured violations rendered into specific repair instructions, not from retrying harder. Full tables: run the bench, or see the blog post.
| piece | file | job |
|---|---|---|
Engine |
src/selfcorrect/engines/ |
produce a candidate, given the task + prior feedback |
Validator |
src/selfcorrect/validators.py, invoices/rules.py |
deterministic checks → structured Violations (Decimal money math, schema, business rules) |
Critic |
src/selfcorrect/critic.py |
violations → targeted natural-language repair instructions (rule-based templates by default) |
| loop | src/selfcorrect/loop.py |
bounded retry orchestration + full per-attempt trace |
| bench | src/selfcorrect/bench.py |
OFF vs ON vs ablation; results.json / results.md / SVG |
The core (types, loop, validators, critic, engines/simulated) is domain-agnostic;
everything invoice-specific lives in selfcorrect/invoices/ as a plug-in (schema, business
rules, feedback templates, error catalog, corpus). Tests enforce the boundary.
| engine | cost | needs | use |
|---|---|---|---|
simulated (default) |
$0 | nothing | CI, reproducible benchmarks, demos |
hermes |
$0 | Ollama + ollama pull hermes3 |
real local LLM (NousResearch Hermes 3 8B) |
anthropic |
API usage | pip install "selfcorrect[anthropic]" + ANTHROPIC_API_KEY |
Claude — worker claude-opus-4-8, critic claude-haiku-4-5 |
# one-time: install Ollama from https://ollama.com, then
ollama pull hermes3
uv run python -m selfcorrect demo --engine hermes --task inv_004
uv run python -m selfcorrect bench --engine hermesHermes 3 is structured-output trained; the engine constrains generation with a JSON
schema via Ollama's format parameter and talks to localhost:11434 using only the
standard library.
from selfcorrect import SelfCorrectingAgent
from selfcorrect.engines import get_engine
from selfcorrect.invoices import build_critic, build_validator, load_tasks
agent = SelfCorrectingAgent(
engine=get_engine("anthropic"), # claude-opus-4-8 via structured outputs
validator=build_validator(),
critic=build_critic(),
max_attempts=3,
)
result = agent.run(load_tasks()[3])src/selfcorrect/ the framework (zero runtime dependencies)
engines/ simulated | hermes (Ollama) | anthropic (optional extra)
invoices/ the domain plug-in: schema, rules, templates, corpus, error catalog
tests/ 77 tests: unit, determinism, e2e lift bounds, import-boundary
examples/run_demo.py scripted walkthrough
docs/ the blog post (GitHub Pages)
uv sync --dev
uv run pytest -q # 77 tests, no network
uv run ruff check src tests examplesMIT — see LICENSE.