Point a coding agent at a UI screenshot. Get back a pixel-perfect HTML/CSS replica.
LLMs write plausible CSS from a screenshot but can't tell whether the result actually looks right. This repo fixes that with the simplest pattern in agent engineering: give the model an objective signal and let it iterate until that signal says "done".
It's a direct implementation of the Ralph loop for UI replication:
target.png ──> agent edits index.html ──> render via headless Chromium
▲ │
└──── feedback.md ◄── pixel diff + element blame
│
match ≥ 99.5%? ──> done
The agent is replaceable. The harness provides what any agent needs to converge: an objective score it cannot argue with, and feedback specific enough to act on.
A verbatim transcript. The agent here is a scripted stand-in that applies
three progressively better drafts — the loop only sees "a CLI that edits
index.html", so it runs without an LLM (see
Works with any agent):
$ python loop.py examples/pricing-card/target.png --agent "sh ../demo-agent/agent.sh" --run-dir runs/demo
target examples/pricing-card/target.png (800x560)
run dir runs/demo
agent sh ../demo-agent/agent.sh
done at 99.5% pixel match, max 8 iterations
--- iteration 1/8 ----------------------------------------
applied draft 1
match 93.78% (27,883 px off, agent 0s, eval 2.1s) (new best)
--- iteration 2/8 ----------------------------------------
applied draft 2
match 98.77% (5,528 px off, agent 0s, eval 1.1s) (new best)
--- iteration 3/8 ----------------------------------------
applied draft 3
match 100.00% (0 px off, agent 0s, eval 0.7s) (new best)
converged: 100.00% >= 99.50% after 3 iteration(s)
result runs/demo/index.html
compare runs/demo/feedback/side_by_side.png
Each iteration, the harness renders index.html in headless Chromium, pixel-diffs it against the target, translates the diff into actionable feedback, and hands that to a fresh agent run. The loop stops when the render matches the target, not when the agent thinks it's done.
A raw score of 93.8% gives an agent nothing to act on. The evaluator attributes mismatched pixels to the DOM elements that own them, reports their computed styles, and flags target content that has no counterpart in the render at all. From feedback/feedback.md:
match: **93.78%** — 27,883 of 448,000 pixels differ (viewport 800x560, anti-aliasing ignored)
status: NOT CONVERGED (needs >= 99.50%)
## Element blame — your rendered elements, worst first
1. `div.card`
box (240,104) 320x353 · 14% of its own box mismatched · 59% of all mismatch
bg #ffffff · color #000000 · font 16px/400 · pad 28px · radius 12px
2. `button.cta` — "Start free trial"
box (268,388) 264x41 · 93% of its own box mismatched · 36% of all mismatch
bg #6d28d9 · color #ffffff · font 15px/600 · pad 12px 0px · radius 8px
...
unattributed: 41% of the mismatch lies under none of your elements — content from the target is missing or misplaced.
The same mismatch as the evaluator draws it (feedback/diff.png — red over a
faded target):
Wrong purple, narrower card, missing text. The feedback names the guilty elements with their computed styles, and the unattributed line flags missing content. Alongside it: diff.png with red highlighted mismatch and the composite side_by_side.png.
git clone https://github.com/Marvin0212/ui-agent-loop && cd ui-agent-loop
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
playwright install chromium
python make_targets.py # re-render targets with your local fonts
python loop.py examples/pricing-card/target.pngOr bring your own screenshot:
python loop.py ~/Desktop/some-ui.png --iters 10 --threshold 0.99Runs are saved under runs/ with full iteration history. Interrupt anytime, resume with --run-dir.
An "agent" is anything that reads a task from stdin and edits files in its cwd:
| agent | --agent |
|---|---|
| Claude Code | claude -p --dangerously-skip-permissions |
| Cursor CLI | cursor-agent -p |
| OpenAI Codex CLI | codex exec --full-auto - |
| smoke test | cp ../../examples/pricing-card/reference.html index.html |
The smoke test copies the ground-truth HTML and converges at 100% in one iteration: a free end-to-end test of the render/diff/blame pipeline at zero token cost.
Three targets of increasing complexity, each with the reference.html used to render it. The agent never sees this reference file:
 |
 |
 |
pricing-card 800x560 |
login 1080x700 |
dashboard 1280x800 |
~500 lines of Python across two files.
loop.py runs the outer loop in about 160 lines. No planner, no orchestration graph, no conversation memory. All state lives in files in the run directory, which makes every iteration inspectable and any run resumable.
evaluate.py is the evaluator of around 350 lines:
- Render: Playwright at the target's viewport, animations and text caret disabled for deterministic pixels.
- Diff: pixelmatch with anti-aliasing detection so font edges don't poison the score.
- Blame: intersects the mismatch mask with DOM bounding boxes, attributing bad pixels to their owners. Unaccounted-for mismatch is flagged as missing content.
- Verdict: one number is the convergence criterion, the loop's stop condition, and the exit code.
loop.py outer loop: run agent -> evaluate -> feedback -> repeat
evaluate.py render via Playwright -> diff via pixelmatch -> blame -> feedback.md
PROMPT.md task contract every agent iteration receives
make_targets.py re-render examples/*/reference.html -> target.png locally
examples/ three targets: pricing-card, login, dashboard
runs/ one workspace per run, gitignored
- Fonts are machine-specific. System fonts render differently across OSes.
make_targets.pyregenerates targets locally for honest scores. - Static, single viewport. One resolution, no hover states, no interactivity.
- Pixel-perfect != good HTML. The metric measures visual fidelity, not code quality.