Founder of trapstreet.run — a public eval platform helping people find the best AI solutions through evidence.
Pick a task. Run it locally. Submit your score.
Most AI benchmarks measure a model. I care about the whole thing you actually ship — the model plus the prompt, the scaffold, the tools, the retries. So I'm building a place where you point that whole thing at a real task, run it on your own machine, and put the score on a public board.
Every number on a board is a real run, not a claim.
🧱 The platform — @trapstreet
|
|
|
|
|
The open tasks themselves — inputs, expected outputs, and a judge. Legal contract extraction, debugging a payout pipeline, camera-trap species ID, an agent playing Minecraft for a diamond. |
|
|
Agent skills, so your coding agent can do the setup for you — install the CLI, build a solution against a task, or author a new task. |
|
|
A DeepSeek Harness plugin — check which plugins actually loaded, and look up public boards without leaving your session. |
Point your coding agent at it — installs into Claude Code, Cursor, Codex, Cline, Amp, Antigravity and 70+ more:
npx skills add trapstreet/trapstreet-skillsThen just say:
| Say this | What happens |
|---|---|
| "set up trapstreet" | Installs and authorizes the tp CLI. One time — local scoring needs no account. |
| "build a solution for <task>" | Writes trap.yaml and the solver, runs it locally, submits when you're happy. From scratch, around code you have, or by adapting someone else's repo. |
| "make a task that evaluates <X>" | Interviews you on what counts as correct and where ground truth comes from, writes the judge and grader, then calibrates until the scores mean something. |
Prefer the CLI directly?
uv tool install trap-cli # the command is `tp`
tp auth login # authorize this machine, once
tp run && tp submit # from any directory with a trap.yamlTasks live in their author's own repository, not mine — publish from anywhere public and register it on the site.
I co-host OpenSourceIRL — open-source drinks in London, with OpenCues. No talks, no set demos, no badges. Bring a laptop if you want to show something; nobody minds if you don't.
| v1.2.0 | 31 Jul 2026 · London · 92 guests | with Supabase |
| v1.0.0 | 10 Jul 2026 · Soho, London · 79 guests | with Shipyard |
Also co-hosted Monologue #3 — Coding Harness (12 Jun 2026, Shoreditch, 35 guests) — a session on agentic dev environments, eval frameworks and orchestration patterns, with Monologue AI+HCI.
Next one goes out on the OpenSourceIRL calendar — subscribe there.
Building something you want on a board? Come say hi in Discord.