chore(eval): bring the mapping evaluation harness into the repo

[murdore]

Five scripts that score the project mapper against the human labels in videos/labels.json instead of running it. They were the only measurement tooling this project had, and they lived in videos/ — which is gitignored, so a single git clean -x would have taken all of them.

Script	Measures
npm run eval:judge	End-to-end P/R/F1 with the real judge, held-out folds
npm run eval:facets	Derives project facets from confirmed videos (k-means → one sentence per cluster)
npm run eval:portfolio	Prefilter reach and threshold fit — the ceiling the judge works under
npm run eval:triage	Whether the triage gate discards videos the mapper wanted
npm run eval:relabel	The human’s self-agreement, by re-serving old videos blind

These are not tests, and are not wired into CI

The suite in src/__tests__ injects fake judge/embed functions and never touches the network, which is what keeps it fast and deterministic. These do the opposite on purpose — they call the real model, because the question they answer is how good the real judge is. A full eval:judge arm is roughly 165 paid Vertex calls and needs four gitignored private inputs (search.db, labels.json, triage.json, projects.json). A fresh clone cannot run them, and that is the intended state rather than a gap to close.

What changed in the port

Only the repo root. Each script hardcoded an absolute path to one developer’s checkout; that is now path.resolve(import.meta.dirname, ".."). Logic is otherwise untouched — including the parts biome dislikes — so the diff against the originals stays reviewable. (biome check . already fails on main, and lint-staged only runs biome format, so this introduces no new gate failure.)

Why it matters beyond durability

docs/eval-harness.md records the two measured cases where adding facets raised prefilter reach while lowering end-to-end F1 — most starkly, auto-derived Breeze facets took its false positives from 1 to 17, the worst of any project in that session. That caution previously existed only as a comment inside a gitignored file.

Verification: npm run build and the full suite (55 files, 476 tests) pass via the pre-commit hook. eval:relabel (the one harness making no paid calls) runs correctly post-port — 30 videos, 77 candidate decisions. Every ${R}-relative path resolves. markdownlint clean on the new doc; the README edit adds 0 new errors. Scanned for absolute paths and credentials — clean.

[github-actions]

Single Commit Policy — COMPLIANT

Status: 1 commit | Valid format | No merge commits | Ready for merge

Commit details

• Hash: d0fee17f88771951668fdc9593b7626793bc0795
• Message: chore(eval): bring the mapping evaluation harness into the repo
• Author: Sachin Sharma

---

Automated validation — dopamine single-commit enforcement