study(capture-coverage): closed-loop eval of recon claims vs independent geometry

[obvious-autobuild]

Capture-coverage study: closed-loop evaluation against independent geometry

What this adds

experiments/capture-coverage/ — a closed-loop study that drives the real recon worker with simulated capture bundles and audits its scene claims against an independent, analytically computed visibility truth:

• sim.py / scenes.py — procedural worlds (walls, openings, occluders, ground) and a deterministic capture simulator (rendered depth, reported poses, confidence, noise).
• reference.py — independent analytic visibility oracle (depth-tested ray casting, two-position evidence bar).
• judge.py — audits the worker's claims: CLEAR_UNSEEN, EVIDENCE_DEFICIT, UNSOUND_PASS, with an uncertainty summary that reports unobserved wall/ground samples as unknown, never observed.
• runner.py / run.py — 18-scenario frozen-manifest grid with replay witnesses (commit, input bundle hash, result hash over canonical judgement content).
• 12 reference-truth tests; runs gitignored.

Results (commit cdc743c)

Full grid completed DONE:0. Highlights:

• Judge never certifies an unobserved region. Degraded inputs surface as EVIDENCE_DEFICIT (baseline already shows 1 — the worker's own evidence check) or CLEAR_UNSEEN (frames-skip claims clear overheads where no frame could see).
• drift-cross-track produces 10 EVIDENCE_DEFICIT — cross-track drift walks the views off the claimed spans.
• one-frame-depth: the worker refuses loudly before claiming anything — no overclaim, though the preregistered single-view audit path was not exercised (the worker cannot produce a scene from one frame at all).
• Replay verification: drift-cross-track and frames-skip witnesses reproduced byte-identically.

Known open item

One unreproduced worker output from the first grid run is preserved under fixtures/repro-anomaly/ and documented in the README: recon produced a different scene on a byte-identical bundle once; it is deterministic under repeated runs, different thread counts, and stale-output reruns in the current environment, but the cause is unexplained. Replays report mismatch rather than papering over it.

Not done

No merge, no deploy, no workflow/settings changes. Runs artifacts stay out of the repo.

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 67bde2d1-a5ab-4d08-bcca-135fa2a7fe47

---

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}