test(recon): offline reconstruction-repeatability harness with committed replay evidence

[obvious-autobuild]

What

An offline repeatability harness for Hunter's recon at PR #220's frozen snapshot (experiments/reconstruction-repeatability/ only; no other files touched).

• Ten frozen cases in manifest.json: identity (repeat, warm cache, reordered frames, re-encoded JPEGs, unrelated history), guarded history (pose edit after a warm cache, interrupted model run), sensitivity (intrinsics shift, measured not graded), and the history-dependent wall (same-size cache edit with key.json untouched; duplicate keyframe id refused).
• No weights, no GPU, no network. MoGe-2 is replaced by an analytic stub (deterministic in the frame id) and recon.depth.rescale by an identity pass; the real moge_cache key path, cache files, and the full pipeline run as-is.
• run.py --manifest executes all cases and commits records to results/; run.py --replay re-executes and compares byte-for-byte against the committed record.

Findings (committed as evidence)

Result	Cases
pass — outputs byte-identical	baseline-repeat, warm-cache, unrelated-history, interrupted-run
pass — JSONs identical, mesh digest moves	image-reencoded (vertex colors carry the re-encoded image bytes)
pass — guarded history lands back on fresh	stale-cache-pose-edit, interrupted-run
refused	frame-id-duplicate (DuplicateFrameId)
fail — the finding	same-size-cache-edit: a cached map overwritten in place (same shape/dtype, key untouched) reroutes the pipeline and lands 0.5 ft off the fresh result; a third run on the tampered cache reproduces the wrong answer — a stable, history-dependent wall.
measured	intrinsics-shift (key changes, no stale reuse)

Replay agrees 10 of 10 cases.

Checks

uv run --project experiments/reconstruction-repeatability pytest experiments/reconstruction-repeatability/tests -q
# 10 passed
uv run --project experiments/reconstruction-repeatability ruff check experiments/reconstruction-repeatability
# All checks passed
uv run --project experiments/reconstruction-repeatability python experiments/reconstruction-repeatability/run.py --replay experiments/reconstruction-repeatability/results
# replay agreed 10 of 10

Not run

• Live MoGe-2 (weights/GPU) — the stub stands in; invariance of a real model to recompression is not claimed.
• Nothing merged, nothing deployed; draft PR into obv/basescanning-004 only.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>

Handoff

Complete. Replay is a gate: any witness record that fails to reproduce byte-for-byte makes run.py --replay exit 1 and name the case; manifest-digest drift is refused; and case membership is checked exactly and completely before any per-case comparison — the committed and fresh name lists must be equal and duplicate identities are refused, closing the truncation where a deleted witness passed as 9-of-9 or an appended forgery as 10-of-10. Five controls drive the actual CLI as a subprocess — clean witnesses replay to exit 0; a deliberately mutated cached result (a wall-metric figure rewritten inside the baseline witness record) replays to exit 1 with the case DIFFER-marked; digest drift, witness deletion, and witness forgery are each refused with the offending case named.

Remaining: live MoGe-2 (weights/GPU) is unrun by design; the same-size cache-edit finding (0.5 ft, stable when repeated) needs a product decision on cache poisoning.

Review: flagged for Hunter — the judgment calls to check are the mutation point chosen for the negative control (wall_metric_runs[0].ground_height_ft inside the baseline witness record) and whether the 0.5 ft same-size-cache-edit behavior should be graded fail (current) or reclassified as a measured sensitivity once a fix lands.

Checks (from repo root):

uv run --project experiments/reconstruction-repeatability pytest experiments/reconstruction-repeatability/tests -q
# 15 passed in ~2 min — 10 witnesses + 5 replay-gate controls (each control replays the full manifest)

uv run --project experiments/reconstruction-repeatability ruff check experiments/reconstruction-repeatability
# All checks passed

uv run --project experiments/reconstruction-repeatability python experiments/reconstruction-repeatability/run.py --replay experiments/reconstruction-repeatability/results
# replay agreed on 10 of 10 cases — exit 0; a mutated, deleted, or forged witness exits 1 naming the case

Resume command (fresh checkout, branch obv/products-l2-recon-repeatability-20261010, base obv/basescanning-004):

uv run --project experiments/reconstruction-repeatability python experiments/reconstruction-repeatability/run.py --replay experiments/reconstruction-repeatability/results

🔗 Obvious Project · 🧵 Obvious Thread

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: bffc2c6b-b68e-4686-83e7-81f762bc4d5b

---

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}

[obvious-autobuild]

Flagging for Hunter's review — the handoff in the PR body names the two judgment calls: the negative control's mutation point (wall_metric_runs[0].ground_height_ft in the baseline witness record) and whether the 0.5 ft same-size-cache-edit stays graded fail or becomes a measured sensitivity once a fix lands. Replay is now a gate: any witness disagreement exits nonzero.