Emit run-manifest.json per review (single-file replay ledger) — FID Lottery audit #1

[MTCMarkFranco]

Emit run-manifest.json per review (single-file replay ledger)

Motivation

From the "Production LLM Reliability — Lessons from the FID Lottery" audit (fid-lottery-audit.md): the single biggest observability gap in lambda-rag is that the replay tuple for any given review is scattered across the report, the section-fact sidecars, the ruleset file, git tags, and CLI logs. The report calls this artifact the most important one to build first — "without replay, every reliability incident becomes anecdotal."

Locked Oracle Phase 1 (PR #178) already captures the ingredients (ModelSnapshot, tokens, PromptHash, PromptVersion, LockedOracleSettings.Fingerprint()). What's missing is one file that ties them together per review run.

What to build

Emit run-manifest.json next to every review report. Content-addressed, canonical JSON, one file per review.

Required fields

{
  "manifestVersion": "1.0.0",
  "runId": "<guid or content-hash>",
  "timestampUtc": "2026-07-06T04:12:00Z",
  "engine": {
    "version": "<informational version string>",
    "gitSha": "<git rev-parse HEAD>",
    "assemblyVersion": "<CLI assembly version>"
  },
  "input": {
    "docPath": "<relative>",
    "docHash": "<sha256 of raw bytes>",
    "docKind": "docx|pdf|md",
    "declaredDomain": "architecture-review"
  },
  "ruleset": {
    "path": "<relative>",
    "hash": "<sha256 canonical>",
    "authoredDomain": "architecture-review",
    "ruleCount": 364,
    "promptVersion": "1.1.0"
  },
  "facts": {
    "sidecars": [
      { "sectionPath": "1.2.3", "fingerprint": "<sha256>", "modelSnapshot": "gpt-5.4-mini-2026-03-17", "tokensIn": 812, "tokensOut": 341 }
    ],
    "totalTokensIn": 0,
    "totalTokensOut": 0,
    "extractor": {
      "kind": "FoundrySectionFactExtractor",
      "settings": "<LockedOracleSettings.Fingerprint()>",
      "endpoint": "<host-only, no keys>",
      "deployment": "gpt-5.4-mini"
    }
  },
  "embedder": {
    "kind": "AzureFoundry",
    "model": "text-embedding-3-large",
    "cacheHits": 0,
    "cacheMisses": 0
  },
  "verdicts": {
    "pass": 0, "fail": 0, "gap": 0, "na": 0, "total": 0
  },
  "elapsed": { "totalMs": 0, "factsMs": 0, "evalMs": 0 },
  "refusal": null
}

Definition of done

• RunManifest record in LambdaRag.Core.
• RunManifestWriter emits canonical JSON with stable key ordering.
• ReviewCommand populates + writes run-manifest.json next to report.md.
• SectionFactSidecarIO returns enough metadata to populate the facts.sidecars array (already close — just needs an aggregate accessor).
• Unit test: same inputs → byte-identical run-manifest.json (Pillar 1 compliance).
• Idempotency test: run-manifest.json file itself has stable hash across re-runs (excluding timestampUtc and elapsed.* which are ledger fields, not fingerprint fields).
• docs/DETERMINISM.md cross-references the manifest as the canonical replay tuple.

Scope boundary

• Not a redesign of the report. run-manifest.json is a sibling artifact to report.md.
• Not telemetry aggregation — that is a separate issue (append-only run-telemetry.jsonl).
• Not a new fingerprint contract for reproducibility gating — timestamp/elapsed are ledger-only.

Estimated effort

~1 day. All ingredients exist; this is composition + serialization + a test.