feat: verify local artifacts against persisted report evidence

[seonghobae]

Buyer-visible outcome

RankWeave 0.14.0 adds exact local artifact verification for persisted pairwise and candidate-family v2 reports.

rankweave verify-artifacts \
  --report comparison.json \
  --baseline-run baseline.run \
  --candidate-run candidate.run \
  --qrels qrels.txt

Candidate-family verification uses ordered, repeatable --candidate ID=PATH arguments. Console and python -m rankweave entrypoints emit byte-identical UTF-8 JSON.

Decision and trust contract

• exit 0: every SHA-256 digest and raw byte count matches;
• exit 1: the report is usable but one or more explicit artifacts differ;
• exit 2: usage, filesystem, bounded-read, UTF-8, JSON, evidence-shape, or candidate-order failure.

The output is path-free and payload-free. A match proves only equality with unsigned evidence in the supplied report. It is not producer authentication, signature verification, provenance verification, an attestation, trusted-execution evidence, or a SLSA-level claim.

Implementation and interoperability

• Pure standard-library ArtifactVerificationRecord, ArtifactVerificationReport, and verify_report_artifacts APIs.
• Pairwise v2 verification requires baseline, candidate, and qrels bytes.
• Candidate-family v2 verification requires exact agreement among report order, evidence order, and caller mapping order.
• SHA-256 and byte counts are compared independently.
• Frozen public records reject contradictory public state.
• Persisted reports reject duplicate JSON member names and non-standard NaN or infinity values under the RFC 8259 boundary.
• New strict Draft 2020-12 schema rankweave.artifact-verification.v1 is discoverable through Python and rankweave schema --report-type verification --schema-version v1.
• Runtime remains Python 3.10+, dependency-free, store-agnostic, standalone-usable, and suitable for naruon or another MSA consumer.

Documentation and standards

README, CLI documentation, schema documentation, dedicated verification documentation, AGENTS.md, ARCHITECTURE.md, CLAUDE.md, CHANGELOG.md, and research references are synchronized.

APA 7 references record RFC 8259, NIST FIPS 180-4, and SLSA specification v1.2. The documentation distinguishes exact-byte integrity comparison from authenticated provenance verification.

Exact-head verification

Current head: 5db2d60ec2d41a89859b2740da7a39cfa932b569

The current merge ref passed:

• Python 3.10, 3.11, 3.12, and 3.13;
• Ruff and compileall;
• 539 tests on every Python matrix entry;
• 1,317/1,317 production statements and 376/376 production branches, 100%;
• production docstring enforcement;
• verification core, malformed evidence, hostile public state, strict JSON, and JSON Schema tests;
• wheel build and required-resource inspection;
• isolated installed-wheel Python API smoke;
• installed console/module pairwise and family verification parity;
• installed mismatch exit-1 behavior;
• dependency consistency;
• Security Scan;
• SAST Semgrep;
• CodeRabbit success status;
• zero unresolved current review threads.

All temporary finalization and repair scaffolding has been removed. No tag, GitHub Release, package publication, signature, attestation, or SLSA claim is created before protected merge.

[coderabbitai]

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 21 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a7d5dee9-f8fe-4008-9d5d-718cb8aae0e0

📥 Commits

Reviewing files that changed from the base of the PR and between b7aaf31 and 5db2d60.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (28)

• .github/workflows/ci.yml
• .github/workflows/repair-pr27-cli.yml
• AGENTS.md
• ARCHITECTURE.md
• CHANGELOG.md
• CLAUDE.md
• README.md
• docs/artifact-verification.md
• docs/cli.md
• docs/report-schemas.md
• docs/research/README.md
• docs/superpowers/plans/2026-08-05-report-artifact-verification.md
• docs/superpowers/specs/2026-08-05-report-artifact-verification-design.md
• pyproject.toml
• src/rankweave/__init__.py
• src/rankweave/artifact_verification.py
• src/rankweave/cli.py
• src/rankweave/report_schemas.py
• src/rankweave/schemas/artifact-verification-v1.schema.json
• tests/test_artifact_verification.py
• tests/test_artifact_verification_defensive.py
• tests/test_report_schemas.py
• tests/test_schema_cli.py
• tests/test_strict_verification_json.py
• tests/test_verification_projection.py
• tests/test_verification_schema.py
• tests/test_verify_artifacts_cli.py
• tests/test_version.py

---

_{Comment @coderabbitai help to get the list of available commands.}