-
Notifications
You must be signed in to change notification settings - Fork 45
feat(diff): add vouch diff for claim/page revisions
#86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| # vouch diff — claim/page revision diff | ||
|
|
||
| ## Problem | ||
|
|
||
| When a claim is superseded or a page is revised, there is no way to see *what | ||
| changed* between the old and new artifact. You can `show` each one and compare | ||
| by eye, but nothing renders the delta. ROADMAP 0.1 lists `vouch diff | ||
| <id-old> <id-new>` for exactly this. | ||
|
|
||
| ## Goal | ||
|
|
||
| `vouch diff <id-old> <id-new>` shows what changed between two claim revisions | ||
| or two page revisions — field-level changes plus a line-diff of the long text | ||
| field. Read-only: no writes, no proposals, no audit events. | ||
|
|
||
| ## Decisions | ||
|
|
||
| - **Auto-detect kind.** Resolve both ids as claims; if that fails, resolve both | ||
| as pages. Mismatched kinds or unknown ids are a clear error — no `--kind` | ||
| flag. | ||
| - **Semantic fields only.** Diff the fields that carry meaning; hide | ||
| always-churning metadata (`id`, `created_at`, `updated_at`, | ||
| `last_confirmed_at`, `approved_by`). | ||
| - **Line-diff the long text.** `claim.text` / `page.body` render as a | ||
| `difflib` unified diff; everything else as `field: old → new`. | ||
| - **CLI-only.** Read-only inspection; does not touch the `kb.*` capability set. | ||
|
|
||
| ## Components — `src/vouch/diff.py` | ||
|
|
||
| ### `DiffError(Exception)` | ||
| Raised for unknown ids and mismatched kinds. | ||
|
|
||
| ### `FieldChange` (dataclass) | ||
| `field: str, old, new` — one changed scalar/list field. | ||
|
|
||
| ### `ArtifactDiff` (dataclass) | ||
| `kind: str, old_id: str, new_id: str, changes: list[FieldChange], | ||
| text_diff: list[str]`. | ||
|
|
||
| ### `diff_artifacts(store, old_id, new_id) -> ArtifactDiff` | ||
| - **Kind resolution:** try `store.get_claim` on both ids → both succeed ⇒ | ||
| `kind="claim"`. Otherwise try `store.get_page` on both → `kind="page"`. If an | ||
| id resolves to neither, raise `DiffError("unknown artifact: <id>")`. If one is | ||
| a claim and the other a page, raise | ||
| `DiffError("cannot diff claim against page")`. | ||
| - **changes:** for each semantic field whose value differs, append a | ||
| `FieldChange`. The long text field is handled separately (not in `changes`). | ||
| - **text_diff:** `list(difflib.unified_diff(old_text.splitlines(), | ||
| new_text.splitlines(), lineterm=""))` for `claim.text` / `page.body`; empty | ||
| when unchanged. | ||
|
|
||
| Field sets (long text field rendered as `text_diff`, the rest as changes): | ||
| - **Claim** — text *(diff)*; type, status, confidence, evidence, entities, | ||
| tags, supersedes, superseded_by, contradicts, scope. | ||
| - **Page** — body *(diff)*; title, type, status, claims, entities, sources, | ||
| tags. | ||
|
|
||
| ## CLI — `vouch diff OLD NEW [--json]` | ||
|
|
||
| Follows existing patterns (`_load_store`, `_cli_errors`, `_emit_json`). | ||
|
|
||
| Human output: | ||
| ``` | ||
| diff claim <old> → <new> | ||
| status: working → stable | ||
| confidence: 0.7 → 0.9 | ||
| evidence: ['s1'] → ['s1', 's2'] | ||
| text: | ||
| --- a | ||
| +++ b | ||
| -old wording | ||
| +new wording | ||
| ``` | ||
| - `--json` → `_emit_json` of the `ArtifactDiff` as a dict. | ||
| - No differences → prints `no differences`. | ||
|
|
||
| ## Error handling | ||
|
|
||
| - Unknown id (neither claim nor page) → `DiffError` → clean CLI `Error:` line. | ||
| - Mismatched kinds → `DiffError` → clean CLI `Error:` line. | ||
|
|
||
| ## Testing (TDD) | ||
|
|
||
| - `diff_artifacts`: two claims differing in status/confidence → matching | ||
| `FieldChange`s; a text change → `text_diff` contains `-`/`+` lines; identical | ||
| claims → empty `changes` and `text_diff`; two pages differing in title/body; | ||
| unknown id → `DiffError`; claim-vs-page → `DiffError`. | ||
| - CLI: `vouch diff a b` prints the changed fields; `--json` emits a dict with | ||
| `kind`/`changes`; unknown id → clean `Error:`; identical → `no differences`. | ||
|
|
||
| ## Non-goals | ||
|
|
||
| - Following supersede chains automatically (caller passes both ids). | ||
| - Diffing entities/relations/sources (claims and pages only, per ROADMAP). | ||
| - MCP/JSONL parity (`kb.*` surface unchanged). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| """Claim/page revision diff — `vouch diff <id-old> <id-new>`. | ||
|
|
||
| Read-only: shows what changed between two claim revisions or two page | ||
| revisions. Field-level changes for scalars/lists, plus a line-diff of the long | ||
| text field (`claim.text` / `page.body`). No writes, no proposals, no audit. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import difflib | ||
| from dataclasses import dataclass | ||
| from enum import Enum | ||
| from typing import Any | ||
|
|
||
| from .models import Claim, Page | ||
| from .storage import ArtifactNotFoundError, KBStore | ||
|
|
||
| # (long-text field rendered as a line diff, scalar/list fields shown as old→new) | ||
| _CLAIM_TEXT = "text" | ||
| _CLAIM_FIELDS = [ | ||
| "type", "status", "confidence", "evidence", "entities", "tags", | ||
| "supersedes", "superseded_by", "contradicts", "scope", | ||
| ] | ||
| _PAGE_TEXT = "body" | ||
| _PAGE_FIELDS = ["title", "type", "status", "claims", "entities", "sources", "tags"] | ||
|
|
||
|
|
||
| class DiffError(ValueError): | ||
| """Raised for unknown ids or mismatched artifact kinds. | ||
|
|
||
| Subclasses ValueError so the CLI's `_cli_errors()` renders it as a clean | ||
| `Error:` line instead of a traceback. | ||
| """ | ||
|
|
||
|
|
||
| @dataclass | ||
| class FieldChange: | ||
| field: str | ||
| old: Any | ||
| new: Any | ||
|
|
||
|
|
||
| @dataclass | ||
| class ArtifactDiff: | ||
| kind: str | ||
| old_id: str | ||
| new_id: str | ||
| changes: list[FieldChange] | ||
| text_diff: list[str] | ||
|
|
||
|
|
||
| def _kind_of(store: KBStore, artifact_id: str) -> str | None: | ||
| try: | ||
| store.get_claim(artifact_id) | ||
| return "claim" | ||
| except ArtifactNotFoundError: | ||
| pass | ||
| try: | ||
| store.get_page(artifact_id) | ||
| return "page" | ||
| except ArtifactNotFoundError: | ||
| return None | ||
|
|
||
|
|
||
| def _norm(value: Any) -> Any: | ||
| return value.value if isinstance(value, Enum) else value | ||
|
|
||
|
|
||
| def _line_diff(old: str, new: str) -> list[str]: | ||
| return list(difflib.unified_diff( | ||
| old.splitlines(), new.splitlines(), lineterm="", | ||
| )) | ||
|
|
||
|
|
||
| def diff_artifacts(store: KBStore, old_id: str, new_id: str) -> ArtifactDiff: | ||
| """Diff two same-kind artifacts (both claims or both pages) by id.""" | ||
| old_kind = _kind_of(store, old_id) | ||
| if old_kind is None: | ||
| raise DiffError(f"unknown artifact: {old_id}") | ||
| new_kind = _kind_of(store, new_id) | ||
| if new_kind is None: | ||
| raise DiffError(f"unknown artifact: {new_id}") | ||
| if old_kind != new_kind: | ||
| raise DiffError(f"cannot diff {old_kind} against {new_kind}") | ||
|
|
||
| old: Claim | Page | ||
| new: Claim | Page | ||
| if old_kind == "claim": | ||
| old, new = store.get_claim(old_id), store.get_claim(new_id) | ||
| fields, text_field = _CLAIM_FIELDS, _CLAIM_TEXT | ||
| else: | ||
| old, new = store.get_page(old_id), store.get_page(new_id) | ||
| fields, text_field = _PAGE_FIELDS, _PAGE_TEXT | ||
|
|
||
| changes: list[FieldChange] = [] | ||
| for field in fields: | ||
| o, n = _norm(getattr(old, field)), _norm(getattr(new, field)) | ||
| if o != n: | ||
| changes.append(FieldChange(field=field, old=o, new=n)) | ||
|
|
||
| text_diff = _line_diff(getattr(old, text_field), getattr(new, text_field)) | ||
| return ArtifactDiff( | ||
| kind=old_kind, old_id=old_id, new_id=new_id, | ||
| changes=changes, text_diff=text_diff, | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| """Claim/page revision diff — `vouch diff`.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from pathlib import Path | ||
|
|
||
| import pytest | ||
| from click.testing import CliRunner | ||
|
|
||
| from vouch.cli import cli | ||
| from vouch.diff import ArtifactDiff, DiffError, diff_artifacts | ||
| from vouch.models import Claim, ClaimStatus, Page | ||
| from vouch.storage import KBStore | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def store(tmp_path: Path) -> KBStore: | ||
| return KBStore.init(tmp_path) | ||
|
|
||
|
|
||
| def _claim(store: KBStore, cid: str, **kw: object) -> Claim: | ||
| src = store.put_source(b"e") | ||
| fields = {"id": cid, "text": "t", "evidence": [src.id]} | ||
| fields.update(kw) | ||
| return store.put_claim(Claim(**fields)) # type: ignore[arg-type] | ||
|
|
||
|
|
||
| # --- diff_artifacts ------------------------------------------------------- | ||
|
|
||
|
|
||
| def test_diff_claims_reports_changed_scalar_fields(store: KBStore) -> None: | ||
| _claim(store, "c1", status=ClaimStatus.WORKING, confidence=0.7) | ||
| _claim(store, "c2", status=ClaimStatus.STABLE, confidence=0.9) | ||
| d = diff_artifacts(store, "c1", "c2") | ||
| assert isinstance(d, ArtifactDiff) | ||
| assert d.kind == "claim" | ||
| changed = {c.field: (c.old, c.new) for c in d.changes} | ||
| assert changed["status"] == ("working", "stable") | ||
| assert changed["confidence"] == (0.7, 0.9) | ||
|
|
||
|
|
||
| def test_diff_claims_text_change_produces_line_diff(store: KBStore) -> None: | ||
| _claim(store, "c1", text="the old wording") | ||
| _claim(store, "c2", text="the new wording") | ||
| d = diff_artifacts(store, "c1", "c2") | ||
| assert any(line.startswith("-the old wording") for line in d.text_diff) | ||
| assert any(line.startswith("+the new wording") for line in d.text_diff) | ||
| # text is rendered as a diff, not as a scalar FieldChange | ||
| assert "text" not in {c.field for c in d.changes} | ||
|
|
||
|
|
||
| def test_diff_identical_claims_has_no_changes(store: KBStore) -> None: | ||
| _claim(store, "c1", text="same", status=ClaimStatus.STABLE) | ||
| _claim(store, "c2", text="same", status=ClaimStatus.STABLE) | ||
| d = diff_artifacts(store, "c1", "c2") | ||
| assert d.changes == [] | ||
| assert d.text_diff == [] | ||
|
|
||
|
|
||
| def test_diff_pages_reports_title_and_body(store: KBStore) -> None: | ||
| store.put_page(Page(id="p1", title="Old", body="line one")) | ||
| store.put_page(Page(id="p2", title="New", body="line two")) | ||
| d = diff_artifacts(store, "p1", "p2") | ||
| assert d.kind == "page" | ||
| changed = {c.field: (c.old, c.new) for c in d.changes} | ||
| assert changed["title"] == ("Old", "New") | ||
| assert any(line.startswith("+line two") for line in d.text_diff) | ||
|
|
||
|
|
||
| def test_diff_unknown_id_raises(store: KBStore) -> None: | ||
| _claim(store, "c1") | ||
| with pytest.raises(DiffError, match="unknown artifact: nope"): | ||
| diff_artifacts(store, "c1", "nope") | ||
|
|
||
|
|
||
| def test_diff_mismatched_kinds_raises(store: KBStore) -> None: | ||
| _claim(store, "c1") | ||
| store.put_page(Page(id="p1", title="P", body="b")) | ||
| with pytest.raises(DiffError, match="cannot diff"): | ||
| diff_artifacts(store, "c1", "p1") | ||
|
|
||
|
|
||
| # --- CLI ------------------------------------------------------------------ | ||
|
|
||
|
|
||
| def test_cli_diff_prints_changed_fields( | ||
| store: KBStore, monkeypatch: pytest.MonkeyPatch, | ||
| ) -> None: | ||
| monkeypatch.chdir(store.root) | ||
| _claim(store, "c1", text="old", status=ClaimStatus.WORKING) | ||
| _claim(store, "c2", text="new", status=ClaimStatus.STABLE) | ||
| res = CliRunner().invoke(cli, ["diff", "c1", "c2"]) | ||
| assert res.exit_code == 0, res.output | ||
| assert "status: working" in res.output and "stable" in res.output | ||
| assert "-old" in res.output and "+new" in res.output | ||
|
|
||
|
|
||
| def test_cli_diff_json(store: KBStore, monkeypatch: pytest.MonkeyPatch) -> None: | ||
| monkeypatch.chdir(store.root) | ||
| _claim(store, "c1", status=ClaimStatus.WORKING) | ||
| _claim(store, "c2", status=ClaimStatus.STABLE) | ||
| res = CliRunner().invoke(cli, ["diff", "c1", "c2", "--json"]) | ||
| assert res.exit_code == 0, res.output | ||
| import json | ||
| payload = json.loads(res.output) | ||
| assert payload["kind"] == "claim" | ||
| assert any(c["field"] == "status" for c in payload["changes"]) | ||
|
|
||
|
|
||
| def test_cli_diff_identical_says_no_differences( | ||
| store: KBStore, monkeypatch: pytest.MonkeyPatch, | ||
| ) -> None: | ||
| monkeypatch.chdir(store.root) | ||
| _claim(store, "c1", text="same") | ||
| _claim(store, "c2", text="same") | ||
| res = CliRunner().invoke(cli, ["diff", "c1", "c2"]) | ||
| assert res.exit_code == 0, res.output | ||
| assert "no differences" in res.output | ||
|
|
||
|
|
||
| def test_cli_diff_unknown_id_clean_error( | ||
| store: KBStore, monkeypatch: pytest.MonkeyPatch, | ||
| ) -> None: | ||
| monkeypatch.chdir(store.root) | ||
| _claim(store, "c1") | ||
| res = CliRunner().invoke(cli, ["diff", "c1", "nope"]) | ||
| assert res.exit_code != 0 | ||
| assert "Traceback" not in res.output | ||
| assert "unknown artifact: nope" in res.output |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a language tag to the fenced code block.
Line 63 starts a fenced block without a language, which triggers markdownlint MD040.
🧩 Suggested fix
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
@docs/superpowers/specs/2026-05-25-vouch-diff-design.mdaround lines 63 - 73,The fenced code block showing the diff (the triple-backtick block containing
"diff claim → " and the lines starting with status:, confidence:,
evidence:, text:, --- a, +++ b, etc.) is missing a language tag and triggers
markdownlint MD040; update the opening fence from
totext (or anotherappropriate language like diff) so the block becomes ```text diff claim →
... to satisfy MD040 and preserve the displayed content.