Fix/required field validators

CHANGELOG.md

@@ -6,6 +6,32 @@ All notable changes to vouch are documented here. Format follows
 
 ## [Unreleased]
 
+### Added
+- `vouch fsck` performs deep consistency checks beyond `vouch doctor`:
+  orphaned embeddings, dangling supersede/contradict chains, decided
+  proposals whose artifact is missing, and FTS5 index-vs-file drift
+  (orphan rows, missing rows, status drift). Read-only; reports findings
+  with object ids. `--fix` is intentionally out of scope (#96).
+- `vouch migrate` checks, dry-runs, and applies on-disk KB format migrations,
+  preserving audit history and rebuilding derived indexes after successful
+  upgrades.
+- `vouch expire` garbage-collects stale pending proposals: dry-run by default,
+  `--apply` moves them to `decided/` with `decision_reason: expired`, emits
+  `proposal.expire` audit events, and honors `review.expire_pending_after_days`
+  in `config.yaml` (default 90; `0` disables). `kb.expire` on MCP/JSONL.
+- `vouch init --template <name>` seeds a domain starter pack. The default `starter` template is unchanged; the new `gittensor` template seeds a small, cited, approved KB about Gittensor (SN74) contribution scoring (1 source, 1 entity, 7 claims — merged-PR rewards, PAT verification, scoring factors, sybil-resistance, repo allow-list policy, issue-solving multiplier, and emission split) so a fresh KB in a Gittensor repo has retrievable context on day one. Templates are an in-code registry — future packs plug in the same way.
+- Structured JSON logging via `VOUCH_LOG_FORMAT=json`. When set, the
+  `vouch` logger emits one JSON object per line with `level`, `logger`,
+  `event`, and any structured extras (e.g. `actor`, `object_ids`) passed
+  through stdlib `extra=`. Unset (or any other value) keeps the existing
+  human-readable format — no behaviour change beyond formatting. Wired
+  into the CLI, MCP server, and JSONL server entry points. `VOUCH_LOG_FORMAT`
+  was already documented in `ROADMAP.md` and `adapters/generic-mcp/README.md`
+  but had no implementation (#97).
+
+### Fixed
+- `discover_root()` now honours `VOUCH_KB_PATH=/abs/path/.vouch` and returns the parent root, instead of always walking up from cwd. The env var was already documented in `adapters/generic-mcp/README.md` but wasn't wired into the code — closing the doc-vs-code drift removes the `"cwd": "..."` ceremony hosts like Claude Desktop need today to point at a specific KB.
+
 ## [0.1.0] — 2026-05-26
 
 ### Packaging
@@ -15,20 +41,62 @@ All notable changes to vouch are documented here. Format follows
   Trusted Publishing (OIDC).
 
 ### Added
+- `vouch approve <id1> <id2> …` approves multiple proposals in one
+  non-interactive call for CI and backlog clearing (#93). Default is
+  all-or-nothing: every id is validated as an approvable pending proposal
+  before any is written, so a typo or already-decided id aborts the batch
+  without approving anything. `--keep-going` switches to best-effort
+  (approve what you can, report the rest, exit non-zero on partial failure).
+  One audit event is still recorded per approved artifact. Complements the
+  interactive `vouch review` queue.
+- Friendlier CLI output (#54, track 2): colourised `vouch status` / `lint` /
+  `doctor` / `search` (honours `NO_COLOR`, `FORCE_COLOR`, and TTY detection);
+  `--json` on `vouch lint` and `vouch search` for machine-readable output
+  while the default stays human-readable; progress callbacks on the long ops
+  (`rebuild_index`, `doctor`, bundle `export`/`import_apply`) surfaced as
+  status lines on interactive terminals; and `vouch index` / `vouch export`
+  now report a clean `Error:` instead of a traceback on a malformed artifact.
+- `vouch sync-check` and `vouch sync-apply` reconcile another `.vouch`
+  directory or bundle by importing only non-conflicting durable artifacts and
+  reporting conflicts without overwriting reviewed knowledge.
+- `vouch pending --json` emits pending proposals as structured JSON for shell
+  scripts, CI checks, and multi-agent review dashboards.
+- `vouch diff <id-old> <id-new>` shows what changed between two claim revisions or two page revisions — field-level changes plus a line-diff of the long text/body. Auto-detects the artifact kind and hides always-churning metadata. Read-only; supports `--json`.
 - Seed a cited starter source and claim during `vouch init`, print first-run
   next steps, and document a 30-second onboarding tour (#54).
+- Add `vouch review`, a guided CLI queue for approving, rejecting, skipping,
+  or dry-running pending proposals without bypassing the review gate.
 
 ### Fixed
+- `bundle.import_check` / `import_apply` and `sync.sync_check` / `sync_apply` now enforce the Source content-addressing invariant: a `sources/<sha>/content` member must hash to `<sha>`, and a `sources/<sha>/meta.yaml` must carry a matching `id`/`hash`. Previously the import side trusted the bundle's directory layout, so a manifest-consistent bundle could land a Source whose content did not match its claimed id — `verify_source` would report `stored_ok=False` only after the import had already succeeded with a clean `bundle.import` audit event. The per-file sha256 gate (#74) only proves bytes match the manifest; this closes the write-time counterpart of the `verify.verify_source` detection.
 - Add `put_relation_idempotent()` to `KBStore` and use it in `supersede()` and `contradict()` so retrying after a partial failure converges to a consistent state instead of raising `ValueError`.
 - Raise `ProposalError("forbidden_self_approval")` in `proposals.approve()` when `approved_by == proposal.proposed_by`, enforcing the review-gate guarantee documented in the README and CONTRIBUTING.
 - `crystallize()` now sets `review.approver_role: trusted-agent` context so single-agent sessions can be crystallized without hitting the `forbidden_self_approval` guard (#47).
+- Narrow `except Exception` to `except ArtifactNotFoundError` in `propose_claim()` evidence validation so I/O and parse errors propagate with their original type instead of being masked as `unknown source/evidence id` (#48).
 - Bundle import rejects tar members whose path escapes `kb_dir`
   (CVE-2007-4559, #9). Previously a crafted `.tar.gz` with a member
   named `../../evil.txt` could write outside `.vouch/`; the manifest
   allow-list did not prevent this because the manifest lives inside
   the same tarball. `import_apply`, `import_check`, and `export_check`
   now validate every member path and raise on unsafe names.
 - Fix `vouch search` CLI: assign backend label per code path so substring fallback results are no longer mislabelled as `fts5`; update stale docstring to reflect multi-backend search surface (#52).
+- Close the review-gate bypass in `sessions.crystallize` (#76). The
+  durable session-summary page wrote `sess.task`, `sess.note`, and
+  `sess.agent` verbatim into rendered markdown, letting an agent
+  land arbitrary content into `pages/` by calling
+  `kb.session_start(task=...)` and getting any one claim approved
+  via crystallize. The summary body now contains only fields the
+  proposing agent cannot influence (session id, server-clock
+  timestamps, list of approved artifact ids). The
+  `session.crystallize` audit event now also includes the summary
+  page id in `object_ids` when a page is written, so `vouch audit`
+  truthfully attributes the write.
+- `context._retrieve` now honors `retrieval.backend` in `config.yaml`
+  instead of always running embeddings first (#92). Accepts `auto`
+  (default — embedding → FTS5 → substring), `embedding`, `fts5`, or
+  `substring`; a legacy `retrieval.backends` list is still read for
+  back-compat. `vouch init` now writes `retrieval.backend: auto`, and the
+  README/ROADMAP describe the actual behavior.
 - `vouch crystallize` now indexes its session-summary page into FTS5 so it
   surfaces from `vouch search` / `kb.search` / `kb.context` without a
   `vouch index` rebuild. Previously the summary was written via
@@ -42,6 +110,18 @@ All notable changes to vouch are documented here. Format follows
   a silent no-op. Existing Linux/macOS bundles are unchanged (their paths
   were already POSIX); Windows bundles produced before this fix should be
   re-exported.
+- `kb.context` no longer returns claims whose status is `archived`,
+  `superseded`, or `redacted` (#78). Two compounding bugs were combining
+  to leak retracted knowledge back to agents: `build_context_pack` had
+  no status filter, and `store.update_claim` only refreshed the
+  embedding cache without keeping `claims_fts.status` in sync — so even
+  after `lifecycle.archive` / `supersede` / `contradict`, the FTS5 row
+  kept its first-index status and `kb.search` / `kb.context` matched the
+  retracted claim. `update_claim` now re-indexes the FTS5 row (mirroring
+  what `proposals.approve` does on first index), and
+  `build_context_pack` drops retracted claims from the assembled pack.
+  `CONTESTED` claims continue to surface so contradictions remain
+  visible.
 - `bundle.import_check` and `bundle.import_apply` now verify each tar
   member's `sha256` against `manifest.json` (#74). Previously the
   per-file hash was only enforced by `export_check`; the import side
@@ -53,6 +133,26 @@ All notable changes to vouch are documented here. Format follows
   with between `import_check` and the apply re-open is rejected
   before anything reaches disk and the audit log does not record
   a `bundle.import` event.
+- `Claim.evidence` now enforces "at least one citation" at the model
+  layer via a `@field_validator` (#81). Previously the
+  README-documented guarantee ("Claims must cite sources … a claim
+  without at least one Source/Evidence id is a validation error")
+  was enforced only in `proposals.propose_claim`, so every other
+  write path — direct `store.put_claim`, `store.update_claim`, and
+  `bundle.import_apply` via `_validate_content` — silently accepted
+  `evidence: []` and landed an uncited claim. The validator closes
+  all three paths at once; `store.update_claim` additionally
+  re-validates via `Claim.model_validate(...)` before persisting so
+  in-place mutation (`c.evidence = []; store.update_claim(c)`)
+  also raises before the YAML hits disk. **Migration note:** because
+  the validator also fires when claims are read back, a KB that
+  already has an uncited `claims/<id>.yaml` on disk from before this
+  fix would otherwise crash `vouch lint` / `vouch doctor` with a
+  `pydantic.ValidationError`. `vouch lint` now iterates `claims/`
+  per-file and surfaces unparseable / uncited YAMLs as
+  `invalid_claim` findings ("edit the YAML to add a citation, or
+  delete the file") instead of bailing out — so existing KBs get a
+  clean repair list rather than a traceback.
 
 ## [0.0.1] — 2026-05-17
 

README.md

@@ -62,7 +62,9 @@ vouch status                     # one-line summary
 vouch pending                    # list pending proposals
 vouch show <id>                  # full details
 vouch approve <id>               # → durable artifact
+vouch approve <id> <id> ...    # approve several reviewed proposals at once
 vouch reject <id> --reason "..."
+vouch expire --apply                  # optional: clear stale pending proposals
 
 # 4. commit
 git add .vouch/ && git commit -m "kb: approve auth-uses-jwt"
@@ -143,11 +145,15 @@ vouch capabilities                          # emit the JSON capabilities descrip
 vouch status [--json]                       # KB counts + pending proposals
 vouch lint [--stale-days N]                 # user-actionable problems
 vouch doctor                                # full sweep incl. source verification
+vouch fsck                                  # deep consistency: indexes, lifecycle, decided
+vouch migrate [--check] [--dry-run]         # upgrade .vouch/ format safely
 
 vouch pending                               # list pending proposals
+vouch review [--limit N] [--type KIND]      # guided proposal review queue
 vouch show <proposal-id>
-vouch approve <proposal-id> [--reason ...]
+vouch approve <proposal-id>... [--reason ...] [--keep-going]
 vouch reject <proposal-id> --reason "..."
+vouch expire [--apply] [--days N] [--json]   # GC stale pending proposals
 
 vouch propose-claim --text ... --source ... [--type ...] [--confidence X]
 vouch propose-page --title ... [--body -] [--claim ID ...]
@@ -176,6 +182,8 @@ vouch export --out path.tar.gz
 vouch export-check path.tar.gz
 vouch import-check path.tar.gz
 vouch import-apply path.tar.gz [--on-conflict skip|overwrite|fail]
+vouch sync-check PATH_OR_BUNDLE
+vouch sync-apply PATH_OR_BUNDLE [--on-conflict fail|skip|propose]
 
 vouch serve [--transport stdio|jsonl]
 ```
@@ -254,11 +262,11 @@ vouch import-apply kb.tar.gz --on-conflict skip  # apply (default skip; never de
 | Area | Current support |
 |------|-----------------|
 | Knowledge base | `.vouch/` folder, YAML claims/entities/relations/evidence/sessions, markdown pages with frontmatter, JSONL audit log, content-addressed sources |
-| CLI | `init`, `discover`, `capabilities`, `status`, `lint`, `doctor`, `pending`, `show`, `approve`, `reject`, `propose-{claim,page,entity,relation}`, `source add`, `source verify`, `supersede`, `contradict`, `archive`, `confirm`, `cite`, `session {start,end}`, `crystallize`, `search`, `context`, `index`, `audit`, `export`, `export-check`, `import-check`, `import-apply`, `serve` |
+| CLI | `init`, `discover`, `capabilities`, `status`, `lint`, `doctor`, `fsck`, `pending`, `show`, `approve`, `reject`, `propose-{claim,page,entity,relation}`, `source add`, `source verify`, `supersede`, `contradict`, `archive`, `confirm`, `cite`, `session {start,end}`, `crystallize`, `search`, `context`, `index`, `audit`, `export`, `export-check`, `import-check`, `import-apply`, `serve` |
 | Tool servers | MCP over stdio + JSONL over stdin/stdout, same `kb.*` surface across both transports, capabilities + knowledge-capability descriptor |
 | Schemas | 13 JSON Schemas (Draft 2020-12) generated from pydantic in [schemas/](schemas/), plus hand-maintained `bundle.manifest` and `jsonl-envelope` schemas |
 | Write safety | review-gated writes via [proposed/](spec/review-gate.md), `dry_run:true` previews, host trust required for `approve`/`reject`, atomic exclusive-create storage, path-traversal blocked on source intake and bundle import |
-| Retrieval | SQLite FTS5 + substring fallback; optional semantic backends (`all-mpnet-base-v2`, `MiniLM-L6`, fastembed-BGE) behind install extras; context packs with citations + quality gate |
+| Retrieval | `retrieval.backend` in `config.yaml` selects the path: `auto` (default — embedding → FTS5 → substring), `embedding`, `fts5`, or `substring`. Semantic backends (`all-mpnet-base-v2`, `MiniLM-L6`, fastembed-BGE) ship behind install extras; `auto` degrades to FTS5 when they aren't installed. Context packs with citations + quality gate |
 | Lifecycle | `supersede`, `contradict`, `archive`, `confirm`, `cite` — direct mutations, all audited |
 | Portability | tar.gz bundles with per-file sha256 `manifest.json`, `export-check`, `import-check`, `import-apply` with skip/overwrite/fail conflict modes |
 | Audit | append-only `audit.log.jsonl`, per-event actor (`VOUCH_AGENT`), object ids, dry-run flag, reversible flag |
@@ -268,7 +276,7 @@ vouch import-apply kb.tar.gz --on-conflict skip  # apply (default skip; never de
 
 ## Status
 
-Pre-1.0. What's *not* in this implementation: vector embeddings (BM25/FTS5 only), per-runtime adapter templates, benchmark fixtures, multi-agent sync, scopes beyond a single field on Claim/Source. If a hole matters to you, file an issue.
+Pre-1.0. What's *not* in this implementation: per-runtime adapter templates, benchmark fixtures, multi-agent sync, scopes beyond a single field on Claim/Source. If a hole matters to you, file an issue.
 
 ## License
 

ROADMAP.md

@@ -6,8 +6,11 @@ promise. Items marked **[VEP]** require a written proposal in
 
 ## 0.1 — surface stabilises (next)
 
-- Vector embeddings as an *optional* retrieval backend alongside FTS5.
-  Default stays FTS5; embeddings opt-in via `config.yaml`. **[VEP]**
+- Vector embeddings as a retrieval backend alongside FTS5 (shipped).
+  Retrieval is controlled by `retrieval.backend` in `config.yaml`:
+  `auto` (default) tries embedding → FTS5 → substring, gracefully
+  degrading to FTS5 when the embeddings extras aren't installed; set
+  `embedding`, `fts5`, or `substring` to pin a single path.
 - `vouch diff <id-old> <id-new>` for claim/page revisions.
 - `vouch approve --batch` for reviewing N proposals in one transaction.
 - HTTP transport (`vouch serve --transport http`) behind a localhost