You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
reviewers scan the pending queue linearly, and at a glance every item carries the same apparent weight. the ones that most deserve a hard look — a claim far from anything already in the kb, a claim that contradicts a pile of approved ones, a claim scraping by with the barest evidence — are exactly the ones that blend in. the raw signal for the near-duplicate direction already exists (find_similar_on_propose attaches non-blocking warnings on propose_claim, and embeddings/dedup.py runs cross-artifact cosine scans over embedding_index), but nothing surfaces the outlier direction: proposals that are anomalous rather than redundant.
this asks for an advisory anomaly annotation on pending proposals. it computes a small set of heuristics — nearest-neighbor cosine distance to the approved corpus (an item with no neighbor above a floor is an outlier), a contradiction count against approved claims, and an evidence-thinness signal — and attaches the result as a read-side annotation on the pending item. it is a hint for the human reviewer. it never rejects, approves, blocks, or rewrites anything.
proposed surface
a read-only kb.flag_anomalies method plus a --flag-anomalies opt-in on the pending listing:
vouch flag-anomalies [--threshold N] [--json] — scores every pending proposal and prints the flagged ones with their reason codes, worst-first. read-only; no writes.
vouch list-pending --flag-anomalies — the same scores attached inline to each pending row, so a reviewer sees them without a second command.
the annotation reuses the warning-dict shape already returned by find_similar_on_propose — a list of {code, ...} entries per proposal. proposed codes:
far_from_corpus — best cosine to any approved artifact is below a floor (nearest-neighbor is far). computed against the same search_embedding / embedding_index path embeddings/dedup.py uses.
contradicts_many — count of approved claims this proposal would contradict (over kb.contradict's existing notion) exceeds a threshold.
config lives under the existing review.* block in .vouch/config.yaml (mirroring review.similarity_threshold): e.g. review.anomaly.far_from_corpus_floor, review.anomaly.contradiction_count, review.anomaly.min_evidence. resolution follows the similarity_threshold(store) pattern in embeddings/similarity.py.
embedding-derived codes degrade gracefully when the embeddings extra is absent — the same ImportError swallow propose_claim already does around find_similar_on_propose, so a base install still gets the non-embedding codes (thin_evidence, contradicts_many).
review gate & scope
this is a read-side advisory only. it computes nothing durable: no proposal, no artifact, no status transition. the flag is presentation metadata attached at read time, exactly like the warnings on propose_claim and unlike anything that writes. because it emits no proposal and touches no artifact, it goes nowhere near proposals.approve() — and it must never gain a "quarantine" or "auto-reject outliers" mode, which would be a write path bypassing the human gate. background or scheduled callers may run it to compute hints; they still cannot act on them, and no code path here approves or rejects. everything stays local: the scores come from the on-disk kb and the local state.db embedding index, no network.
scoring logic (thresholds, contradiction counting, corpus-distance) belongs in a new scoring helper called from the read handlers, not in storage.py, which stays pure I/O.
adding kb.flag_anomalies touches the four registration sites: @mcp.tool() in src/vouch/server.py, _h_flag_anomalies + HANDLERS["kb.flag_anomalies"] in src/vouch/jsonl_server.py, METHODS in src/vouch/capabilities.py, and the vouch flag-anomalies command in src/vouch/cli.py, plus tests/test_flag_anomalies.py. (if the --flag-anomalies flag on kb.list_pending alone covers the need, only the flag + tests are required; decide during design.)
feat: propose-time similarity warnings for duplicate claims #147 (propose-time similarity warnings) and kb.dedup_scan — those detect the redundant direction (too close to something that exists). this detects the inverse (too far from everything, an outlier). shared cosine machinery, opposite predicate.
pending proposals can be scored for anomaly and returned with per-proposal reason codes, worst-first, read-only.
far_from_corpus reuses the existing search_embedding / embedding_index path and produces no code when the embeddings extra is absent.
thin_evidence and contradicts_many are computed without embeddings so a base install still gets non-embedding flags.
thresholds resolve from review.anomaly.* in .vouch/config.yaml, defaulting sanely when unset, following the similarity_threshold(store) resolution pattern.
the feature emits no proposal, writes no artifact, and performs no status transition — verified by an audit-log assertion that a scoring run adds no mutation events.
scoring logic lives outside storage.py (a scoring helper), keeping the I/O layer pure.
if a new kb.* method is added, all four registration sites are updated with parity and test_capabilities passes.
tests/test_flag_anomalies.py covers: an obvious outlier flagged, a normal in-cluster claim not flagged, thin-evidence flagged, graceful degradation with no embedder, and the no-mutation invariant.
reviewers scan the pending queue linearly, and at a glance every item carries the same apparent weight. the ones that most deserve a hard look — a claim far from anything already in the kb, a claim that contradicts a pile of approved ones, a claim scraping by with the barest evidence — are exactly the ones that blend in. the raw signal for the near-duplicate direction already exists (
find_similar_on_proposeattaches non-blockingwarningsonpropose_claim, andembeddings/dedup.pyruns cross-artifact cosine scans overembedding_index), but nothing surfaces the outlier direction: proposals that are anomalous rather than redundant.this asks for an advisory anomaly annotation on pending proposals. it computes a small set of heuristics — nearest-neighbor cosine distance to the approved corpus (an item with no neighbor above a floor is an outlier), a contradiction count against approved claims, and an evidence-thinness signal — and attaches the result as a read-side annotation on the pending item. it is a hint for the human reviewer. it never rejects, approves, blocks, or rewrites anything.
proposed surface
a read-only
kb.flag_anomaliesmethod plus a--flag-anomaliesopt-in on the pending listing:vouch flag-anomalies [--threshold N] [--json]— scores every pending proposal and prints the flagged ones with their reason codes, worst-first. read-only; no writes.vouch list-pending --flag-anomalies— the same scores attached inline to each pending row, so a reviewer sees them without a second command.find_similar_on_propose— a list of{code, ...}entries per proposal. proposed codes:far_from_corpus— best cosine to any approved artifact is below a floor (nearest-neighbor is far). computed against the samesearch_embedding/embedding_indexpathembeddings/dedup.pyuses.contradicts_many— count of approved claims this proposal would contradict (overkb.contradict's existing notion) exceeds a threshold.thin_evidence— evidence list is at or below a floor.propose_claimalready requires at least one evidence id; this is the softer "technically cited but suspiciously thin" case (see bug: Claim model has no min-evidence validator — uncited claims land via bundle import, put_claim, update_claim #81, below).review.*block in.vouch/config.yaml(mirroringreview.similarity_threshold): e.g.review.anomaly.far_from_corpus_floor,review.anomaly.contradiction_count,review.anomaly.min_evidence. resolution follows thesimilarity_threshold(store)pattern inembeddings/similarity.py.ImportErrorswallowpropose_claimalready does aroundfind_similar_on_propose, so a base install still gets the non-embedding codes (thin_evidence,contradicts_many).review gate & scope
this is a read-side advisory only. it computes nothing durable: no proposal, no artifact, no status transition. the flag is presentation metadata attached at read time, exactly like the
warningsonpropose_claimand unlike anything that writes. because it emits no proposal and touches no artifact, it goes nowhere nearproposals.approve()— and it must never gain a "quarantine" or "auto-reject outliers" mode, which would be a write path bypassing the human gate. background or scheduled callers may run it to compute hints; they still cannot act on them, and no code path here approves or rejects. everything stays local: the scores come from the on-disk kb and the localstate.dbembedding index, no network.scoring logic (thresholds, contradiction counting, corpus-distance) belongs in a new scoring helper called from the read handlers, not in
storage.py, which stays pure I/O.adding
kb.flag_anomaliestouches the four registration sites:@mcp.tool()insrc/vouch/server.py,_h_flag_anomalies+HANDLERS["kb.flag_anomalies"]insrc/vouch/jsonl_server.py,METHODSinsrc/vouch/capabilities.py, and thevouch flag-anomaliescommand insrc/vouch/cli.py, plustests/test_flag_anomalies.py. (if the--flag-anomaliesflag onkb.list_pendingalone covers the need, only the flag + tests are required; decide during design.)adjacent issues, with the distinction stated:
kb.dedup_scan— those detect the redundant direction (too close to something that exists). this detects the inverse (too far from everything, an outlier). shared cosine machinery, opposite predicate.acceptance criteria
far_from_corpusreuses the existingsearch_embedding/embedding_indexpath and produces no code when the embeddings extra is absent.thin_evidenceandcontradicts_manyare computed without embeddings so a base install still gets non-embedding flags.review.anomaly.*in.vouch/config.yaml, defaulting sanely when unset, following thesimilarity_threshold(store)resolution pattern.storage.py(a scoring helper), keeping the I/O layer pure.kb.*method is added, all four registration sites are updated with parity andtest_capabilitiespasses.tests/test_flag_anomalies.pycovers: an obvious outlier flagged, a normal in-cluster claim not flagged, thin-evidence flagged, graceful degradation with no embedder, and the no-mutation invariant.