Repository navigation
Research agent P1: N-source corroboration + contradiction surfacing + KB promotion gate #12623
Description
Part of #12621 (Phase 1 — precision layer + quarantine release).
Goal
Add multi-source corroboration so facts earn confidence, surface disagreement instead of hiding it, and promote corroborated facts from the quarantined research collection into the general KB.
Scope
Extend services/claim_verifier.py (do not fork): upgrade the current 2-source arbitration (ClaimVerifier.verify, claim_verifier.py:365) to N-independent-source verification.
- Gather ≥K independent sources for each material claim (materiality tag comes from synthesis; verify only asserted claims — cost control).
- Agreement → raise fact confidence (function of independent-source count + independence).
- Disagreement → create a
contradictsRelation (fact↔fact) and set the fact's verification /requires_human_reviewfield (reuse existing flag). - Add a refutation prompt ("what evidence would contradict this?") — port the pattern the
deep-researchskill already uses.
Promotion gate: facts in the research collection that reach the corroboration/confidence threshold are promoted (moved/copied) into the general KB collection so chat/RAG can see them. Below threshold → stay quarantined. Never assert single-source claims as verified.
Acceptance criteria
- A material claim with ≥K agreeing independent sources gets raised confidence + is promoted to the general KB collection.
- Conflicting sources produce a
contradictsrelation +requires_human_review, and the claim is NOT promoted. -
/researchresponse confidence/contradictions[]reflects the corroboration outcome. - Single-source claims are never marked verified.
- Unit tests for agreement / disagreement / below-threshold paths; integration test asserting promotion visibility in RAG.
Depends on
Phase 0 (#12622) — facts + research collection must exist.
Activity
mrveiss commentedon Jul 30, 2026
Opened PR #13011 implementing N-source corroboration + contradiction surfacing + KB promotion gate. See PR body for premise verification (deep-research skill pattern doesn't exist as described; ported the real refutation-prompt pattern from
agent_loop/pre_action_verifier.pyinstead), what was built, and quarantine-boundary proof.feat(research): N-source corroboration + contradiction surfacing + KB…
github-actions commentedon Jul 30, 2026
PR #13011 (merged to
Dev_new_gui) references this issue with a close keyword.If this issue is fully resolved, close it manually. If work remains, no action is needed.
mrveiss commentedon Jul 30, 2026
Closed by PR #13011, merged to
Dev_new_guias1db2568e5.Delivered: N-source corroboration extending
ClaimVerifierin place (not forked) —SourceAgreement,CorroborationResult,corroborate(),classify_agreement(); contradiction surfacing ascontradictsfact↔fact relations; and the promotion gate releasing corroborated facts from theresearchquarantine collection into general KB visibility. Seven tunables (K threshold, confidence-formula parameters, promotion threshold, collection name, search top-k, classify timeout) are env-sourcedssot_configconstants, not literals.A latent defect made this feature a no-op, and was caught only by probing the mechanism.
knowledge/facts.py::update_fact()synced ChromaDB only whencontentalso changed. Promotion is by definition a metadata-only update — so every promotion would have been written to Redis and never reached the vector store. The gate would have reported success while promoting nothing, and no test at the time would have failed. Fixed in-scope via_sync_fact_metadata_in_chromadb, withknowledge/facts_metadata_sync_test.pydriving the real production path rather than a mock.Two of this issue's own pointers were wrong:
deep-researchskill" — no such skill has prompt logic.skills/builtin/research/SKILL.mdis an unrelated slash-command skill, andonboarding/presets.py's"deep-research"entry is a starter-preset system-prompt string. The genuine adversarial-refutation pattern lives inagent_loop/pre_action_verifier.py(structured response, low temperature, fail-open); ported that, applied to claim-vs-source instead of action-vs-safety.requires_human_reviewexisted as a result-object field (VerifiedClaim,ConflictResolutionResult,GroundedResponse, LLCreview_gate) but had never been fact metadata. Reused the name for the new fact-level flag rather than inventing a second vocabulary.Confirmed as described:
contradictsinknowledge/relations.py:43(KB_RELATION_TYPES) viacreate_fact_relation(), andClaimVerifier.verifyatservices/claim_verifier.py:365as the 2-source arbitration to extend.Quarantine boundary proven end-to-end, not described.
services/research/quarantine_boundary_test.pydrives a real where-filter through the production code path: quarantined fact excluded, promoted fact included, legacy fact lacking thecollectionfield still visible, disputed fact still excluded.memory_adapter._match_wheregained$ne/$eq(its own docstring invited the extension) so the proof runs without live ChromaDB, whichconftest.pystubs globally.Verification: py_compile /
flake8 --max-line-length=120/black --check --line-length=120clean on every touched file;cp-swapped baseline (nevergit stash) gave an identical failing set before and after, with 22 new tests and zero regressions;tests/test_startup_imports.py350/350 unchanged; a 3898-test sweep acrossservices/+knowledge/showed every file this PR touches 100% green, with the pre-existing failures confirmed environmental (livebackend_urlfixture, npu_pipeline, rag_service_kb_synthesis).feat(research): LLM planner loop - decompose, skip-known, prune, satu…