Skip to content

fix(knowledge): exclude quarantined research facts at every reachable KB search site (#13009) - #13029

Merged
mrveiss merged 1 commit into
Dev_new_guifrom
issue-13009
Jul 30, 2026
Merged

mrveiss merged 1 commit into
Dev_new_guifrom
issue-13009

Conversation

@mrveiss

@mrveiss mrveiss commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Closes #13009

Thinking Path

Read #13009 + the two PRs that established the quarantine: #13010 (#12622, added the {"collection": {"$ne": "research"}} filter at async_chat_workflow.py::_execute_kb_search, the one confirmed-live chat-RAG surface at the time) and #13011 (#12623, the corroboration/promotion gate that later releases facts from quarantine).

Enumerated every reachable KB-read call site repo-wide (kb.search()/knowledge_base.search(), plus the raw collection.query() sites internal to the knowledge/ package itself, which are the implementation of kb.search() and out of scope). Judged each for (a) general-purpose/chat vs. research-internal, (b) production reachability, (c) whether it already threads a caller-supplied filters argument.

#13009's own 3 named candidates, resolved:

  • ai_hardware_accelerator.py:834 (_gpu_semantic_search) — not reachable. Dispatched only via AIHardwareAccelerator.process_task(task_type="semantic_search"), and the only constructor of such a task, accelerated_semantic_search(), has zero callers anywhere in the repo.
  • utils/hybrid_search.py (328/379/472, HybridSearchEngine) — reachable but not a leak. get_hybrid_search_engine() is only ever called from utils/system_validator.py's diagnostic self-test, with hardcoded synthetic queries ("Python programming tutorial", "test query"); the validator never surfaces fact content back to any response, only score/keyword-count presence.
  • advanced_rag_optimizer.py:395 (_perform_semantic_search) — genuine, reachable, working leak. Traced: chat_workflow/manager.py::_init_knowledge_service → RAGService(kb) → AdvancedRAGOptimizer.advanced_search() → _perform_semantic_search() → unfiltered kb.search(). ChatWorkflowManager is instantiated from api/chat.py's live chat endpoints and initialization/lifespan.py — this is a second, parallel live chat-RAG surface alongside the one Research agent P0: findings→KB facts + grounded /research synthesis #12622 already fixed.

11 more genuine, reachable, working sites found, all general-purpose reads that must exclude quarantined facts (none are the #12622/#12623 research-internal corroboration path, which correctly keeps reading the quarantine collection unfiltered): api/chat.py (process_chat_message citations — #10548 RAG grounding on the primary /chat//chat/message endpoints; _enhance_with_knowledge_base — AI-Stack chat pipeline), api/agent.py (_enhance_context_with_kb — goal-orchestration context; comprehensive_research_task — a general multi-agent task despite its name, not the quarantine pipeline), api/ai_stack_integration.py (rag_query/chat/knowledge_search — admin-gated, but the quarantine guarantee isn't "admin-only", it's "invisible until promoted"), api/knowledge_ai_stack.py::rag_search, and api/knowledge_search_aggregator.py (_search_facts/get_llm_context/_get_facts_for_graph — this module's own docstring calls it "the single entry point for all knowledge retrieval").

Several apparent call sites turned out pre-existing broken or dead, independent of this issue — fixing the quarantine there would be moot (no content is ever returned today) and is out of scope for #13009, so filed separately with file:line evidence: #13024 (stale n_results/search_mode kwargs no longer accepted by the canonical KnowledgeBase.search() signature, silently caught by broad except blocks — agents/kb_librarian/librarian.py, agents/knowledge_retrieval_agent.py, api/chat_knowledge.py, services/grounded_agent.py), #13025 (KBLibrarianAgent.search_knowledge passes limit= which triggers "Enhanced" mode returning a Dict, but the caller iterates it as a List — every non-empty search silently returns []), #13026 (api/knowledge_mcp.py's local KB singleton passes a config_manager kwarg KnowledgeBase.__init__ doesn't accept — raises TypeError on first use), #13027 (tools/tool_registry.py's singleton is never constructed with a knowledge_base, so the LLM tool-calling KB-search tool always reports "unavailable"), #13028 (dead code: ai_hardware_accelerator._gpu_semantic_search and KnowledgeRetrievalAgent have zero production callers).

What Changed

  • New knowledge/quarantine.py: extracted RESEARCH_QUARANTINE_FILTER from async_chat_workflow.py's private module constant now that there are 12 real call sites (per Other unfiltered KB.search() call sites may leak #12622 quarantined research facts #13009's own guidance: share the filter once there are 2+ sites, don't scatter copies). Reuses config.research_quarantine_collection (Research agent P1: N-source corroboration + contradiction surfacing + KB promotion gate #12623's SSOT constant) — never a literal.
  • Applied filters=RESEARCH_QUARANTINE_FILTER at all 12 confirmed-genuine sites: advanced_rag_optimizer.py, api/chat.py (×2), api/agent.py (×2), api/ai_stack_integration.py (×3), api/knowledge_ai_stack.py, api/knowledge_search_aggregator.py (×3).
  • async_chat_workflow.py now imports the shared constant instead of defining its own private copy.
  • Research-internal readers (services/claim_verifier.py, services/research/orchestrator.py, services/research/planner.py) were not touched — they exist to read the quarantine collection for corroboration.

Verification

  • py_compile, flake8 --max-line-length=120, black --check --line-length=120: clean on every touched/new file.
  • Central filter-mechanism proof (knowledge/quarantine_test.py, 4 tests): drives a real (non-mocked) InMemoryClient where-filter, same pattern as feat(research): N-source corroboration + contradiction surfacing + KB promotion gate (#12623) #13011's quarantine_boundary_test.py — a quarantined fact is excluded, a promoted fact (metadata flipped) is included, a legacy fact with no collection field stays included, and the filter dict is asserted to derive from config.research_quarantine_collection (not hardcoded).
  • Per-site wiring proof (11 new tests across api/agent_test.py, api/chat_test.py, api/ai_stack_integration_test.py, api/knowledge_ai_stack_test.py, api/knowledge_search_aggregator_test.py, plus 1 updated assertion in the pre-existing advanced_rag_optimizer_test.py): each calls the real function/route coroutine directly (no FastAPI TestClient needed — these are plain async defs; Depends(...) defaults are simply overridden with explicit mock kwargs) and asserts kb.search(...) was invoked with filters=RESEARCH_QUARANTINE_FILTER.
  • Research-internal regression proof: reran tests/services/test_claim_verifier.py, services/research/orchestrator_test.py, services/research/planner_test.py, services/research/quarantine_boundary_test.py, tests/unit/chat_workflow/test_async_chat_workflow_kb_search.py — all still pass unmodified, confirming the promotion/corroboration gate still sees quarantined facts.
  • cp-swapped baseline diff (not git stash, per worktree rules): swapped all 8 modified files back to pre-PR (HEAD), reran the same test set: 106 passed (pre-PR test file also reverted, so no false failure from the new filters= assertion). Restored PR files: 121 passed (106 + 15 new/updated). Identical baseline, zero regressions.
  • tests/test_startup_imports.py: 350/350 passed, unchanged before/after.

Model Used

Claude Sonnet 5

… general-purpose KB search site (#13009)

Audited every kb.search()/knowledge_base.search() call site repo-wide against
#12622's quarantine guarantee. Two of the three sites named in #13009 turned
out inert (ai_hardware_accelerator._gpu_semantic_search has zero callers;
utils/hybrid_search is reachable only via a diagnostic with synthetic
queries and never surfaces content). The audit found 11 more genuinely
reachable, working, general-purpose sites with the same unfiltered-search
gap as the one #12622 fixed in async_chat_workflow.py, spanning the primary
/chat and /chat/message endpoints, AI-Stack chat/RAG endpoints, agent-goal
context enhancement, and the knowledge-search-aggregator "single entry
point for all knowledge retrieval".

Extracted the filter from async_chat_workflow.py's private constant into
knowledge/quarantine.py (RESEARCH_QUARANTINE_FILTER) now that there are 12
real call sites, and applied it at each. Several other apparent call sites
(agents/kb_librarian_agent.py, agents/knowledge_retrieval_agent.py,
tools/tool_registry.py, api/knowledge_mcp.py) were found to be pre-existing
broken or unreachable independent of this issue -- filed separately as
#13024-#13028 rather than layering a quarantine fix on dead code.
@github-actions

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No hardcoded values detected that have SSOT config equivalents!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant