Skip to content

Other unfiltered KB.search() call sites may leak #12622 quarantined research facts #13009

Description

@mrveiss

Context

#12622 adds ResearchOrchestrator, which lands web-research findings as KB facts tagged metadata.collection="research" — quarantined until a future promotion gate (#12623) reviews them. The PR excludes these from the confirmed live chat-RAG injection path (autobot-backend/async_chat_workflow.py::_execute_kb_search) by passing a {"collection": {"$ne": "research"}} filter to kb.search().

Grepping for other kb.search() / self.kb.search() call sites with no metadata filter turned up a few more that would also surface quarantined facts once they vectorize into the shared ChromaDB index, if/when these paths are live:

  • autobot-backend/ai_hardware_accelerator.py:834 — _gpu_semantic_search, an unfiltered kb.search(query=query, top_k=top_k). Docstring explicitly says it "match[es] the pattern used in async_chat_workflow", i.e. it's meant to be an equivalent general-semantic-search surface.
  • autobot-backend/advanced_rag_optimizer.py:395 — _perform_semantic_search, an unfiltered self.kb.search(query, top_k=limit).
  • autobot-backend/utils/hybrid_search.py:472 — a kb.search(query, top_k=min(top_k*2, 20)) call with no filters argument at all (unlike the two other call sites in the same file at lines 328 and 379, which accept a caller-supplied filters dict).

Ask

For each: confirm whether it is reachable from a genuinely general-purpose chat/RAG surface (vs. an inert/experimental path). If reachable, apply the same quarantine-exclusion filter used in async_chat_workflow.py (or better, extract a shared RESEARCH_QUARANTINE_FILTER helper once there are 2+ real call sites, per autobot_shared.ssot_config.config.research_quarantine_collection).

Not fixed in #12622 itself to avoid widening that PR's blast radius into subsystems whose production-reachability wasn't independently verified there.

Activity

  1. github-actions commented on Jul 30, 2026

    @github-actions
    Contributor

    PR #13029 (merged to Dev_new_gui) references this issue with a close keyword.

    fix(knowledge): exclude quarantined research facts at every reachable KB search site (#13009)

    If this issue is fully resolved, close it manually. If work remains, no action is needed.

  2. mrveiss commented on Jul 30, 2026

    @mrveiss
    OwnerAuthor

    Fixed by PR #13029, merged to Dev_new_gui as 704780ba8.

    The answer to this issue's question was no — the fixed site was not the only one. It was one of twelve.

    Exhaustive repo-wide enumeration of KB read sites, each classified as general-purpose (must exclude quarantined facts) or research-internal (must still see them):

    Genuine, reachable, working leaks — 12, now filtered:

    • advanced_rag_optimizer.py:395 — the most consequential. Reachable via chat_workflow/manager.py → RAGService → AdvancedRAGOptimizer, i.e. a second live chat-RAG surface running in parallel with the one Research agent P0: findings→KB facts + grounded /research synthesis #12622 fixed.
    • api/chat.py — process_chat_message citations, _enhance_with_knowledge_base
    • api/agent.py — _enhance_context_with_kb, comprehensive_research_task
    • api/ai_stack_integration.py — rag_query, chat, knowledge_search
    • api/knowledge_ai_stack.py::rag_search
    • api/knowledge_search_aggregator.py — _search_facts, get_llm_context, _get_facts_for_graph

    Named in this issue but NOT leaks: ai_hardware_accelerator.py:834 is dead code (zero callers into its dispatch path); utils/hybrid_search.py:328/379/472 is reachable only from a diagnostic self-test with synthetic queries that never surfaces content. Reporting these as non-issues rather than filtering them, since scattering redundant filters would obscure which sites actually matter.

    Fix shape: the filter now lives once, in knowledge/quarantine.py as RESEARCH_QUARANTINE_FILTER, reusing config.research_quarantine_collection rather than a literal. All 12 sites import it — including #12622's original fix in async_chat_workflow.py, which previously carried a local dict literal. A security filter copy-pasted across twelve call sites is precisely how eleven of them drift out of sync.

    Verification, in both directions:

    • knowledge/quarantine_test.py (new) proves the mechanism end-to-end through a real InMemoryClient where-filter: quarantined excluded, promoted included, legacy facts lacking the collection field still included — the case that would otherwise silently hide the entire pre-existing KB.
    • 11 new + 1 updated per-site tests prove each fixed call site actually passes the filter.
    • The research-internal reads are proven untouched: claim_verifier, research/orchestrator, research/planner and quarantine_boundary_test all rerun unmodified and still pass, confirming the corroboration and promotion gate can still see quarantined facts. Over-applying the filter would have blinded the gate to the very facts it exists to evaluate.
    • cp-swapped baseline: identical results before/after (106 → 121 passing, no regressions); startup-import smoke 350/350 unchanged.

    Filed, not patched here — several sites were already broken for reasons unrelated to quarantine, and applying a filter to code that never returns content would have been moot: #13024, #13025, #13026, #13027, #13028 (stale n_results/search_mode kwargs the canonical KnowledgeBase.search() no longer accepts; a return-type mismatch that silently swallows all results; a broken singleton constructor; a singleton that never receives its KB dependency).

    A correction has been posted on #12622 recording that its stated guarantee held only for the single site it touched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions