Skip to content

bug: kb_search stamps a single backend label on all results — per-result backend tag is incorrect when fallback is triggered #51

Description

@Tet-9

Description

kb_search in server.py determines the backend value once based
on which code branch executed, then stamps that single value uniformly
across every result in the returned list:

hits = index_db.search(store.kb_dir, query, limit=limit)
backend = "fts5"
if not hits:
    hits = store.search_substring(query, limit=limit)
    backend = "substring"
return [
    {"kind": k, "id": i, "snippet": s, "score": sc, "backend": backend}
    for k, i, s, sc in hits
]

The backend tag is supposed to tell the caller which retrieval engine
produced each result — this is load-bearing information for agents and
tooling that need to interpret relevance scores correctly (FTS5 scores
are BM25-ranked; substring scores are raw hit counts). Stamping a
single global label breaks that contract entirely.

The bug has two concrete failure modes:

  1. When state.db is absent or FTS5 raises, the except branch runs
    and backend is correctly set to "substring" — but if FTS5
    returns an empty list instead of raising, backend stays "fts5"
    and substring results are mislabelled.
  2. Any future code path that merges results from both backends will
    mislabel every hit from whichever backend ran second.

Steps to Reproduce

  1. Ensure state.db exists but is empty so FTS5 returns no hits
  2. Call kb_search with a query that matches via substring
  3. Observe: all returned results carry "backend": "fts5" instead of
    "backend": "substring"

Actual Behavior

All results share one backend label determined by control flow, not
by which backend produced each individual hit.

Expected Behavior

Each hit should carry the backend tag of the backend that actually
produced it. The label must be attached at the point the hit is
generated, not assigned globally after retrieval.

Environment

  • Python 3.11+
  • vouch pre-1.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions