bug(kb): search_by_metadata examines at most 500 facts, chosen by Redis key order, and reports success either way #17713

search_by_metadata examines at most 500 facts, chosen by Redis key order, and reports success either way

Found while building #17546's removal path, which addresses a Knowledge Base document by its
file_path metadata. Filed separately because the limitation belongs to a shared capability,
not to the watch folder — and it is reachable from a public HTTP route.

The mechanism

autobot-backend/knowledge/metadata.py:

async def _get_filtered_fact_keys(self) -> List[str]:
    fact_keys = await asyncio.to_thread(self.redis_client.keys, "fact:*")
    return [k for k in fact_keys if not any(x in ... for x in [":tags", ":category", ":all"])]

async def search_by_metadata(self, field_name, value, operator="eq", limit=50):
    fact_keys = await self._get_filtered_fact_keys()
    for key in fact_keys[:500]:
        ...
    return {"status": "success", "fact_ids": matching_facts, "count": len(matching_facts), ...}

Three properties, and the third is the one that makes this more than a cap.

1. It is bounded at 500 regardless of corpus size. A KB with 5,000 facts has 4,500 that this
search cannot reach. limit (default 50) bounds the results; the 500 bounds what is examined,
and only the first is a caller-visible parameter.

2. The 500 are not a stable subset. redis.keys() returns keys in keyspace iteration order,
which is not guaranteed and shifts as keys are added, removed or rehashed. So which 500 facts get
examined can differ between two consecutive identical calls. The same query can find a document
and then not find it, with nothing changed by the caller.

3. Both outcomes report "status": "success". A truncated scan is indistinguishable from an
exhaustive one. There is no truncated flag, no examined count, nothing that says 500 of N were
looked at.

Why this is not only an internal concern

api/knowledge_metadata.py:297 exposes it as an HTTP route, and the handler only branches on
status != "success". A truncated search returns 200 with a partial result set presented as
complete. The route's own docstring advertises operator and limit as the controls; neither
tells the caller what was searched.

The shape

Did not find and could not look are rendered identically, and that is the family this
repository has been finding all week (#15826). It is sharper here than usual because count is a
real number and status is a real success — nothing about the response is malformed. It is a
correct answer to "what matched among the first 500 keys Redis happened to return", presented as an
answer to "what matches".

What #17546 does about it, and why that is not a fix

#17546's remove_watched_file reports not_found to the error counter rather than treating it as
a clean no-op, precisely because the search cannot promise exhaustiveness — so an unremovable
document is visible rather than looking like a file that was never ingested. That is containment
at one call site, not a repair. Every other caller, including the HTTP route, still reads a
truncated result as complete.

Acceptance criteria

• A truncated search is distinguishable from an exhaustive one in the return value and in
  the HTTP response — an examined count and a truncated flag, or an explicit error when the
  corpus exceeds what the scan can cover. The current status: success for both is the defect.
• The scan is either exhaustive for the queried predicate, or bounded in a way the caller
  chooses and can see. If a bound stays, it is a named constant with its reason recorded, not a
  literal [:500] mid-expression.
• Ordering is not left to redis.keys(). Either iterate deterministically or state that the
  subset is arbitrary — a result that changes between identical calls, with no input changed,
  is the property to remove or to disclose.
• redis.keys("fact:*") is reviewed in the same pass. It is an O(N) blocking command against
  the whole keyspace on every call, which is a production hazard independent of the 500 —
  SCAN is the usual answer. Not folded in here: decide it, do not let it ride.
• A test with more than 500 facts asserts that a fact beyond the scan is either found or
  reported as unexamined — never silently absent.

Related

#17546 (found here; contains it at one call site), #15826 (the umbrella: a check that reports
clean having examined nothing), #15165 (knowledge search under load)