tech-debt(kb): six providers for one KnowledgeBase across three independent stores — retire the fork

Summary

One concept — "the knowledge base" — has six providers across five modules and three
independent stores. Two of the providers are heavily used and disagree about which instance
is the instance; the remaining four are a mix of minor, latent and broken.

This is a fork the duplication guard cannot see (#17312: forks are same-concept, not
same-text) and that tools/lint/check_symbol_forks.py only partly sees — it matches by
name, so get_or_create_knowledge_base and get_knowledge_base_async do not cluster with
get_knowledge_base at all despite returning the same concept from a different store.

The six providers

#	Provider	Store	`initialize()`?	Consumers
1	`knowledge/_composed.py:73` `get_knowledge_base()`	module global `_knowledge_base_instance` + `asyncio.Lock`	yes, raises on failure	44 import sites
2	`knowledge_factory.py:133` `get_or_create_knowledge_base(app)`	`app.state.knowledge_base`	yes	~20 `api/knowledge*.py` + `initialization/lifespan.py:39`
3	`knowledge_factory.py:182` `get_knowledge_base_async()`	module global in the same module	yes	3 sites
4	`knowledge_base_factory.py:129` `get_knowledge_base()`	`KnowledgeBaseInitializer._instance` (`:47`, initialized at `:85-86`)	yes	5 sites
5	`utils/resource_factory.py:23` `ResourceFactory.get_knowledge_base()`	writes `app.state.knowledge_base` (`:42`)	no	0 (only the module-local `get_kb` at `:215`, itself unimported)
6	`dependencies.py:49` / `:160`	fresh per request / `dependency_cache`	no (sync — cannot await)	8 endpoints

knowledge_base.py is not a seventh home — it is a documented back-compat facade
re-exporting #1 (knowledge_base.py:18), the same pattern as
constants/threshold_constants.py. Facades are the intended end state here; separate stores
are not.

Why three stores is a correctness problem, not just untidiness

Docs improvements #1 and Docs improvements #2 can both exist in one process. A fact written through an api/knowledge*.py route
(store Docs improvements #2) and a fact written through memory/, llc/ or mcp_server/ (store Docs improvements #1) go to
different KnowledgeBase objects, each with its own ChromaDB client, embedding cache and
ownership manager. autobot_shared.store_authority exists precisely to forbid this: one
store is the system of record and every other copy is a rebuildable projection.
Dev new gui #4 is a third singleton with its own lock and its own failure semantics (returns None and
swallows, where Docs improvements #1 raises).
Two module names differ by one word — knowledge_factory.py vs
knowledge_base_factory.py — and export near-identical names. api/knowledge_mcp.py get_knowledge_base() always raises TypeError (config_manager kwarg not accepted) #13026 (closed) was a direct
consequence: a caller reached the wrong get_knowledge_base and every request raised
TypeError on an unaccepted config_manager kwarg.
Dev new gui #5 would write an uninitialized instance into app.state.knowledge_base, i.e. corrupt
store Docs improvements #2 for every later reader. It is unwired today, so this is latent.

Scope split

The live half — #6 hands 8 endpoints an instance whose search raises, and every call site
swallows it — is filed and fixed separately as the bug. This issue is the consolidation
half and should not be bundled with it: it changes which store ~25 call sites read.

Proposed end state

One store. app.state.knowledge_base, owned by initialization/lifespan.py, is the natural
choice — it is the one a FastAPI process can scope to its own lifetime, and it already has the
retry-cooldown and re-initialize logic (#3094/#3106). Non-request code needs a request-free
accessor over the same store, which is what #1 and #3 are each independently reinventing.

Order, smallest blast radius first:

knowledge_base_factory (Dev new gui #4, 5 consumers) delegates to the canonical accessor; keep the
module as a facade, do not delete it.
knowledge_factory.get_knowledge_base_async (Fix frontend errors and remove unused code #3, 3 consumers) and
knowledge/_composed.get_knowledge_base (Docs improvements #1, 44 sites) reconcile onto one global, so the
request-free path and the app-state path cannot diverge.
get_or_create_knowledge_base (Docs improvements #2) becomes a thin app-scoped wrapper over it.
Extend check_symbol_forks.py or add a dedicated guard so a seventh provider cannot be
added silently — the current name-matching detector would not have caught any of Docs improvements #2–Dev new gui #5.

Acceptance criteria

exactly one object holds the live KnowledgeBase; every other accessor reads it
knowledge_base_factory, knowledge_factory, knowledge/_composed, dependencies and
utils/resource_factory all resolve to that one object, each kept as a wired facade
no provider can return an instance with initialized is False
a guard fails if a new module introduces an independent KnowledgeBase() construction
outside the canonical accessor
the two near-identical module names are disambiguated, or one becomes an explicit facade
over the other with the reason recorded

Live half filed as #18121 — dependencies.get_knowledge_base (provider #6 in the table) hands 8 mounted endpoints an instance whose ensure_initialized() raises, and every call site swallows it. Fixing that one does not reduce the store count; it only stops the sixth provider inventing a seventh instance per request.

The construction population, measured

The provider table in the description undercounts the problem. Sweeping every
KnowledgeBase( construction in non-test backend code:

total constructions                               25
inside the canonical factories                     4
outside them, initialize() within 8 lines          3
outside them, no initialize() anywhere in the file 18

The 18:

agents/kb_librarian_agent.py:49                  agents/knowledge_retrieval_agent.py:142
agents/librarian_assistant.py:53                 api/chat_knowledge_manager.py:152
api/knowledge_debug.py:53                        api/knowledge_debug.py:177
api/knowledge_eval.py:38                         api/knowledge_eval.py:73
api/workflow.py:115                              knowledge_sync_incremental.py:241
npu_semantic_search.py:247                       orchestrator.py:180
services/advanced_workflow/coordinator.py:45     utils/error_boundary_examples.py:194
utils/operation_timeout_integration.py:444       utils/system_validator.py:171
utils/system_validator.py:897                    worker_node.py:122

What this measurement does and does not say. It says: no initialize() call on that
instance's own name appears anywhere in the module that constructs it. It does not say all
18 are broken. Three things it cannot see, each of which would make a site fine:

the instance is handed to another object that initializes it;
only methods with their own lazy-init guard are ever called on it (base.py:563,
index.py:159 have one — the basic search path at
search_components/basic_vector_search.py does not, it calls ensure_initialized(), which
raises);
the code path is dead.

Several look like the third case — utils/error_boundary_examples.py is named as examples, and
utils/system_validator.py is a diagnostic. So the honest count of live defects among the 18 is
not yet known and needs per-site triage; what is established is that 18 sites construct the
object in a way that is only safe by accident.

This is why the guard in the acceptance criteria matters more than any individual fix here: the
population is large enough that fixing sites one at a time will not converge, and a baseline
ratchet over these 18 is the mechanism that makes it converge. That baseline is also what the
#18121 fix deliberately did not add — that PR is the three providers and the eight live
endpoints, nothing wider.

Method, for anyone re-running it: match (?:^|[^A-Za-z_.])KnowledgeBase\(, exclude
_test.py/tests//conftest/test_ and class KnowledgeBase(, then for each hit search the
same file for <assigned-name>.initialize() and <assigned-name>.ensure_initialized(.

review(18130): answer all eleven CodeRabbit threads — one KB store, d…

review(18130): round 3 — a cooldown withheld a healthy store, and aut…

Two divergence paths remain outside knowledge_factory.py, found while closing a hole in this issue's guard on #18130. Recording rather than fixing, because they are in the startup path and that is a different risk class than the factory refactor.

initialization/lifespan.py::_init_knowledge_base:

knowledge_base = await get_or_create_knowledge_base(app)
app.state.knowledge_base = knowledge_base          # :720
...
except Exception as kb_error:
    app.state.knowledge_base = None                # :749

Line 720 is redundant on the happy path — get_or_create_knowledge_base already published the instance to both stores — but it is not harmless. When that call returns None (retry cooldown active, or init returned False), this writes None over the projection while the module singleton may still hold a live initialised instance. peek_knowledge_base() then reports an instance and app.state.knowledge_base reports none, which is exactly the asymmetry this issue exists to remove, one file over.

Line 749 does the same unconditionally on any exception.

Why I did not fix it here: #18130 is a fork-consolidation PR, lifespan.py is the startup path, and a wrong change there fails closed on boot rather than degrading. It also wants a decision I should not take alone — whether a failed KB init should clear the projection (making the two stores disagree) or leave the last good instance in place (making a failed init invisible to a caller reading app state). Neither is obviously right; the second is what the singleton already does.

What #18130 does deliver for this issue:

_publish_knowledge_base is the single writer of both stores, and the adoption path now goes through it instead of assigning the projection directly
the guard reports three sets — who calls the publisher, who declares the singleton global, and who writes the app.state projection — with the third added because a creator could call the publisher and then assign a rival instance, which passed the first two checks while the stores diverged
the matcher takes source text and has fixtures for the consolidated shape, a creator that bypasses the publisher, a second writer of the global, and that mixed path

The guard reads knowledge_factory.py only, so it cannot see the two sites above. That is a stated blind spot, not an implied clean result — which is the distinction this issue's own acceptance criteria ask for.

review(18130): the publisher is the only writer of the projection, an…

refactor(forks): retire fifteen forks across eight issues, each onto …

Owner decision (2026-10-09): on a failed init, keep the last good instance and record the failure — do not clear the projection.

So initialization/lifespan.py stops writing app.state.knowledge_base = None:

knowledge_base = await get_or_create_knowledge_base(app)
app.state.knowledge_base = knowledge_base     # :720  overwrites with None on failure
...
except Exception as kb_error:
    app.state.knowledge_base = None           # :749

Both sites currently make the two stores disagree whenever the factory returns None while the singleton still holds a live instance — cooldown active, or initialize() returned False.

The decision gives three properties, in the order they were weighed:

the stores never disagree — the projection is only ever written by _publish_knowledge_base, which refactor(forks): retire fifteen forks across eight issues, each onto one canonical home (#18121) #18130 already made the single writer inside the factory
readers keep serving from a working KB through a transient ChromaDB blip, instead of a working instance being discarded because one re-init attempt failed
the failure stays observable, through the existing failure timestamp and logging rather than by nulling the store

The rejected option was clearing both stores so they agree on None. It was rejected on availability: a transient blip would drop a working knowledge base, and the agreement it buys is agreement on the worse value. "Stores agree" was never the only goal — they agree under this decision too.

Implementation note for the PR: line 720's assignment is redundant on the happy path, because get_or_create_knowledge_base already published the instance to both stores. So the fix is to delete both writes rather than to guard them, which also removes the last two projection writers outside the publisher and lets the #18130 guard widen from one module to the pair.

fix(tests): the projection matcher sees module-level writes to app.st…

fix(backend): the app-state knowledge-base projection has exactly one…

Progress: #18146 merged as b3049e1. It delivers the projection-single-writer part of this issue. After it, app.state.knowledge_base has one non-test writer, _publish_knowledge_base in knowledge_factory.py, and the guard repo_tests/one_knowledge_base_store_18122_test.py fails on any second writer. No acceptance criterion is ticked by this PR alone. Still open: the facades in knowledge_base_factory, _composed and dependencies; no provider returning an instance with initialized=False; a guard against independent KnowledgeBase() construction; disambiguating the module names. Guard gaps found in review: #18147.

refactor(forks): retire fifteen forks across eight issues, each onto …

fix(backend): the app-state knowledge-base projection has exactly one…

Navigation Menu

Repository navigation

tech-debt(kb): six providers for one KnowledgeBase across three independent stores — retire the fork #18122

Summary

The six providers

Why three stores is a correctness problem, not just untidiness

Scope split

Proposed end state

Acceptance criteria

The construction population, measured

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

tech-debt(kb): six providers for one KnowledgeBase across three independent stores — retire the fork #18122

Description

Summary

The six providers

Why three stores is a correctness problem, not just untidiness

Scope split

Proposed end state

Acceptance criteria

Activity

mrveiss commented on Oct 8, 2026

mrveiss commented on Oct 8, 2026

The construction population, measured

mrveiss commented on Oct 9, 2026

mrveiss commented on Oct 9, 2026

mrveiss commented on Oct 9, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions