Problem
GET /api/knowledge_base/facts/by_category returns every fact in each category with no upper bound across categories (per-category limit defaults to 100, but all KnowledgeCategory values are fetched at once). As the KB grows this becomes a latency/memory cliff: the handler batch-fetches and JSON-serialises the full category set on every cache miss, and the frontend KnowledgeBrowser builds the entire tree in one pass.
Follow-up to #12370, which removed full_content from each list item (~98% payload cut). The remaining scaling risk is the unbounded number of items.
Proposed change
- Backend: add cursor/limit pagination to
facts/by_category (e.g. ?cursor=&limit=), returning a next_cursor + has_more — mirror the existing /entries cursor contract in useKnowledgeBrowser.fetchEntriesPage so the frontend pattern is reused, not reinvented.
- Frontend: wire
KnowledgeBrowser category expansion to infinite-scroll / "Load more" using the returned cursor (the component already has loadMore/hasMoreEntries scaffolding for the user-knowledge mode — extend it to the by-category tree).
Scope / notes
- Keep the per-fact detail endpoint (
GET /fact/{fact_key}) as the lazy-load source (unchanged by this work).
- Preserve the 60s Redis cache; cache per
(category, cursor, limit) key.
- No data-loss risk — read-path only.
priority: medium. Discovered during #12370.
Problem
GET /api/knowledge_base/facts/by_categoryreturns every fact in each category with no upper bound across categories (per-categorylimitdefaults to 100, but allKnowledgeCategoryvalues are fetched at once). As the KB grows this becomes a latency/memory cliff: the handler batch-fetches and JSON-serialises the full category set on every cache miss, and the frontendKnowledgeBrowserbuilds the entire tree in one pass.Follow-up to #12370, which removed
full_contentfrom each list item (~98% payload cut). The remaining scaling risk is the unbounded number of items.Proposed change
facts/by_category(e.g.?cursor=&limit=), returning anext_cursor+has_more— mirror the existing/entriescursor contract inuseKnowledgeBrowser.fetchEntriesPageso the frontend pattern is reused, not reinvented.KnowledgeBrowsercategory expansion to infinite-scroll / "Load more" using the returned cursor (the component already hasloadMore/hasMoreEntriesscaffolding for the user-knowledge mode — extend it to the by-category tree).Scope / notes
GET /fact/{fact_key}) as the lazy-load source (unchanged by this work).(category, cursor, limit)key.priority: medium. Discovered during #12370.