RAPTOR tree is built for every document and discarded — no loader, no retrieval path, no metric

Problem

The RAPTOR tree is built for every document that has embeddings, at a cost of one LLM
call per cluster per level, and then discarded. Nothing reads it.

Evidence

grep -rn 'raptor_tree' across the repo returns three non-test hits, and they are the
declaration, the assignment, and its own log line:

autobot-backend/knowledge/pipeline/base.py:32 — self.raptor_tree: Dict | None = None
autobot-backend/knowledge/pipeline/cognifiers/summarizer.py:107 —
context.raptor_tree = await self.build_raptor_tree(chunks, context.embeddings)
summarizer.py:108 — the log line reporting how many levels were built

Confirming there is no sink:

No loader reads it. autobot-backend/knowledge/pipeline/loaders/ contains
chromadb_loader.py, sqlite_loader.py, mesh_seeder.py, redis_graph_loader.py;
none reference raptor_tree.
PipelineResult (base.py:35-53) has no field for it, so it is not even counted in
metrics — the waste is invisible.
autobot-backend/knowledge/summary_search.py retrieves by
level in {chunk, section, document} — the flat three-tier path produced by
recursive_summarizer.py, not the L0/L1/L2 tree.
No loader serialises the context generically (no vars(context) / asdict(context)
path exists), so it is not being persisted incidentally either.

Proposed approach

Persist tree nodes through chromadb_loader.py with a level field (L0/L1/L2)
and their source_chunk_ids, so the existing SummarySearchService machinery is
reused rather than a parallel store introduced.
Extend summary_search.py (search_summaries, drill_down) to traverse tree levels
alongside the existing chunk/section/document levels.
Add a raptor_nodes counter to PipelineResult so a future regression to
build-and-discard is visible in metrics rather than silent.

Risks / constraints

Depends on the provenance fix in RAPTOR tree nodes are built with an empty source_chunk_ids, so no tree node can cite its sources #14968 landing first — wiring a tree whose nodes cannot cite
their sources would have to be redone.
Level naming must not collide with the existing SummaryLevel values.
If the tree is judged not worth retrieving against, the correct outcome is still not
to leave it built-and-discarded: either wire it or stop building it, decided
explicitly.

Acceptance criteria

Tree nodes are persisted and retrievable by level
A retrieval against an upper level can drill down to L0 chunks via source_chunk_ids
PipelineResult reports the node count
A test asserts nodes are readable back from the store, not merely that the loader
was called

Part of #14967

Blocked by: #14968 (provenance must land first)
Blocks: #14972

Implementation order for the whole umbrella is in #14967.

This issue's premise does not hold, and the correction is load-bearing for sizing it. Found while implementing #14968 (PR #16916).

The RAPTOR tree is built for every document that has embeddings, at a cost of one LLM call per cluster per level, and then discarded. Nothing reads it.

It is not built, so there is nothing to discard and no cost being paid.

_summarize_groups passed level=SummaryLevel.SECTION, but SummaryLevel is Literal["chunk", "section", "document"] — a type alias with no members. Literal[...].SECTION raises AttributeError on the first call, and process() swallows it:

except Exception as exc:
    logger.warning("RAPTOR tree building failed (non-fatal): %s", exc)

So build_raptor_tree has been throwing before producing a single node, at warning level, for as long as that call has existed. Every level above L0 is absent — not empty, not uncited, absent.

Two consequences for this issue:

The impact paragraph overstates the waste. There are no wasted LLM calls per cluster per level, because no clustering summarisation completes. The real cost was a silent warning in the logs.
AC2 was unreachable regardless of persistence. "A retrieval against an upper level can drill down to L0 chunks via source_chunk_ids" needs upper levels to exist and to carry provenance. fix(kb): the RAPTOR tree is built at all, and its nodes cite their chunks (#14968) #16916 delivers both — the level fix so the tree builds, and the provenance fix so its nodes cite their chunks. Before that, a loader written to this issue's spec would have persisted nothing and passed its own tests against an empty tree.

So this issue is now correctly scoped for the first time: the tree really will exist after #16916 merges, and it really will have no sink. The four acceptance criteria stand as written; they just describe work that was previously impossible rather than merely undone.

Recommend it be picked up after #16916, not alongside it. I hold the claim on this issue but am not implementing it — it is a persistence feature (loader, retrieval path, PipelineResult field), too large for one honest review pass beside the fix, and it needs #16916's output to test against. Releasing the claim so it is visible as available; happy to take it if nobody else does.

Classification (for work assignment — not a fix proposal)

Scope: kb (secondary chromadb): persisting and retrieving the RAPTOR tree.
Primary files/dirs:
- autobot-backend/knowledge/pipeline/base.py (verified on main; raptor_tree declared at :32)
- autobot-backend/knowledge/pipeline/cognifiers/summarizer.py (verified on main; assigned at :107)
- autobot-backend/knowledge/pipeline/loaders/chromadb_loader.py (verified on main)
- autobot-backend/knowledge/summary_search.py (verified on main)
Umbrella: no. It is a child of Hierarchical retrieval and table ingest: two capabilities built and discarded, three quality gaps #14967 (open) and has no sub-issues of its own.
Blocked by: the body names RAPTOR tree nodes are built with an empty source_chunk_ids, so no tree node can cite its sources #14968, which is now closed (completed via merged PR fix(kb): the RAPTOR tree is built at all, and its nodes cite their chunks (#14968) #16916), so the named blocker is cleared. Nothing else is visible in the issue. Note that a separate triage (triage(v0.9.0): release criticality of the 123 open issues — 26 blocking, 36 small, 18 umbrellas, 41 misfiled, 2 undetermined #17639) recorded this issue as needing an explicit "wire it or stop building it" decision. That is a scope question, not a blocker recorded here.
Read vs inferred: Read the issue body, its 1 comment (which records that fix(kb): the RAPTOR tree is built at all, and its nodes cite their chunks (#14968) #16916 made the tree build at all) and RAPTOR tree nodes are built with an empty source_chunk_ids, so no tree node can cite its sources #14968's state, and confirmed the paths on main. grep raptor_tree on main still finds only the declaration, the assignment and a log line; no loader reads it.
Undetermined: none.

Generated by Claude Code

Navigation Menu

Repository navigation

RAPTOR tree is built for every document and discarded — no loader, no retrieval path, no metric #14969

Problem

Evidence

Proposed approach

Risks / constraints

Acceptance criteria

Classification (for work assignment — not a fix proposal)

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

RAPTOR tree is built for every document and discarded — no loader, no retrieval path, no metric #14969

Description

Problem

Evidence

Proposed approach

Risks / constraints

Acceptance criteria

Activity

mrveiss commented on Sep 17, 2026

mrveiss commented on Sep 28, 2026

Classification (for work assignment — not a fix proposal)

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions