Skip to content

bug(kb): exception-driven _fallback_chunking also bypasses subclass metadata enrichment #14480

Open

Description

@mrveiss

Summary

Same defect class as #14467, different trigger. SemanticChunkerBase._fallback_chunking() — the path chunk_text() falls back to on any exception in the primary chunking pipeline — builds each chunk's metadata directly in _create_fallback_chunk():

metadata={"fallback_chunking": True, **(metadata or {})}

This never calls _build_chunk_metadata() / _extra_chunk_metadata(), so a chunk produced via the exception fallback is missing chunk_index, total_chunks, embedding_model, chunking_method, and any subclass-contributed key such as optimization_version (see GPUSemanticChunker._extra_chunk_metadata()) — the same shape gap #14467 fixed for the zero/one-sentence path.

Where it was found

While fixing #14467 (single/zero-sentence chunks bypassing enrichment), the same audit covered every return path out of chunk_text(). _fallback_chunking() is the third one (autobot-backend/utils/semantic_chunker_base.py, _create_fallback_chunk and _fallback_chunking), and it has an identical gap. Kept out of that PR to keep the fix scoped to the reported symptom.

Consumer impact

Same conclusion as #14467: no current production consumer reads the subclass-contributed keys off chunk.metadata (knowledge_sync_incremental.py builds its own metadata dict from base_metadata and does not read chunk.metadata; advanced_rag_optimizer.py never calls chunk_text() directly). No stored data is affected today, but the inconsistency is a live seam for a future subclass or consumer.

What is needed

Route _create_fallback_chunk() through the same _enrich_chunks_with_metadata()/_build_chunk_metadata() call the other two paths now use, so all three return paths out of chunk_text() agree on one enrichment point.

Acceptance criteria

  • A chunk produced via the exception-driven fallback path carries the same subclass metadata keys as one from the primary path
  • Asserted via a test that exercises chunk_text() with a stubbed exception in the primary pipeline (not _create_fallback_chunk() directly)

Refs #14467

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions