Why Does Haystack Stop Grouping Related Chunks After Adding Metadata?

[prashant-qualwebs]

Need help!
I am using Haystack for retrieving relevant chunks from documents. When a user sends a query, the system returns the top 3 most relevant chunks from the complete document.

Now, I have added some metadata to the documents. For example, each section belongs to a specific chunk_id and index_id. After adding this metadata, when I run the same query again, the system only returns results at the section level.

Previously, the response could include multiple related parts together (for example, two sections combined in one answer). But now, it does not return those related parts together anymore—it only returns individual section-wise results.

Does anyone have an idea where I might be making a mistake? Or is this expected behavior? Is it possible to get combined results again?

[ibondarenko1]

The behaviour change is expected for the way you've added metadata - the retriever is now treating each chunk as a self-contained unit instead of letting siblings travel together. Two patterns to get the combined results back.

1. The cause: per-chunk metadata implicitly disabled the parent-document grouping.

Haystack's default retrievers (InMemoryEmbeddingRetriever, QdrantEmbeddingRetriever, etc.) return raw Document objects. When you had no per-chunk metadata, multiple chunks from the same source could rank in the top-3 and your downstream prompt assembled them. Once each chunk has a unique chunk_id, the embeddings still rank correctly, but if anywhere in your pipeline you have a unique_by or filter that dedups on chunk_id, you now get exactly one section per top-3 slot.

Quick check - print the raw retriever output before any postprocessing:

from haystack import Pipeline
result = retriever.run(query_embedding=embedding, top_k=10)
for d in result['documents']:
    print(d.meta.get('chunk_id'), d.score, d.content[:80])

If you see neighbouring chunk_ids in the top-10 here but only single sections in the final answer, the grouping was lost downstream, not at retrieval.

2. Switch to a parent-document retrieval pattern.

The cleanest fix when each chunk has a chunk_id and an index_id is: retrieve at chunk level, then expand to parent at the post-processor stage:

from haystack import component, Document
from typing import List

@component
class ChunkExpander:
    def __init__(self, document_store, neighbour_radius: int = 1):
        self.store = document_store
        self.r = neighbour_radius

    @component.output_types(documents=List[Document])
    def run(self, documents: List[Document]) -> dict:
        seen, out = set(), []
        for d in documents:
            idx = d.meta.get('index_id')
            cid = d.meta.get('chunk_id')
            if (idx, cid) in seen: continue
            # pull this chunk + neighbours within the same index_id
            siblings = self.store.filter_documents(filters={
                "operator": "AND",
                "conditions": [
                    {"field": "meta.index_id", "operator": "==", "value": idx},
                    {"field": "meta.chunk_id", "operator": ">=", "value": cid - self.r},
                    {"field": "meta.chunk_id", "operator": "<=", "value": cid + self.r},
                ]
            })
            for s in siblings:
                if (s.meta['index_id'], s.meta['chunk_id']) not in seen:
                    seen.add((s.meta['index_id'], s.meta['chunk_id']))
                    out.append(s)
        return {"documents": out}

Add this between retriever and prompt builder. Now ranking happens at chunk granularity but the prompt sees neighbouring sections together.

3. If you need actual section merging (concatenation), do it in a separate component.

@component
class SectionMerger:
    @component.output_types(documents=List[Document])
    def run(self, documents: List[Document]) -> dict:
        by_index = {}
        for d in documents:
            by_index.setdefault(d.meta['index_id'], []).append(d)
        merged = []
        for idx, chunks in by_index.items():
            chunks.sort(key=lambda c: c.meta['chunk_id'])
            content = "\n\n".join(c.content for c in chunks)
            best_score = max(c.score or 0 for c in chunks)
            merged.append(Document(content=content, meta={'index_id': idx},
                                    score=best_score))
        merged.sort(key=lambda d: d.score, reverse=True)
        return {"documents": merged}

This collapses all retrieved chunks per index_id into one combined Document, scored by the highest-ranked chunk. Good when the LLM benefits from continuous prose; less good when chunks are intentionally heterogeneous.

4. Increase top_k at the retriever, narrow at the postprocessor.

If you currently call retriever with top_k=3, you have no headroom to expand. Set retriever top_k=20, run the expander, then take the top 3 expanded groups. The retrieval cost is negligible at the embedding-store level.

5. Confirm metadata isn't filtering before retrieval.

If you added the metadata as actual filter clauses (e.g., MetaFieldRanker, or pre-retrieval filters={"meta.section_type": "..."}) instead of passive payload, the search space shrinks and natural co-occurrence drops. Run with no filters first to confirm the embedding model still ranks neighbours together.

Recipe: dump raw retriever output to confirm rankings still co-locate -> add a ChunkExpander component for neighbour expansion using index_id+chunk_id -> raise top_k at the retriever -> add a SectionMerger if you want one prose block per section -> drop pre-retrieval metadata filters unless you need them as hard constraints.