v2.31.0

⭐️ Highlights

📦 Slimming down Haystack core ahead of 3.0

This release begins the migration of many components out of haystack core and into dedicated integration packages, in preparation for Haystack 3.0. Components with heavy or optional dependencies — including all SentenceTransformers embedders and rankers, the Hugging Face API components, the legacy Generators, TikaDocumentConverter, AzureOCRDocumentConverter, the Whisper transcribers, the OpenAPI connectors, the spaCy and Transformers extractors/classifiers/routers, SerperDevWebSearch, SearchApiWebSearch, DocumentLanguageClassifier/TextLanguageRouter, and the Datadog and OpenTelemetry tracers — are now deprecated and will be removed in 3.0.

Each component continues to work as before for now, and moving to the new package is a one-line import change once you install it. See the Deprecation Notes below for the full list and per-component migration snippets. For example:

# Before
from haystack.components.embedders import SentenceTransformersTextEmbedder

# After: pip install sentence-transformers-haystack
from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder

🔀 Type-preserving routing with ConditionalRouter

ConditionalRouter now accepts an output_passthrough: True flag on a route. When set, the route's output is treated as a plain variable name rather than a Jinja2 template, and the value is passed straight through from the pipeline inputs. This lets you route complex, non-basic types such as dataclasses and Pydantic models without the Jinja2 rendering silently converting them to their string representation.

from haystack.components.routers import ConditionalRouter

routes = [
    {
        "condition": "{{query.intent == 'search'}}",
        "output": "query",            # variable name, not a Jinja2 template
        "output_name": "search_query",
        "output_type": ParsedQuery,
        "output_passthrough": True,
    },
]

router = ConditionalRouter(routes)
result = router.run(query=query)
assert result["search_query"] is query   # same object, type preserved

⬆️ Upgrade Notes

• DocumentNDCGEvaluator now matches documents by their content field by default instead of their auto-generated id. Previously, ground truth and retrieved documents were matched only if they had identical id values, which rarely happened in practice since IDs are generated independently for each Document instance. As a result, NDCG scores computed with this evaluator may change for existing pipelines. To keep the previous id-based matching behavior, pass document_comparison_field="id" when constructing the evaluator.

🚀 New Features

• Added native asynchronous support (run_async) to LLMEvaluator, FaithfulnessEvaluator, and ContextRelevanceEvaluator. This allows concurrent evaluation loops inside async applications like FastMCP or FastAPI without blocking the main event loop, while automatically falling back to thread workers for synchronous chat generators.
• Added optional YAML frontmatter extraction to MarkdownToDocument. When initialized with extract_frontmatter=True, YAML frontmatter at the beginning of a Markdown file is removed from the converted content and added to Document.meta.

⚡️ Enhancement Notes

• Added document_comparison_field parameter to DocumentNDCGEvaluator, consistent with DocumentMAPEvaluator, DocumentMRREvaluator, and DocumentRecallEvaluator. Users can now match documents by "content", "id", or any "meta.<key>" field when calculating NDCG scores.

• Add output_passthrough option to ConditionalRouter. When output_passthrough: True is set in a route, the output field is treated as a plain variable name instead of a Jinja2 template, and the value is passed directly from the pipeline inputs to the route output. This allows routing of complex non-basic types such as dataclasses and Pydantic models without unwanted Jinja2 template processing.

  Without output_passthrough, the router renders output as a Jinja2 template, which converts the value to its string representation. Custom types cannot survive that round-trip:

  # Without output_passthrough — the object is silently converted to a string
  routes = [
      {
          "condition": "{{True}}",
          "output": "{{query}}",
          "output_name": "out",
          "output_type": ParsedQuery,
      }
  ]
  router = ConditionalRouter(routes)
  result = router.run(query=ParsedQuery(text="hello", intent="search", entities=[]))
  # result["out"] == "ParsedQuery(text='hello', intent='search', entities=[])"
  # ^^^ str, not ParsedQuery — the object was destroyed

  Set output_passthrough: True to skip Jinja2 entirely and pass the value directly from kwargs:

  from haystack.components.routers import ConditionalRouter
  from dataclasses import dataclass, field
  
  @dataclass
  class ParsedQuery:
      text: str
      intent: str        # "search" | "chat"
      entities: list[str] = field(default_factory=list)
  
  routes = [
      {
          "condition": "{{query.intent == 'search'}}",
          "output": "query",           # variable name, not a Jinja2 template
          "output_name": "search_query",
          "output_type": ParsedQuery,
          "output_passthrough": True,
      },
      {
          "condition": "{{query.intent == 'chat'}}",
          "output": "query",
          "output_name": "chat_query",
          "output_type": ParsedQuery,
          "output_passthrough": True,
      },
  ]
  
  router = ConditionalRouter(routes)
  query = ParsedQuery(text="What is Haystack?", intent="search", entities=["Haystack"])
  result = router.run(query=query)
  
  assert isinstance(result["search_query"], ParsedQuery)  # type preserved
  assert result["search_query"] is query                  # same object, no copying

• Added an opt-in expand_reference_ranges parameter to AnswerBuilder. When enabled, reference ranges like [6-10] and comma-separated ranges like [1-3,7-9] are expanded to the corresponding document indices in RAG answers. The feature is disabled by default to preserve existing parsing behavior.

⚠️ Deprecation Notes

• AzureOCRDocumentConverter is deprecated and will be removed from Haystack in version 3.0. It is moving to the azure-form-recognizer-haystack package. To continue using it, install the package with pip install azure-form-recognizer-haystack and update your import as follows:

  from haystack_integrations.components.converters.azure_form_recognizer import AzureOCRDocumentConverter

• DatadogTracer is deprecated and will be removed from Haystack in version 3.0. It is moving to the datadog-haystack package. To continue using it, install the package with pip install datadog-haystack and add the DatadogConnector component to your pipeline to enable tracing, or update your import as follows:

  from haystack_integrations.tracing.datadog import DatadogTracer

  Note that, starting with Haystack 3.0, Datadog tracing is no longer auto-enabled when ddtrace is installed. Use the DatadogConnector component to enable it.

• OpenAIGenerator, AzureOpenAIGenerator, HuggingFaceAPIGenerator, and HuggingFaceLocalGenerator have been deprecated and will be removed in Haystack 3.0. Generators living in Haystack Core Integrations will also be removed in Haystack 3.0.

  Their chat counterparts (OpenAIChatGenerator, AzureOpenAIChatGenerator, HuggingFaceAPIChatGenerator, HuggingFaceLocalChatGenerator) are the replacement. Starting from Haystack 2.30.0, all ChatGenerators also accept a plain str as input to make the transition easier.

  How to migrate:

  Direct usage (running a generator from Python code)

  Before:

  from haystack.components.generators import OpenAIGenerator
  
  gen = OpenAIGenerator()
  result = gen.run("What is NLP?")
  text = result["replies"][0]   # str
  meta = result["meta"][0]      # dict with model metadata

  After:

  from haystack.components.generators.chat import OpenAIChatGenerator
  
  gen = OpenAIChatGenerator()
  result = gen.run("What is NLP?")   # str input accepted directly
  reply = result["replies"][0]       # ChatMessage
  text = reply.text                  # str
  meta = reply.meta                  # dict with model metadata (now on the message)

  System prompt

  Before:

  from haystack.components.generators import OpenAIGenerator
  
  gen = OpenAIGenerator(system_prompt="You are concise.")
  result = gen.run("What is NLP?")

  After:

  from haystack.components.generators.chat import OpenAIChatGenerator
  from haystack.dataclasses import ChatMessage
  
  gen = OpenAIChatGenerator()
  result = gen.run([
      ChatMessage.from_system("You are concise."),
      ChatMessage.from_user("What is NLP?"),
  ])

  Pipeline usage

  Pipelines that connected PromptBuilder (output: str) to a legacy Generator continue to work unchanged when you swap in a ChatGenerator. The Haystack pipeline type system automatically converts str to list[ChatMessage] at the connection edge.

  Before:

  from haystack.components.generators import OpenAIGenerator
  from haystack.components.builders import PromptBuilder
  
  pipeline.add_component("prompt_builder", PromptBuilder(template=prompt_template))
  pipeline.add_component("llm", OpenAIGenerator())
  pipeline.connect("prompt_builder", "llm")   # str -> str

  After, minimal change (smart connection still works):

  from haystack.components.generators.chat import OpenAIChatGenerator
  from haystack.components.b...

v2.31.0-rc2

v2.31.0-rc2

v2.31.0-rc1

v2.31.0-rc1

v2.30.2

🐛 Bug Fixes

• Fixed the Agent exiting prematurely under the default exit_conditions=["text"]. The agent now only stops when the last message is an assistant message with non-empty text (or when no tool invoker is configured). Previously, if the LLM produced an invalid tool call that was discarded, the resulting assistant message with empty text and no tool calls would trigger an exit, preventing the agent from recovering. The agent now continues looping so the model can recover on the next iteration.

v2.30.2-rc1

v2.30.2-rc1

v2.30.1

⚡️ Enhancement Notes

• AzureOpenAIChatGenerator now accepts a Secret for the azure_endpoint and api_version parameters in addition to a plain string. This makes it possible to resolve these values from environment variables at runtime, for example with Secret.from_env_var("AZURE_OPENAI_ENDPOINT"), so the same serialized pipeline can switch between environments (e.g. dev and prod) by changing environment variables instead of the pipeline definition.

v2.30.1-rc1

v2.30.1-rc1

v2.30.0

⭐️ Highlights

🐍 Syntax-aware Python code splitting with PythonCodeSplitter

The new PythonCodeSplitter is a syntax-aware splitter for Python source files, built for code-RAG and code-search pipelines where naive line-based splitting tends to cut through functions and lose structural context. It parses sources with the ast module and greedily merges units, such as module docstring, import blocks, top-level functions, class headers, methods, and nested classes, into chunks of roughly max_effective_lines, keeping whole functions and methods together. For functions that exceed oversized_factor * max_effective_lines, it falls back to a line-based secondary split with overlap.

Two options make the resulting chunks more useful downstream: strip_docstrings=True moves docstrings into chunk metadata, and preserve_class_definition=True prepends the enclosing class signature to chunks whose members live in a later chunk. Each chunk also carries rich metadata including start_line, end_line, unit_kinds, include_classes, decorators, docstrings, source_id, and split_id.

from haystack.components.preprocessors import PythonCodeSplitter

splitter = PythonCodeSplitter(
    max_effective_lines=80,
    strip_docstrings=True,
    preserve_class_definition=True,
)
result = splitter.run(documents=[doc])

💬 Pass a plain string to any ChatGenerator

All Haystack ChatGenerator components now accept a plain string for the messages parameter in addition to a list of ChatMessage objects. The string is automatically wrapped in a ChatMessage with the user role. This makes switching from a Generator to a ChatGenerator a one-line change. The change applies to AzureOpenAIChatGenerator, AzureOpenAIResponsesChatGenerator, FallbackChatGenerator, HuggingFaceAPIChatGenerator, HuggingFaceLocalChatGenerator, OpenAIChatGenerator, and OpenAIResponsesChatGenerator, and will soon be rolled out to the ChatGenerators in Haystack Core Integrations.

from haystack.components.generators.chat import OpenAIChatGenerator

generator = OpenAIChatGenerator()

# passing a string is equivalent to passing [ChatMessage.from_user("...")]
response = generator.run("What's Natural Language Processing?")
print(response["replies"][0].text)

⬆️ Upgrade Notes

• DALLEImageGenerator has been updated to account for OpenAI's retirement of the DALL-E models. The default model is now gpt-image-2 (previously dall-e-3). To migrate:

  • Update model value: besides gpt-image-2, gpt-image-1 and gpt-image-1-mini are also supported.
  • Update quality value: the new accepted values are auto, high, medium, or low (previously standard or hd).
  • Update size value: the new accepted values are 1024x1024, 1024x1536, 1536x1024, or auto. gpt-image-2 also supports arbitrary sizes.
  • The response_format parameter is now ignored. The component always returns base64-encoded JSON.

  # Before
  llm.run([message], my_callback)
  
  # After
  llm.run(messages=[message], streaming_callback=my_callback)

🚀 New Features

• Introduced the PythonCodeSplitter component, a syntax-aware splitter for Python source files:

  • Parses sources with the ast module and merges units (module docstring, import blocks, top-level functions, class headers, methods, nested classes, and remaining statements) greedily into chunks of roughly max_effective_lines.
  • Keeps whole functions and methods together; falls back to a line-based secondary split (using DocumentSplitter) with overlap only for functions whose effective length exceeds oversized_factor * max_effective_lines.
  • Optionally strips docstrings into chunk metadata via strip_docstrings=True, and prepends the enclosing class signature to chunks whose members live in a later chunk via preserve_class_definition=True.
  • Emits per-chunk metadata including start_line, end_line, unit_kinds, include_classes, decorators, docstrings, source_id, and split_id.

• All Haystack ChatGenerator components now also accept a plain string for the messages parameter in addition to a list of ChatMessage objects. The string is automatically converted into a list containing a ChatMessage with the user role. This is done to simplify switching from Generators to ChatGenerators; Generators might be removed in Haystack 3.0.

  This applies to AzureOpenAIChatGenerator, AzureOpenAIResponsesChatGenerator, FallbackChatGenerator, HuggingFaceAPIChatGenerator, HuggingFaceLocalChatGenerator, OpenAIChatGenerator, and OpenAIResponsesChatGenerator.

  The same change will be soon applied to ChatGenerators available in Haystack Core Integrations.

  Example:

  from haystack.components.generators.chat import OpenAIChatGenerator
  
  generator = OpenAIChatGenerator()
  
  # passing a string is equivalent to passing [ChatMessage.from_user("...")]
  response = generator.run("What's Natural Language Processing?")
  print(response["replies"][0].text)

⚡️ Enhancement Notes

• Added run_async to TextEmbeddingRetriever, MultiQueryEmbeddingRetriever, and MultiQueryTextRetriever. These components now execute natively as coroutines in AsyncPipeline, delegating to each wrapped component's run_async when available and falling back to a thread executor otherwise.
• Fix grammar in the AzureOpenAIGenerator and AzureOpenAIChatGenerator docstring code examples ("<this a model name..." → "<this is a model name...") so that copy-pasted snippets read correctly.
• Update ToolsType to improve type checking for the tools parameter. Any class that inherits from either Tool or Toolset is now accepted in any sequence (list, tuple, etc).
• Pipeline.draw() and Pipeline.show() now validate the Mermaid server response before writing it to disk. The response body is checked against the expected output format (PNG, JPEG, WebP, SVG, or PDF) via its magic-byte signature, and the Content-Type header is checked as well. If the response is empty or does not match the requested format, a PipelineDrawingError is raised and no file is written. This prevents a misconfigured or untrusted server_url from causing arbitrary content (for example an HTML error page) to be saved verbatim to the output path.

🐛 Bug Fixes

• Prevent Document.from_dict() from mutating the input dictionary during deserialization.
• Prevent DocumentLanguageClassifier from crashing when Document.content=None by marking them as unmatched and logging a warning.
• Fixed a bug where Agent would not exit when the model emitted multiple tool calls in a single turn and the configured exit-condition tool was not the first one in the list. Previously, only the first tool call in each assistant message was checked against exit_conditions, so a reply like [search, finish] (with exit_conditions=["finish"]) would silently fail to stop the loop and keep iterating until max_agent_steps was reached. Since parallel tool calls are now the norm for frontier models, this could quietly turn a single successful turn into dozens of wasted LLM calls. The Agent now inspects every tool call in the message, so the exit condition is honored regardless of ordering.
• Fix AnswerBuilder.run() mutating the meta dict of input Document objects. source_index (and referenced when reference_pattern is set) are now only added to the document copies inside GeneratedAnswer.documents, not to the originals.
• Fixed DocumentJoiner in concatenate mode so that documents with a score of exactly 0.0 are no longer treated as unscored during deduplication. Previously a truthiness check coerced score=0.0 to -inf, which could cause a worse, negatively-scored duplicate to be kept instead of the 0.0-scored document. The merge mode was updated to the same explicit is not None check for consistency; its observable behavior is unchanged.
• Fixed in-place mutation of ExtractedAnswer.meta in ExtractiveReader._add_answer_page_number when the answer's meta was None. Now uses dataclasses.replace to avoid triggering the dataclass mutation warning.
• Fixed ExtractiveReader raising ValueError when the number of valid answer spans for a sequence was smaller than answers_per_seq (for example with short documents or when answers_per_seq exceeded the number of upper-triangular, non-masked (start, end) token pairs). _postprocess now filters the per-sequence probabilities by the same validity mask it already applied to the start/end token indices, so the three structures always have matching lengths.
• HierarchicalDocumentSplitter no longer mutates the metadata of the input Document. _add_meta_data now returns a new Document with a copied meta dict via dataclasses.replace instead of writing __block_size, __parent_id, __children_ids and __level onto the caller's Document.
• Fixed a bug in LLMMetadataExtractor.run_async where the asyncio.Semaphore intended to bound concurrent LLM calls to max_workers was acquired once around the outer gather(...) call instead of inside each task. As a result, max_workers had no effect in run_async and all LLM requests for a batch were issued simultaneously. The semaphore is now acquired per task, so max_workers correctly caps in-flight requests.
• expand_page_range() now raises a ValueError: too many values to unpack when a page range string contained more than one hyphen (e.g. "10-20-30"). The parser now validates the format and raises a clear ValueError with an explanatory message for invalid inputs.
• LLMMetadataExtractor now raises a clear ValueError when the prompt contains no template variables. Previously this case raised an unhelpful IndexError: list index out of range. The error message now consistently expl...

v2.30.0-rc1

v2.30.0-rc1

v2.29.0

⭐️ Highlights

🔍 Combine Retrievers with MultiRetriever and TextEmbeddingRetriever

Two new retriever components make it easier to build hybrid search pipelines. MultiRetriever runs multiple text retrievers in parallel and merges their results into a single deduplicated list, ranked by reciprocal rank fusion by default. You can selectively enable or disable individual retrievers at runtime using the active_retrievers parameter. This is useful when you want to skip the embedding retriever for short or keyword-only queries, for example.

TextEmbeddingRetriever wraps an embedding-based retriever together with a text embedder into a single component, making it compatible with MultiRetriever by implementing the TextRetriever protocol. Here's how to combine BM25 and embedding retrieval in a single component:

from haystack.components.retrievers import MultiRetriever, TextEmbeddingRetriever
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever, InMemoryEmbeddingRetriever
from haystack.components.embedders import SentenceTransformersTextEmbedder

retriever = MultiRetriever(
    retrievers={
        "bm25": InMemoryBM25Retriever(document_store=doc_store),
        "embedding": TextEmbeddingRetriever(
            retriever=InMemoryEmbeddingRetriever(document_store=doc_store),
            text_embedder=SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"),
        ),
    },
    top_k=3,
)

# Run all retrievers
result = retriever.run(query="green energy sources")

# Run only the BM25 retriever
result = retriever.run(query="green energy sources", active_retrievers=["bm25"])

⬆️ Upgrade Notes

• LLM.run and LLM.run_async no longer accept messages and streaming_callback as positional arguments — they must now be passed as keyword arguments. Update any direct calls accordingly:

  # Before
  llm.run([message], my_callback)
  
  # After
  llm.run(messages=[message], streaming_callback=my_callback)

🚀 New Features

• Add run_async to CacheChecker, enabling it to be used in AsyncPipeline without blocking the event loop.

⚡️ Enhancement Notes

• Document the input ordering behavior of auto-promoted lazy variadic sockets in Pipeline.connect(). When multiple senders are connected to the same list-typed receiver socket, ordering depends on the pipeline class. With Pipeline, items are ordered alphabetically by sender component name (because Pipeline.run() schedules components in alphabetical order for deterministic execution), not by the order of connect() calls. With AsyncPipeline, no ordering is guaranteed, since components in different branches may run in parallel. The docstrings now point users to a dedicated joiner component when they need explicit ordering.
• Add join_mode parameter to the experimental MultiRetriever component, supporting "reciprocal_rank_fusion" (default) and "concatenate". Reciprocal Rank Fusion merges the ranked result lists from all retrievers into a single deduplicated list ordered by RRF score. The underlying RRF logic is extracted into a shared utility _reciprocal_rank_fusion in haystack.utils.misc, which is now also used by DocumentJoiner.
• LLM now supports two usage modes:
  1. Template-variable mode: provide a user_prompt with Jinja2 variables (e.g. {{ query }}).
     Those variables become pipeline inputs and messages is optional. The rendered user_prompt
     is always appended after any messages provided at runtime.
  2. Pass-through mode: omit user_prompt or provide one with no template variables. messages
     becomes a required input, allowing a fully-constructed list of ChatMessages to be passed from upstream.

🐛 Bug Fixes

• Fixed a bug in NamedEntityExtractor where the spaCy/Thinc device state was not correctly restored after execution, potentially affecting the device configuration of other spaCy components in the same process.
• Preserve resumable snapshots when some inputs or outputs are non-serializable. Haystack now omits only the failing top-level fields (for example non-serializable callbacks or runtime objects) instead of replacing the whole payload with an empty dictionary. This applies both to agent sub-component inputs (chat_generator and tool_invoker) and to pipeline-level inputs, original_input_data, and pipeline_outputs captured by _create_pipeline_snapshot. When every field fails to serialize, the snapshot still stores a structurally valid empty payload ({"serialization_schema": {"type": "object", "properties": {}}, "serialized_data": {}}) so that resuming the snapshot does not raise DeserializationError — for example when resuming from a ToolBreakpoint where the sub-component's inputs are not strictly required.
• Fixed tools_strict=True in OpenAIChatGenerator to recursively apply additionalProperties: false and required to all nested objects in tool parameter schemas. Previously only the top-level object was transformed, causing OpenAI's strict mode to reject tools with nested parameters.

💙 Big thank you to everyone who contributed to this release!

@Aftabbs, @albertodiazdurana, @anakin87, @ArkaD171717, @bilgeyucel, @bogdankostic, @davidsbatista, @FuturMix, @julian-risch, @kacperlukawski, @ritikraj2425, @saivedant169, @shaun0927, @sjrl, @SyedShahmeerAli12