feat: add tool result offload compactor

[sjrl]

Related Issues

• fixes https://github.com/deepset-ai/haystack-private/issues/524

Proposed Changes:

Added the experimental ToolResultOffloadCompactor for use with CompactionHook. It keeps fresh tool output directly in the Agent's context, then writes eligible older results to a ToolResultStore only when context compaction is triggered. The compacted conversation retains a reference and configurable preview, allowing the Agent to retrieve the full stored output later with a read tool scoped to the same store. Image and file parts of a result are offloaded too when the store sets supports_binary_content.

The store-writing and pointer-building logic moved out of ToolResultOffloadHook into haystack/hooks/tool_result_offloading/utils.py so the hook and the compactor share the same reference format and tool_result_offloaded metadata. The oldest-first replace-until-target loop that ToolResultPruningCompactor already had now lives in haystack/hooks/compaction/utils.py, and both compactors use it.

from typing import Annotated

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.hooks.compaction import CompactionHook, ToolResultOffloadCompactor
from haystack.hooks.tool_result_offloading import FileSystemToolResultStore
from haystack.tools import tool

store = FileSystemToolResultStore(root="tool_results")


@tool
def read_offloaded_result(path: Annotated[str, "Path of an offloaded tool result"]) -> str:
    """Read back the full content of an offloaded tool result."""
    content = store.read(path)
    if isinstance(content, bytes):
        return f"'{path}' holds {len(content)} bytes of binary content and cannot be read as text."
    return content


hook = CompactionHook(compactor=ToolResultOffloadCompactor(store=store), context_window=400_000)
agent = Agent(
    chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-nano"),
    tools=[web_search, read_offloaded_result],
    hooks={"before_llm": [hook]},
)

web_search stands in for any tool whose results can grow large. For results with images or files, pass a provider token counter such as OpenAITokenCounter to CompactionHook; the default ApproximateTokenCounter charges a flat rate per image and file.

How did you test it?

New tests

Notes for the reviewer

The docs page for this feature is in a separate PR: #13119

Known limitations, tracked as follow-ups:

• The compactor always writes to the store it was created with and does not read a per-run store from hook_context like ToolResultOffloadHook does, so a multi-user server currently needs an Agent per run: https://github.com/deepset-ai/haystack-private/issues/591
• A failing store.write stops the Agent run instead of leaving the result in context. This applies to ToolResultOffloadHook too: https://github.com/deepset-ai/haystack-private/issues/592

Known limitation: when a tool call has no id, the store key falls back to the message's position in the conversation. If another compactor (e.g. SlidingWindowCompactor) drops messages between two compaction rounds in the same run, a later result can land on the same position and overwrite a stored entry that an earlier reference still points to. Providers almost always set tool call ids, so this is left as is for now.

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
haystack-docs	Ignored	Preview	Oct 9, 2026 9:44am UTC

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/hooks/compaction
__init__.py
tool_result_offloading.py					164
tool_result_pruning.py
utils.py
haystack/hooks/tool_result_offloading
hooks.py					253
utils.py
haystack/hooks/tool_result_offloading/types
protocol.py
Project Total

This report was generated by python-coverage-comment-action

[sjrl]

Changes to this file are for moving common functionality out into utils.py

[sjrl]

This file contains shared utility functions that used to live in hooks.py The utility functions are used in tool_result_offloading.py

[bogdankostic]

Looks good in general, just found one case where the keys of distinct tool calls might collide.

[bogdankostic]

web_search is not defined here, so this code block can't be used out of the box. Should we add a comment about this?

[sjrl]

Added a comment noting web_search stands in for the user's own tool.

[bogdankostic]

I don't think this happens often in practice, but with OllamaChatGenerator, for example, tool calls don't have an ID, so we would fall back to f'call{index}'. Two separate runs on the same Agent can then get the same keys for results at the same position, with the second run overwriting the first one's results. If the first conversation is continued in a later run, e.g. a chat app passing the history back, the model would read the other run's result.

[sjrl]

Thanks, good catch. Keys now get a random suffix so separate runs never share a key, and I added a regression test that runs compact() twice on the same store.

[bogdankostic]

LGTM