Motivation
Work on #2092, based on Erik Tews's tool-failure use case, validated the current architectural boundary:
- tools report structured facts;
- hooks decide whether the agent continues or terminates;
StepEvent::ToolResult exposes structured outcomes and typed extensions;
- the scratchpad supports run-scoped cooperation between hooks.
The example works with the current API, but implementing and testing it exposed several opportunities to make this pattern safer and easier for users.
Potential improvements
1. Make typed metadata easier to attach to tool errors
Today, attaching ToolResultExtensions to a failed tool execution requires overriding Tool::call_structured, manually calling classify_error, and returning Ok(ToolReturn::failed(...)) with extensions.
Consider a richer error-classification API that can provide both the standard ToolFailure and typed result extensions. Possible shapes include:
- an additional
error_extensions(&Self::Error) -> ToolResultExtensions method; or
- a richer classification/report type containing the failure and extensions.
The goal is to let a tool report machine-readable error context without taking responsibility for agent lifecycle decisions.
2. Provide per-tool-call scratchpad state
The scratchpad is keyed by type and shared across the run. When tool calls execute concurrently, users must build and correctly maintain their own map keyed by internal_call_id.
A call-scoped facility conceptually similar to:
ctx.scratchpad().for_call(internal_call_id)
could make the safe pattern easier and prevent implementations that accidentally use the latest unrelated result. Any design should preserve the documented concurrency semantics and avoid relying on tool completion order.
3. Add a hook testing harness to test-utils
HookContext is intentionally constructed by the runner and cannot be created by downstream tests. As a result, testing custom hooks and scratchpad cooperation requires driving an entire agent with MockCompletionModel.
A focused test harness could support:
- setting run metadata and the current turn;
- dispatching selected
StepEvents;
- registering a
HookStack in order;
- inspecting the resulting
Flow and scratchpad state;
- exercising concurrent/correlated tool-call scenarios.
Full mock-runner tests would remain valuable, but simple hook policies would become easier to test directly.
4. Clarify when to use ToolResult data directly versus the scratchpad
A hook handling StepEvent::ToolResult can inspect outcome and extensions directly. The scratchpad is primarily useful when:
- multiple hooks cooperate;
- state must survive across different event types;
- failures or budgets accumulate across a run.
Documentation and examples should make this distinction explicit so users do not copy data into the scratchpad when the same hook can act on the event directly.
5. Highlight per-event hook ordering versus cross-call concurrency
For a single ToolResult, hooks run in registration order. With concurrent tools, events from different calls may interleave while sharing the same scratchpad.
This distinction is already documented, but it is easy to implement an unsafe policy such as reading ledger.last(). A prominent example should demonstrate correlation by internal_call_id and explain that registration order does not imply global ordering across calls.
6. Consider graceful completion separately from cancellation
Flow::Terminate currently surfaces as PromptError::PromptCancelled, which is appropriate for aborting after a fatal tool failure. Some applications may instead want to stop the loop successfully with an application-provided final result or explanation.
A distinct concept such as Flow::Finish may be worth discussing. This is a broader product/API decision and should not be coupled to the other ergonomic improvements without considering:
- streaming and non-streaming parity;
- transcript persistence;
- final response types;
- hook-stack short-circuiting;
- telemetry semantics.
Suggested prioritization
The most immediately actionable items appear to be:
- structured extensions on classified errors;
- safer call-scoped scratchpad ergonomics;
- a downstream hook testing harness.
The documentation changes can accompany those improvements. Graceful completion should be evaluated independently as a larger API decision.
Reference
Motivation
Work on #2092, based on Erik Tews's tool-failure use case, validated the current architectural boundary:
StepEvent::ToolResultexposes structured outcomes and typed extensions;The example works with the current API, but implementing and testing it exposed several opportunities to make this pattern safer and easier for users.
Potential improvements
1. Make typed metadata easier to attach to tool errors
Today, attaching
ToolResultExtensionsto a failed tool execution requires overridingTool::call_structured, manually callingclassify_error, and returningOk(ToolReturn::failed(...))with extensions.Consider a richer error-classification API that can provide both the standard
ToolFailureand typed result extensions. Possible shapes include:error_extensions(&Self::Error) -> ToolResultExtensionsmethod; orThe goal is to let a tool report machine-readable error context without taking responsibility for agent lifecycle decisions.
2. Provide per-tool-call scratchpad state
The scratchpad is keyed by type and shared across the run. When tool calls execute concurrently, users must build and correctly maintain their own map keyed by
internal_call_id.A call-scoped facility conceptually similar to:
could make the safe pattern easier and prevent implementations that accidentally use the latest unrelated result. Any design should preserve the documented concurrency semantics and avoid relying on tool completion order.
3. Add a hook testing harness to
test-utilsHookContextis intentionally constructed by the runner and cannot be created by downstream tests. As a result, testing custom hooks and scratchpad cooperation requires driving an entire agent withMockCompletionModel.A focused test harness could support:
StepEvents;HookStackin order;Flowand scratchpad state;Full mock-runner tests would remain valuable, but simple hook policies would become easier to test directly.
4. Clarify when to use
ToolResultdata directly versus the scratchpadA hook handling
StepEvent::ToolResultcan inspectoutcomeandextensionsdirectly. The scratchpad is primarily useful when:Documentation and examples should make this distinction explicit so users do not copy data into the scratchpad when the same hook can act on the event directly.
5. Highlight per-event hook ordering versus cross-call concurrency
For a single
ToolResult, hooks run in registration order. With concurrent tools, events from different calls may interleave while sharing the same scratchpad.This distinction is already documented, but it is easy to implement an unsafe policy such as reading
ledger.last(). A prominent example should demonstrate correlation byinternal_call_idand explain that registration order does not imply global ordering across calls.6. Consider graceful completion separately from cancellation
Flow::Terminatecurrently surfaces asPromptError::PromptCancelled, which is appropriate for aborting after a fatal tool failure. Some applications may instead want to stop the loop successfully with an application-provided final result or explanation.A distinct concept such as
Flow::Finishmay be worth discussing. This is a broader product/API decision and should not be coupled to the other ergonomic improvements without considering:Suggested prioritization
The most immediately actionable items appear to be:
The documentation changes can accompany those improvements. Graceful completion should be evaluated independently as a larger API decision.
Reference