Description
Bug description
When using an Anthropic ChatModel
with a streaming chat response (StreamResponseSpec#chatResponse
into Flux<ChatResponse>
), tools that take in zero parameters are not invoked.
The JSON content returned by Anthropic's text/event-stream response looks like:
{"type":"message_start","message":{"id":"msg_01CNBhjoXJNRjoDHX5jLhWa5","type":"message","role":"assistant","model":"claude-3-7-sonnet-20250219","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":3154,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":3}} }
...
{"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_01Pi8emvHyK6qoLUVjcnmDCr","name":"hello_world","input":{}} }
{"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""} }
{"type":"content_block_stop","index":1 }
When parsing the event with type EventType.CONTENT_BLOCK_STOP
, the ToolUseAggregationEvent
is not returned due to the partialJson
being an empty String. In my IDE, this looks like:
where you can see the !eventAggregator.isEmpty()
only passes if partialJson
has a non-empty value. That's resulting in StreamHelper#mergeToolUseEvents
returning the ContentBlockStopEvent
instead of the ToolUseAggregationEvent
being finished and returned.
In my opinion, either the isEmpty
check should still return true if the partialJson
is empty, or if we need to avoid the squashIntoContentBlock()
call, a different definition should be checked if eventAggregatgion
applies on ContentBlockStop
(this would require the getToolContentBlocks()
check to be modified downstream as well).
Environment
spring-ai 1.0.0-M7
Amazon Corretto JDK 21
Steps to reproduce
Using Anthropic AutoConfiguration, invoke:
chatClient.prompt().user("Call the hello_world tool")).stream().chatResponse();
with a simple nullary tool defined:
@Tool(name = "hello_world")
String helloWorld() {
return "Hi spring-ai!";
}
Set a breakpoint in that tool, or in StreamHelper as per the above screenshot. You will see that the tool is never invoked.
Expected behavior
Nullary tools should still be invoked when streaming -- I note that the tool call works normally the same as a function with positive arity (still running into #2670) if I manually set the partialJson
to "{}"
in the debugger
This seems similar to this same issue involving Suppliers with Bedrock Converse (#1878), though the code is separate. While AnthropicApi has its own StreamHelper class; Bedrock Converse has a similar check for isEmpty() in ConverseApiUtils with its own forked ToolUseAggregationEvent class.