Skip to content

AnthropicApi does not invoke nullary function tools when using streaming #2735

Open
@asw12

Description

@asw12

Bug description
When using an Anthropic ChatModel with a streaming chat response (StreamResponseSpec#chatResponse into Flux<ChatResponse>), tools that take in zero parameters are not invoked.

The JSON content returned by Anthropic's text/event-stream response looks like:

{"type":"message_start","message":{"id":"msg_01CNBhjoXJNRjoDHX5jLhWa5","type":"message","role":"assistant","model":"claude-3-7-sonnet-20250219","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":3154,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":3}}   }
...
{"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_01Pi8emvHyK6qoLUVjcnmDCr","name":"hello_world","input":{}}            }
{"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}       }
{"type":"content_block_stop","index":1   }

When parsing the event with type EventType.CONTENT_BLOCK_STOP, the ToolUseAggregationEvent is not returned due to the partialJson being an empty String. In my IDE, this looks like:
Image
where you can see the !eventAggregator.isEmpty() only passes if partialJson has a non-empty value. That's resulting in StreamHelper#mergeToolUseEvents returning the ContentBlockStopEvent instead of the ToolUseAggregationEvent being finished and returned.

In my opinion, either the isEmpty check should still return true if the partialJson is empty, or if we need to avoid the squashIntoContentBlock() call, a different definition should be checked if eventAggregatgion applies on ContentBlockStop (this would require the getToolContentBlocks() check to be modified downstream as well).

Environment
spring-ai 1.0.0-M7
Amazon Corretto JDK 21

Steps to reproduce
Using Anthropic AutoConfiguration, invoke:

chatClient.prompt().user("Call the hello_world tool")).stream().chatResponse();

with a simple nullary tool defined:

@Tool(name = "hello_world")
String helloWorld() {
	return "Hi spring-ai!";
}

Set a breakpoint in that tool, or in StreamHelper as per the above screenshot. You will see that the tool is never invoked.

Expected behavior
Nullary tools should still be invoked when streaming -- I note that the tool call works normally the same as a function with positive arity (still running into #2670) if I manually set the partialJson to "{}" in the debugger

This seems similar to this same issue involving Suppliers with Bedrock Converse (#1878), though the code is separate. While AnthropicApi has its own StreamHelper class; Bedrock Converse has a similar check for isEmpty() in ConverseApiUtils with its own forked ToolUseAggregationEvent class.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions