Skip to content

fix(provider): parallel tool calls with distinct ids no longer merge into one slot - #21

Open
juicycleff wants to merge 2 commits into
mainfrom
fix/accumulator-parallel-tool-calls
Open

juicycleff wants to merge 2 commits into
mainfrom
fix/accumulator-parallel-tool-calls

Conversation

@juicycleff

Copy link
Copy Markdown
Contributor

If you stream a turn where the model calls two tools at once through Anthropic, Bedrock, Gemini Live or the OpenAI realtime API, you get one tool call back instead of two. The first call's arguments come back with the second call's JSON glued onto the end, and the second call is gone. The next request that replays that turn then fails with tool_use.input: Input should be an object.

The cause is in provider/accumulate.go. Those four streams send every tool-call delta as a one-element slice, so every parallel call arrives at index 0. When the second call's id wasn't known yet, mergeTool fell back to whatever was parked at index 0 and merged into it. The index fallback now applies only when the incoming delta has no id, the parked call has no id, or the two ids match. A delta with a different id opens its own slot under the first free key, and toolOrder still records the order the calls arrived in. OpenAI-style streams, which name a call once and then send id-less fragments at the same index, merge the same way they did before.

Tests

  • provider/accumulate_parallel_test.go feeds the accumulator two calls with distinct ids at index 0, plus a trailing empty delta, and checks you get two calls that each keep their own arguments. A second test pins the OpenAI-style path: two named calls followed by id-less fragments still merge by index.
  • providers/anthropic/stream_parallel_tools_test.go replays the Anthropic wire sequence that broke (two tool_use blocks, each streaming input_json_delta) through the mock server.

We ran both distinct-id tests against the old accumulate.go and they fail with the bug itself: got 1 tool calls, want 2, and the one call carries {"id": "layer_a", "minutes": 30}{"id": "mk-1"} as its arguments. The id-less test passes before and after, as it should.

Checked on this branch: go test -race over ./provider/... and every module under providers/, then go build ./... && go test ./..., then golangci-lint run ./... on a fresh cache with 0 issues. Mind you, each provider is its own module in go.work, so go test ./providers/... from the root matches nothing. You have to run them module by module.

Against the open dependabot PRs

None of them touch these files. #20 is the docs site's npm lockfile, #12 and #11 are workflow files. #19 bumps templ, the mongo driver and grpc in go.mod, and accumulate.go imports only the standard library, so the two can merge in either order.

…into one slot

Anthropic, Bedrock, Gemini Live and the OpenAI realtime stream emit every tool-call delta as a one-element slice, so every parallel call arrives at index 0. The accumulator's index fallback matched the first slot for the second call's id, glued the second call's JSON onto the first call's arguments and dropped the second call. Resuming that turn then failed with 'tool_use.input: Input should be an object'. A delta whose id differs from the call already parked at its index now opens its own slot under the first free key; id-less fragments keep merging by index.

Tests: go test ./provider/ ./providers/anthropic/ (RED: 1 tool call with concatenated arguments; GREEN after the fix), go test ./... clean
govulncheck fails CI on go.opentelemetry.io/otel/sdk v1.44.0, which the
extension reaches through forge's dashboard init. The same advisory covers
the otlptrace exporters at v1.43.0, so they move to v1.45.0 too, and the
otel core modules follow in lockstep. grpcsrv and _examples/grpc replace
nexus with the root module, so their go.mod files are tidied to match or
GOWORK=off builds ask for a tidy.

Checked: govulncheck on go1.26.6 reports 0 affecting vulnerabilities, go
build and go test pass in the workspace, every nested module vets with
GOWORK=off, golangci-lint reports 0 issues.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant