Repository navigation
fix(provider): parallel tool calls with distinct ids no longer merge into one slot - #21
Open
juicycleff wants to merge 2 commits into
Open
juicycleff wants to merge 2 commits into
juicycleff wants to merge 2 commits into
Conversation
…into one slot Anthropic, Bedrock, Gemini Live and the OpenAI realtime stream emit every tool-call delta as a one-element slice, so every parallel call arrives at index 0. The accumulator's index fallback matched the first slot for the second call's id, glued the second call's JSON onto the first call's arguments and dropped the second call. Resuming that turn then failed with 'tool_use.input: Input should be an object'. A delta whose id differs from the call already parked at its index now opens its own slot under the first free key; id-less fragments keep merging by index. Tests: go test ./provider/ ./providers/anthropic/ (RED: 1 tool call with concatenated arguments; GREEN after the fix), go test ./... clean
govulncheck fails CI on go.opentelemetry.io/otel/sdk v1.44.0, which the extension reaches through forge's dashboard init. The same advisory covers the otlptrace exporters at v1.43.0, so they move to v1.45.0 too, and the otel core modules follow in lockstep. grpcsrv and _examples/grpc replace nexus with the root module, so their go.mod files are tidied to match or GOWORK=off builds ask for a tidy. Checked: govulncheck on go1.26.6 reports 0 affecting vulnerabilities, go build and go test pass in the workspace, every nested module vets with GOWORK=off, golangci-lint reports 0 issues.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If you stream a turn where the model calls two tools at once through Anthropic, Bedrock, Gemini Live or the OpenAI realtime API, you get one tool call back instead of two. The first call's arguments come back with the second call's JSON glued onto the end, and the second call is gone. The next request that replays that turn then fails with
tool_use.input: Input should be an object.The cause is in
provider/accumulate.go. Those four streams send every tool-call delta as a one-element slice, so every parallel call arrives at index 0. When the second call's id wasn't known yet,mergeToolfell back to whatever was parked at index 0 and merged into it. The index fallback now applies only when the incoming delta has no id, the parked call has no id, or the two ids match. A delta with a different id opens its own slot under the first free key, andtoolOrderstill records the order the calls arrived in. OpenAI-style streams, which name a call once and then send id-less fragments at the same index, merge the same way they did before.Tests
provider/accumulate_parallel_test.gofeeds the accumulator two calls with distinct ids at index 0, plus a trailing empty delta, and checks you get two calls that each keep their own arguments. A second test pins the OpenAI-style path: two named calls followed by id-less fragments still merge by index.providers/anthropic/stream_parallel_tools_test.goreplays the Anthropic wire sequence that broke (twotool_useblocks, each streaminginput_json_delta) through the mock server.We ran both distinct-id tests against the old
accumulate.goand they fail with the bug itself:got 1 tool calls, want 2, and the one call carries{"id": "layer_a", "minutes": 30}{"id": "mk-1"}as its arguments. The id-less test passes before and after, as it should.Checked on this branch:
go test -raceover./provider/...and every module underproviders/, thengo build ./... && go test ./..., thengolangci-lint run ./...on a fresh cache with 0 issues. Mind you, each provider is its own module ingo.work, sogo test ./providers/...from the root matches nothing. You have to run them module by module.Against the open dependabot PRs
None of them touch these files. #20 is the docs site's npm lockfile, #12 and #11 are workflow files. #19 bumps templ, the mongo driver and grpc in
go.mod, andaccumulate.goimports only the standard library, so the two can merge in either order.