Skip to content

[BUG] Azure streaming completions merge parallel tool calls into one corrupted call #7486

Description

@BlueX888

AzureCompletion._process_streaming_update accumulates streamed tool calls keyed on the position of each tool_call inside choice.delta.tool_calls (enumerate(...)), instead of on the index field of the tool call itself.

Azure AI Inference is OpenAI-compatible: in streaming mode it emits one tool_call entry per chunk, and that entry carries its own index field identifying which call the delta belongs to. With parallel tool calls, chunks for the second call arrive with index: 1 but sit at position 0 within their own chunk. So enumerate yields 0 for every chunk and all deltas collapse into a single accumulator slot.

The executor therefore receives one tool call instead of N, with the id of the first call, the name of the last, and the argument fragments concatenated in arrival order (usually invalid JSON).

Steps to Reproduce

from unittest.mock import patch
from azure.ai.inference.models import StreamingChatCompletionsUpdate
from crewai.llms.providers.azure.completion import AzureCompletion


def chunk(index, call_id, name, arguments):
    return StreamingChatCompletionsUpdate(
        id="chatcmpl-1", model="gpt-4o", created=None,
        choices=[{
            "index": 0,
            "delta": {"role": "assistant", "content": None, "tool_calls": [{
                "index": index, "id": call_id, "type": "function",
                "function": {"name": name, "arguments": arguments},
            }]},
            "finish_reason": None,
        }],
    )


chunks = [
    chunk(0, "call_aaa", "get_weather", '{"city":'),
    chunk(0, "call_aaa", "get_weather", '"Paris"}'),
    chunk(1, "call_bbb", "get_time", '{"tz":'),
    chunk(1, "call_bbb", "get_time", '"UTC"}'),
]

llm = AzureCompletion(model="gpt-4o", api_key="k",
                      endpoint="https://test.openai.azure.com", stream=True)
with patch.object(llm._client, "complete") as m:
    m.return_value = iter(chunks)
    result = llm.call([{"role": "user", "content": "Weather and time in Paris?"}])

print(result)

Four chunks, alternating between the two parallel calls, exactly as the Azure streaming API delivers them.

Expected behavior

Two tool calls, each intact:

[
    {"id": "call_aaa", "type": "function",
     "function": {"name": "get_weather", "arguments": '{"city":"Paris"}'}},
    {"id": "call_bbb", "type": "function",
     "function": {"name": "get_time", "arguments": '{"tz":"UTC"}'}},
]

Screenshots/Code snippets

Actual output — one call, wrong name, and the two argument fragments concatenated into invalid JSON:

[{'id': 'call_aaa', 'type': 'function',
  'function': {'name': 'get_time', 'arguments': '{"city":"Paris"}{"tz":"UTC"}'}}]

Root cause is lib/crewai/src/crewai/llms/providers/azure/completion.py around line 1000:

for idx, tool_call in enumerate(choice.delta.tool_calls):
    if idx not in tool_calls:
        tool_calls[idx] = {...}

For reference, the OpenAI provider in this repo already keys on the wire index (tool_call.index, e.g. lib/crewai/src/crewai/llms/providers/openai/completion.py (see the tool_index = tool_call.index aggregation)), so the two providers disagree on parallel tool calls.

Additional context

  • The Azure StreamingChatResponseToolCallUpdate model is a MutableMapping and does not declare index as an attribute (getattr(tc, "index") raises AttributeError), but the raw key survives deserialization, so tool_call.get("index") reads it correctly.
  • Anything that issues two or more tool calls in one Azure streaming turn is affected; only the first call's id and only the last call's name survive, and the concatenated arguments fail json.loads.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions