[Bug]: Streaming Anthropic Web Search broken in 1.80.6 and 1.80.7

[RensDimmendaal]

What happened?

When using litellm with anthropic web search and stream=True, trailing {} are added to the results. This is not the case when calling Anthropic directly. My guess is this was an existing issue, but wasn't yet observed because search results were not included in the stream chunk builder because their index was -1?

Here's an ipython snippet that uses uv to to reproduce the result in different versions litellm.

versions = ["1.80.7", "1.80.6", "1.80.5"]
for v in versions:
    script = f'''# /// script
# requires-python = ">=3.12"
# dependencies = ["litellm=={v}"]
# ///
from litellm import completion, stream_chunk_builder

resp = completion(
    model="claude-sonnet-4-5-20250929",
    messages=[{{"role": "user", "content": "Search the web and tell me a fun otter fact."}}],
    stream=True,
    web_search_options={{"search_context_size": "low"}}
)

chunks = []
for chunk in resp:
    chunks.append(chunk)
    if hasattr(chunk.choices[0].delta, 'tool_calls') and chunk.choices[0].delta.tool_calls:
        tc = chunk.choices[0].delta.tool_calls[0]
        print(f"index={{getattr(tc, 'index', None)}} id={{getattr(tc, 'id', None)}} func={{tc.function}}")

result = stream_chunk_builder(chunks)
print("Stream Chunk Builder Tool Calls:", result.choices[0].message.tool_calls)
'''
    with open(f"test_litellm_{v}.py", "w") as f: f.write(script)
    print(f"==== Litellm=={v}: ====")
    !uv run test_litellm_{v}.py 2>/dev/null

==== Litellm==1.80.7: ====
index=0 id=srvtoolu_01ToAk6amujEpVrkC7r59P8X func=Function(arguments='', name='web_search')
index=0 id=None func=Function(arguments='', name=None)
index=0 id=None func=Function(arguments='{"quer', name=None)
index=0 id=None func=Function(arguments='y": ', name=None)
index=0 id=None func=Function(arguments='"fun', name=None)
index=0 id=None func=Function(arguments=' o', name=None)
index=0 id=None func=Function(arguments='tte', name=None)
index=0 id=None func=Function(arguments='r facts"}', name=None)
index=0 id=None func=Function(arguments='{}', name=None)
index=0 id=None func=Function(arguments='{}', name=None)
Stream Chunk Builder Tool Calls: [ChatCompletionMessageToolCall(function=Function(arguments='{"query": "fun otter facts"}{}{}', name='web_search'), id='srvtoolu_01ToAk6amujEpVrkC7r59P8X', type='function')]
==== Litellm==1.80.6: ====
index=0 id=srvtoolu_01RFrfcBjwG2VS3eYWJzgjZa func=Function(arguments='', name='web_search')
index=0 id=None func=Function(arguments='', name=None)
index=0 id=None func=Function(arguments='{"qu', name=None)
index=0 id=None func=Function(arguments='ery": "fun ', name=None)
index=0 id=None func=Function(arguments='otter fac', name=None)
index=0 id=None func=Function(arguments='ts"}', name=None)
index=0 id=None func=Function(arguments='{}', name=None)
index=0 id=None func=Function(arguments='{}', name=None)
Stream Chunk Builder Tool Calls: [ChatCompletionMessageToolCall(function=Function(arguments='{"query": "fun otter facts"}{}{}', name='web_search'), id='srvtoolu_01RFrfcBjwG2VS3eYWJzgjZa', type='function')]
==== Litellm==1.80.5: ====
index=-1 id=None func=Function(arguments='', name=None)
index=-1 id=None func=Function(arguments='{"qu', name=None)
index=-1 id=None func=Function(arguments='ery":', name=None)
index=-1 id=None func=Function(arguments=' "f', name=None)
index=-1 id=None func=Function(arguments='un otter', name=None)
index=-1 id=None func=Function(arguments=' facts', name=None)
index=-1 id=None func=Function(arguments='"}', name=None)
index=-1 id=None func=Function(arguments='{}', name=None)
index=-1 id=None func=Function(arguments='{}', name=None)
Stream Chunk Builder Tool Calls: []

Using Anthropic directly does not add the trailing {}:

curl https://api.anthropic.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4-5-20250929",
    "max_tokens": 1024,
    "stream": true,
    "messages": [{"role": "user", "content": "Search the web and tell me a fun otter fact."}],
    "tools": [{"type": "web_search_20250305", "name": "web_search", "max_uses": 1}]
  }'

event: message_start
data: {"type":"message_start","message":{"model":"claude-sonnet-4-5-20250929","id":"msg_01TmkihjeV9Ffgq9PiuD2vLc","type":"message","role":"assistant","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":2225,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":1,"service_tier":"standard"}}            }

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"server_tool_use","id":"srvtoolu_01Wmv5uX99hvYYhQRcm5g9Eq","name":"web_search","input":{}}    }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":""}   }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"qu"}    }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"ery\": \"f"}               }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"un ot"}   }

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"ter facts\"}"}            }

event: content_block_stop
data: {"type":"content_block_stop","index":0    }

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.80.7

Twitter / LinkedIn details

No response

[KeremTurgutlu]

Additional Info

There is even a bigger issue than the trailing parentheses which is that web_search being returned as a tool call that appears like it should be invoked. Claude API can distinguish between server tool calls (ServerToolUseBlock) and custom tool calls (ToolUseBlock) so that user can know when to generate the tool result.

[jph00]

@krrishdholakia @Chesars IIUC this can be closed now - fixed by #17737

[krrishdholakia]

thanks @jph00