Skip to content

Conversation

@Zhikaiiii
Copy link
Contributor

@Zhikaiiii Zhikaiiii commented Oct 7, 2025

Purpose

  1. Fix bug of qwen3_xml tool parser mentioned in [Frontend] Add a new xml-based tool parser for qwen3-coder #25028 (comment)
  2. Fix bug of qwen3_xml tool parser when missing open tool_call tag mentioned in [Frontend] Add a new xml-based tool parser for qwen3-coder #25028 (comment)
  3. The corresponding test have been added.

Test Plan

We run the unit-test to ensure the parser result is correct.

pytest -v -s tests/tool_use/test_qwen3coder_tool_parser.py

We also launch an api-server and use a tool-call curl to ensure the problem is fixed.

vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct --dtype auto --served-model-name qwen3_coder \
    --enable-auto-tool-choice --tool-call-parser qwen3_xml --max-model-len 131072
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3_coder",
    "messages": [
      {
        "role": "user",
        "content": "北京天气怎么样?"
      }
    ],
    "stream": true,
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "获取指定城市的天气信息",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "城市名称,例如 北京、杭州"
              }
            },
            "required": ["location"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'

Test Result

Unit test results:

collected 19 items                                                                                                                                                                              

tests/tool_use/test_qwen3coder_tool_parser.py ...................                                                                                                                         [100%]

======================================================================================= warnings summary ========================================================================================
vllm/__init__.py:7
  /tuatara-repo/dev_workspace/wuzhikai.wzk/code_repo/vllm/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash:
  No module named 'vllm._version'
    from .version import __version__, __version_tuple__  # isort:skip

<frozen importlib._bootstrap>:241
  <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:241
  <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================ 19 passed, 3 warnings in 7.38s =================================================================================

The respons of api-server

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}],"prompt_token_ids":null}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"tool_calls":[{"id":"chatcmpl-tool-d4cfb0be25674db7939e2f08390975c8","type":"function","index":0,"function":{"name":"get_weather","arguments":""}}]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null,"tool_calls":[{"id":"chatcmpl-tool-d4cfb0be25674db7939e2f08390975c8","type":"function","index":0,"function":{"name":null,"arguments":"{\"location\": \""}}]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"tool_calls":[{"id":"chatcmpl-tool-d4cfb0be25674db7939e2f08390975c8","type":"function","index":0,"function":{"name":null,"arguments":"北京"}}]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"tool_calls":[{"id":"chatcmpl-tool-d4cfb0be25674db7939e2f08390975c8","type":"function","index":0,"function":{"name":null,"arguments":""}}]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"tool_calls":[{"id":"chatcmpl-tool-d4cfb0be25674db7939e2f08390975c8","type":"function","index":0,"function":{"name":null,"arguments":"\""}}]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":null},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"tool_calls":[{"id":"chatcmpl-tool-d4cfb0be25674db7939e2f08390975c8","type":"function","index":0,"function":{"name":null,"arguments":"}"}}]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"tool_calls":[{"id":"chatcmpl-tool-d4cfb0be25674db7939e2f08390975c8","type":"function","index":0,"function":{"name":null,"arguments":""}}]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-9b7984d842c943638ce8675c4688997f","object":"chat.completion.chunk","created":1759831811,"model":"qwen3_coder","choices":[{"index":0,"delta":{"content":""},"logprobs":null,"finish_reason":"tool_calls","stop_reason":null,"token_ids":null}]}

data: [DONE]

Signed-off-by: Zhikaiiii <1658973216@qq.com>
@mergify mergify bot added frontend qwen Related to Qwen models tool-calling labels Oct 7, 2025
@mergify
Copy link

mergify bot commented Oct 7, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Zhikaiiii.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Qwen3 XML tool parser to fix a bug, likely related to streaming tool calls, by improving its compatibility with serving_chat.py. The changes include switching to a centralized make_tool_call_id utility for consistency and adding state tracking attributes (prev_tool_call_arr, streamed_args_for_tool) that are now correctly managed in both streaming and non-streaming contexts. While the fix for streaming appears correct, I've identified a critical type inconsistency in the non-streaming extract_tool_calls method that could lead to runtime errors.

Comment on lines 1105 to 1108
self.prev_tool_call_arr.append({
"name": "",
"arguments": {}
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

There is a type inconsistency in how the arguments field is handled for self.prev_tool_call_arr. It is initialized as a dictionary ({}), but later at line 1116, it is assigned a string value from tool_call.function.arguments. This mismatch can lead to TypeError exceptions in downstream code that expects a consistent type. The corresponding logic in the extract_tool_calls_streaming method correctly initializes arguments as an empty string (""). To ensure type consistency and prevent potential runtime errors, you should initialize arguments as an empty string here as well.

Suggested change
self.prev_tool_call_arr.append({
"name": "",
"arguments": {}
})
self.prev_tool_call_arr.append({
"name": "",
"arguments": ""
})

Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: Zhikaiiii <1658973216@qq.com>
self.streamed_args_for_tool[tool_index] += (
tool_call.function.arguments
)
return result

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add corresponding test case to better testing and coverage

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: Zhikaiiii <1658973216@qq.com>
Copy link
Collaborator

@chaunceyjiang chaunceyjiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@chaunceyjiang chaunceyjiang added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 14, 2025
@chaunceyjiang chaunceyjiang merged commit 9354660 into vllm-project:main Oct 15, 2025
49 checks passed
bbartels pushed a commit to bbartels/vllm that referenced this pull request Oct 16, 2025
Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: bbartels <benjamin@bartels.dev>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
Signed-off-by: Zhikaiiii <1658973216@qq.com>
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
Signed-off-by: Zhikaiiii <1658973216@qq.com>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
0xrushi pushed a commit to 0xrushi/vllm that referenced this pull request Oct 26, 2025
Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
0xrushi pushed a commit to 0xrushi/vllm that referenced this pull request Oct 26, 2025
Signed-off-by: Zhikaiiii <1658973216@qq.com>
Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

frontend qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed tool-calling

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants