Add DeepSeek-R1-0528 function call chat template

[Xu-Wenqing]

DeepSeek-R1-0528 model support function call, add function call chat template.

Usage:

vllm serve ... --enable-auto-tool-choice --tool-call-parser deepseek_v3 --chat-template examples/tool_chat_template_deepseekr1.jinja

Function Call Test
Use Berkeley Function Calling Leaderboard to evaluate function call template here.

Evaluation Result:
🦍 Model: DeepSeek-R1-0528
🔍 Running test: simple
✅ Test completed: simple. 🎯 Accuracy: 0.9325
Number of models evaluated: 100%|███████████████████████████████████████████| 1/1 [00:00<00:00, 41.24it/s]
📈 Aggregating data to generate leaderboard score table...
🏁 Evaluation completed. See /Users/xuwenqing/function_call_eval/score/data_overall.csv for overall evaluation results on BFCL V3.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[Xu-Wenqing]

@aarnphm @DarkLight1337

[Xu-Wenqing]

#18931

[markluofd]

I use the following command to start server

vllm serve /model     --tensor-parallel-size 8     --pipeline-parallel-size 2     --trust-remote-code     --gpu-memory-utilization 0.92     --enable-auto-tool-choice     --tool-call-parser deepseek_v3                     --chat-template /home/work/easyedge/llm/tool_chat_template_deepseekr1.jinja     --max-model-len 98304     --host 0.0.0.0     --port 8669     --served-model-name DeepSeek-R1     --uvicorn-log-level info

and curl the server with the body

{
    "messages": [
        {
            "content": "",
            "role": "system"
        },
        {
            "content": "what's the magic function of 5?",
            "role": "user"
        }
    ],
    "model": "",
    "max_tokens": 2560,
    "stream": false,
    "temperature": 0.7,
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "magic_function",
                "description": "Applies a magic function to an input.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "input": {
                            "type": "integer"
                        }
                    },
                    "required": [
                        "input"
                    ]
                }
            }
        }
    ]
}

but I got the result as follows:

{
    "id": "chatcmpl-83f63afeb0644f1990e2c865cd08f7f2",
    "object": "chat.completion",
    "created": 1748572697,
    "model": "DeepSeek-R1",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "reasoning_content": null,
                "content": "<think>\nWe are given a function called \"magic_function\" that takes an integer input.\n The user query is: \"what's the magic function of 5?\"\n We should call the magic_function with input 5.\n We will output the function call in the required format.\n</think>\n",
                "tool_calls": []
            },
            "logprobs": null,
            "finish_reason": "tool_calls",
            "stop_reason": null
        }
    ],
    "usage": {
        "prompt_tokens": 160,
        "total_tokens": 238,
        "completion_tokens": 78,
        "prompt_tokens_details": null
    },
    "prompt_logprobs": null,
    "kv_transfer_params": null
}

it seems like tool parser failed to contract the tool_calls parameters, have I use the wrong command ?

[Xu-Wenqing]

I use the following command to start server

vllm serve /model     --tensor-parallel-size 8     --pipeline-parallel-size 2     --trust-remote-code     --gpu-memory-utilization 0.92     --enable-auto-tool-choice     --tool-call-parser deepseek_v3                     --chat-template /home/work/easyedge/llm/tool_chat_template_deepseekr1.jinja     --max-model-len 98304     --host 0.0.0.0     --port 8669     --served-model-name DeepSeek-R1     --uvicorn-log-level info

and curl the server with the body

{
    "messages": [
        {
            "content": "",
            "role": "system"
        },
        {
            "content": "what's the magic function of 5?",
            "role": "user"
        }
    ],
    "model": "",
    "max_tokens": 2560,
    "stream": false,
    "temperature": 0.7,
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "magic_function",
                "description": "Applies a magic function to an input.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "input": {
                            "type": "integer"
                        }
                    },
                    "required": [
                        "input"
                    ]
                }
            }
        }
    ]
}

but I got the result as follows:

{
    "id": "chatcmpl-83f63afeb0644f1990e2c865cd08f7f2",
    "object": "chat.completion",
    "created": 1748572697,
    "model": "DeepSeek-R1",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "reasoning_content": null,
                "content": "<think>\nWe are given a function called \"magic_function\" that takes an integer input.\n The user query is: \"what's the magic function of 5?\"\n We should call the magic_function with input 5.\n We will output the function call in the required format.\n</think>\n",
                "tool_calls": []
            },
            "logprobs": null,
            "finish_reason": "tool_calls",
            "stop_reason": null
        }
    ],
    "usage": {
        "prompt_tokens": 160,
        "total_tokens": 238,
        "completion_tokens": 78,
        "prompt_tokens_details": null
    },
    "prompt_logprobs": null,
    "kv_transfer_params": null
}

it seems like tool parser failed to contract the tool_calls parameters, have I use the wrong command ?

@markluofd you can add tool_choice="required" in your request.

[markluofd]

failed too, seems like the response is not a json format😂
request as:

{
    "messages": [
        {
            "content": "",
            "role": "system"
        },
        {
            "content": "what's the magic function of 5?",
            "role": "user"
        }
    ],
    "model": "",
    "max_tokens": 2560,
    "stream": false,
    "temperature": 0.7,
    "tool_choice": "required",
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "magic_function",
                "description": "Applies a magic function to an input.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "input": {
                            "type": "integer"
                        }
                    },
                    "required": [
                        "input"
                    ]
                }
            }
        }
    ]
}

response as:

{
    "object": "error",
    "message": "1 validation error for list[function-wrap[__log_extra_fields__()]]\n  Invalid JSON: EOF while parsing a string at line 1 column 3 [type=json_invalid, input_value='[{\"', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.11/v/json_invalid",
    "type": "BadRequestError",
    "param": null,
    "code": 400
}

I extract the prompt from vllm log as:

Received request chatcmpl-95a8d630d8f64a8cad7548c5057c59d5: prompt: '<｜begin▁of▁sentence｜>\n    You may call one or more functions to assist with the user query.\n\n    Here are the available functions:\n{"type": "function", "function": {"name": "magic_function", "description": "Applies a magic function to an input.", "parameters": {"type": "object", "properties": {"input": {"type": "integer"}}, "required": ["input"]}}}\n    For function call returns, you should first print <｜tool▁calls▁begin｜>\n    For each function call, you should return object like:\n\n    <｜tool▁call▁begin｜>function<｜tool▁sep｜><function_name>\n```json\n<function_arguments_in_json_format>\n```<｜tool▁call▁end｜>\n    At the end of function call returns, you should print <｜tool▁calls▁end｜><｜end▁of▁sentence｜>\n<｜User｜>what\'s the magic function of 5?    <｜Assistant｜>\n', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.7, top_p=0.95, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=2560, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=GuidedDecodingParams(json={'type': 'array', 'minItems': 1, 'items': {'type': 'object', 'anyOf': [{'properties': {'name': {'type': 'string', 'enum': ['magic_function']}, 'parameters': {'type': 'object', 'properties': {'input': {'type': 'integer'}}, 'required': ['input']}}, 'required': ['name', 'parameters']}]}}, regex=None, choice=None, grammar=None, json_object=None, backend=None, backend_was_auto=False, disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, whitespace_pattern=None, structural_tag=None), extra_args=None), prompt_token_ids: None, prompt_embeds shape: None, lora_request: None, prompt_adapter_request: None

[mgoin]

Seems reasonable to me, thanks

[NaiveYan]

I'm encountering an issue when trying to use the DeepSeek-R1-0528-Qwen3-8B model. It appears unsupported, returning Error 400:

{'object': 'error', 
'message': 'DeepSeek-V3 Tool parser could not locate tool call start/end tokens in the tokenizer! None', 
'type': 'BadRequestError', 
'param': None, 
'code': 400}

[houseroad]

Thanks for adding the support, @Xu-Wenqing. Btw, could you paste the test results in the PR description? Also do you want to include the updated version in this PR? It's fine to have another one to include the updated template.

[alllexx88]

I'm experiencing the same with DeepSeek-R1-0528-Qwen3-8B as @NaiveYan is. IDK if it helps, but here's an ollama chat template that has tool calling working with this model: https://ollama.com/okamototk/deepseek-r1:8b/blobs/e94a8ecb9327
Thanks

[wukaixingxp]

Not sure if it is a intentional, but this chat-template has some spaces at the beginning of some lines.. should we clean it up? the current version looks like this and here is my cleaned version. Let me know if it is helpful.

[wukaixingxp]

@Xu-Wenqing I just tested with llama-stack-eval on your PR and it seems the non_streaming tool call failed as well, can you double check if it is a model problem or chat_template problem? CC: @houseroad

_____________ test_chat_non_streaming_tool_calling[deepseek-ai/DeepSeek-R1-0528-basic] ______________
request = <FixtureRequest for <Function test_chat_non_streaming_tool_calling[deepseek-ai/DeepSeek-R1-0528-basic]>>
openai_client = <openai.OpenAI object at 0x7f86edc81610>, model = 'deepseek-ai/DeepSeek-R1-0528'
provider = 'vllm'
verification_config = {'cerebras': ProviderConfig(provider='cerebras', base_url='https://api.cerebras.ai/v1', api_key_var='CEREBRAS_API_KEY'...4-maverick-17b-128e-instruct', canonical_id='Llama-4-Maverick-Instruct')], test_exclusions={}, self_hosted=False), ...}
case = {'case_id': 'basic', 'expected': {'tool_arguments': {'location': 'San Francisco'}, 'tool_name': 'get_weather'}, 'input...rs': {'additionalProperties': False, 'properties': {...}, 'required': [...], 'type': 'object'}}, 'type': 'function'}]}}
    @pytest.mark.parametrize(
        "case",
        chat_completion_test_cases["test_tool_calling"]["test_params"]["case"],
        ids=case_id_generator,
    )
    def test_chat_non_streaming_tool_calling(request, openai_client, model, provider, verification_config, case):
        if "xfail" in case and case["xfail"]:
            pytest.xfail(case["xfail"]["reason"])
    
        test_name_base = get_base_test_name(request)
        if should_skip_test(verification_config, provider, model, test_name_base):
            pytest.skip(f"Skipping {test_name_base} for model {model} on provider {provider} based on config.")
    
        response = openai_client.chat.completions.create(
            model=model,
            messages=case["input"]["messages"],
            tools=case["input"]["tools"],
            stream=False,
        )
    
        assert response.choices[0].message.role == "assistant"
>       assert len(response.choices[0].message.tool_calls) == 1
E       assert 0 == 1
E        +  where 0 = len([])
E        +    where [] = ChatCompletionMessage(content='<think>\nWe are given a user query: "What\'s the weather like in San Francisco?"\n We have one function available: get_weather, which requires a location (city and country).\n The user mentioned "San Francisco", but note that the function expects a city and country.\n Since the user did not specify the country, we can assume it\'s San Francisco, USA because it\'s a well-known city.\n However, note that the function description says "City and country e.g. Bogotá, Colombia". \n We should provide the location in the format: "San Francisco, USA".\n\n But note: the example format uses a comma and then the country. We\'ll use the same.\n\n However, let\'s consider: the function might be designed to handle just the city if the country is obvious? \n But the specification says it requires a location in the format of "City and country". So we must provide both.\n\n Since the user didn\'t specify, we\'ll use "San Francisco, USA" as the most common one.\n\n Therefore, we will call the function get_weather with the argument: {"location": "San Francisco, USA"}\n\n We must format the function call as:\n\nfunction\x1eget_weather\n```json\n{"location":"San Francisc...ser query? \n\n Yes, because we are calling the function.\n\n We do not provide any other text.\n\n So the assistant\'s response is exactly:\n\n   \\x1cfunction\\x1fget_weather\n   ```json\n   {"location": "San Francisco, USA"}\n   ```\\x1e\\x1d\n\n But note: the GS (U+001D) is after the RS (U+001E). And we have no newline between the RS and GS? That\'s acceptable.\n\n However, the instruction says "At the end of function call returns, you should print \x1d", meaning that the GS comes immediately after the function call block (which ends with RS).\n\n So we do not put a newline between RS and GS.\n\n Therefore, we output the string as:\n\n   [FS] + "function" + [US] + "get_weather" + "\\n```json\\n" + "{\\"location\\": \\"San Francisco, USA\\"}" + "\\n```" + [RS] + [GS]\n\n But note: the JSON part: we are writing a string with a comma and space. That is acceptable.\n\n We are done.\n</think>\nI need to get the current temperature for San Francisco. I\'ll use the provided function.\n\nfunction\x1eget_weather\n```json\n{"location": "San Francisco, USA"}\n```\x1e', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=[], reasoning_content=None).tool_calls
E        +      where ChatCompletionMessage(content='<think>\nWe are given a user query: "What\'s the weather like in San Francisco?"\n We have one function available: get_weather, which requires a location (city and country).\n The user mentioned "San Francisco", but note that the function expects a city and country.\n Since the user did not specify the country, we can assume it\'s San Francisco, USA because it\'s a well-known city.\n However, note that the function description says "City and country e.g. Bogotá, Colombia". \n We should provide the location in the format: "San Francisco, USA".\n\n But note: the example format uses a comma and then the country. We\'ll use the same.\n\n However, let\'s consider: the function might be designed to handle just the city if the country is obvious? \n But the specification says it requires a location in the format of "City and country". So we must provide both.\n\n Since the user didn\'t specify, we\'ll use "San Francisco, USA" as the most common one.\n\n Therefore, we will call the function get_weather with the argument: {"location": "San Francisco, USA"}\n\n We must format the function call as:\n\nfunction\x1eget_weather\n```json\n{"location":"San Francisc...ser query? \n\n Yes, because we are calling the function.\n\n We do not provide any other text.\n\n So the assistant\'s response is exactly:\n\n   \\x1cfunction\\x1fget_weather\n   ```json\n   {"location": "San Francisco, USA"}\n   ```\\x1e\\x1d\n\n But note: the GS (U+001D) is after the RS (U+001E). And we have no newline between the RS and GS? That\'s acceptable.\n\n However, the instruction says "At the end of function call returns, you should print \x1d", meaning that the GS comes immediately after the function call block (which ends with RS).\n\n So we do not put a newline between RS and GS.\n\n Therefore, we output the string as:\n\n   [FS] + "function" + [US] + "get_weather" + "\\n```json\\n" + "{\\"location\\": \\"San Francisco, USA\\"}" + "\\n```" + [RS] + [GS]\n\n But note: the JSON part: we are writing a string with a comma and space. That is acceptable.\n\n We are done.\n</think>\nI need to get the current temperature for San Francisco. I\'ll use the provided function.\n\nfunction\x1eget_weather\n```json\n{"location": "San Francisco, USA"}\n```\x1e', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=[], reasoning_content=None) = Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='<think>\nWe are given a user query: "What\'s the weather like in San Francisco?"\n We have one function available: get_weather, which requires a location (city and country).\n The user mentioned "San Francisco", but note that the function expects a city and country.\n Since the user did not specify the country, we can assume it\'s San Francisco, USA because it\'s a well-known city.\n However, note that the function description says "City and country e.g. Bogotá, Colombia". \n We should provide the location in the format: "San Francisco, USA".\n\n But note: the example format uses a comma and then the country. We\'ll use the same.\n\n However, let\'s consider: the function might be designed to handle just the city if the country is obvious? \n But the specification says it requires a location in the format of "City and country". So we must provide both.\n\n Since the user didn\'t specify, we\'ll use "San Francisco, USA" as the most common one.\n\n Therefore, we will call the function get_weather with the argument: {"location": "San Francisco, USA"}\n\n We must format the function call as:\n..., because we are calling the function.\n\n We do not provide any other text.\n\n So the assistant\'s response is exactly:\n\n   \\x1cfunction\\x1fget_weather\n   ```json\n   {"location": "San Francisco, USA"}\n   ```\\x1e\\x1d\n\n But note: the GS (U+001D) is after the RS (U+001E). And we have no newline between the RS and GS? That\'s acceptable.\n\n However, the instruction says "At the end of function call returns, you should print \x1d", meaning that the GS comes immediately after the function call block (which ends with RS).\n\n So we do not put a newline between RS and GS.\n\n Therefore, we output the string as:\n\n   [FS] + "function" + [US] + "get_weather" + "\\n```json\\n" + "{\\"location\\": \\"San Francisco, USA\\"}" + "\\n```" + [RS] + [GS]\n\n But note: the JSON part: we are writing a string with a comma and space. That is acceptable.\n\n We are done.\n</think>\nI need to get the current temperature for San Francisco. I\'ll use the provided function.\n\nfunction\x1eget_weather\n```json\n{"location": "San Francisco, USA"}\n```\x1e', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=[], reasoning_content=None), stop_reason=None).message
llama_stack_evals/functional_tests/openai_api/test_chat_completion.py:250: AssertionError
___ test_chat_non_streaming_multi_turn_tool_calling[deepseek-ai/DeepSeek-R1-0528-text_then_tool] ____
request = <FixtureRequest for <Function test_chat_non_streaming_multi_turn_tool_calling[deepseek-ai/DeepSeek-R1-0528-text_then_tool]>>
openai_client = <openai.OpenAI object at 0x7f86edb0d190>, model = 'deepseek-ai/DeepSeek-R1-0528'
provider = 'vllm'
verification_config = {'cerebras': ProviderConfig(provider='cerebras', base_url='https://api.cerebras.ai/v1', api_key_var='CEREBRAS_API_KEY'...4-maverick-17b-128e-instruct', canonical_id='Llama-4-Maverick-Instruct')], test_exclusions={}, self_hosted=False), ...}
case = {'case_id': 'text_then_tool', 'expected': [{'answer': ['sol'], 'num_tool_calls': 0}, {'num_tool_calls': 1, 'tool_argum...], 'type': 'object'}}, 'type': 'function'}]}, 'tool_responses': [{'response': "{'response': '70 degrees and foggy'}"}]}
    @pytest.mark.parametrize(
        "case",
        chat_completion_test_cases.get("test_chat_multi_turn_tool_calling", {}).get("test_params", {}).get("case", []),
        ids=case_id_generator,
    )
    def test_chat_non_streaming_multi_turn_tool_calling(request, openai_client, model, provider, verification_config, case):
        """
        Test cases for multi-turn tool calling.
        Tool calls are asserted.
        Tool responses are provided in the test case.
        Final response is asserted.
        """
    
        test_name_base = get_base_test_name(request)
        if should_skip_test(verification_config, provider, model, test_name_base):
            pytest.skip(f"Skipping {test_name_base} for model {model} on provider {provider} based on config.")
    
        # Create a copy of the messages list to avoid modifying the original
        messages = []
        tools = case["input"]["tools"]
        # Use deepcopy to prevent modification across runs/parametrization
        expected_results = copy.deepcopy(case["expected"])
        tool_responses = copy.deepcopy(case.get("tool_responses", []))
        input_messages_turns = copy.deepcopy(case["input"]["messages"])
    
        # keep going until either
        # 1. we have messages to test in multi-turn
        # 2. no messages but last message is tool response
        while len(input_messages_turns) > 0 or (len(messages) > 0 and messages[-1]["role"] == "tool"):
            # do not take new messages if last message is tool response
            if len(messages) == 0 or messages[-1]["role"] != "tool":
                new_messages = input_messages_turns.pop(0)
                # Ensure new_messages is a list of message objects
                if isinstance(new_messages, list):
                    messages.extend(new_messages)
                else:
                    # If it's a single message object, add it directly
                    messages.append(new_messages)
    
            # --- API Call ---
            response = openai_client.chat.completions.create(
                model=model,
                messages=messages,
                tools=tools,
                stream=False,
            )
    
            # --- Process Response ---
            assistant_message = response.choices[0].message
            messages.append(assistant_message.model_dump(exclude_unset=True))
    
            assert assistant_message.role == "assistant"
    
            # Get the expected result data
            expected = expected_results.pop(0)
            num_tool_calls = expected["num_tool_calls"]
    
            # --- Assertions based on expected result ---
>           assert len(assistant_message.tool_calls or []) == num_tool_calls, (
                f"Expected {num_tool_calls} tool calls, but got {len(assistant_message.tool_calls or [])}"
            )
E           AssertionError: Expected 1 tool calls, but got 0
E           assert 0 == 1
E            +  where 0 = len(([] or []))
E            +    where [] = ChatCompletionMessage(content='<think>\nWe are given a user query: "What\'s the weather like in San Francisco?"\n We have a function called "get_weather" that requires a "location" parameter.\n The location should be the city and state (both required). Since the user only said "San Francisco", we should assume the state is California (CA) because that\'s the most famous one.\n However, note that the function description says "The city and state (both required)". So we must provide both.\n We can use "San Francisco, CA" as the location.\n\n Steps:\n 1. We are going to call the function "get_weather" with the argument: {"location": "San Francisco, CA"}\n 2. We must format the function call exactly as specified.\n\n Format:\n   - Start with: `function_call`\n   - Then on the next line: ```json\n     { ... }\n     ```\n   - End with: `end_function_call`\n\n But note: the example format provided was:\n      •#function_call•#\n      ```json\n      { ... }\n      ```\n      •#end_function_call•#\n\n However, the actual tokens we are to use are:\n      •#function_call•#   -> at the beginning of the function call\n      •#end_function_call•# -> at the end\n\n But note: the example in the ...nside the JSON. However, the function call format we are to use is defined as:\n\n      •#function_call•#\n      ```json\n      {\n        "name": "function_name",\n        "arguments": {\n          "arg1": "value1",\n          ...\n        }\n      }\n      ```\n      •#end_function_call•#\n\n Actually, the instructions did not specify the exact JSON structure? But the common practice in function calling is to have a JSON object with two keys: "name" and "arguments". And "arguments" is another object with the parameters.\n\n Therefore, we output:\n\n •#function_call•#\n```json\n{\n  "name": "get_weather",\n  "arguments": {\n    "location": "San Francisco, CA"\n  }\n}\n```\n •#end_function_call•#\n</think>\nTo retrieve the weather information for San Francisco, I\'ll call the `get_weather` function with the required location parameter. Since the function requires both city and state, I\'ll use "San Francisco, CA" as the location.\n\n •#function_call•#\n```json\n{"name": "get_weather", "arguments": {"location": "San Francisco, CA"}}\n```\n •#end_function_call•#', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=[], reasoning_content=None).tool_calls
llama_stack_evals/functional_tests/openai_api/test_chat_completion.py:462: AssertionError

[Xu-Wenqing]

@houseroad @wukaixingxp @markluofd @NaiveYan @alllexx88 Sorry for the late reply. The past few days were Chinese Dragon Boat Festival, I didn’t check messages. I’ll try out the chat template on some test datasets again. Meanwhile, seems DeepSeek update DeepSeek-R1-0528 chat template: https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/commit/4236a6af538feda4548eca9ab308586007567f52#d2h-846292 I will also update the template here.

[Zongru-Wang]

Error code: 400 - {'object': 'error', 'message': 'DeepSeek-V3 Tool parser could not locate tool call start/end tokens in the tokenizer! None', 'type': 'BadRequestError', 'param': None, 'code': 400}， if i use langchain will tool call agent, it will get the following error.

vllm serve /deepseek-ai/DeepSeek-R1-0528-Qwen3-8B --tensor-parallel-size 8 --host 0.0.0.0 --port 10001 --api-key none --rope-scaling '{"factor": 2.0, "original_max_position_embeddings": 32768, "rope_type": "yarn"}' --gpu-memory-utilization 0.9 --enable-reasoning --reasoning-parser deepseek_r1 --guided_decoding_backend guidance --enable-auto-tool-choice --tool-call-parser deepseek_v3 --chat-template /home/ubuntu/wzr/LLM-MODELS/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B/tool_chat_template_deepseekr1.jinja --served-model-name DeepSeek-R1

[Zongru-Wang]

Error code: 400 - {'object': 'error', 'message': 'DeepSeek-V3 Tool parser could not locate tool call start/end tokens in the tokenizer! None', 'type': 'BadRequestError', 'param': None, 'code': 400}， if i use langchain will tool call agent, it will get the following error.

vllm serve /deepseek-ai/DeepSeek-R1-0528-Qwen3-8B --tensor-parallel-size 8 --host 0.0.0.0 --port 10001 --api-key none --rope-scaling '{"factor": 2.0, "original_max_position_embeddings": 32768, "rope_type": "yarn"}' --gpu-memory-utilization 0.9 --enable-reasoning --reasoning-parser deepseek_r1 --guided_decoding_backend guidance --enable-auto-tool-choice --tool-call-parser deepseek_v3 --chat-template /home/ubuntu/wzr/LLM-MODELS/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B/tool_chat_template_deepseekr1.jinja --served-model-name DeepSeek-R1

I tried --tool-call-parser deepseek_v3 --chat-template examples/tool_chat_template_deepseekr1.jinja, and got Error code: 400 - {'object': 'error', 'message': 'DeepSeek-V3 Tool parser could not locate tool call start/end tokens in the tokenizer! None', 'type': 'BadRequestError', 'param': None, 'code': 400}, if I use --tool-call-parser hermes, vllm backend shows: The following fields were present in the request but ignored: {'function_call'}.

I am using langchain agent to make tool calls, QWQ-32B, Qwen3 serires works fine for me.

[hmellor]

How would this look in the docs?

Suggested change

	* `deepseek-ai/DeepSeek-V3-0324`
	* `deepseek-ai/DeepSeek-R1-0528`
	* `deepseek-ai/DeepSeek-V3-0324` (`--tool-call-parser deepseek_v3 --chat-template examples/tool_chat_template_deepseekv3.jinja`)
	* `deepseek-ai/DeepSeek-R1-0528` (`--tool-call-parser deepseek_v3 --chat-template examples/tool_chat_template_deepseekr1.jinja`)

[Xu-Wenqing]

@hmellor Updated the markdown file.

[Xu-Wenqing]

@houseroad Updated the chat template, and added test results in description.

[houseroad]

Thanks!

[menardorama]

Hi,
I have tested this PR with https://huggingface.co/deepseek-ai/DeepSeek-R1-0528 and tool calling is never working

 vllm serve deepseek-ai/DeepSeek-R1-0528  --port 8000 --trust-remote-code --tensor-parallel-size 8 --enable-reasoning --reasoning-parser deepseek_r1 --tool-call-parser deepseek_v3 --enable-auto-tool-choice --chat-template /vllm-templates/tool_chat_template_deepseekr1_v2.jinja"

Is anybody got success using this model and this PR ?

[qdivan]

Hi, I have tested this PR with https://huggingface.co/deepseek-ai/DeepSeek-R1-0528 and tool calling is never working

 vllm serve deepseek-ai/DeepSeek-R1-0528  --port 8000 --trust-remote-code --tensor-parallel-size 8 --enable-reasoning --reasoning-parser deepseek_r1 --tool-call-parser deepseek_v3 --enable-auto-tool-choice --chat-template /vllm-templates/tool_chat_template_deepseekr1_v2.jinja"

Is anybody got success using this model and this PR ?

I got successed in tool calling. maybe you did not put the template file in the right path.

[menardorama]

/vllm-templates/tool_chat_template_deepseekr1_v2.jinja

Well that's an absolute path and I can read the file.

You are using the same model ?

I am running vllm 0.9.0.1 on NVIDIA B200 if that helps...

[qdivan]

/vllm-templates/tool_chat_template_deepseekr1_v2.jinja

Well that's an absolute path and I can read the file.

You are using the same model ?

I am running vllm 0.9.0.1 on NVIDIA B200 if that helps...

I use docker and I put the template in /data/vllm-extra/tool_chat_template_deepseekr1.jinja

and use docker run to start the model and it worked.

you can follow the bash:

docker run -d \
  --name vllm-deepseek-r1 \
  --restart unless-stopped \
  --runtime=nvidia \
  --health-cmd="wget -qO- http://localhost:12345/v1/models >/dev/null 2>&1 || exit 1" \
  --health-interval=30s \
  --health-timeout=5s \
  --health-retries=3 \
  --health-start-period=300s \
  --gpus all \
  -p 12345:12345 \
  --ipc=host \
  --log-driver json-file \
  --log-opt max-size=100m \
  --log-opt max-file=3 \
  -v /data1/models:/data1/models \
  -v /data/vllm-cache:/data/vllm-cache \
  -v /data/app/vllm/log:/workspace/logs \
  -v /data/vllm-extra/tool_chat_template_deepseekr1.jinja:/vllm-workspace/examples/tool_chat_template_deepseekr1.jinja:ro \
  -e VLLM_CACHE_ROOT=/data/vllm-cache \
  -e VLLM_WORKER_MULTIPROC_METHOD=spawn \
  -e VLLM_MARLIN_USE_ATOMIC_ADD=1 \
  -e HF_HUB_OFFLINE=1 \
  -e VLLM_USE_MODELSCOPE=true \
  -e OMP_NUM_THREADS=1 \
  -e VLLM_USE_V1=1 \
  vllm/vllm-openai:latest \
  --host 0.0.0.0 \
  --port 12345 \
  --model /data1/models/DeepSeek-R1-0528 \
  --served-model-name deepseek-reasoning \
  --tensor-parallel-size 8 \
  --gpu-memory-utilization 0.98 \
  --max-model-len 131071 \
  --max-seq-len-to-capture 8192 \
  --max-num-seqs 16 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser deepseek_v3 \
  --chat-template examples/tool_chat_template_deepseekr1.jinja \
  --trust-remote-code