[Frontend] Fix multiple values for keyword argument error (#10075)

[DIYer22]

Fix #10075 like:

curl -X POST "http://39.105.21.95:12481/v1/chat/completions" \
     -H "Content-Type: application/json" \
     -d '{
           "model": "meta-llama/Meta-Llama-3-8B-Instruct",
           "messages": [
             {
               "role": "user",
               "content": "tell me a common saying"
             },
             {
               "role": "assistant",
               "content": "Here is a common saying about apple. An apple a day, keeps"
             }
           ],
           "add_generation_prompt": false, "chat_template_kwargs":{"continue_final_message": true}
         }'

Internal Server Error%  

Error message on server

TypeError: vllm.entrypoints.chat_utils.apply_hf_chat_template() got multiple values for keyword argument 'continue_final_message'

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[DIYer22]

I've tried many times and still don't know how to write the code to pass the mypy type check, can anyone help me? 😢

[DarkLight1337]

Thanks for fixing, please resolve the type errors.

[DarkLight1337]

Suggested change

	_chat_template_kwargs = dict(
	chat_template=chat_template,
	add_generation_prompt=add_generation_prompt,
	continue_final_message=continue_final_message,
	tools=tool_dicts,
	documents=documents,
	)
	_chat_template_kwargs = ChatTemplateKwargs(
	chat_template=chat_template,
	add_generation_prompt=add_generation_prompt,
	continue_final_message=continue_final_message,
	tools=tool_dicts,
	documents=documents,
	)

[DarkLight1337]

Does this suggestion work?

[DIYer22]

Not work, still can't pass the type check of mypy.

[DarkLight1337]

Is this intended? This argument wasn't there before.

[DIYer22]

Yes, for pass mypy type checking. When using **dict in function, you need to explicitly call all the typed parameters defined by the function, otherwise it will not pass the mypy type check.

[DarkLight1337]

Can you add a note here to indicate this?

[DarkLight1337]

The code looks good; can you add a test to verify this behavior?

[DIYer22]

@DarkLight1337 I finally found a simple, scientific way to pass mypy type check, please review it.

This change is too simple, and there was no test code for this before, so I don't think it's necessary to add new testcase.

[DIYer22]

I have deployed this fixed version of the branch on my public server. You can test it directly like this

curl -X POST "http://39.105.21.95:12481/v1/chat/completions" \
     -H "Content-Type: application/json" \
     -d '{
           "model": "meta-llama/Meta-Llama-3-8B-Instruct",
           "messages": [
             {
               "role": "user",
               "content": "tell me a common saying"
             },
             {
               "role": "assistant",
               "content": "Here is a common saying about apple. An apple a day, keeps"
             }
           ],
           "add_generation_prompt": false, "chat_template_kwargs":{"continue_final_message": true}
         }'

[DarkLight1337]

Ok, as long as the code works for you.

[DarkLight1337]

Waiting for CI to be fixed internally before starting the merge process.