how to set this chat_template in server?

[wac81]

how to set this chat_template in openchat?
because i watched output it's difference from ./server and python -m llama.cpp.server. then i thought, may is difference chat_template made this?

openchat chat_template:
Using gguf chat template: {{ bos_token }}{% for message in messages %}{{ 'GPT4 Correct ' + message['role'].title() + ': ' + message['content'] + '<|end_of_turn|>'}}{% endfor %}{% if add_generation_prompt %}{{ 'GPT4 Correct Assistant:' }}{% endif %}

how to set chat_template in ./server with --chat-template

[phymbert]

Have a look to https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template

[ngxson]

We don’t support custom chat template for now, but you can use /completions endpoint (not /chat/completions) and input the formatted chat with the template that you made

[wac81]

I have an extension of that problem, ./server and python -m llama.cpp.server startup get inconsistent results, server is way worse and less controllable, I observed the python -m startup method, it should be using gguf's chat_template, I don't know if I'm right or not, but the result is very different, I am using openchat-0106.

I have an extension of that problem, ./server and python -m llama.cpp.server startup get inconsistent results, server is way worse and less controllable, I observed the python -m startup method, it should be using gguf's chat_template, I don't know if I'm right or not, but the result is very different, I am using openchat-0106.

[wac81]

Have a look to https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template

I've definitely seen the link, but thanks anyway

[teleprint-me]

You can manually set the chat template using gguf-py. Look at set-metadata.py in examples. I would go into more detail, but I have to go to work. I can post later if you're still having trouble understanding how to do it.

[ngxson]

@teleprint-me He's using custom self-made chat template {{ 'GPT4 Correct ' + message['role'].title()

This is not a common chat template that is supported by llama.cpp, so it won't work. Please see discussion: #5922 (comment)

[teleprint-me]

Yes, I understand how the template works. It doesn't change anything. My point stands.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.