Skip to content

feature request: Support user-defined conversation template #408

Closed
@gaocegege

Description

@gaocegege

We need to change the conversation template when we use our fine-tuned MPT 30b model.

https://github.com/vllm-project/vllm/blob/main/vllm/entrypoints/openai/api_server.py#L66

I think this feature is important especially when we use vLLM in production (with fine-tuned models).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions