[Feature]: Does the latest version of vllm (0.3.3) support lora inference for wizardcoder and deepseek models

[ivanzfb]

🚀 The feature, motivation and pitch

In our actual project, we need to use wizardcoder/deepseek's loraadapters for reasoning. The latest version of vllm (0.3.3) is the support plan for this wizardcoder/deepseek's loraadapters?

Alternatives

No response

Additional context

No response

[simon-mo]

Wizardcoder is uses GPTBigCode, there's already a feature request here #3362
Deepseek coder is just Llama, which we do support!

See https://docs.vllm.ai/en/latest/models/supported_models.html for LoRA support status.

[ivanzfb]

Wizardcoder is uses GPTBigCode, there's already a feature request here #3362 Deepseek coder is just Llama, which we do support!

See https://docs.vllm.ai/en/latest/models/supported_models.html for LoRA support status.

But I used the official lora infrence method to ensure that the vocabulary exceeded the limit of the error

[simon-mo]

Ah this is a bug indeed. Please open a new github issue under Model saying LoRA for Deepseek is not supported due to vocab size limit.