Does vllm support vicuna-13b-v1.5-16k ?

[Extremys]

The model seems to loop in its outputs

I'm using:
vllm 0.1.2
transformers 4.31.0
launched through the FastChat (commit b0462aa )/vllm wrapper (worker)
Any idea on what's going on? Thanks.

[DiscordJim]

Having same issue with vLLM 0.1.3.

[LiuXiaoxuanPKU]

Currently, vllm does not support 16k because the model uses rope embedding as shown here. We are actively working on it as show in pr #555, vllm will support vicuna-13b-v1.5-16k after that pr.

[Zhuqln]

thanks a lot for your effort. i use the pr and works well.
i have another question about long-context. when i use more than 2.5k-len context then i get an warning about exceeds limit.
it means vllm can not afford long-context ?

[hmellor]

Closing as vLLM now supports RoPE.