Support for longchat-7b-16k

[yguo33]

It would be great, if you can add support for longchat(from Fastchat) models, which have 16k context length:

https://github.com/lm-sys/FastChat

https://github.com/lm-sys/FastChat/blob/6d06351542bc0c3701d54619e6df4c26aa91a260/fastchat/model/llama_condense_monkey_patch.py#L10C18-L10C18

[srka99]

Is this implemented in #555?

[WoosukKwon]

@srka99 Yes. Thanks for pinging this!

[pseudotensor]

Does this also allow support of the vicuna based long context models like lmsys/vicuna-13b-v1.5-16k ? I can't tell.

https://huggingface.co/lmsys/vicuna-13b-v1.5-16k

The answer is yes, it seems to work fine.