supporting superhot models?

[bryanhpchiang]

specifically:

• https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test

i've confirmed that i can load the model in vllm and successfully generate completions but the context length is still only 2048.

[bryanhpchiang]

looking into it

not sure where this error is coming from actually. couldn't find it in the codebase

i tried changing this when setting up the llm but didn't help

[wheel-is]

any luck?

[hmellor]

Closing this issue as stale as there has been no discussion in the past 3 months.

If you are still experiencing the issue you describe, feel free to re-open this issue.