Skip to content

Add AliBi context scaling into vllm for Baichuan13B #686

Closed
@lucasjinreal

Description

@lucasjinreal

Hello,

any consideration add Alibi context scaling using ntk like method to support baichuan13b much more context length input?

there were some work did it: https://github.com/keezen/ntk_alibi

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions