New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

exllama重写了 Llama 的 HF 变压器实现 #18

Open

ziwang-com opened this issue May 30, 2023 · 0 comments

Owner

ziwang-com commented May 30, 2023

https://github.com/turboderp/exllama
埃克斯拉玛
重写了 Llama 的 HF 变压器实现，目标如下：

设计用于量化砝码
快速且节省内存的推理（不仅仅是注意力）
跨多个设备映射
内置（多） LoRA 支持
时髦采样函数的配套库

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment