New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tracking] Quantization Integration #120

Open

1 of 3 tasks

stellaraccident opened this issue Oct 31, 2023 · 0 comments

Open

1 of 3 tasks

[tracking] Quantization Integration #120

stellaraccident opened this issue Oct 31, 2023 · 0 comments

Labels

Milestone

Contributor

stellaraccident commented Oct 31, 2023 •

edited

Loading

Turbine aims to support integration with multiple quantization workflows, generally meeting model development where it is at on a case by case basis.

Ad-hoc quantization via AOT transforms and weight import (built-out in the context of llama2) (implemented in Adds support to use brevitas quantized weights for stateless_llama #179)
AutoGPTQ Integration (built out in the context of Falcon180B)
General floating point type demotion and fine tuning

stellaraccident added the tracking-issue label

stellaraccident added this to the Path to V1 milestone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment