You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+7-2Lines changed: 7 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -170,14 +170,19 @@ For *most* developers you probably want to skip building custom C++/CUDA extensi
170
170
USE_CPP=0 pip install -e .
171
171
```
172
172
173
-
## Integrations
173
+
## OSS Integrations
174
174
175
175
We're also fortunate to be integrated into some of the leading open-source libraries including
176
176
1. Hugging Face transformers with a [builtin inference backend](https://huggingface.co/docs/transformers/main/quantization/torchao) and [low bit optimizers](https://github.com/huggingface/transformers/pull/31865)
177
-
2. Hugging Face diffusers best practices with torch.compile and torchao [standalone repo](https://github.com/sayakpaul/diffusers-torchao)
177
+
2. Hugging Face diffusers best practices with torch.compile and torchao in a standalone repo[diffusers-torchao](https://github.com/sayakpaul/diffusers-torchao)
178
178
3. Mobius HQQ backend leveraged our int4 kernels to get [195 tok/s on a 4090](https://github.com/mobiusml/hqq#faster-inference)
179
+
4.[TorchTune](https://github.com/pytorch/torchtune) for our QLoRA and QAT recipes
180
+
5.[torchchat](https://github.com/pytorch/torchtune) for post training quantization
181
+
6.[SGLang](https://github.com/sgl-project/sglang/pull/1341) for LLM inference quantization
179
182
180
183
## Videos
184
+
*[Keynote talk at GPU MODE IRL](https://youtu.be/FH5wiwOyPX4?si=VZK22hHz25GRzBG1&t=1009)
185
+
*[Low precision dtypes at PyTorch conference](https://youtu.be/xcKwEZ77Cps?si=7BS6cXMGgYtFlnrA)
181
186
*[Slaying OOMs at the Mastering LLM's course](https://www.youtube.com/watch?v=UvRl4ansfCg)
182
187
*[Advanced Quantization at CUDA MODE](https://youtu.be/1u9xUK3G4VM?si=4JcPlw2w8chPXW8J)
0 commit comments