Merge pull request #8 from arnavgarg1/add_lorax

Add LoRAX to usage options in README.
NetEase-FuXi · Jan 29, 2024 · 8877e96 · 8877e96
2 parents 71adb5e + 3da5b49
commit 8877e96
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -76,6 +76,11 @@ see [this](https://github.com/huggingface/text-generation-inference/pull/1018)
 --quantize eetq
 ```
 
+4. Use EETQ in [LoRAX](https://github.com/predibase/lorax). See [docs](https://predibase.github.io/lorax/guides/quantization/#eetq) here.
+```bash
+lorax-launcher --model-id mistralai/Mistral-7B-v0.1 --quantize eetq ...
+```
+
 ## Examples
 
 Model: