Skip to content

Commit d05e8d4

Browse files
thesuesmgoin
andcommitted
Update docs/source/quantization/bnb.rst
Co-authored-by: Michael Goin <michael@neuralmagic.com>
1 parent d5387a3 commit d05e8d4

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

docs/source/quantization/bnb.rst

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,10 @@ Below are the steps to utilize BitsAndBytes with vLLM.
1515
1616
vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint.
1717

18-
Read quantized checkpoint.
19-
2018
You can find bitsandbytes quantized models on https://huggingface.co/models?other=bitsandbytes.
2119
And usually, these repositories have a config.json file that includes a quantization_config section.
2220

21+
Read quantized checkpoint.
2322
--------------------------
2423

2524
.. code-block:: python

0 commit comments

Comments
 (0)