Skip to content

Commit

Permalink
[Doc]: Update tensorizer docs to include vllm[tensorizer] (vllm-proje…
Browse files Browse the repository at this point in the history
…ct#7889)

Co-authored-by: Kaunil Dhruv <dhruv.kaunil@gmail.com>
Signed-off-by: Amit Garg <mitgarg17495@gmail.com>
  • Loading branch information
2 people authored and garg-amit committed Oct 28, 2024
1 parent e733e2e commit e1336b3
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion docs/source/serving/tensorizer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,7 @@ shorter Pod startup times and CPU memory usage. Tensor encryption is also suppor

For more information on CoreWeave's Tensorizer, please refer to
`CoreWeave's Tensorizer documentation <https://github.com/coreweave/tensorizer>`_. For more information on serializing a vLLM model, as well a general usage guide to using Tensorizer with vLLM, see
the `vLLM example script <https://docs.vllm.ai/en/stable/getting_started/examples/tensorize_vllm_model.html>`_.
the `vLLM example script <https://docs.vllm.ai/en/stable/getting_started/examples/tensorize_vllm_model.html>`_.

.. note::
Note that to use this feature you will need to install `tensorizer` by running `pip install vllm[tensorizer]`.

0 comments on commit e1336b3

Please sign in to comment.