Skip to content

Commit

Permalink
[Doc] Add note for docker user (#4340)
Browse files Browse the repository at this point in the history
Co-authored-by: Simon Mo <simon.mo@hey.com>
  • Loading branch information
youkaichao and simon-mo authored Apr 24, 2024
1 parent aae0824 commit 2768884
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions docs/source/serving/deploying_with_docker.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,6 @@ To run vLLM:
--env "HUGGING_FACE_HUB_TOKEN=<secret>" \
vllm/vllm-openai <args...>
.. note::

vLLM docker image is currently designed to be run under the root user (contribution welcomed for changing this!). It will try to load library at runtime under the root user's home directory, e.g. `/root/.config/vllm/nccl/cu12/libnccl.so.2.18.1` . If you are running the container under a different user, you may need to change the permissions of the library (and all the parent directories) to allow the user to access it. Then run vLLM with environment variable `VLLM_NCCL_SO_PATH=/root/.config/vllm/nccl/cu12/libnccl.so.2.18.1` .

0 comments on commit 2768884

Please sign in to comment.