Skip to content

Commit

Permalink
[Doc] Dockerfile instructions for optional dependencies and dev trans…
Browse files Browse the repository at this point in the history
…formers (vllm-project#13699)
  • Loading branch information
DarkLight1337 authored Feb 22, 2025
1 parent c904fdd commit 8354f66
Showing 1 changed file with 30 additions and 0 deletions.
30 changes: 30 additions & 0 deletions docs/source/deployment/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,36 @@ container to access the host's shared memory. vLLM uses PyTorch, which uses shar
memory to share data between processes under the hood, particularly for tensor parallel inference.
:::

:::{note}
Optional dependencies are not included in order to avoid licensing issues (e.g. <gh-issue:8030>).

If you need to use those dependencies (having accepted the license terms),
create a custom Dockerfile on top of the base image with an extra layer that installs them:

```Dockerfile
FROM vllm/vllm-openai:v0.7.3

# e.g. install the `audio` and `video` optional dependencies
# NOTE: Make sure the version of vLLM matches the base image!
RUN uv pip install --system vllm[audio,video]==0.7.3
```

:::

:::{tip}
Some new models may only be available on the main branch of [HF Transformers](https://github.com/huggingface/transformers).

To use the development version of `transformers`, create a custom Dockerfile on top of the base image
with an extra layer that installs their code from source:

```Dockerfile
FROM vllm/vllm-openai:latest

RUN uv pip install --system git+https://github.com/huggingface/transformers.git
```

:::

(deployment-docker-build-image-from-source)=

## Building vLLM's Docker Image from Source
Expand Down

0 comments on commit 8354f66

Please sign in to comment.