feature request: Dockerfile

[gaocegege]

I think it would be better if we could provide dockerfile in vLLM, for distributed serving, and single-gpu serving.

Then we could use it on Kubernetes or other container-based environments.

[gaocegege]

We have a reference file: https://github.com/kemingy/vllm-env/blob/main/Dockerfile for MPT 30b.

[samos123]

I am happy to submit a PR using this Dockerfile too: https://github.com/substratusai/vllm-docker

The approach I took is slightly different and rely on runtime image instead of devel NVIDIA cuda image. The devel image isn't needed but a lot bigger in size.

[theycallmeloki]

I'd like to help and want to specifically target self hosted systems to bring up the required mpirun environment to quickly run these in parallel for inferencing

I think particularly now it uses Ray which is working great for me but I would like to add a bit of service discovery as well so we can know they can bootstrap talking together like how they do in the mpi operator in k8s land

[simon-mo]

Closed by #1350