Request for creation of a wheel for vllm

[dhritiman]

Hi vllm team,

We are looking to use vllm. One of the issues we are facing is that pip install for the project fails if CUDA is not installed on the build host. Can we have a wheel for vllm ? From the installation page I understand that "vLLM is a Python library that also contains some C++ and CUDA code. This additional code requires compilation on the user’s machine.". But can we not have platform specific wheels like for CentOS, macOS etc ?

Looking forward to hear from you.

Thanks,
Dhritiman

[zhuohan123]

Hi Dhritiman, this is actually a frequent ask from our users (e.g., #139). We are not very experienced on building wheels. If possible, could you take a look into this and maybe contribute? That would be a great contribution and very helpful for vLLM.

[WoosukKwon]

Hi @dhritiman, in the latest release and henceforth, vLLM is published with pre-built CUDA binaries (for Linux machines). Please try out pip install vllm and re-opens this issue if it does not work.