Plans to make the installation work on Windows without WSL?

[alexandre-ist]

I get the following error during install:

No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\

Due to my company's policy we can't have WSL enabled on our GPU machine, are there any plans for making installs possible on Windows without WSL?

Thank you,

[ABooth01]

I would like to know this too as I have the same challenge.

[jpeig]

Why not install CUDA? @ABooth01 @alexandre-ist

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.1.0/local_installers/cuda-repo-wsl-ubuntu-12-1-local_12.1.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-1-local_12.1.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-12-1-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

[alexandre-ist]

We don't have access to WSL or any Linux option, so can't run your code.

In my case we do have CUDA installed, however the vllm pip install fails on Windows.
The solution so far seems to be using WSL, which many companies do not enable for "security" reasons...
(https://github.com/vllm-project/vllm/issues/778

[jim-plus]

The first obstacle under plain Windows appears to be cmake failing to find a CUDA runtime despite CUDA_HOME being set. This may be something as simple as the difference between Unix versus Windows filespecs.

[MoonRide303]

@hmellor Why it's not planned? All the major components are natively supported on Windows:

• Python,
• PyTorch,
• CUDA,
• CMake.

Windows has ~70% market share of the desktop OS. Other apps for both text and visual models (llama.cpp, ComfyUI, etc.) work just fine, with GPU acceleration. Why you want majority of the desktops users to be unable to use vLLM? Isn't your motto "Easy, fast, and cheap LLM serving for everyone"?

[hmellor]

vLLM is not really meant for desktop use, so Windows's market share isn't very important here. This is why most development effort in vLLM goes towards improving vLLM for use in the cloud/server.

If somebody contributes Windows support it'll likely be accepted, but it's not a priority for vLLM to develop this feature.

[ABooth01]

@hmellor Understood but I was using windows machines in AWS, so not desktop use but still cloud based as per company requirements at that time. Seems limited to ignore Windows machines desktop or otherwise? I have since managed to migrate everything over to Linux machines now so no longer an issue.

[hmellor]

Oh I see, sorry for the misunderstanding.

I believe this feature request was originally for desktop use and I had closed it as stale (which GitHub calls not planned) as there has been no progress or discussion in many months.

If a new feature request for Windows server was made then we could leave that as long as it takes.