Could not build wheels for vllm, which is required to install pyproject.toml-based projects

[ABooth01]

This appears to be an issue with CUDA version requiring 11.7. However, there are security concerns with this version of CUDA requiring me to use version 12.2 is there any intention of updating vLLM to work with the latest version of CUDA?

[latishab]

Building wheels for collected packages: vllm
Building wheel for vllm (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for vllm (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [157 lines of output]
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.3'
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-311
creating build\lib.win-amd64-cpython-311\vllm
copying vllm\block.py -> build\lib.win-amd64-cpython-311\vllm
copying vllm\config.py -> build\lib.win-amd64-cpython-311\vllm
copying vllm\logger.py -> build\lib.win-amd64-cpython-311\vllm
copying vllm\outputs.py -> build\lib.win-amd64-cpython-311\vllm
copying vllm\sampling_params.py -> build\lib.win-amd64-cpython-311\vllm
copying vllm\sequence.py -> build\lib.win-amd64-cpython-311\vllm
copying vllm\utils.py -> build\lib.win-amd64-cpython-311\vllm
copying vllm_init_.py -> build\lib.win-amd64-cpython-311\vllm
creating build\lib.win-amd64-cpython-311\vllm\core
copying vllm\core\block_manager.py -> build\lib.win-amd64-cpython-311\vllm\core
copying vllm\core\policy.py -> build\lib.win-amd64-cpython-311\vllm\core
copying vllm\core\scheduler.py -> build\lib.win-amd64-cpython-311\vllm\core
copying vllm\core_init_.py -> build\lib.win-amd64-cpython-311\vllm\core
creating build\lib.win-amd64-cpython-311\vllm\engine
copying vllm\engine\arg_utils.py -> build\lib.win-amd64-cpython-311\vllm\engine
copying vllm\engine\async_llm_engine.py -> build\lib.win-amd64-cpython-311\vllm\engine
copying vllm\engine\llm_engine.py -> build\lib.win-amd64-cpython-311\vllm\engine
copying vllm\engine\ray_utils.py -> build\lib.win-amd64-cpython-311\vllm\engine
copying vllm\engine_init_.py -> build\lib.win-amd64-cpython-311\vllm\engine
creating build\lib.win-amd64-cpython-311\vllm\entrypoints
copying vllm\entrypoints\api_server.py -> build\lib.win-amd64-cpython-311\vllm\entrypoints
copying vllm\entrypoints\llm.py -> build\lib.win-amd64-cpython-311\vllm\entrypoints
copying vllm\entrypoints_init_.py -> build\lib.win-amd64-cpython-311\vllm\entrypoints
creating build\lib.win-amd64-cpython-311\vllm\model_executor
copying vllm\model_executor\input_metadata.py -> build\lib.win-amd64-cpython-311\vllm\model_executor
copying vllm\model_executor\model_loader.py -> build\lib.win-amd64-cpython-311\vllm\model_executor
copying vllm\model_executor\utils.py -> build\lib.win-amd64-cpython-311\vllm\model_executor
copying vllm\model_executor\weight_utils.py -> build\lib.win-amd64-cpython-311\vllm\model_executor
copying vllm\model_executor_init_.py -> build\lib.win-amd64-cpython-311\vllm\model_executor
creating build\lib.win-amd64-cpython-311\vllm\transformers_utils
copying vllm\transformers_utils\config.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils
copying vllm\transformers_utils\tokenizer.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils
copying vllm\transformers_utils_init_.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils
creating build\lib.win-amd64-cpython-311\vllm\worker
copying vllm\worker\cache_engine.py -> build\lib.win-amd64-cpython-311\vllm\worker
copying vllm\worker\worker.py -> build\lib.win-amd64-cpython-311\vllm\worker
copying vllm\worker_init_.py -> build\lib.win-amd64-cpython-311\vllm\worker
creating build\lib.win-amd64-cpython-311\vllm\entrypoints\openai
copying vllm\entrypoints\openai\api_server.py -> build\lib.win-amd64-cpython-311\vllm\entrypoints\openai
copying vllm\entrypoints\openai\protocol.py -> build\lib.win-amd64-cpython-311\vllm\entrypoints\openai
copying vllm\entrypoints\openai_init_.py -> build\lib.win-amd64-cpython-311\vllm\entrypoints\openai
creating build\lib.win-amd64-cpython-311\vllm\model_executor\layers
copying vllm\model_executor\layers\activation.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers
copying vllm\model_executor\layers\attention.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers
copying vllm\model_executor\layers\layernorm.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers
copying vllm\model_executor\layers\rotary_embedding.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers
copying vllm\model_executor\layers\sampler.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers
copying vllm\model_executor\layers_init_.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers
creating build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\aquila.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\baichuan.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\bloom.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\falcon.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\gpt2.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\gpt_bigcode.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\gpt_j.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\gpt_neox.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\internlm.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\llama.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\mistral.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\mpt.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\opt.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models\qwen.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
copying vllm\model_executor\models_init_.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\models
creating build\lib.win-amd64-cpython-311\vllm\model_executor\parallel_utils
copying vllm\model_executor\parallel_utils\communication_op.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\parallel_utils
copying vllm\model_executor\parallel_utils\layers.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\parallel_utils
copying vllm\model_executor\parallel_utils\parallel_state.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\parallel_utils
copying vllm\model_executor\parallel_utils\utils.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\parallel_utils
copying vllm\model_executor\parallel_utils_init_.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\parallel_utils
creating build\lib.win-amd64-cpython-311\vllm\model_executor\quantization_utils
copying vllm\model_executor\quantization_utils\awq.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\quantization_utils
copying vllm\model_executor\quantization_utils\base.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\quantization_utils
copying vllm\model_executor\quantization_utils_init_.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\quantization_utils
creating build\lib.win-amd64-cpython-311\vllm\model_executor\layers\quantized_linear
copying vllm\model_executor\layers\quantized_linear\awq.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers\quantized_linear
copying vllm\model_executor\layers\quantized_linear_init_.py -> build\lib.win-amd64-cpython-311\vllm\model_executor\layers\quantized_linear
creating build\lib.win-amd64-cpython-311\vllm\transformers_utils\configs
copying vllm\transformers_utils\configs\aquila.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils\configs
copying vllm\transformers_utils\configs\baichuan.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils\configs
copying vllm\transformers_utils\configs\falcon.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils\configs
copying vllm\transformers_utils\configs\mpt.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils\configs
copying vllm\transformers_utils\configs\qwen.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils\configs
copying vllm\transformers_utils\configs_init_.py -> build\lib.win-amd64-cpython-311\vllm\transformers_utils\configs
running build_ext
C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\torch\utils\cpp_extension.py:359: UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified
warnings.warn(f'Error checking compiler version for {compiler}: {error}')
Traceback (most recent call last):
File "C:\Users\SNS\anaconda3\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in
main()
File "C:\Users\SNS\anaconda3\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\SNS\anaconda3\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\build_meta.py", line 434, in build_wheel
return self._build_with_temp_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\build_meta.py", line 419, in build_with_temp_dir
self.run_setup()
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
exec(code, locals())
File "", line 252, in
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_init.py", line 103, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\core.py", line 201, in run_commands
dist.run_commands()
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\dist.py", line 969, in run_commands
self.run_command(cmd)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\dist.py", line 989, in run_command
super().run_command(command)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\wheel\bdist_wheel.py", line 364, in run
self.run_command("build")
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\dist.py", line 989, in run_command
super().run_command(command)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\command\build.py", line 131, in run
self.run_command(cmd_name)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\dist.py", line 989, in run_command
super().run_command(command)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 88, in run
_build_ext.run(self)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\setuptools_distutils\command\build_ext.py", line 345, in run
self.build_extensions()
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\torch\utils\cpp_extension.py", line 499, in build_extensions
_check_cuda_version(compiler_name, compiler_version)
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\torch\utils\cpp_extension.py", line 383, in _check_cuda_version
torch_cuda_version = packaging.version.parse(torch.version.cuda)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\pkg_resources_vendor\packaging\version.py", line 52, in parse
return Version(version)
^^^^^^^^^^^^^^^^
File "C:\Users\SNS\AppData\Local\Temp\pip-build-env-5_6jbzx7\overlay\Lib\site-packages\pkg_resources_vendor\packaging\version.py", line 196, in init
match = self._regex.search(version)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for vllm
Failed to build vllm
ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects

(base) C:\Users\SNS>

[adityamiskin]

Facing the same issue but for cuda version = 11.8

[gstarwd]

same error

[bacoco]

Same problem

[ugref]

Same error with cuda v12.3

[latishab]

Same error with cuda v12.3

Same, I have an error with cuda v12.3

[ugref]

Retried with changed CUDA version to 11.7.1, same problem occurs

[DoodleBears]

Same when I try it on Windows. But, it works fine in WSL.

As far as I know,

• NVIDIA Driver will show a maximum capable version the CUDA Toolkit can be installed in right-up corner when using nvidia-smi
• I install vllm in WSL using Anaconda on my Windows 11 with
  • CUDA Toolkit = 11.8 with pytorch 2.0.1 working fine
  • CUDA Toolkit = 11.7 with pytorch 2.0.1 working fine
• I also install conda install -y cudatoolkit-dev -c conda-forge

[Siegi96]

any fix for that?

[xiexq2019]

any fix for that?

[raygeAI]

any fix for that?

[evolu8]

Still failing. Thoughts?

[darkbloodevil]

waiting for fix

[WaddleKing]

i am still waiting for fix, it is new years please i beg

[SuperMalinge]

same issues here