[Feature]: Support kv-cache-dtype fp8 without nvcc

[samos123]

🚀 The feature, motivation and pitch

Currently the following error is thrown if nvcc is not installed:

WARNING 05-08 01:34:59 utils.py:313] Not found nvcc in /usr/local/cuda. Skip cuda version check!
Traceback (most recent call last):                                                                              
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,                                                                  
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)                                                                                     
  File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 159, in <module>
    engine = AsyncLLMEngine.from_engine_args(                                                                   
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/async_llm_engine.py", line 341, in from_engine_args
    engine_config = engine_args.create_engine_config()
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py", line 471, in create_engine_config
    cache_config = CacheConfig(self.block_size,
  File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 310, in __init__
    self._verify_cache_dtype()
  File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 333, in _verify_cache_dtype
    if nvcc_cuda_version < Version("11.8"):
TypeError: '<' not supported between instances of 'NoneType' and 'Version'

However, we should be able to do a version check without depending on nvcc. So let's remove this dependency.

Alternatives

No response

Additional context

No response

[samos123]

I can make the PR if this would be accepted

[youkaichao]

Is it this one? #4173

[samos123]

I guess reverting that PR might be all that's needed. That PR seems to be the reason why nvcc dependency was introduced.

edit: on a closer look, what I said is not true. The PR doesn't fix this nor did it introduce the nvcc dependency. The dependency seems to be introduced previously.

[samos123]

So I think the PR almost fixes this. However, the version check is still done and relies on nvcc is my understanding? So you can't use fp8 unless you install nvcc even with that PR.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!