Skip to content
This repository was archived by the owner on Oct 11, 2024. It is now read-only.

Commit c1e7a79

Browse files
AnyISalInRobert Shaw
authored andcommitted
[Bugfix] Fix the fp8 kv_cache check error that occurs when failing to obtain the CUDA version. (vllm-project#4173)
Signed-off-by: AnyISalIn <anyisalin@gmail.com>
1 parent b496ac2 commit c1e7a79

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm/config.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -385,7 +385,8 @@ def _verify_cache_dtype(self) -> None:
385385
elif self.cache_dtype == "fp8":
386386
if not is_hip():
387387
nvcc_cuda_version = get_nvcc_cuda_version()
388-
if nvcc_cuda_version < Version("11.8"):
388+
if nvcc_cuda_version is not None \
389+
and nvcc_cuda_version < Version("11.8"):
389390
raise ValueError(
390391
"FP8 is not supported when cuda version is"
391392
"lower than 11.8.")

0 commit comments

Comments
 (0)