Skip to content

Commit a88bb9b

Browse files
authored
[Bugfix] Fix the fp8 kv_cache check error that occurs when failing to obtain the CUDA version. (#4173)
Signed-off-by: AnyISalIn <anyisalin@gmail.com>
1 parent 6f1df80 commit a88bb9b

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm/config.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -353,7 +353,8 @@ def _verify_cache_dtype(self) -> None:
353353
elif self.cache_dtype == "fp8":
354354
if not is_hip():
355355
nvcc_cuda_version = get_nvcc_cuda_version()
356-
if nvcc_cuda_version < Version("11.8"):
356+
if nvcc_cuda_version is not None \
357+
and nvcc_cuda_version < Version("11.8"):
357358
raise ValueError(
358359
"FP8 is not supported when cuda version is"
359360
"lower than 11.8.")

0 commit comments

Comments
 (0)