Skip to content

Commit e5cb02e

Browse files
committed
only enable cuda platform
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
1 parent 1da28d9 commit e5cb02e

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm/v1/core/kv_cache_utils.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -801,7 +801,8 @@ def get_kv_cache_config(vllm_config: VllmConfig,
801801
The generated KVCacheConfigs
802802
"""
803803
check_enough_kv_cache_memory(vllm_config, kv_cache_spec, available_memory)
804-
if vllm_config.cache_config.disable_hybrid_allocator:
804+
if (vllm_config.cache_config.disable_hybrid_allocator
805+
or vllm_config.device_config.device.type != "cuda"):
805806
unify_hybrid_kv_cache_specs(kv_cache_spec)
806807
if is_kv_cache_type_uniform(kv_cache_spec):
807808
# KV cache of all layers are the same, which is true for

0 commit comments

Comments
 (0)