Skip to content

Commit 79fcfc9

Browse files
committed
[Bugfix] Respect num-gpu-blocks-override in v1
Signed-off-by: Jon Swenson <jmswen@gmail.com>
1 parent 29fa5ca commit 79fcfc9

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

vllm/v1/core/kv_cache_utils.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -660,6 +660,7 @@ def get_num_blocks(vllm_config: VllmConfig, num_layers: int,
660660
logger.info(
661661
"Overriding num_gpu_blocks=%d with "
662662
"num_gpu_blocks_override=%d", num_blocks, num_gpu_blocks_override)
663+
return num_gpu_blocks_override
663664
return num_blocks
664665

665666

0 commit comments

Comments
 (0)