Skip to content

Commit 424b3dc

Browse files
Adolfo Victoriafacebook-github-bot
authored andcommitted
Respect passed in device overrides in engine args
Summary: Signed-off-by: Adolfo Victoria adovi@meta.com When calling `create_engine_config` we don't respect the device set in engine args and instead grab it from the runtime platform directly. This logic is also extraneous since the default behavior (i.e. device=auto) is to set the device to the runtime platform device if auto is passed inside of `__post_init__` inside of `DeviceConfig` Differential Revision: D75496412
1 parent b78f844 commit 424b3dc

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/engine/arg_utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -979,7 +979,7 @@ def create_engine_config(
979979
from vllm.platforms import current_platform
980980
current_platform.pre_register_and_update()
981981

982-
device_config = DeviceConfig(device=current_platform.device_type)
982+
device_config = DeviceConfig(device=self.device)
983983
model_config = self.create_model_config()
984984

985985
# * If VLLM_USE_V1 is unset, we enable V1 for "supported features"

0 commit comments

Comments
 (0)