Skip to content

[FIX]Patch run-cluster.sh (fix for #28328) #30002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
evberrypi wants to merge 3 commits into
base: main
Choose a base branch
from
+4 −24
Open

[FIX]Patch run-cluster.sh (fix for #28328) #30002

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fc13dac
Patch run-cluster.sh
evberrypi Dec 3, 2025
55b981b
Build VLLM_HOST_IP from additional args
evberrypi Dec 4, 2025
46322ad
remove extra for loop
evberrypi Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 4 additions & 24 deletions examples/online_serving/run_cluster.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,32 +78,13 @@ trap cleanup EXIT
# while workers connect to the head's address.
RAY_START_CMD="ray start --block"
if [ "${NODE_TYPE}" == "--head" ]; then
RAY_START_CMD+=" --head --port=6379"
RAY_START_CMD+=" --head --node-ip-address=${HEAD_NODE_ADDRESS} --port=6379"
else
RAY_START_CMD+=" --address=${HEAD_NODE_ADDRESS}:6379"
fi

# Parse VLLM_HOST_IP from additional args if present.
# This is needed for multi-NIC configurations where Ray needs explicit IP bindings.
VLLM_HOST_IP=""
Copy link
Collaborator

@ruisearch42 ruisearch42 Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate a bit why VLLM_HOST_IP does not work? I think it was designed for picking an IP when there are multiple:

vllm/vllm/envs.py

Line 519 in 4c2e10e

# of the current node, when the node has multiple network interfaces.

for arg in "${ADDITIONAL_ARGS[@]}"; do
if [[ $arg == "-e" ]]; then
continue
fi
if [[ $arg == VLLM_HOST_IP=* ]]; then
VLLM_HOST_IP="${arg#VLLM_HOST_IP=}"
break
RAY_START_CMD+=" --address=${HEAD_NODE_ADDRESS}:6379"
if [ -n "${VLLM_HOST_IP}" ]; then
RAY_START_CMD+=" --node-ip-address=${VLLM_HOST_IP}"
fi
done

# Build Ray IP environment variables if VLLM_HOST_IP is set.
# These variables ensure Ray binds to the correct network interface on multi-NIC systems.
RAY_IP_VARS=()
if [ -n "${VLLM_HOST_IP}" ]; then
RAY_IP_VARS=(
-e "RAY_NODE_IP_ADDRESS=${VLLM_HOST_IP}"
-e "RAY_OVERRIDE_NODE_IP_ADDRESS=${VLLM_HOST_IP}"
Comment on lines -104 to -105
Copy link
Collaborator

@ruisearch42 ruisearch42 Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ray doesn't have these env vars? So removing them should be good here. Any reason they were added in 28328 in the first place?

)
fi

# Launch the container with the assembled parameters.
Expand All @@ -118,6 +99,5 @@ docker run \
--shm-size 10.24g \
--gpus all \
-v "${PATH_TO_HF_HOME}:/root/.cache/huggingface" \
"${RAY_IP_VARS[@]}" \
"${ADDITIONAL_ARGS[@]}" \
"${DOCKER_IMAGE}" -c "${RAY_START_CMD}"