Fix v1 supported oracle for worker-cls and worker-extension-cls #15324

hijkzzz · 2025-03-22T07:58:13Z

No description provided.

github-actions · 2025-03-22T07:58:21Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

vllm/engine/arg_utils.py

youkaichao · 2025-03-22T08:21:17Z

full error log:

INFO 03-22 01:13:06 [__init__.py:256] Automatically detected platform cuda.
2025-03-22 01:13:12,038 INFO worker.py:1832 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
(pid=1621476) INFO 03-22 01:13:17 [__init__.py:256] Automatically detected platform cuda.
(MyLLM pid=1621476) INFO 03-22 01:13:24 [config.py:585] This model supports multiple tasks: {'reward', 'embed', 'classify', 'score', 'generate'}. Defaulting to 'generate'.
(MyLLM pid=1621476) INFO 03-22 01:13:24 [config.py:1693] Chunked prefill is enabled with max_num_batched_tokens=16384.
(MyLLM pid=1621476) WARNING 03-22 01:13:24 [cuda.py:96] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
(MyLLM pid=1621476) WARNING 03-22 01:13:26 [utils.py:2176] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/getting_started/troubleshooting.html#python-multiprocessing for more information. Reason: In a Ray actor and can only be spawned
(MyLLM pid=1621476) INFO 03-22 01:13:29 [__init__.py:256] Automatically detected platform cuda.
(MyLLM pid=1621476) INFO 03-22 01:13:31 [core.py:54] Initializing a V1 LLM engine (v0.7.3.dev876+g2fa0e1396) with config: model='facebook/opt-125m', speculative_config=None, tokenizer='facebook/opt-125m', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=2048, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=None, served_model_name=facebook/opt-125m, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=False, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[],"max_capture_size":0}
(MyLLM pid=1621476) 2025-03-22 01:13:33,275     INFO worker.py:1832 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8266 
(MyLLM pid=1621476) INFO 03-22 01:13:34 [ray_utils.py:316] Using the existing placement group
...
(MyLLM pid=1621476) ERROR 03-22 01:13:34 [core.py:343]     ray._private.state.state.placement_group_table(pg_id)["bundles"].values()
(MyLLM pid=1621476) ERROR 03-22 01:13:34 [core.py:343] KeyError: 'bundles'

@comaniac it seems passing the placement group is not enough. the new process fails to recognize the old ray cluster.

2d1e722 should fix it.

Signed-off-by: youkaichao <youkaichao@gmail.com>

youkaichao · 2025-03-22T08:43:42Z

vllm/engine/arg_utils.py

-        if self.worker_cls != EngineArgs.worker_cls:
-            _raise_or_fallback(feature_name="--worker-cls",
-                               recommend_to_remove=False)
-            return False
-
-        if self.worker_extension_cls != EngineArgs.worker_extension_cls:
-            _raise_or_fallback(feature_name="--worker-extension-cls",
-                               recommend_to_remove=False)
-            return False
-


cc @robertgshaw2-redhat for visibility.

youkaichao · 2025-03-22T08:45:44Z

.buildkite/test-pipeline.yaml

  - pushd ../examples/offline_inference
-  - python3 rlhf.py
-  - RAY_DEDUP_LOGS=0 python3 rlhf_colocate.py
+  - VLLM_ENABLE_V1_MULTIPROCESSING=0 python3 rlhf.py


in v1, LLM.collective_rpc is broken in the default case, because self.llm_engine.model_executor is in a different process.

cc @njhill @robertgshaw2-redhat if you can help fix it. i'm disabling VLLM_ENABLE_V1_MULTIPROCESSING for the test right now.

Do you think we should fall back to this automatically if the user selects --worker-cls?

we don't need to fall back. worker-cls and rlhf can be separate things.

youkaichao · 2025-03-22T08:46:29Z

vllm/utils.py

+        # even we choose to spawn, we need to pass to the subprocess
+        # the ray address, so that it knows to connect to the ray cluster.
+        # env vars are inherited by subprocesses, even if we use spawn.
+        import ray
+        os.environ["RAY_ADDRESS"] = ray.get_runtime_context().gcs_address


this is a follow-up after #14705

Ah make sense. I only tested the Ray Serve use case before. cc @ruisearch42

ruisearch42

LG

ruisearch42 · 2025-03-23T02:00:05Z

vllm/utils.py

+        # even we choose to spawn, we need to pass to the subprocess
+        # the ray address, so that it knows to connect to the ray cluster.
+        # env vars are inherited by subprocesses, even if we use spawn.
+        import ray


ray is not always installed for vllm, we need a try-except like is_in_ray_actor()

this is guarded under is_in_ray_actor(), so it is safe.

Yeah that's right, realized it afterwards

vllm/utils.py

Signed-off-by: youkaichao <youkaichao@gmail.com>

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com>

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com> Signed-off-by: Wes Medford <wryanmedford@gmail.com>

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com> Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com>

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com> Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>

Fix arg_utils.py for OpenRLHF using vLLM V1 engine

48a4da8

youkaichao reviewed Mar 22, 2025

View reviewed changes

vllm/engine/arg_utils.py Outdated Show resolved Hide resolved

Update arg_utils.py

effacb0

youkaichao changed the title ~~Fix arg_utils.py for OpenRLHF using vLLM V1 engine~~ Fix v1 supported oracle for worker-cls and worker-extension-cls Mar 22, 2025

youkaichao added 3 commits March 22, 2025 16:30

fix cluster init

2d1e722

Signed-off-by: youkaichao <youkaichao@gmail.com>

add comments

80626e5

Signed-off-by: youkaichao <youkaichao@gmail.com>

fix tests

df8a73e

Signed-off-by: youkaichao <youkaichao@gmail.com>

youkaichao reviewed Mar 22, 2025

View reviewed changes

mergify bot added the ci/build label Mar 22, 2025

youkaichao reviewed Mar 22, 2025

View reviewed changes

youkaichao requested a review from robertgshaw2-redhat March 22, 2025 17:09

ruisearch42 reviewed Mar 23, 2025

View reviewed changes

update comments

55075bf

Signed-off-by: youkaichao <youkaichao@gmail.com>

comaniac approved these changes Mar 23, 2025

View reviewed changes

youkaichao mentioned this pull request Mar 23, 2025

[Bug]: LLM.collective_rpc is broken in v1 by default #15349

Closed

1 task

youkaichao merged commit 0661cfe into vllm-project:main Mar 23, 2025
5 of 8 checks passed

youkaichao mentioned this pull request Mar 23, 2025

[ci/build] fix broken tests in LLM.collective_rpc #15350

Merged

erictang000 pushed a commit to erictang000/vllm that referenced this pull request Mar 25, 2025

Fix v1 supported oracle for worker-cls and worker-extension-cls (vllm…

70e1cdd

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com>

ckhordiasma mentioned this pull request Apr 17, 2025

[do not merge] pr test for nm changes into 2.20 red-hat-data-services/vllm#107

Closed

lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Apr 29, 2025

Fix v1 supported oracle for worker-cls and worker-extension-cls (vllm…

ef20763

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com>

shreyankg pushed a commit to shreyankg/vllm that referenced this pull request May 3, 2025

Fix v1 supported oracle for worker-cls and worker-extension-cls (vllm…

d282cb5

…-project#15324) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: youkaichao <youkaichao@gmail.com>

Uh oh!

Fix v1 supported oracle for worker-cls and worker-extension-cls #15324

Fix v1 supported oracle for worker-cls and worker-extension-cls #15324

Uh oh!

Conversation

hijkzzz commented Mar 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 22, 2025

Uh oh!

Uh oh!

youkaichao commented Mar 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ruisearch42 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hijkzzz commented Mar 22, 2025 •

edited by github-actions bot

Loading

youkaichao commented Mar 22, 2025 •

edited

Loading