[Frontend] Add /collective_rpc API endpoint

[22quinn]

Purpose

Currently most RL frameworks interact with vLLM via LLM or AsyncLLM class, like the example here:

vllm/examples/offline_inference/rlhf.py

Line 112 in 8ea0c27

handle = llm.collective_rpc.remote(

In order for RL framework to interact with vLLM via the API server, a /collective_rpc endpoint is needed.

This endpoint accepts a json body with the following fields. For security reason, we do not handle (de-)serialization but accepts only strings.

• method: str
• args: list[str]
• kwargs: dict[str, str]
• timeout: Optional[float]

Test Plan

PYTHONPATH="/data/users/user/gitrepos/vllm" pytest tests/entrypoints/openai/test_collective_rpc.py

Test Result

• test_get_model_name passed
• test_return_none passed
• test_echo_args_kwargs passed

(Optional) Documentation Update

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request introduces a new /collective_rpc API endpoint. The changes include adding an abstract collective_rpc method to the EngineClient protocol and implementing the corresponding endpoint in the API server.

My review has identified a few issues:

1. A critical issue where the new abstract method collective_rpc appears to be unimplemented in the concrete engine classes provided in the context, which would lead to runtime errors.
2. A high-severity issue in the API endpoint implementation related to unsafe access to the request body, which could cause unhandled exceptions.
3. Another high-severity issue with how the results from the RPC call are serialized into the JSON response, which would lead to incorrect output for structured data.

Please see the detailed comments for suggestions on how to address these points.

[gemini-code-assist]

Using str(result) for serialization is problematic for structured data types like dictionaries or lists. It converts them into a string representation (e.g., "{'a': 1}") instead of a proper JSON object. This can lead to unexpected behavior on the client side. It's better to append the result directly and let JSONResponse handle the serialization. This will correctly serialize JSON-compatible types and raise an error for non-serializable types, which is more explicit.

Suggested change

	response.append(str(result))
	response.append(result)

[mgoin]

I think we should make the /collective_rpc endpoint only added if a CLI argument is set. I worry about the concern of security-conscious users seeing an "rpc" endpoint by default. Then we could also expand the support of the endpoint if it is opt-in

[22quinn]

I think we should make the /collective_rpc endpoint only added if a CLI argument is set. I worry about the concern of security-conscious users seeing an "rpc" endpoint by default. Then we could also expand the support of the endpoint if it is opt-in

This endpoint is currently gated behind VLLM_SERVER_DEV_MODE=1, similar to /reset_prefix_cache, /sleep, /wake_up. What kind of extra support you're thinking of?

[mgoin]

Ah I wasn't aware of that gate, that sounds sufficient to me.

[DarkLight1337]

LGTM

[DarkLight1337]

Please fix the failing tests though

[22quinn]

@DarkLight1337 The full version of entrypoints-test-api-server passed (only the fastcheck version failed) before your rebase. Same goes for openai-api-correctness. Probably safe to force merge?

[DarkLight1337]

I'll just retry the tests