[Bug]: LLM.collective_rpc is broken in v1 by default

[youkaichao]

Your current environment

v0.8.1

🐛 Describe the bug

see #15324 (comment) for details.

because self.llm_engine.model_executor is in a different process.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[youkaichao]

the current fix is to set VLLM_ENABLE_V1_MULTIPROCESSING=0

[rajveerb]

Is this true just for the latest release?

[youkaichao]

Is this true just for the latest release?

yes. welcome to help!

[wwl2755]

I also got a similar issue when I was trying to fix the load-format. Do you see a particular reason why it happens? VLLM_ENABLE_V1_MULTIPROCESSING=0 does not fix in that situation.

Details in: #14532 (comment)

[wwl2755]

Maybe this can be closed as llm.collective_rpc() can be supported by #15444

[youkaichao]

close as fixed by #15444