Skip to content
This repository was archived by the owner on Aug 7, 2025. It is now read-only.

Commit 048aa53

Browse files
authored
Set vllm multiproc method to spawn (#3310)
* Use spawn instead of fork method for vllm * Fix lint
1 parent d6ea6e7 commit 048aa53

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

ts/torch_handler/vllm_handler.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import asyncio
22
import logging
3+
import os
34
import pathlib
45
import time
56
from unittest.mock import MagicMock
@@ -41,6 +42,8 @@ def initialize(self, ctx):
4142
ctx.model_yaml_config.get("handler", {})
4243
)
4344

45+
os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
46+
4447
self.vllm_engine = AsyncLLMEngine.from_engine_args(vllm_engine_config)
4548

4649
self.adapters = ctx.model_yaml_config.get("handler", {}).get("adapters", {})

0 commit comments

Comments
 (0)