Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix] set OMP_NUM_THREADS to 1 by default when using the multiproc_gpu_executor #6109

Merged
merged 3 commits into from
Jul 3, 2024
Merged

[Bugfix] set OMP_NUM_THREADS to 1 by default when using the multiproc_gpu_executor #6109

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ea3b340
fix: set OMP_NUM_THREADS when using MP executor
tjohnson31415 Jul 3, 2024
c562066
review: nit format
tjohnson31415 Jul 3, 2024
9705c67
fixup: fix review suggestion syntax
tjohnson31415 Jul 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fixup: fix review suggestion syntax 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
  • Loading branch information
@tjohnson31415
tjohnson31415 committed Jul 3, 2024
commit 9705c67a4a961e5527c9797ad35191bc6fc12267
2 changes: 1 addition & 1 deletion vllm/executor/multiproc_gpu_executor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ def _init_executor(self) -> None:

# Set OMP_NUM_THREADS to 1 if it is not set explicitly, avoids CPU
# contention amongst the shards
if "OMP_NUM_THREADS" is not in os.environ:
if "OMP_NUM_THREADS" not in os.environ:
os.environ["OMP_NUM_THREADS"] = "1"

assert world_size <= cuda_device_count_stateless(), (
Expand Down
Loading