[Frontend]Reduce vLLM's import time

[Chen-0210]

This PR optimizes the import time of from vllm import LLMand fix this issue #14924
The majority of the changes only involve reordering the import statements. SoI think this change will not affect the core functionality and can reduce import time.

Comparison

time python3 -c "import vllm"

Before:

INFO 03-23 16:46:03 [__init__.py:260] No platform detected, vLLM is running on UnspecifiedPlatform

real    0m5.923s
user    0m12.976s
sys     0m5.908s

After:

The time is mainly costed in two parts

1. import torch takes 1.5–2s
2. from openai import xxx takes around 0.5s. I don't know how to fix these since they are used in complex ways.

real    0m2.957s
user    0m3.570s
sys     0m5.284s

time vllm -v

Before:

INFO 03-23 16:49:14 [__init__.py:260] No platform detected, vLLM is running on UnspecifiedPlatform
0.8.1

real    0m6.389s
user    0m13.483s
sys     0m5.955s

After:

time vllm -v can be optimized in vllm/entrypoints/cli, and want to implemente it in a separate pr.

INFO 03-23 16:48:52 [__init__.py:260] No platform detected, vLLM is running on UnspecifiedPlatform
0.1.dev5099+gff47aab

real    0m4.430s
user    0m4.870s
sys     0m5.731s

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Chen-0210.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Chen-0210.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[davidxia]

I squashed all the commits into one, rebased on top of latest main, fixed conflicts, and force pushed from 1393d49 to dd1de69.

[Chen-0210]

@davidxia Could you please run the pre-commit hook? I noticed some lint issues that need to be fixed.

[davidxia]

I reran the benchmarks with the same setup. The main branch's times are here.

commit 36a0bc1e5bf5621ab42accca24ee454d9e18583e

python -c 'import vllm' 52% faster!

$ hyperfine 'python -c "import vllm"' --warmup 3 --runs 20 --export-markdown out.md
Benchmark 1: python -c "import vllm"
  Time (mean ± σ):      3.504 s ±  0.119 s    [User: 3.947 s, System: 0.366 s]
  Range (min … max):    3.306 s …  3.891 s    20 runs

Command	Mean [s]	Min [s]	Max [s]	Relative
python -c "import vllm"	3.504 ± 0.119	3.306	3.891	1.00

vllm --version

$ hyperfine 'vllm --version' --warmup 3 --runs 20 --export-markdown out.md
Benchmark 1: vllm --version
  Time (mean ± σ):      9.853 s ±  0.329 s    [User: 9.643 s, System: 0.973 s]
  Range (min … max):    9.515 s … 10.599 s    20 runs

Command	Mean [s]	Min [s]	Max [s]	Relative
vllm --version	9.853 ± 0.329	9.515	10.599	1.00

[davidxia]

rebased and resolved conflicts

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Chen-0210.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[davidxia]

Looks like conflicts again. I can help fix again, but just wanted to know from maintainers if this PR is still something you think is worth doing and maybe a cursory review before we spend too much time on this.

[reidliu41]

lol, Just tested the latest code, since currently, it will load more and more things..

vllm --help
INFO 05-10 16:58:03 [__init__.py:248] Automatically detected platform cuda.
INFO 05-10 16:58:10 [config.py:760] This model supports multiple tasks: {'reward', 'score', 'generate', 'embed', 'classify'}. Defaulting to 'generate'.
WARNING 05-10 16:58:10 [config.py:2019] max_model_len was is not set. Defaulting to arbitrary value of 8192.
WARNING 05-10 16:58:10 [config.py:2025] max_num_seqs was is not set. Defaulting to arbitrary value of 128.
INFO 05-10 16:58:11 [config.py:760] This model supports multiple tasks: {'reward', 'score', 'generate', 'embed', 'classify'}. Defaulting to 'generate'.
INFO 05-10 16:58:12 [config.py:760] This model supports multiple tasks: {'reward', 'score', 'generate', 'embed', 'classify'}. Defaulting to 'generate'.
INFO 05-10 16:58:13 [config.py:760] This model supports multiple tasks: {'reward', 'score', 'generate', 'embed', 'classify'}. Defaulting to 'generate'.
usage: vllm [-h] [-v] {chat,complete,serve,bench,collect-env} ...

vLLM CLI

positional arguments:
  {chat,complete,serve,bench,collect-env}
    chat                Generate chat completions via the running API server.
    complete            Generate text completions based on the given prompt via the running API server.
    serve               Start the vLLM OpenAI Compatible API server.
    bench               vLLM bench subcommand.
    collect-env         Start collecting environment information.

options:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit


$ vllm --version
INFO 05-10 14:25:32 [__init__.py:248] Automatically detected platform cuda.
ERROR 05-10 14:30:34 [config.py:108] Error retrieving file list: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/facebook/opt-125m/tree/main?recursive=True&expand=False (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f9b59e508f0>, 'Connection to huggingface.co timed out. (connect timeout=None)'))"), '(Request ID: b598ef4d-db37-4a4a-aa4b-6e0e4fd561a8)'), retrying 1 of 2

[aarnphm]

I would suggest to try at smaller scale iteratively, let's start at multimodal directory?

[davidxia]

I would suggest to try at smaller scale iteratively, let's start at multimodal directory?

Sounds good. I'll make a separate PR soon.

[davidxia]

I would suggest to try at smaller scale iteratively, let's start at multimodal directory?

@aarnphm #18031

[github-actions]

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!

[github-actions]

This pull request has been automatically closed due to inactivity. Please feel free to reopen if you intend to continue working on it. Thank you!