Skip to content

Conversation

noooop
Copy link
Contributor

@noooop noooop commented Sep 11, 2025

Purpose

  • ci envs for convenient testing
    • VLLM_CI_NO_SKIP
    • VLLM_CI_DTYPE
    • VLLM_CI_HEAD_DTYPE
    • VLLM_CI_HF_DTYPE

e.g. testing Flex Attention, you don't need to manually modify dtype=float32 #24089

VLLM_CI_DTYPE=float32 pytest tests/models/language/pooling_mteb_test/test_st_projector.py::test_embed_models_mteb[model_info1] 
VLLM_CI_DTYPE=float32 pytest tests/models/language/pooling_mteb_test/test_gte.py::test_rerank_models_mteb[model_info0]

A model family has many models with the same architecture. By default, a model family tests only one model.
Through VLLM_CI_NO_SKIP, all models can be tested.

VLLM_CI_NO_SKIP=1 pytest tests/models/language/pooling_mteb_test/test_baai.py::test_embed_models_mteb[model_info1]

VLLM_CI_HEAD_DTYPE & VLLM_CI_HF_DTYPE #24567

VLLM_CI_HF_DTYPE="float32" VLLM_CI_HEAD_DTYPE="model" pytest -s -vvv tests/models/language/generation_ppl_test/test_gpt.py

hitchhiking

cc @DarkLight1337 @maxdebayser

Test Plan

keep ci green

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mergify mergify bot added ci/build qwen Related to Qwen models labels Sep 11, 2025
@noooop
Copy link
Contributor Author

noooop commented Sep 11, 2025

cc @DarkLight1337

What do you think about this split?

@DarkLight1337
Copy link
Member

DarkLight1337 commented Sep 11, 2025

Let's do this one directory at a time, to avoid creating many merge conflicts

@noooop noooop changed the title [CI] Split pooling from entrypoints Test & Split mteb test from Language Models Test [CI] Add ci_envs for convenient testing Sep 11, 2025
Copy link

mergify bot commented Sep 11, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @noooop.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Copy link

mergify bot commented Sep 11, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @noooop.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: wang.yuqi <noooop@126.com>
@mergify mergify bot removed the needs-rebase label Sep 11, 2025
@noooop noooop marked this pull request as ready for review September 11, 2025 14:15
@noooop noooop changed the title [CI] Add ci_envs for convenient testing [CI] Add ci_envs for convenient local testing Sep 11, 2025
@noooop
Copy link
Contributor Author

noooop commented Sep 12, 2025

@DarkLight1337

Could you please review this pr.

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <noooop@126.com>
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) September 12, 2025 07:23
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 12, 2025
@DarkLight1337 DarkLight1337 merged commit d21a36f into vllm-project:main Sep 12, 2025
50 checks passed
@noooop noooop deleted the split_test branch September 12, 2025 09:52
skyloevil pushed a commit to skyloevil/vllm that referenced this pull request Sep 13, 2025
BoyuanFeng pushed a commit to BoyuanFeng/vllm that referenced this pull request Sep 14, 2025
dsxsteven pushed a commit to dsxsteven/vllm_splitPR that referenced this pull request Sep 15, 2025
cboss6 pushed a commit to cboss6/vllm that referenced this pull request Sep 16, 2025
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
cboss6 pushed a commit to cboss6/vllm that referenced this pull request Sep 16, 2025
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
langc23 pushed a commit to zte-riscv/vllm that referenced this pull request Sep 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/build qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants