[1/N][Refactor] Refactor code to adapt with vllm main #3612

MengqingCao · 2025-10-22T03:11:00Z

What this PR does / why we need it?

This is the step 1 of refactoring code to adapt with vllm main, and this pr aligned with vllm-project/vllm@17c540a

refactor deepseek to the latest code arch as of vllm-project/vllm@17c540a
bunches of fixes due to vllm changes
- Fix AscendScheduler __post_init__, caused by [Bugfix]: Clean up chunked prefill logging when using whisper vllm#25075
- Fix AscendScheduler init got an unexpected arg block_size, caused by [bugfix][DCP] fix block_size of hash in DCP prefix caching vllm#26296
- Fix KVCacheManager get_num_common_prefix_blocks arg, caused by fix(v1/kv_cache): resolve async KV transfer bug in cascade attention vllm#23485
- Fix MLAAttention import,caused by Separate MLAAttention class from Attention vllm#25103
- Fix SharedFusedMoE import, caused by [Model] Apply shared experts overlap optimization to all models with shared experts vllm#26145
- Fix LazyLoader improt, caused by [Chore] Separate out vllm.utils.import_utils vllm#27022
- Fix vllm.utils.swap_dict_values improt, caused by [Chore] Separate out vllm.utils.collections vllm#26990
- Fix Backend enum import, caused by [Attention] Move Backend enum into registry vllm#25893
- Fix CompilationLevel renaming to CompilationMode issue introduced by [Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change compilation level to compilation mode, deprecation compilation level vllm#26355
- Fix fused_moe ops, caused by [ROCm][FEAT] Fuse DeepSeek shared experts into AITER fused_moe ops vllm#24097
- Fix bert model because of inputs_embeds, caused by [Bugfix] Token type and position embeddings fail to be applied to inputs_embeds vllm#25922
- Fix MRope because of get_input_positions_tensor to get_mrope_input_positions, caused by [Refactor]: Use M-RoPE interface directly while defining model class instead of maintaining model specific M-RoPE implementation in mrope.py vllm#24172
- Fix splitting_ops changes introduced by [torch.compile] Make inductor partition rules respect splitting_ops #25691 vllm#25845
- Fix multi-modality changes introduced by [Bugfix] Merge MM embeddings by index instead of token IDs vllm#16229
- Fix lora bias dropping issue introduced by Remove LoRA bias support vllm#25807
- Fix structured ouput break introduced by [Core] Streamline some structured output related code vllm#26737

Does this PR introduce any user-facing change?

How was this patch tested?

CI passed with existing test.

vLLM version: v0.11.0rc3
vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0

github-actions · 2025-10-22T03:11:15Z

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

A PR should do only one thing, smaller PRs enable faster reviews.
Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

gemini-code-assist

Code Review

This pull request refactors the code to adapt with the vllm main branch. The changes include removing an unused pre-commit hook, adding version-dependent logic for scheduler initialization and compilation level, and updating attention mechanisms for compatibility with different vllm versions. The review focuses on identifying potential issues related to version compatibility and code maintainability, specifically targeting high and critical severity issues.

tests/ut/kv_connector/test_mooncake_connector.py

vllm_ascend/lora/punica_npu.py

github-actions · 2025-10-22T06:21:56Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-10-22T06:21:57Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-10-22T06:21:57Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

vllm_ascend/worker/worker_v1.py

vllm_ascend/platform.py

MengqingCao · 2025-10-22T09:19:07Z

vllm_ascend/utils.py

 def version_check():
    """check if torch_npu version >= dev20250919"""
-    import re
+    import re  # noqa


make sense in performance

weijinqian0 · 2025-10-22T09:21:49Z

add fill_(0) in attenion for vllm-project/vllm#26680

vllm_ascend/torchair/torchair_mla.py

vllm_ascend/torchair/models/torchair_deepseek_v2.py

github-actions · 2025-10-23T09:27:24Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

whx-sjtu

Very hard work!

vllm_ascend/attention/sfa_v1.py

whx-sjtu

LGTM

github-actions · 2025-10-24T02:35:23Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Icey <1790571317@qq.com> Co-authored-by: Icey <1790571317@qq.com>

Signed-off-by: MengqingCao <cmq0113@163.com>

* fix torchair deepseekv2 modeling when q_lora_rank is None * add fill_(0) for attn output in dummy run * add comments for NPUWorker.device Signed-off-by: MengqingCao <cmq0113@163.com>

Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao · 2025-10-24T06:13:29Z

This pr or the updated vllm code maybe introduce some synchornize operations somewhere, which breaks aclgraph in mtp scenario. But I think this pr is more important, thus recommand to merge this firstly after ci passed. And I will fix the above issue later

…ion tests (#3729) ### What this PR does / why we need it? Enable the unit tests that #3612 skipped. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Unit tests. - vLLM main: vllm-project/vllm@17c540a Signed-off-by: gcanlin <canlinguosdu@gmail.com>

### What this PR does / why we need it? [UT] fix ut test for test_utils that #3612 skipped. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? vLLM version: v0.11.0rc3 vLLM main: vllm-project/vllm@17c540a - vLLM version: v0.11.0rc3 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: Meihan-chen <jcccx.cmh@gmail.com>

github-actions bot added module:tests module:ops module:core module:quantization labels Oct 22, 2025

MengqingCao changed the title ~~[Refactor] Refactor code to adapt with vllm main~~ [1/N][Refactor] Refactor code to adapt with vllm main Oct 22, 2025

gemini-code-assist bot reviewed Oct 22, 2025

View reviewed changes

tests/ut/kv_connector/test_mooncake_connector.py Outdated Show resolved Hide resolved

vllm_ascend/lora/punica_npu.py Show resolved Hide resolved

vllm_ascend/lora/punica_npu.py Show resolved Hide resolved

MengqingCao added ready read for review ready-for-test start test by label for PR labels Oct 22, 2025

github-actions bot added the merge-conflicts label Oct 22, 2025

MengqingCao force-pushed the refactor_ds_1022 branch from 5a6a209 to d606f30 Compare October 22, 2025 06:42

github-actions bot removed the merge-conflicts label Oct 22, 2025

MengqingCao commented Oct 22, 2025

View reviewed changes

vllm_ascend/worker/worker_v1.py Show resolved Hide resolved

MengqingCao commented Oct 22, 2025

View reviewed changes

vllm_ascend/platform.py Show resolved Hide resolved

MengqingCao commented Oct 22, 2025

View reviewed changes

whx-sjtu reviewed Oct 23, 2025

View reviewed changes

vllm_ascend/torchair/torchair_mla.py Outdated Show resolved Hide resolved

vllm_ascend/torchair/models/torchair_deepseek_v2.py Outdated Show resolved Hide resolved

github-actions bot added the merge-conflicts label Oct 23, 2025

MengqingCao force-pushed the refactor_ds_1022 branch from e7bc4b6 to f23817d Compare October 23, 2025 12:05

github-actions bot removed the merge-conflicts label Oct 23, 2025

whx-sjtu suggested changes Oct 24, 2025

View reviewed changes

vllm_ascend/attention/sfa_v1.py Show resolved Hide resolved

vllm_ascend/attention/sfa_v1.py Show resolved Hide resolved

whx-sjtu approved these changes Oct 24, 2025

View reviewed changes

github-actions bot added the merge-conflicts label Oct 24, 2025

MengqingCao and others added 2 commits October 24, 2025 02:59

[Refactor][DS32] Refactor code to adapt with vllm main

b1b99b4

Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Icey <1790571317@qq.com> Co-authored-by: Icey <1790571317@qq.com>

fix kvcache and remove useless skip on ut

303bc2c

Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao added 16 commits October 24, 2025 02:59

fix torchair deepseek

4fda979

Signed-off-by: MengqingCao <cmq0113@163.com>

fix lint

b8d805b

Signed-off-by: MengqingCao <cmq0113@163.com>

fix SchedulerDynamicBatch

3b74c7f

Signed-off-by: MengqingCao <cmq0113@163.com>

fix new scheduler

43859a8

Signed-off-by: MengqingCao <cmq0113@163.com>

update sfa by replace q_a_proj by fused_qkv_a_proj

c4143ba

Signed-off-by: MengqingCao <cmq0113@163.com>

fix torchair mla/sfa by replacing q_a_proj by fused_qkv_a_proj

9b7af2a

Signed-off-by: MengqingCao <cmq0113@163.com>

tiny fix

3a563ce

Signed-off-by: MengqingCao <cmq0113@163.com>

fix sfa

233a985

Signed-off-by: MengqingCao <cmq0113@163.com>

fix sfa eager

fedd3bd

Signed-off-by: MengqingCao <cmq0113@163.com>

Many fixes

6cc3c33

* fix torchair deepseekv2 modeling when q_lora_rank is None * add fill_(0) for attn output in dummy run * add comments for NPUWorker.device Signed-off-by: MengqingCao <cmq0113@163.com>

fix ut

1523557

Signed-off-by: MengqingCao <cmq0113@163.com>

revert torchair refactor

b133df2

Signed-off-by: MengqingCao <cmq0113@163.com>

revert torchair sfa changes

4cb8d85

Signed-off-by: MengqingCao <cmq0113@163.com>

fix mm and skip tests

28ab290

Signed-off-by: MengqingCao <cmq0113@163.com>

fix kv_no_split contiguous

317726f

Signed-off-by: MengqingCao <cmq0113@163.com>

fixes on mm && e2e

05599bc

Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao force-pushed the refactor_ds_1022 branch from 6512d4b to 05599bc Compare October 24, 2025 05:35

github-actions bot removed the merge-conflicts label Oct 24, 2025

MengqingCao added 2 commits October 24, 2025 05:53

skip ut

e9ff866

Signed-off-by: MengqingCao <cmq0113@163.com>

skip ut

713c723

Signed-off-by: MengqingCao <cmq0113@163.com>

wangxiyuan approved these changes Oct 24, 2025

View reviewed changes

wangxiyuan merged commit cea0755 into vllm-project:main Oct 24, 2025
28 checks passed

This was referenced Oct 24, 2025

[Refactor][main CI] Refactor code to align with vllm main #3504

Closed

Refactor ds 1020 rebase main new #3558

Closed

[DeepSeek] DeepSeek adapt to newer vLLM main #3592

Closed

gcanlin mentioned this pull request Oct 25, 2025

[UT][fix] Add missing get_ascend_config mock to NPUWorker initialization tests #3729

Merged

Meihan-chen mentioned this pull request Oct 27, 2025

[UT] fix skip ut test for test_utils #3803

Merged

[1/N][Refactor] Refactor code to adapt with vllm main #3612

[1/N][Refactor] Refactor code to adapt with vllm main #3612

Uh oh!

Conversation

MengqingCao commented Oct 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

Uh oh!

Uh oh!

MengqingCao Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

weijinqian0 commented Oct 22, 2025

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

whx-sjtu left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

whx-sjtu left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 24, 2025

Uh oh!

MengqingCao commented Oct 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

MengqingCao commented Oct 22, 2025 •

edited by github-actions bot

Loading

whx-sjtu left a comment •

edited

Loading