Remove LoRA bias support #25807

ashwin-phadke · 2025-09-27T17:03:55Z

#23892 : Removes support for LoRA bias completely from vLLM. LoRA bias was added in #5733.

Looks like layers.py has been kind of replaced with the layers directory, this needs some review.

FIX #23892

Purpose

Test Plan

Update all tests that reference LoRA bias and re run them

Test Result

Essential Elements of an Effective PR Description Checklist

In progress:

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com>

jeejeelee · 2025-09-29T00:18:35Z

vllm/config/__init__.py

@@ -60,17 +60,97 @@



Don't modify unrelated code

This seems like a bad merge which entirely reverts all the configu modularisation work

Yes, I believe it is a bad merge too, working on it.

mergify · 2025-09-30T16:15:07Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ashwin-phadke.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com>

ashwin-phadke · 2025-09-30T16:29:29Z

This should solve it.

ashwin-phadke · 2025-10-06T16:20:19Z

@jeejeelee @hmellor any feedback on this one?
I noticed my comments on section x removed made it through, so I will remove those, other than that do you guys see any issues that must be addressed? anything missing or done incorrectly?
Let me know.

jeejeelee · 2025-10-07T13:54:06Z

Very sorry for the delayed feedback due to the holiday. Can you resolve these conflicts?

hmellor · 2025-10-07T14:00:48Z

See https://vllm-dev.slack.com/archives/C07R5Q1Q2BB/p1759663228844749 for detailed instructions on resolving the conflicts

mergify · 2025-10-09T09:27:18Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ashwin-phadke.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

jeejeelee

@ashwin-phadke Thank you for contribution.

jeejeelee · 2025-10-10T02:37:07Z

@Yikun @xuechendi We are cleaning up lora bias related code (see: #23892), which may lead to the plugin LoRA failure

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com> Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: Dhruvil Bhatt <bhattdbh@amazon.com>

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com> Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: bbartels <benjamin@bartels.dev>

Signed-off-by: MengqingCao <cmq0113@163.com>

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Icey <1790571317@qq.com>

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com> Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com> Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

### What this PR does / why we need it? This is the step 1 of refactoring code to adapt with vllm main, and this pr aligned with vllm-project/vllm@17c540a 1. refactor deepseek to the latest code arch as of vllm-project/vllm@17c540a 2. bunches of fixes due to vllm changes - Fix `AscendScheduler` `__post_init__`, caused by vllm-project/vllm#25075 - Fix `AscendScheduler` init got an unexpected arg `block_size`, caused by vllm-project/vllm#26296 - Fix `KVCacheManager` `get_num_common_prefix_blocks` arg, caused by vllm-project/vllm#23485 - Fix `MLAAttention` import,caused by vllm-project/vllm#25103 - Fix `SharedFusedMoE` import, caused by vllm-project/vllm#26145 - Fix `LazyLoader` improt, caused by vllm-project/vllm#27022 - Fix `vllm.utils.swap_dict_values` improt, caused by vllm-project/vllm#26990 - Fix `Backend` enum import, caused by vllm-project/vllm#25893 - Fix `CompilationLevel` renaming to `CompilationMode` issue introduced by vllm-project/vllm#26355 - Fix fused_moe ops, caused by vllm-project/vllm#24097 - Fix bert model because of `inputs_embeds`, caused by vllm-project/vllm#25922 - Fix MRope because of `get_input_positions_tensor` to `get_mrope_input_positions`, caused by vllm-project/vllm#24172 - Fix `splitting_ops` changes introduced by vllm-project/vllm#25845 - Fix multi-modality changes introduced by vllm-project/vllm#16229 - Fix lora bias dropping issue introduced by vllm-project/vllm#25807 - Fix structured ouput break introduced by vllm-project/vllm#26737 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? CI passed with existing test. - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Icey <1790571317@qq.com> Co-authored-by: Icey <1790571317@qq.com>

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com> Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com> Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>

Remove LoRA bias support

d1ec886

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com>

mergify bot added the tpu Related to Google TPUs label Sep 27, 2025

ashwin-phadke mentioned this pull request Sep 27, 2025

[RFC]: Remove LoRA bias #23892

Closed

3 tasks

ashwin-phadke added 2 commits September 28, 2025 18:29

Remove LoRA bias from layers

1b8a9de

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com>

Changes to function calls

8ace17f

Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com>

ashwin-phadke marked this pull request as ready for review September 28, 2025 13:58

ashwin-phadke requested review from ProExpertProg, WoosukKwon, hmellor, houseroad, jeejeelee, mgoin, robertgshaw2-redhat, simon-mo, tlrmchlsmth, yewentao256 and youkaichao as code owners September 28, 2025 13:58

jeejeelee reviewed Sep 29, 2025

View reviewed changes

mergify bot added the needs-rebase label Sep 30, 2025

Merge branch 'main' into remove-lora-bias-support

69f22ba

Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com>

mergify bot removed the needs-rebase label Sep 30, 2025

hmellor requested a review from jeejeelee September 30, 2025 17:05

mergify bot added the needs-rebase label Oct 9, 2025

Fix conflict

7e3dc4a

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

jeejeelee requested review from DarkLight1337, NickLucche and aarnphm as code owners October 10, 2025 02:32

jeejeelee approved these changes Oct 10, 2025

View reviewed changes

jeejeelee added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 10, 2025

jeejeelee added 2 commits October 10, 2025 08:00

Fix dummy weights

a8919d3

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Merge branch 'main' into remove-lora-bias-support

17b8710

jeejeelee enabled auto-merge (squash) October 10, 2025 09:20

jeejeelee merged commit ab196ed into vllm-project:main Oct 10, 2025
55 checks passed

This was referenced Oct 13, 2025

Fix lora CI tests due to an vLLM change. vllm-project/tpu-inference#839

Merged

Fix lora tests failure in TPU CI due to the removal of LoRA bias #26723

Merged

MengqingCao added a commit to MengqingCao/vllm-ascend-fork that referenced this pull request Oct 17, 2025

fix lora vllm-project/vllm#25807

f129db5

Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao pushed a commit to MengqingCao/vllm-ascend-fork that referenced this pull request Oct 18, 2025

Many fixes

3478f9d

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao pushed a commit to MengqingCao/vllm-ascend-fork that referenced this pull request Oct 19, 2025

Many fixes

44eb471

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao pushed a commit to MengqingCao/vllm-ascend-fork that referenced this pull request Oct 20, 2025

Many fixes

31a4b67

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

wxsIcey added a commit to wxsIcey/vllm-ascend that referenced this pull request Oct 20, 2025

Many fixes

7d78442

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao pushed a commit to MengqingCao/vllm-ascend-fork that referenced this pull request Oct 20, 2025

Many fixes

f31f5b0

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao pushed a commit to MengqingCao/vllm-ascend-fork that referenced this pull request Oct 20, 2025

Many fixes

81f6b58

* fix bert model * fix guided decoding * revert skipped e2e test * fix lora vllm-project/vllm#25807 * fix vl Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao mentioned this pull request Oct 22, 2025

[1/N][Refactor] Refactor code to adapt with vllm main vllm-project/vllm-ascend#3612

Merged

Uh oh!

Remove LoRA bias support #25807

Remove LoRA bias support #25807

Uh oh!

Conversation

ashwin-phadke commented Sep 27, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

jeejeelee Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

hmellor Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

ashwin-phadke Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

mergify bot commented Sep 30, 2025

Uh oh!

ashwin-phadke commented Sep 30, 2025

Uh oh!

ashwin-phadke commented Oct 6, 2025

Uh oh!

jeejeelee commented Oct 7, 2025

Uh oh!

hmellor commented Oct 7, 2025

Uh oh!

mergify bot commented Oct 9, 2025

Uh oh!

jeejeelee left a comment

Choose a reason for hiding this comment

Uh oh!

jeejeelee commented Oct 10, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ashwin-phadke commented Sep 27, 2025 •

edited by github-actions bot

Loading