Skip to content

[Bugfix] Fix the lm_head in gpt_bigcode in lora mode #6357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 61 commits into from
May 26, 2025
Merged
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
ca4efbb
Fix the lm_head in gptbigcode in lora mode
maxdebayser Jul 12, 2024
5d2cd9e
Enforce no quantization for lm_head
maxdebayser Jul 12, 2024
c5769a5
intervene in fight between yapf and isort
maxdebayser Jul 12, 2024
2d172e0
address review comments
maxdebayser Jul 13, 2024
b04d65b
address review comments
maxdebayser Jul 13, 2024
bc0bfcb
address review comments
maxdebayser Jul 13, 2024
73434eb
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 13, 2024
0153ee9
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 15, 2024
ed6d951
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 16, 2024
04abf1b
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 17, 2024
07aa7e4
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 17, 2024
daf566f
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 18, 2024
462deea
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 23, 2024
2f36f3b
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 24, 2024
3fb641b
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 24, 2024
c690724
fix superclass method signature change
maxdebayser Jul 24, 2024
4b5dbd5
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 25, 2024
1a8d282
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 29, 2024
685bd96
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 30, 2024
2d1cc82
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 30, 2024
5c1fe68
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Jul 31, 2024
31b6e93
Merge remote-tracking branch 'remotes/upstream/main' into gpt_bigcode…
maxdebayser Aug 1, 2024
5ac5547
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 2, 2024
b05e5c4
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 5, 2024
d11655a
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 6, 2024
7450a1d
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 7, 2024
5f6ac9e
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 10, 2024
b87657d
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 12, 2024
b6867de
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 15, 2024
b4abc03
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser Aug 20, 2024
ef9f79d
fix merge mistake
maxdebayser Aug 20, 2024
4700613
fix merge mistake
maxdebayser Aug 20, 2024
3f4037e
fix merge mistake
maxdebayser Aug 20, 2024
c809ed4
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 26, 2024
33dd909
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Aug 27, 2024
c89dcf0
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser Sep 2, 2024
e31b2b6
add default prefix to get_quant_method
maxdebayser Sep 2, 2024
ea5ea7d
skip loading of lm_head only if tie_word_embeddings is True
maxdebayser Sep 2, 2024
10246be
Make weight tie work with quantization
maxdebayser Sep 2, 2024
9cef81e
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Sep 4, 2024
98a0269
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Sep 10, 2024
fb3eef0
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Sep 24, 2024
6028806
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Sep 26, 2024
db57383
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Oct 8, 2024
77e1966
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Oct 14, 2024
0b81edf
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Oct 15, 2024
b7558fa
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser Nov 12, 2024
7d6f1e9
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Nov 14, 2024
e43d46b
Merge branch 'main' into gpt_bigcode_lora
maxdebayser Feb 28, 2025
08f3360
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Mar 3, 2025
fad78ab
address review comments
maxdebayser Mar 3, 2025
1634ef0
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Mar 13, 2025
86c195e
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Mar 17, 2025
994ec18
Merge branch 'vllm-project:main' into gpt_bigcode_lora
maxdebayser Mar 21, 2025
a6f0308
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser Apr 23, 2025
8ee4962
Simplify the fix
maxdebayser Apr 23, 2025
1395572
Revert other changes
maxdebayser Apr 23, 2025
1d9b323
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser Apr 24, 2025
311a97c
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser May 13, 2025
ac01fee
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser May 15, 2025
e28e496
Merge branch 'upstream_main' into gpt_bigcode_lora
maxdebayser May 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions vllm/model_executor/models/gpt_bigcode.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,12 +272,6 @@ def load_weights(self, weights: Iterable[tuple[str,
class GPTBigCodeForCausalLM(nn.Module, SupportsLoRA, SupportsPP):
packed_modules_mapping = {"c_attn": ["c_attn"]}

# LoRA specific attributes
embedding_modules = {
"wte": "input_embeddings",
"lm_head": "output_embeddings",
}

def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):
super().__init__()
config = vllm_config.model_config.hf_config
Expand Down Expand Up @@ -330,8 +324,11 @@ def compute_logits(

def load_weights(self, weights: Iterable[tuple[str,
torch.Tensor]]) -> set[str]:
skip_prefixes = None
if self.config.tie_word_embeddings:
skip_prefixes = ["lm_head."]
loader = AutoWeightsLoader(
self,
skip_prefixes=(["lm_head."]),
skip_prefixes=skip_prefixes,
)
return loader.load_weights(weights)
return loader.load_weights(weights)