[RFC]: Disallow extra vocab for LoRA

[WoosukKwon]

Motivation.

Currently, vLLM allows each LoRA adapter to define its own additional vocabulary:

vllm/vllm/config/__init__.py

Lines 2456 to 2460 in 65197a5

	lora_extra_vocab_size: int = 256
	"""Maximum size of extra vocabulary that can be present in a LoRA adapter
	(added to the base model vocabulary)."""
	lora_vocab_padding_size: ClassVar[int] = current_platform\
	.get_lora_vocab_padding_size()

However, this introduces significant complexity because:

1. We can no longer assume a single tokenizer per model (since each LoRA adapter can have its own tokenizer).
2. The size of the unembedding layer becomes ambiguous.

Proposed Change.

Since this feature appears to be rarely used, I propose removing it. Going forward, vLLM will assume that all LoRA adapters for a given model share the same vocabulary.

Feedback Period.

1 week

CC List.

@jeejeelee

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[cjackal]

+1 on discontinuing extra vocab.

What's worse (on UX wise, not DX wise) about status quo of LoRA extra vocab feature is that --lora-extra-vocab-size CLI argument does not accept 0, c.f. #15036. It gives an impression that extra vocab is a must-have for LoRA adapters (which goes against "conventions"). If extra vocab feature is seldom-used, it would be better to deprecate the --lora-extra-vocab-size argument to avoid this kind of possible misunderstanding.

[jeejeelee]

Completely agree

[ayushsatyam146]

If moving ahead with this issue is finalised, can I work on it? If yes can someone please assign it to me?
@jeejeelee @WoosukKwon

[WoosukKwon]

@ayushsatyam146 In principle, we should wait for 1 week to get feeback before actually deleting it. However, please feel free to work in advance. If there's no blocker, we will merge your PR next week.

[ayushsatyam146]

Makes sense! thanks for clarification @WoosukKwon. I will start working on this.

[jeejeelee]

@ayushsatyam146 There's already a PR in progress, see #23540

[njhill]

This also sounds good to me but should we give a bit more time to confirm that it really doesn't have much usage?

We should at least ask/announce in relevant slack channels, community call, etc. to try to find folks. We could also put in release notes of our next release, that we're considering removal in the following release.

[cjackal]

This also sounds good to me but should we give a bit more time to confirm that it really doesn't have much usage?

We should at least ask/announce in relevant slack channels, community call, etc. to try to find folks. We could also put in release notes of our next release, that we're considering removal in the following release.

AFAIK most(every?) LoRA training framework does not even touch the embedding layer by default, and unlike other deprecated features (e.g. per-request guided decoding) whose usage are hard to collect (and thus we may hear longer and actively), LoRA extra vocab is (and must be) inscribed in the LoRA adapter, so I guess it would suffice to make stats over huggingface-uploaded adapters like base_model:gpt-oss, base_model:llama-3?

I just checked it for ~20 just for fun, haven't find one with embedding layers included in adapter_models.safetensors yet.

[jeejeelee]

This also sounds good to me but should we give a bit more time to confirm that it really doesn't have much usage?

We should at least ask/announce in relevant slack channels, community call, etc. to try to find folks. We could also put in release notes of our next release, that we're considering removal in the following release.

I think we can use #23635 - first remind users that it's already deprecated, see what user feedback we get, and if there's no user feedback later on, maybe we can consider removing it