Major issues with transformers version causing rubbish generations with Gemma3 family using vllm

[AbdelrahmanHagrass]

System Info

.

Who can help?

Major issues with transformers version when used with Gemma3 family models on vllm. The output generations are incorrect and not usable—appears to be due to incompatibility or regression in transformers. Please advise on compatible versions or fixes.

Example: Generations are rubbish or meaningless compared to expected outputs.

Tested with latest vllm and Gemma3 models.

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

.

Expected behavior

.

[zucchini-nlp]

Hey @AbdelrahmanHagrass , do you mean when using model_impl='transformers' with vLLM? I believe it is a known issue with sliding window, though ig it was fixed recently by @hmellor

[hmellor]

@AbdelrahmanHagrass could you share more information about what you're running? A minimal script or a vllm serve ... command

[AbdelrahmanHagrass]

No i use the normal vllm serve , without changing the model_impl, it gives very random outputs. These guys have the same issue ,
vllm-project/vllm#20341

[AbdelrahmanHagrass]

So i should install the dev version or is it in the latest pip installation , also should i install the latest transformers version normally or a which version should make things go smother

[hmellor]

The linked PR solves the issue mentioned in that issue