[Bug]: vllm overrides transformer's Autoconfig for mllama

[lyuqin-scale]

Your current environment

vllm 0.6.2

Model Input Dumps

No response

🐛 Describe the bug

This line overrides transformer's autoconfig for mllama, which should be removed

vllm/vllm/transformers_utils/config.py

Line 41 in e5dc713

_CONFIG_REGISTRY_OVERRIDE_HF: Dict[str, Type[PretrainedConfig]] = {

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[heheda12345]

We override it only for changing is_encoder_decoder=True, as described here. Other parts are not modified.

[lyuqin-scale]

@heheda12345 but it does change the behavior of AutoConfig if vllm is imported together

[lyuqin-scale]

comparing with

[lyuqin-scale]

due to the above, transformer's AutoModelForPreTraining.from_pretrained failed

[lyuqin-scale]

and if we don't import vLLM, it loads successfully. However, we need to import both transformers and vLLM, and use transformer to load model at runtime for some tasks

[heheda12345]

For a quick fix, you can do something like here and here

We are thinking of how to make it cleaner.

[lyuqin-scale]

We've had temporary workaround by explicitly using transformers.MllamaForConditionalGeneration to load the mllama model. But we believe a fix from vLLM side could benefit the other vLLM users who aren't aware of this override