[Bugfix] Fix broken ViT attention selection for Blackwell device #30731
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request addresses a crash in ViT attention backend selection by replacing an assertion with a fallback mechanism. When an unsupported backend is specified, it now logs an error and proceeds with automatic selection instead of crashing. My review focuses on refining this new logic to prevent incorrect error logging when no backend is specified (a valid use case for auto-selection) and adjusting the log severity to better reflect the recoverable nature of the situation.
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
| selected_backend = get_current_vllm_config().attention_config.backend | ||
| if attn_backend is None: | ||
| attn_backend = selected_backend |
There was a problem hiding this comment.
After second thought, I think we have better decouple ViT attention backend and backbone attention backend config.
…m-project#30731) Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
) Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> (cherry picked from commit e94384b)
…m-project#30731) Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Signed-off-by: Nathan Price <nathan@abridge.com>
3 participants
Purpose
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.