Fix tensor device and dtype placement in Qwen2VL model #26219
Conversation
Summary: Signed-off-by: yuanfengli <yuanfengli@meta.com> Fix device placement issue in Qwen2-VL model by explicitly specifying device and dtype when creating `grid_thw_` tensor to prevent device mismatch errors during GPU inference. Test Plan: Tested with Qwen2-VL model inference on GPU to verify the tensor device placement issue is resolved. Refer to D82781653 Differential Revision: D83903654
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request correctly addresses a device placement issue in the Qwen2-VL model. By explicitly specifying the device and dtype when creating the grid_thw_ tensor, the change prevents device mismatch errors during GPU inference. The fix is well-targeted, effective, and I've confirmed that no similar issues exist elsewhere in the file.
github-actions
bot
added
the
ready
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
…26219) Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Yuanfeng Li <yuanfengli@meta.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
…26219) Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Yuanfeng Li <yuanfengli@meta.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>
…26219) Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Yuanfeng Li <yuanfengli@meta.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
…26219) Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Yuanfeng Li <yuanfengli@meta.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
…26219) Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Yuanfeng Li <yuanfengli@meta.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
…26219) Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Yuanfeng Li <yuanfengli@meta.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
…26219) Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Yuanfeng Li <yuanfengli@meta.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
3 participants
Summary:
Signed-off-by: Yuanfeng Li yuanfengli@meta.com
Fix device placement issue in Qwen2-VL model by explicitly specifying device and dtype when creating
grid_thw_tensor to prevent device mismatch errors during GPU inference.Test Plan:
Tested with Qwen2-VL model inference on GPU to verify the tensor device placement issue is resolved.
Differential Revision: D83903654