[ROCm] Enable DeepEP ROCm as all2allbackend for AMD GPUs. #34692
[ROCm] Enable DeepEP ROCm as all2allbackend for AMD GPUs. #34692lcskrishna wants to merge 3 commits intovllm-project:mainfrom
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
There was a problem hiding this comment.
Code Review
This pull request enables DeepEP on ROCm for AMD GPUs by adding platform-specific logic, particularly for handling the float8_e4m3fnuz data type and adjusting API arguments. The changes are consistent across the modified files. My main feedback is to refactor a piece of duplicated code in all2all.py to improve maintainability.
|
cc: @itej89 @amathews-amd |
This PR is to integrate changes required to run DeepEP as an all2allbackend on AMD GPUs.
co-authored by : @itej89
The following changes are performed:
Next steps:
Purpose
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.