Optimum amd support #464
Optimum amd support #464
Conversation
Rocm Onnxruntime
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
Added comprehensive AMD GPU support to Infinity, enabling model deployment and optimization on AMD MI200/MI300 series GPUs with architecture-specific configurations.
- Added AMD-specific Docker configurations in
/libs/infinity_emb/Docker.template.yamlwith separate build paths for gfx90a/gfx942 (source build) and gfx1100 (pre-built wheel) - Integrated MIGraphX and ROCm execution providers in
/libs/infinity_emb/infinity_emb/transformer/utils_optimum.pyfor AMD GPU optimization - Added AMD deployment documentation in
/docs/docs/deploy.mdwith specific run commands and flags for AMD GPUs - Disabled BetterTransformer for AMD platforms due to compatibility issues
- Added CHECK_OPTIMUM_AMD optional import for AMD-specific optimizations
6 file(s) reviewed, 10 comment(s)
Edit PR Review Bot Settings | Greptile
| # GPU architecture specific installations | ||
| RUN cd /opt/rocm/share/amd_smi && python -m pip wheel . --wheel-dir=/install | ||
| RUN apt update -y && apt install migraphx -y | ||
| RUN if [ "$GPU_ARCH" = "gfx90a" ] || [ "$GPU_ARCH" = "gfx942" ]; then \ |
There was a problem hiding this comment.
logic: Consider adding error handling and logging for failed installations. The script continues silently if any of the installation steps fail.
| elif "MIGraphXExecutionProvider" in available: | ||
| return "MIGraphXExecutionProvider" # swapped order of ROCM and MIGraphX | ||
| elif "ROCMExecutionProvider" in available: |
There was a problem hiding this comment.
logic: MIGraphX is prioritized over ROCM here but reversed in the CUDA section above. Consider using consistent ordering.
| if files_optimized: | ||
| file_optimized = files_optimized[0] | ||
| file_optimized = files_optimized[-1] |
There was a problem hiding this comment.
style: Using files_optimized[-1] could be unstable if multiple optimized versions exist. Consider using version sorting or timestamps.
|
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #464 +/- ##
==========================================
- Coverage 79.04% 78.92% -0.13%
==========================================
Files 42 42
Lines 3408 3417 +9
==========================================
+ Hits 2694 2697 +3
- Misses 714 720 +6 ☔ View full report in Codecov by Sentry. |
Related Issue
Checklist
Additional Notes
Add any other context about the PR here.