feat(ck-tile): enable multi-D GEMM bridge on gfx1250 (MI400) - #10915
Closed
ozturkosu wants to merge 1 commit into
Closed
feat(ck-tile): enable multi-D GEMM bridge on gfx1250 (MI400)#10915ozturkosu wants to merge 1 commit into
ozturkosu wants to merge 1 commit into
Conversation
The merged multi-D GEMM bridge (#9308) ships only an MFMA CI config (warp_tile 32x32x16), which is valid on gfx942/gfx950 but reports as unsupported (status -2/-1) on gfx1250. gfx1250 has no MFMA units; its RDNA4 WMMA fp16/bf16 warp tile is 16x16x32. Add a gfx1250 multi_d CI config that pins the WMMA warp tile (16x16x32) while keeping the multi_d_config D-fusion block, extend the multi_d bridge unit test to assert the gfx1250 config uses WMMA (not MFMA) and retains the multi_d block, and add gfx1250 to the driver's --arch help. Depends on #10904 (shared gfx1250 dispatcher enablement: _SUPPORTED_ARCHES, wave-combo fallbacks, arch_specs_generated.py gfx1250 warp tiles). Validated on an MI400 (gfx1250, ROCm 10.1) node: built the dispatcher static lib for gfx1250 and ran the multi_d sweep with the new CI config (fp16, rcrr, --verify) across all 4 default problems (1024^3, 2048^3, 4096^3, 512x512x8192). 80 measurements verify against the fp32 numpy reference (max_rel ~5e-4), spanning compv3/compv4/mem pipelines, both MultiDAdd/MultiDMultiply ops, and both D-counts, at 26-69 TFLOPs (median 31). Remaining status -1 kernels are unsupported trait combos gracefully surfaced by the bridge, not correctness failures.
Contributor
Author
gfx1250 (MI400) GEMM bridge enablement — status across the stacked PR series
All enablement PRs are stacked on #10904 (shared gfx1250 arch enablement), Python/JSON-only, each with a CPU test. Validated on MI400 / gfx1250, ROCm 10.1. Remaining C++ follow-ups: stream-K cross-workgroup coherency handshake (CDNA atomics → RDNA4/wave32); mx gfx1250 scale-preshuffle (ck_tile already ships |
This was referenced Aug 18, 2026
Contributor
Author
|
Superseded by #10921, which combines the shared gfx1250 enablement (gemm_universal) with the grouped, multi-D, and multi-ABD variant configs+tests into a single PR against develop. Branch retained (not deleted) as the base of the remaining partial PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables the merged multi-D GEMM bridge (#9308) on gfx1250 (MI400).
Stacked on #10904 (
users/muozturk/ck/gemm-universal-gfx1250-enable), which supplies the shared gfx1250 dispatcher enablement. This PR adds only the multi_d-specific piece on top.Does the merged multi_d bridge support gfx1250? No (as merged).
Two reasons the merged #9308 does not run on gfx1250:
gemm_utils._SUPPORTED_ARCHES, the wave-combo fallback tables, andarch_specs_generated.pydid not include gfx1250. This is fixed by the base PR feat(ck-tile): enable gemm_universal GEMM bridge on gfx1250 (MI400) #10904.gemm_multi_d/configs/default_ci_config.json) pinswarp_tile = 32x32x16, an MFMA shape. gfx1250 has no MFMA units; it runs the RDNA4 WMMA path whose fp16/bf16 warp tile is 16x16x32. On gfx1250 the MFMA kernels report as unsupported (status -2/-1) and never run.Changes
gemm_multi_d/configs/default_ci_config_gfx1250.json(new): mirrors the multi_d CI config but pins the WMMA warp tile (16x16x32) and keeps themulti_d_configD-fusion block (elementwise_ops x num_d_tensors).dispatcher/tests/test_multi_d_bridge.py: newTestMultiDGfx1250Configasserting the gfx1250 config exists, uses WMMA (16x16x32, not MFMA 32x32x16), and retains the multi_d block. The pre-existingtest_configs_are_valid_sweepsalso now covers the new config.gemm_multi_d_full_benchmark.py: add gfx1250 to the--archhelp text.No C/C++ changes, so no clang-format needed. The merged multi_d bridge is fp16-only by design, so this enablement targets fp16.
MI400 validation (gfx1250, ROCm 10.1)
Built the dispatcher static lib for
-DGPU_TARGETS=gfx1250and ran:max_rel ~5e-4), evenly across all 4 default problems (1024^3, 2048^3, 4096^3, 512x512x8192) — 20 verified rows per problem.compv3/compv4/mempipelines, bothMultiDAdd/MultiDMultiply, and bothnum_d_tensors1/2.status -1kernels are unsupported trait combos gracefully surfaced by the bridge (not correctness failures); every problem still has passing kernels.Notes
arch_specs_generated.pyis auto-generated fromarch_specs.json, which has no gfx1250 entry. Per the precedent in feat(ck-tile): enable gemm_universal GEMM bridge on gfx1250 (MI400) #10904, the generated file is hand-edited for gfx1250 until the source spec is updated.Test plan
pytest dispatcher/tests/test_multi_d_bridge.py(13 passed)--verify: 80/80 measured kernels correct