feat(hipdnn): add cuDNN frontend compatibility shim with SDPA - #9361
Merged
Conversation
…/hipdnn-cudnn-graph
…/hipdnn-cudnn-graph
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
mousdahl-amd
commented
Jul 13, 2026
mousdahl-amd
commented
Jul 13, 2026
Codecov Report❌ Patch coverage is ❌ Your project status has failed because the head coverage (76.84%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #9361 +/- ##
===========================================
+ Coverage 64.16% 64.77% +0.61%
===========================================
Files 2757 2689 -68
Lines 449555 423281 -26274
Branches 66143 62923 -3220
===========================================
- Hits 288439 274175 -14264
+ Misses 139967 128429 -11538
+ Partials 21149 20677 -472
*This pull request uses carry forward flags. Click here to find out more.
🚀 New features to boost your workflow:
|
Move the cuDNN SDPA attribute surface off the standalone shim type and onto the real hipDNN SdpaAttributes/SdpaBackwardAttributes via a first-wins "unsupported usage" latch on the base Attributes. Shim setters with no hipDNN equivalent now record a reason instead of failing at the call site; the owning node surfaces it at validate(). Shrinks sdpa_attributes.h substantially and adds forward/backward attribute, node, and integration test coverage.
mousdahl-amd
commented
Jul 15, 2026
mousdahl-amd
commented
Jul 15, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an opt-in, header-only cuDNN-frontend compatibility layer to hipDNN (including SDPA) so hipified cuDNN-frontend consumers can build against cuDNN-spelled APIs while lowering onto hipDNN’s native frontend. Alongside the shim, the PR refactors native SDPA attributes/nodes to record unsupported “source-compat” requests and surface them during node/graph validation, and adds host-side tests plus a round-trip sample.
Changes:
- Introduces
hipdnn_compatibility/cudnnshim headers (graph wrapper, error recorder, SDPA aliases, version signal) and wires them into the umbrella include. - Refactors native SDPA attributes/nodes to support cuDNN-spelled setters and deferred “unsupported usage” reporting at
validate()/pre_validate_node(). - Adds shim-focused gtests, a reusable mock-backend fixture, and a host-only SDPA shim round-trip sample gated behind
HIPDNN_ENABLE_CUDNN_COMPATIBILITY && HIPDNN_ENABLE_SDPA.
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| projects/hipdnn/tests/frontend/IntegrationSdpaFwdDescriptorLowering.cpp | Updates SDPA scale setter name to the new set_attn_scale API. |
| projects/hipdnn/tests/frontend/IntegrationSdpaFwdDescriptorLifting.cpp | Updates SDPA scale setter name to the new set_attn_scale API. |
| projects/hipdnn/tests/frontend/IntegrationSdpaBwdLifting.cpp | Updates SDPA scale setter name to the new set_attn_scale API. |
| projects/hipdnn/tests/frontend/IntegrationSdpaBwdDescriptorLowering.cpp | Updates SDPA scale setter name to the new set_attn_scale API. |
| projects/hipdnn/samples/sdpa/SdpaFprop.cpp | Updates SDPA sample to use set_attn_scale. |
| projects/hipdnn/samples/cudnn_shim/SdpaRoundTrip.cpp | Adds host-only shim SDPA “round-trip” validation sample. |
| projects/hipdnn/samples/CMakeLists.txt | Builds/registers the shim SDPA sample as a CTest when both shim+SDPA are enabled. |
| projects/hipdnn/frontend/tests/TestSdpaFwdNode.cpp | Adds tests ensuring unsupported shim setters fail early in pre_validate_node(). |
| projects/hipdnn/frontend/tests/TestSdpaBwdNode.cpp | Adds tests ensuring unsupported shim setters fail early in pre_validate_node(). |
| projects/hipdnn/frontend/tests/TestSdpaBackwardAttributes.cpp | Updates tests for renamed setters and adds coverage for unsupported-usage latch behavior. |
| projects/hipdnn/frontend/tests/TestSdpaAttributes.cpp | Updates tests for renamed setters and adds coverage for unsupported-usage latch behavior. |
| projects/hipdnn/frontend/tests/TestCudnnShimGraphSDPA.cpp | Adds shim graph SDPA tests (host-only validate + mock-backed lowering). |
| projects/hipdnn/frontend/tests/TestCudnnShimGraph.cpp | Adds shim graph wrapper surface and error-recorder behavior tests. |
| projects/hipdnn/frontend/tests/TestCudnnShimError.cpp | Tightens shim error-type aliasing tests/imports. |
| projects/hipdnn/frontend/tests/fake_backend/MockBackendFixture.hpp | Adds reusable mock-backend fixture for shim lowering tests. |
| projects/hipdnn/frontend/tests/CudnnShimTestSupport.hpp | Adds shared test helpers for shim graph SDPA tests (tensor creation + canonical inputs). |
| projects/hipdnn/frontend/tests/CMakeLists.txt | Registers new shim graph tests under the appropriate build flags. |
| projects/hipdnn/frontend/include/hipdnn_frontend/node/SdpaFwdNode.hpp | Surfaces unsupported-usage latch as an early validation error. |
| projects/hipdnn/frontend/include/hipdnn_frontend/node/SdpaBwdNode.hpp | Surfaces unsupported-usage latch as an early validation error. |
| projects/hipdnn/frontend/include/hipdnn_frontend/Graph.hpp | Warns and ignores unfuse_fma hint; adjusts override-shape API gating. |
| projects/hipdnn/frontend/include/hipdnn_frontend/detail/SdpaFwdUnpacker.hpp | Updates unpacker to use renamed SDPA setters. |
| projects/hipdnn/frontend/include/hipdnn_frontend/attributes/SdpaBackwardAttributes.hpp | Adds cuDNN-compat setters + unsupported-usage recording; renames/overloads attn-scale setter. |
| projects/hipdnn/frontend/include/hipdnn_frontend/attributes/SdpaAttributes.hpp | Adds cuDNN-compat setters + unsupported-usage recording; renames/overloads multiple SDPA setters. |
| projects/hipdnn/frontend/include/hipdnn_frontend/attributes/Attributes.hpp | Adds first-wins “unsupported usage” latch to attribute base type. |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/detail/graph_wrapper.h | Implements cuDNN-shaped graph::Graph wrapper with staged lifecycle + deferred error handling. |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/detail/error_recorder.h | Adds reusable first-error-wins deferred error recorder used by the shim graph. |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/cudnn_frontend/sdpa_attributes.h | Provides cuDNN-shaped SDPA aliases to native SDPA attributes (when SDPA enabled). |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/cudnn_frontend/graph_properties.h | Adds cuDNN-expected type aliases in shim graph namespace. |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/cudnn_frontend/graph_interface.h | Adds shim public graph interface include wrapper. |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/cudnn_frontend.h | Extends shim umbrella to include graph interface. |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/cudnn_frontend_version.h | Adds HIPDNN_COMPATIBILITY_CUDNN_FRONTEND macro as a shim-presence signal. |
| projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/cudnn_frontend_utils.h | Adds placeholder KernelCache/DeviceProperties types for source compatibility. |
| projects/hipdnn/CMakeLists.txt | Clarifies the shim option description/comments. |
| dnn-providers/integration-tests/src/integration_tests/sdpa/IntegrationGpuSdpaFwdInference.cpp | Updates integration test to use set_attn_scale. |
| dnn-providers/hip-kernel-provider/src/integration_tests/asm_sdpa_engine/IntegrationGpuSdpaBackward.cpp | Updates integration test to use set_attn_scale. |
Comments suppressed due to low confidence (1)
projects/hipdnn/frontend/include/hipdnn_frontend/attributes/SdpaAttributes.hpp:632
- This renames/removes the previously-available set_attn_scale_value(float) API. Even though call sites in-tree were updated, this is a source-breaking change for any downstream users of the hipDNN SDPA frontend API. Consider keeping set_attn_scale_value(float) as a deprecated forwarding alias to the new cuDNN-spelled set_attn_scale(float) to preserve backwards compatibility.
// cuDNN spells the scalar attention-scale overload set_attn_scale(float);
// it joins the shared_ptr set_attn_scale overloads above.
// NOLINTNEXTLINE(readability-identifier-naming)
SdpaAttributes& set_attn_scale(float value)
{
attn_scale_value = value;
return *this;
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
BrianHarrisonAMD
approved these changes
Jul 20, 2026
BrianHarrisonAMD
left a comment
BrianHarrisonAMD
left a comment
Contributor
There was a problem hiding this comment.
LGTM!
I think this is the ideal form.
For other nodes, if we need the same, then I think we should just extend to add the parity support instead of changing existing, but for SDPA (behind flag still) I think we can adjust to land with the proper names to begin with.
SamuelReeder
approved these changes
Jul 20, 2026
SamuelReeder
left a comment
SamuelReeder
left a comment
Contributor
There was a problem hiding this comment.
LGTM overall! It's worth addressing the deserialization comment before merging.
mousdahl-amd
enabled auto-merge (squash)
July 21, 2026 18:36
SamuelReeder
added a commit
that referenced
this pull request
Jul 22, 2026
…ests set_attn_scale_value(float) was renamed to an overload of set_attn_scale on develop (#9361, cuDNN compat shim), which this branch hasn't picked up yet. Setting the public attn_scale_value field directly compiles against both the old and new setter API.
3 tasks
mousdahl-amd
added a commit
that referenced
this pull request
Aug 7, 2026
## Summary JIRA ID : ALMIOPEN-2041 Feature Flag Protected: Yes - HIPDNN_ENABLE_CUDNN_COMPATIBILITY Completes the cuDNN-frontend graph node surface on the compatibility shim added in #9361, so hipified cuDNN v9 consumers can build the full set of `graph::Graph::*` node methods against hipDNN. Tier-1 nodes with a real hipDNN engine forward to the wrapped frontend graph; the remaining cuDNN v9 nodes are fail-stubs that record `GRAPH_NOT_SUPPORTED` (surfaced at the next `validate()`) while still handing back a live, graph-registered placeholder tensor — so hipified source compiles, survives the idiomatic `node(...)->set_output(true).set_uid(n)` chain, and fails loudly at `validate()` instead of dereferencing null. The shim targets cuDNN frontend **v1.24.0** (`CUDNN_FRONTEND_VERSION == 12400`, declared in `hipdnn_compatibility/cudnn/cudnn_frontend_version.h`). `detail/graph_wrapper.h` `static_assert`s on that macro, so the pin cannot move without a deliberate re-diff of every node signature against upstream `graph_interface.h` / `node_interface.h`. ## Risk Assessment Risk 3 (low-moderate). Header-only and host-only: no kernel selection, dispatch, or shipping default changes. The cuDNN-shaped surface — new public header, ~18 Tier-1 node methods, 22 fail-stub attribute classes — is behind the opt-in `HIPDNN_ENABLE_CUDNN_COMPATIBILITY` flag (default OFF). Two cuDNN spellings land **unguarded** in the native `hipdnn_frontend::graph` namespace regardless of the flag: `Layernorm_backward_attributes` and `Resample_attributes`, `typedef`s of existing types added alongside the already-unguarded `Resample_fwd_attributes`. That is intentional — cuDNN spellings are welcome in the native namespace — not a side effect of the shim. Tier-1 node methods are thin forwarders to existing, already-tested hipDNN engines; the only new runtime behavior is the fail-stub record-and-return-placeholder path, which is host-covered. Also changes the superbuild CI workflow (enables the flag in both jobs). ## ASIC Coverage ASIC-independent. Frontend source-compatibility plumbing that lowers cuDNN-spelled graphs onto the existing hipDNN frontend; changes no kernel selection, support surface, or default behavior. All tests validate host-side with no device; the four samples are host-only graph-build+validate. No multi-arch sweep required. The shim, its tests, and its samples build only with `HIPDNN_ENABLE_CUDNN_COMPATIBILITY=ON`, and this PR is what turns that on in CI: it adds `-DHIPDNN_ENABLE_CUDNN_COMPATIBILITY=ON` to both jobs in `hipdnn-superbuild-ci.yml`, both of which run `ctest --output-on-failure`. Shim coverage is therefore exercised by PR CI on gfx1151/Windows and gfx94X/Linux, not just locally. ## Testing Summary - Node-coverage gtests (`TestCudnnShimGraphNodes.cpp`): Tier-1 nodes build a well-shaped graph that `validate()`s good; Tier-2 fail-stubs record `GRAPH_NOT_SUPPORTED` with the issue-tracker message; a poison test proves a fail-stub overrides an otherwise-valid graph (first-error-wins); a compile-time check that all 39 cuDNN v9 `*_attributes` classes construct and chain `.set_name`. - Fail-stub return-value tests: a single-output stub (`reshape`) and multi-output stubs (`genstats`, `sdpa_fp8`) hand back live placeholders that survive `->set_output(true).set_uid(n)` before `validate()` reports `GRAPH_NOT_SUPPORTED` — the shape that a null return would crash on. - `sdpa_fp8_backward` overload tests: both the 18-tensor FP8 form (7 outputs) and the 17-tensor MXFP8 form (6 outputs) resolve and record, pinning arity and output count against the FE v1.24.0 signatures. - Host-only samples registered as CTests, mirroring upstream cuDNN FE sample names: ConvFprop, Matmul, Layernorm, Pointwise. ## Testing Checklist - [x] hipDNN unit tests (cuDNN shim node coverage) - built with `HIPDNN_ENABLE_CUDNN_COMPATIBILITY=ON`, ran `hipdnn-unit-check` - Status: Passed - [x] cuDNN-shim host-only samples - `ctest -R hipdnn_sample_cudnn_shim_` - Status: Passed - [x] PR CI (builds and runs the shim tests and samples with the flag ON) - GitHub PR checks - Status: Passed ## Technical Changes - Adds the Tier-1 node methods to `detail/graph_wrapper.h` (conv fprop/dgrad/wgrad, batchnorm + backward + inference, layernorm + backward, rmsnorm + backward, matmul, pointwise, reduction, resample, block-scale quantize/dequantize), each forwarding to the wrapped hipDNN graph and flipping the graph to Native mode. - Adds `detail/node_wrappers/unsupported_nodes.h`: a CRTP `UnsupportedAttributes` base plus macros that stamp Tier-2 fail-stub attribute classes and matching node methods recording `GRAPH_NOT_SUPPORTED` for the cuDNN v9 nodes with no hipDNN engine yet. Fail-stubs return graph-registered placeholder tensors (single or array, per the node's upstream return type) so the recorded error surfaces at `validate()` rather than as a null dereference at the call site. - Pins the targeted cuDNN FE version: `detail/graph_wrapper.h` `static_assert`s on `CUDNN_FRONTEND_VERSION` (v1.24.0), making node-signature drift against upstream a build failure rather than a source-compatibility break for consumers. - Corrects `sdpa_fp8_backward` to upstream's two overloads — 18 tensor params returning `array<...,7>` (FP8) and 17 returning `array<...,6>` (MXFP8) — matching the existing `sdpa_fp8` pair. - Aliases the Tier-1 attribute types into `<shim>::graph` in `cudnn_frontend/graph_properties.h`. - Adds cuDNN-spelled aliases `Layernorm_backward_attributes` and `Resample_attributes` on the native attribute headers (unguarded, as with `Resample_fwd_attributes`). - Enables `HIPDNN_ENABLE_CUDNN_COMPATIBILITY=ON` in both jobs of `.github/workflows/hipdnn-superbuild-ci.yml`, so PR CI builds and runs the shim tests and samples. - Wires the node-coverage gtest and four host-only samples behind `HIPDNN_ENABLE_CUDNN_COMPATIBILITY` in the frontend-tests and samples CMake. - Moves the `_mode = Mode::Native;` flip after the forwarded call in all 18 Tier-1 node methods, matching the pre-existing `sdpa`/`sdpa_backward` ordering: a throwing forward can no longer leave the wrapper claiming Native with no node added. - Adds `get_compute_data_type()` to the Tier-2 `UnsupportedAttributes` base, so a stub exposes the same universal accessor pair as the hipDNN attribute types the Tier-1 aliases resolve to. - Guards all four new samples with the same `try`/`catch` as `SdpaRoundTrip.cpp`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a header-only cuDNN-frontend compatibility shim to hipDNN so hipified consumers (e.g. PyTorch's
aten/src/ATen/native/cudnn/MHA.cpp) can build against cuDNN-spelled graph and SDPA APIs and have them lower onto the hipDNN frontend. Implements RFC 0012. The shim, its tests, and its sample are gated behind the opt-inHIPDNN_ENABLE_CUDNN_COMPATIBILITY(andHIPDNN_ENABLE_SDPA) options, both OFF by default.JIRA ID : ALMIOPEN-2039 (ALMIOPEN-2040)
Risk Assessment
Low risk. The change is header-only, entirely behind opt-in build flags that default OFF, and does not touch kernel selection, dispatch behavior, or any shipping default. The SDPA attribute surface was restructured to record unsupported source-compatibility requests on the base attribute type and surface them at node
validate()rather than failing at the call site; this is confined to the frontend attribute/node types and is covered by host-side unit tests.ASIC Coverage
ASIC-independent. This is frontend source-compatibility plumbing that lowers cuDNN-spelled graphs onto the existing hipDNN frontend; it changes no kernel selection, support surface, or default behavior. All tests run against a mock backend with no device, and the sample is host-only. Passing PR CI is sufficient; no dedicated multi-arch sweep is required.
Testing Summary
validate().Testing Checklist
git commit- Status: Passedcmake --preset hipdnn-cudnn-all(enablesHIPDNN_ENABLE_CUDNN_COMPATIBILITY+HIPDNN_ENABLE_SDPA), ranhipdnn-unit-check(7/7 ctests) - Status: Passedctest --test-dir build -R hipdnn_sample_cudnn_shim_sdpa_round_trip- Status: PassedTechnical Changes
projects/hipdnn/frontend/include/hipdnn_compatibility/cudnn/: agraph::Graphwrapper (detail/graph_wrapper.h) mirroring cuDNN's snake_case public spelling and error-recorder pattern, SDPA attributes (cudnn_frontend/sdpa_attributes.h), graph interface/properties, frontend utils, version, and an error recorder.Attributes(recordUnsupported()/hasUnsupportedUsage()/getUnsupportedReason()), so shim setters with no hipDNN equivalent record a reason instead of failing at the call site, and the owning SDPA node surfaces it atvalidate(). This moves the bulk of the SDPA logic off the standalone shimsdpa_attributes.honto the real hipDNNSdpaAttributes/SdpaBackwardAttributes.HIPDNN_ENABLE_CUDNN_COMPATIBILITYandHIPDNN_ENABLE_SDPA(both default OFF) that gate installation, tests, and the sample.TestCudnnShimGraph.cpp,TestCudnnShimGraphSDPA.cpp) and for the SDPA forward/backward attributes and nodes (TestSdpaAttributes.cpp,TestSdpaBackwardAttributes.cpp,TestSdpaFwdNode.cpp,TestSdpaBwdNode.cpp), plus a reusable mock-backend fixture and a host-only SDPA round-trip sample registered as a CTest.