feat(gfx11): add packed-MQ4 and optional CK prefill backend - #616
Draft
HUSRCF wants to merge 14 commits into
Draft
feat(gfx11): add packed-MQ4 and optional CK prefill backend#616HUSRCF wants to merge 14 commits into
HUSRCF wants to merge 14 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add the validated gfx1100 packed-MQ4 prefill stack and an optional, fail-closed FlashAttention CK sidecar for the Qwen3.6/3.8 27B Asym3-K/Q8-V path. The bundle tooling provides a relocatable prebuilt deployment path, so users do not need to compile CK locally.
The optimized routes are restricted to the validated RDNA3 shapes and retain native fallbacks. CK additionally requires the
flash-attn-ckCargo feature and an explicit sidecar path; graph capture, tree masks, multi-slot attention, generic llama paths, unsupported shapes, and unavailable sidecars remain native.The branch is based directly on
beta@80a572c8. The implementation was migrated as separate reviewable commits covering zero-correction fusion, K32 probes, group128 execution, fused SwiGLU packing, activation reload reduction, packed-MQ4 primitives, FP16 FFN intermediates, production routing, and benchmark evidence.Scope
gfx1100, ROCm 7.14On official
beta@80a572c8with the complete packed-MQ4 production flag set, three alternating W7900 process pairs at PP8192 measured native attention748.4 tok/sversus staged CK attention1221.5 tok/s: a 1.6321x paired-median speedup, with 3/3 positive pairs. Each process ran three prefill trials and reported its in-process median. Every pair produced identical greedy token IDs, while decode remained neutral at35.0-35.2 tok/s.The full-stack harness, raw rows, artifact hashes, and reproduction command are committed under
experiments/flash-attn-ck-sidecar/results/beta_w7900_pp8192_fullopt_ab_20260823/. The earlier CK-only migration result (593.7 -> 865.8 tok/s) remains underresults/beta_w7900_pp8192_ab_20260823/for attribution.Which crates does this touch?
kernels/(gfx11 packed-MQ4 kernels and optional CK sidecar)crates/rdna-compute(execution primitives, optional loader, dispatch)crates/hipfire-runtime/crates/hipfire-dispatch(Qwen prefill routing)crates/hipfire-cli/crates/hipfire-daemon(feature forwarding)No Architecture trait change.
Test plan
cargo check --locked -p hipfire-cli --features flash-attn-ckcargo check --locked -p rdna-compute --example test_batched_attn_slots --features flash-attn-ckcargo build --release --locked -p hipfire-daemon --features deltanet,flash-attn-ckcargo build --release -p hipfire-runtime --example bench_qwen35_mq4 --features deltanet,flash-attn-ck$ORIGINdependency, and launcher smoke./scripts/no-gpu-ci.sh: reaches the final legacy suite, then current beta reports two unrelated baseline failures (ensure_binariesexpects the pre-split daemon cargo argv; change-gate lists three moved/dead pflash paths). This PR does not touch those files.Packaging
The bundle contains a CK-enabled CLI and daemon, relocatable dense/staged sidecars, and matching gfx1100 code-object cache. The installer verifies external and internal SHA256, manifest/ABI, sidecar dependencies, kernel inventory, and visible GPU before atomically switching a versioned symlink. Unsupported systems are rejected before activation.