Skip to content

feat(gfx11): add packed-MQ4 and optional CK prefill backend - #616

Draft
HUSRCF wants to merge 14 commits into
warpfront:betafrom
HUSRCF:feat/gfx11-ck-prebuilt-beta
Draft

feat(gfx11): add packed-MQ4 and optional CK prefill backend#616
HUSRCF wants to merge 14 commits into
warpfront:betafrom
HUSRCF:feat/gfx11-ck-prebuilt-beta

Conversation

@HUSRCF

@HUSRCF HUSRCF commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Add the validated gfx1100 packed-MQ4 prefill stack and an optional, fail-closed FlashAttention CK sidecar for the Qwen3.6/3.8 27B Asym3-K/Q8-V path. The bundle tooling provides a relocatable prebuilt deployment path, so users do not need to compile CK locally.

The optimized routes are restricted to the validated RDNA3 shapes and retain native fallbacks. CK additionally requires the flash-attn-ck Cargo feature and an explicit sidecar path; graph capture, tree masks, multi-slot attention, generic llama paths, unsupported shapes, and unavailable sidecars remain native.

The branch is based directly on beta@80a572c8. The implementation was migrated as separate reviewable commits covering zero-correction fusion, K32 probes, group128 execution, fused SwiGLU packing, activation reload reduction, packed-MQ4 primitives, FP16 FFN intermediates, production routing, and benchmark evidence.

Scope

  • Radeon Pro W7900 / exact gfx1100, ROCm 7.14
  • Qwen hybrid prefill, including Hq=24, Hkv=4, D=256 attention
  • packed-MQ4 projection routes plus causal contiguous-prefix CK attention
  • prefill only; decode and speculative verification remain native
  • no prebuilt binary is committed to git

On official beta@80a572c8 with the complete packed-MQ4 production flag set, three alternating W7900 process pairs at PP8192 measured native attention 748.4 tok/s versus staged CK attention 1221.5 tok/s: a 1.6321x paired-median speedup, with 3/3 positive pairs. Each process ran three prefill trials and reported its in-process median. Every pair produced identical greedy token IDs, while decode remained neutral at 35.0-35.2 tok/s.

The full-stack harness, raw rows, artifact hashes, and reproduction command are committed under experiments/flash-attn-ck-sidecar/results/beta_w7900_pp8192_fullopt_ab_20260823/. The earlier CK-only migration result (593.7 -> 865.8 tok/s) remains under results/beta_w7900_pp8192_ab_20260823/ for attribution.

Which crates does this touch?

  • kernels/ (gfx11 packed-MQ4 kernels and optional CK sidecar)
  • crates/rdna-compute (execution primitives, optional loader, dispatch)
  • crates/hipfire-runtime / crates/hipfire-dispatch (Qwen prefill routing)
  • crates/hipfire-cli / crates/hipfire-daemon (feature forwarding)
  • docs / scripts / benchmark evidence

No Architecture trait change.

Test plan

  • cargo check --locked -p hipfire-cli --features flash-attn-ck
  • cargo check --locked -p rdna-compute --example test_batched_attn_slots --features flash-attn-ck
  • cargo build --release --locked -p hipfire-daemon --features deltanet,flash-attn-ck
  • cargo build --release -p hipfire-runtime --example bench_qwen35_mq4 --features deltanet,flash-attn-ck
  • gfx1100 D256 dense CK sidecar source build on ROCm 7.14
  • D256 CPU-reference smoke: MHA, MQA, and GQA; causal and non-causal; default and non-default stream
  • build-capability smoke verifies that only generated D64/D128/D256 dimensions are advertised
  • staged quantized sidecar source build and ABI smoke (ABI=1, workspace=3,145,728 bytes, staged=11,534,336 bytes)
  • bundle package, checksum, clean-prefix install, $ORIGIN dependency, and launcher smoke
  • bundle contract tests: 4/4
  • optional Rust loader tests: 6/6
  • Python suite: 836 passed
  • Rust core tests: 169 passed; Qwen prefill tests: 13 passed; CLI/TUI tests: 153 passed
  • full-stack W7900 PP8192 A/B: 3/3 positive pairs, exact greedy token-ID agreement
  • full ./scripts/no-gpu-ci.sh: reaches the final legacy suite, then current beta reports two unrelated baseline failures (ensure_binaries expects the pre-split daemon cargo argv; change-gate lists three moved/dead pflash paths). This PR does not touch those files.

Packaging

The bundle contains a CK-enabled CLI and daemon, relocatable dense/staged sidecars, and matching gfx1100 code-object cache. The installer verifies external and internal SHA256, manifest/ABI, sidecar dependencies, kernel inventory, and visible GPU before atomically switching a versioned symlink. Unsupported systems are rejected before activation.

@HUSRCF HUSRCF changed the title feat(gfx11): add optional CK prefill sidecar and prebuilt bundle feat(gfx11): add packed-MQ4 and optional CK prefill backend Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant