You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Eval bug: RX 7800 XT Vulkan decode and MTP slower on 84e76d8a2 than c77ae695c; #27952 prefill gain not seen #29410
version: 0.5.0-dev (build 1099, commit 84e76d8a2)
built with MSVC 19.44.35227.0 for x64
Previous build, same machine and flags:
version: 0.4.1-dev (build 953, commit c77ae695c)
84e76d8a2 is 146 commits after c77ae695c and includes #27952. This is not a bisect. Prefill did not show the Strix Halo speedup, and decode / MTP got slower.
Operating systems
Windows 11
Which llama.cpp modules do you know to be affected?
Depths measured: 48000 and 130000 tokens. Acceptance stayed 1.00 on both builds.
Problem description
#27952 reports about +32% prompt processing for Qwen3.6-35B-A3B Q4_K_M on Strix Halo (RDNA3.5), and the new int8 coopmat1 MMQ path is enabled for q4_K on RDNA3. On this discrete RX 7800 XT the same quant did not get faster, and token generation got slower.
MTP acceptance was 1.00 at both depths on both builds, so the draft tokens are accepted. Each step just takes longer. The 48k MTP point fell by about half (55.8 → 25.8 t/s). The no-MTP tg32 point fell about 15% (42.4 → 36.1). pp512 is flat (1174 → 1151), not the +32% from the Strix Halo table.
Vulkan commits in c77ae695c..84e76d8a2 also include #28415 (IQ4_XS), #28822 (IQ3_S), #29139 (symbol visibility), #24406 (Intel Xe flash attention), #29328 (Adreno), #29365 (conv alignment) and #28501 (mul_mat_id hoist 256→512). The daily model is Q4_K_M with 256 experts, so the IQ kernels and the 512-expert hoist should not be the hot path. #27952 is the change that retargets q4_K MMQ on RDNA3. Decode uses the vec path, and #25666 (do not enable MMVQ for speculative decode on AMD) is still open, so the MTP drop may be a different commit in the same range.
Related but not the same report: #28752 is a Linux prefill regression after #28190. Here prompt processing at pp512 did not regress; generation and MTP did.
Happy to retest a single commit or a build with #27952 reverted if that helps narrow it.
Cross-data from the same GPU family — Radeon RX 7900 XTX (gfx1100, 24 GB), Linux, Mesa 26.2 RADV (vs your Windows + AMD proprietary driver), dense Qwen3.8-27B IQ3_S, same machine, interleaved runs. I cannot reproduce the decode/MTP regression on RADV.
llama-bench (-r 3, IQ3_S, Vulkan):
build
pp512 (t/s)
tg128 (t/s)
41c878d70 (b11065-based, predates the #27952 merge)
Acceptance ranges overlap across reps and effective decode speed is equal-or-better on master. Prefill improving also matches the #27952 A/B I posted in #29342 on this card (long-ctx prefill +6.2%, decode flat).
The slowdown not reproducing on Mesa/RADV with a dense 27B suggests it may be specific to the Windows AMD Vulkan driver (your log shows shared memory: 32768 vs 65536 on RADV) and/or the MoE 35B-A3B path — worth a dense model or a Mesa build as a second data point.
It was tuned and tested on Linux only, yes. Especially the 32KB shmem limit on Windows may restrict you to the medium tile here. I don't have a Windows setup to tune it, it might need a smaller L-warptile.
Activity
tczank commentedon Sep 25, 2026
Cross-data from the same GPU family — Radeon RX 7900 XTX (gfx1100, 24 GB), Linux, Mesa 26.2 RADV (vs your Windows + AMD proprietary driver), dense Qwen3.8-27B IQ3_S, same machine, interleaved runs. I cannot reproduce the decode/MTP regression on RADV.
llama-bench (
-r 3, IQ3_S, Vulkan):llama-server MTP probe (
--spec-type draft-mtp --spec-draft-n-max 2, 131072 ctx, q4_0 KV, vision on, greedy 400-token prompts, 4 reps each):Acceptance ranges overlap across reps and effective decode speed is equal-or-better on master. Prefill improving also matches the #27952 A/B I posted in #29342 on this card (long-ctx prefill +6.2%, decode flat).
The slowdown not reproducing on Mesa/RADV with a dense 27B suggests it may be specific to the Windows AMD Vulkan driver (your log shows
shared memory: 32768vs 65536 on RADV) and/or the MoE 35B-A3B path — worth a dense model or a Mesa build as a second data point.0cc4m commentedon Sep 25, 2026
It was tuned and tested on Linux only, yes. Especially the 32KB shmem limit on Windows may restrict you to the medium tile here. I don't have a Windows setup to tune it, it might need a smaller L-warptile.