Skip to content

Eval bug: RX 7800 XT Vulkan decode and MTP slower on 84e76d8a2 than c77ae695c; #27952 prefill gain not seen #29410

Open

Description

@MatheusMenezes08

Name and Version

Regressed build:

version: 0.5.0-dev (build 1099, commit 84e76d8a2)
built with MSVC 19.44.35227.0 for x64

Previous build, same machine and flags:

version: 0.4.1-dev (build 953, commit c77ae695c)

84e76d8a2 is 146 commits after c77ae695c and includes #27952. This is not a bisect. Prefill did not show the Strix Halo speedup, and decode / MTP got slower.

Operating systems

Windows 11

Which llama.cpp modules do you know to be affected?

llama-bench, llama-server (Vulkan)

Hardware

  • GPU: AMD Radeon RX 7800 XT 16 GB (RDNA3, discrete), AMD proprietary Vulkan driver
  • Log line: matrix cores: KHR_coopmat, uma: 0, shared memory: 32768
  • CPU: Ryzen 5 7600X, 32 GB RAM
  • Model: Qwen3.5/3.6 35B-A3B MoE, Q4_K_M, native MTP (Qwopus3.6-35B-A3B-Coder-MTP-Q4_K_M.gguf)

Command line

Empty-context bench (no speculative decoding):

llama-bench -m Qwopus3.6-35B-A3B-Coder-MTP-Q4_K_M.gguf -dev Vulkan0 -ngl 99 -ncmoe 18 -lm none -fa on -p 512 -n 32 -r 2

MTP decode at depth (llama-server, then tg128 on a filled KV):

llama-server -dev Vulkan0 -ngl 99 -lm none --n-cpu-moe 18 -m Qwopus3.6-35B-A3B-Coder-MTP-Q4_K_M.gguf -fa on -b 2048 --ubatch-size 2048 --cache-type-k q4_0 --cache-type-v q4_0 --reasoning off --spec-type draft-mtp --spec-draft-n-max 3 -c 163840 -t 12

Depths measured: 48000 and 130000 tokens. Acceptance stayed 1.00 on both builds.

Problem description

#27952 reports about +32% prompt processing for Qwen3.6-35B-A3B Q4_K_M on Strix Halo (RDNA3.5), and the new int8 coopmat1 MMQ path is enabled for q4_K on RDNA3. On this discrete RX 7800 XT the same quant did not get faster, and token generation got slower.

Test c77ae69 (17 Sep) 84e76d8 (24 Sep)
llama-bench pp512 1174 t/s 1151 ± 9 t/s
llama-bench tg32 (no MTP) 42.4 t/s 36.1 ± 0.6 t/s
MTP n=3, tg128 @ 48k 55.79 t/s 25.80 t/s
MTP n=3, tg128 @ 130k 39.15 t/s 27.14 t/s

MTP acceptance was 1.00 at both depths on both builds, so the draft tokens are accepted. Each step just takes longer. The 48k MTP point fell by about half (55.8 → 25.8 t/s). The no-MTP tg32 point fell about 15% (42.4 → 36.1). pp512 is flat (1174 → 1151), not the +32% from the Strix Halo table.

Vulkan commits in c77ae695c..84e76d8a2 also include #28415 (IQ4_XS), #28822 (IQ3_S), #29139 (symbol visibility), #24406 (Intel Xe flash attention), #29328 (Adreno), #29365 (conv alignment) and #28501 (mul_mat_id hoist 256→512). The daily model is Q4_K_M with 256 experts, so the IQ kernels and the 512-expert hoist should not be the hot path. #27952 is the change that retargets q4_K MMQ on RDNA3. Decode uses the vec path, and #25666 (do not enable MMVQ for speculative decode on AMD) is still open, so the MTP drop may be a different commit in the same range.

Related but not the same report: #28752 is a Linux prefill regression after #28190. Here prompt processing at pp512 did not regress; generation and MTP did.

Happy to retest a single commit or a build with #27952 reverted if that helps narrow it.

Activity

  1. tczank commented on Sep 25, 2026

    @tczank

    Cross-data from the same GPU family — Radeon RX 7900 XTX (gfx1100, 24 GB), Linux, Mesa 26.2 RADV (vs your Windows + AMD proprietary driver), dense Qwen3.8-27B IQ3_S, same machine, interleaved runs. I cannot reproduce the decode/MTP regression on RADV.

    llama-bench (-r 3, IQ3_S, Vulkan):

    build pp512 (t/s) tg128 (t/s)
    41c878d70 (b11065-based, predates the #27952 merge) 943.9 avg 51.11 avg
    a02c7f5 (master, contains #27952) 980.2 avg (+3.8%) 50.98 (flat)

    llama-server MTP probe (--spec-type draft-mtp --spec-draft-n-max 2, 131072 ctx, q4_0 KV, vision on, greedy 400-token prompts, 4 reps each):

    build tg median acceptance (mean, range)
    41c878d70 (production) 79.6 t/s 0.690 (0.669–0.726)
    a02c7f5 (master) 82.0 t/s (+3%) 0.675 (0.664–0.682)

    Acceptance ranges overlap across reps and effective decode speed is equal-or-better on master. Prefill improving also matches the #27952 A/B I posted in #29342 on this card (long-ctx prefill +6.2%, decode flat).

    The slowdown not reproducing on Mesa/RADV with a dense 27B suggests it may be specific to the Windows AMD Vulkan driver (your log shows shared memory: 32768 vs 65536 on RADV) and/or the MoE 35B-A3B path — worth a dense model or a Mesa build as a second data point.

  2. 0cc4m commented on Sep 25, 2026

    @0cc4m
    Contributor

    It was tuned and tested on Linux only, yes. Especially the 32KB shmem limit on Windows may restrict you to the medium tile here. I don't have a Windows setup to tune it, it might need a smaller L-warptile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions