Windows ARM64 Vulkan for ternary Q2_0 models

Run a ternary-weight Q2_0 GGUF (ggml type 42) on the Snapdragon X Elite's Adreno GPU via llama.cpp's Vulkan backend, on Windows ARM64. This repo contains a reproducible build guide, the required CMake patch for the Vulkan backend, a native ARM64 Vulkan SDK recipe, the fix for the vk::CommandBuffer::end: Incomplete context-init crash (GGML_VK_PREFER_HOST_MEMORY=1), and CPU-vs-Vulkan benchmarks. The model GGUF is used unmodified.

Validated on: Snapdragon X Elite (X1E80100, 12-core Oryon), Adreno X1-85, 64 GB unified memory, Windows 11 ARM64.

---

TL;DR / Context

Mainline llama.cpp does not support ggml type 42 (ternary Q2_0). A patched ternary-capable llama.cpp fork adds CPU and partial GPU support, but it ships CPU-only builds and its Vulkan path needs three fixes on Windows ARM64 before it can load a ternary 8B-class model on the Adreno:

1. A one-line CMake patch so ggml-vulkan links SPIRV-Headers::SPIRV-Headers.
2. Either a proper import library for vulkan-1.dll (static build), or the native ARM64 Vulkan SDK (shared build).
3. GGML_VK_PREFER_HOST_MEMORY=1 at runtime to route around a device-local buffer allocation bug during context init.

This repository documents both build paths end to end. The ternary model GGUF is used exactly as shipped.

---

Environment

Item	Value
CPU	Snapdragon X Elite X1E80100, 12-core Oryon (ARMv8, NEON; no SVE in user mode)
GPU	Adreno X1-85 (unified memory)
RAM	64 GB
OS	Windows 11 ARM64
Vulkan SDK	KhronosGroup.VulkanSDK 1.4.350.0 (native ARM64) or shaderc continuous build (x64-emulated glslc)
Toolchain	llvm-mingw aarch64 UCRT, CMake 4.x, Ninja
Headers	Vulkan-Headers, SPIRV-Headers (installed as a CMake package)
Runtime DLLs	libc++.dll, libunwind.dll, libomp.dll, libwinpthread-1.dll from llvm-mingw

---

Step-by-step instructions

All commands are shown in Git Bash. Replace C:\Users\you\project with your own working directory.

1. Clone the patched llama.cpp fork

mkdir -p /c/Users/you/project
cd /c/Users/you/project
git clone --depth 1 --branch ternary https://github.com/<org>/llama.cpp ternary-llama-src
cd ternary-llama-src
git apply ../patches/0001-ggml-vulkan-link-spirv-headers.patch

The ternary branch is the one that adds ggml type 42 support. Replace <org> with the actual fork owner you are using.

2. Install the toolchain

# Native ARM64 Vulkan SDK (recommended)
winget install --id KhronosGroup.VulkanSDK --exact

Also install or unpack:

• llvm-mingw aarch64 UCRT from https://github.com/mstorsjo/llvm-mingw/releases
• CMake and Ninja
• Python 3.12 for ARM64 (optional, for scripting)

3. Stage Vulkan and SPIRV headers

cd /c/Users/you/project
mkdir -p vulkan && cd vulkan
git clone --depth 1 https://github.com/KhronosGroup/Vulkan-Headers.git
git clone --depth 1 https://github.com/KhronosGroup/SPIRV-Headers.git
cd SPIRV-Headers
cmake -B build -DCMAKE_INSTALL_PREFIX=/c/Users/you/project/vulkan/spirv-headers-install
cmake --build build --target install

4. Configure (shared build, native SDK)

export PATH="/c/Users/you/project/llvm-mingw/bin:$PATH"
cd /c/Users/you/project/ternary-llama-src
cmake -S . -B build-vulkan -G Ninja \
  -DCMAKE_C_COMPILER=/c/Users/you/project/llvm-mingw/bin/aarch64-w64-mingw32-clang.exe \
  -DCMAKE_CXX_COMPILER=/c/Users/you/project/llvm-mingw/bin/aarch64-w64-mingw32-clang++.exe \
  -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DGGML_NATIVE=OFF \
  -DCMAKE_C_FLAGS="-D_WIN32_WINNT=0x0A00" -DCMAKE_CXX_FLAGS="-D_WIN32_WINNT=0x0A00" \
  -DVulkan_ROOT="C:/VulkanSDK/1.4.350.0" \
  -DSPIRV-Headers_DIR=/c/Users/you/project/vulkan/spirv-headers-install/share/cmake/SPIRV-Headers

5. Build

cmake --build build-vulkan --target llama-server -j 10

6. Run

GGML_VK_PREFER_HOST_MEMORY=1 ./build-vulkan/bin/llama-server.exe \
  -m /c/Users/you/project/models/<model>-Q2_0.gguf \
  --port 8101 --host 127.0.0.1 -c 8192 -t 4 -ngl 99 --no-mmap --no-warmup

For a background server on Windows, use scripts/launch-adreno-server.ps1.

---

Verification / testing

1. Confirm Vulkan sees the Adreno:

./build-vulkan/bin/llama-server.exe --list-devices

Expected output includes:

Vulkan0: Qualcomm(R) Adreno(TM) X1-85 GPU (36602 MiB)

2. Run a short generation test:

./build-vulkan/bin/llama-cli.exe \
  -m /c/Users/you/project/models/<model>-Q2_0.gguf \
  -p "Say OK" -n 4 --no-warmup -t 4 -c 512 -ngl 99

3. Compare CPU vs Vulkan decode on the same prompt:

# CPU only
./build-vulkan/bin/llama-cli.exe -m /c/Users/you/project/models/<model>-Q2_0.gguf \
  -p "Say OK" -n 32 --no-warmup -t 4 -c 512 -ngl 0

# GPU offload
./build-vulkan/bin/llama-cli.exe -m /c/Users/you/project/models/<model>-Q2_0.gguf \
  -p "Say OK" -n 32 --no-warmup -t 4 -c 512 -ngl 99

On the validation machine, CPU decode was ~14.4 tok/s and Vulkan decode was ~12.6 tok/s for the ternary Q2_0 model.

---

Known limitations

• No Q2_0 Vulkan kernel. The ternary type is dequantized to float on the GPU, so Vulkan decode is slightly slower than the CPU ternary kernel for this specific quantization.
• Windows ARM64 Vulkan tooling is young. The first validated path used an x64-emulated shaderc toolchain; the native ARM64 SDK (1.4.350.0) is now available and recommended.
• Thermal throttling. Sustained all-core load on the X Elite can silently reduce throughput. Treat benchmark numbers as one data point from one machine.
• Unified memory only. The GGML_VK_PREFER_HOST_MEMORY=1 workaround is viable because Adreno on Snapdragon shares system RAM; behavior on discrete GPUs may differ.
• Single-machine validation. All numbers come from one Snapdragon X Elite laptop on one day.

---

Reproduction notes

• GUIDE.md contains the full narrative, the static-build path, the SIGILL root-cause analysis, and the import-library fix.
• patches/0001-ggml-vulkan-link-spirv-headers.patch is the only source patch required.
• scripts/launch-adreno-server.ps1 is a convenience wrapper for a background OpenAI-compatible server.
• Replace all C:\Users\you\project placeholders with your actual working directory.

---

License

Repository content (guide, patch, scripts): PolyForm Noncommercial License 1.0.0.

Component licenses are listed in GUIDE.md.

---

Author: Dr. Lucas Root, Ph.D. — info@lucasroot.com