feat: build llama.cpp with ggml-hexagon backend for Snapdragon NPU (Android + Windows ARM + Linux ARM) #2

Summary

Build llama.cpp with -DGGML_HEXAGON=ON for Snapdragon devices. Upstream merged the ggml-hexagon backend (PR ggml-org#16547) but publishes no pre-built binaries with NPU support.

Target runtimes

Build dependencies

Android (Docker)

• Image: ghcr.io/snapdragon-toolchain/arm64-android:v0.7
• Includes: Android NDK r28b, Hexagon SDK 6.6.0.0, OpenCL SDK, CMake, Ninja
• Host: any (Docker Desktop), --platform linux/amd64
• Build time: ~15-30 min first, ~3-8 min incremental
• Output: pkg-snapdragon/ with lib/ (HTP skels) + bin/ (llama-server, llama-cli, llama-bench)

Windows ARM64 (native)

• Host: ARM64 Windows (self-hosted runner or cross-compile from x64)
• MSVC 2026 arm64 + LLVM/Clang (via winget)
• Hexagon SDK 6.6+ (free, Qualcomm account required)
  • URL: https://softwarecenter.qualcomm.com/catalog/item/Hexagon_SDK?version=6.6.0.0
  • CI-optimized: https://github.com/snapdragon-toolchain/hexagon-sdk
• OpenCL SDK 2.3+ (Adreno GPU)
  • CI-optimized: https://github.com/snapdragon-toolchain/opencl-sdk
• Qualcomm NPU driver: https://softwarecenter.qualcomm.com/catalog/item/Qualcomm_HND
• Extra: Windows test-signing mode (bcdedit /set TESTSIGNING ON) + self-signed cert for NPU driver

Linux ARM64 (cross-compile)

• Image: ghcr.io/snapdragon-toolchain/arm64-linux:v0.1
• Host: Linux x86_64 only (cross-compilation)
• Currently no OpenCL support on Linux Snapdragon (GGML_OPENCL=OFF)

Asset naming

Follow existing pattern:

llama-{tag}-bin-{platform}-{backend}-{arch}.{ext}

Examples:

llama-daily-2026-06-20-bin-android-hexagon-arm64.tar.gz
llama-daily-2026-06-20-bin-win-hexagon-arm64.zip
llama-daily-2026-06-20-bin-linux-hexagon-arm64.tar.gz

Proposed CI workflow

name: Build Hexagon NPU runtimes

on:
  schedule:
    - cron: '0 2 * * *'
  workflow_dispatch:

jobs:
  android:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build in Docker toolchain
        run: |
          docker run --rm -u $(id -u):$(id -g) \
            -v $(pwd):/workspace \
            --platform linux/amd64 \
            ghcr.io/snapdragon-toolchain/arm64-android:v0.7 \
            bash -c "
              cp docs/backend/snapdragon/CMakeUserPresets.json .
              cmake --preset arm64-android-snapdragon-release -B build-snapdragon
              cmake --build build-snapdragon -j \$(nproc)
              cmake --install build-snapdragon --prefix pkg-snapdragon
            "
      - name: Package
        run: |
          TAG=$(date +daily-%Y-%m-%d)
          tar czf "llama-$TAG-bin-android-hexagon-arm64.tar.gz" -C pkg-snapdragon .
      - uses: softprops/action-gh-release@v2
        with:
          tag_name: daily-${{ github.event.repository.updated_at }}
          files: llama-*-hexagon-*.tar.gz
          prerelease: true

  windows-arm64:
    runs-on: [self-hosted, windows, arm64]
    # ... requires ARM64 Windows runner

  linux-arm64:
    runs-on: ubuntu-latest
    # ... uses arm64-linux Docker toolchain

Runtime usage

# Android
adb push pkg-snapdragon /data/local/tmp/
adb shell "cd /data/local/tmp/pkg-snapdragon && \
  LD_LIBRARY_PATH=./lib ADSP_LIBRARY_PATH=./lib \
  ./bin/llama-server --no-mmap --device HTP0 -m /model.gguf"

# Windows
$env:ADSP_LIBRARY_PATH=".\lib"
.\bin\llama-server.exe --device HTP0 -m model.gguf

# Linux
LD_LIBRARY_PATH=./lib ADSP_LIBRARY_PATH=./lib \
  ./bin/llama-server --device HTP0 -m model.gguf

Runtime environment variables

Hardware reference

Known limitations

Related

• Tracking in npu-optimize: feat: Qualcomm Hexagon NPU support (Android + Windows ARM + Linux ARM) npu-optimize#5
• Upstream docs: docs/backend/snapdragon/README.md
• Upstream PRs: Add experimental ggml-hexagon backend for the Hexagon NPU ggml-org/llama.cpp#16547 (ggml-hexagon), hexagon: improved Op queuing, buffer and cache management ggml-org/llama.cpp#21705 (op queuing), hexagon: enable offloading to Hexagon on Windows on Snapdragon ggml-org/llama.cpp#19150 (Windows)
• Snapdragon toolchain: https://github.com/snapdragon-toolchain
• Qualcomm Hexagon SDK (free): https://softwarecenter.qualcomm.com/catalog/item/Hexagon_SDK
• Perf benchmarks: https://specpicks.com/reviews/llama-cpp-snapdragon-hexagon-npu-benchmarks-2026