Compile bug: Error generating vulkan shaders for mul_mm.comp

[tomoverlund]

Git commit

$ git rev-parse HEAD
81bc4d3

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

Running on freshly updated Debian unstable on AMD 64, using a fresh git clone of llama.cpp. Trying to compile with Vulkan support. Packages libvulkan-dev (1.4.341.0-1) and glslc (2026.1-1) are installed. Running on a single core, non-concurrent build.

Build steps:

$ cmake -B build -DGGML_VULKAN=1
$ cmake --build build --config Release

I get an error at this point:

[ 15%] Generate vulkan shaders for mul_mm.comp
Error executing command for matmul_id_subgroup_q8_0_f32_aligned_fp32: Failed to fork process
Error executing command for matmul_id_subgroup_q8_0_f16_fp32: Failed to fork process
[..]

followed by a long list of similar failures.

Other shaders seemed to compile okay. The previous successful build step was:
[ 14%] Generate vulkan shaders for mul_mat_vecq.comp
and the next successful build step was:
[ 15%] Generate vulkan shaders for mul_mm_cm2.comp

The build continues on, and eventually fails at:
[ 67%] Linking CXX executable ../bin/test-tokenizer-0
/usr/bin/x86_64-linux-gnu-ld.bfd: ../bin/libggml-vulkan.so.0.9.8: undefined reference to `matmul_id_subgroup_q6_k_f32_f16acc_len'

First Bad Commit

No response

Compile command

$ cmake -B build -DGGML_VULKAN=1
$ cmake --build build --config Release

Relevant log output

See main description.

[github-actions]

This issue might be similar or related to the following issue(s):

• Compile bug: Vulkan backend, glslc shader compilation errors (Ubuntu 24.04.3) #16860: Vulkan backend, glslc shader compilation errors on Ubuntu 24.04.3
• Compile bug: Vulkan fails to compile after f117be1 #17164: Vulkan fails to compile after f117be1 due to glslc check
• Misc. bug: Running Vulkan backend on Intel DG1 outputs gibberish due to commit 38eaf32 #17302: Vulkan backend on Intel DG1 outputs gibberish due to commit 38eaf32
• Compile bug: (windows/vulkan/debug build) add_rms_data and add_rms_len not found #16502: (windows/vulkan/debug build) add_rms_data and add_rms_len not found

These issues all involve Vulkan shader compilation failures or linking errors similar to the mul_mm.comp shader generation issues reported here.

This comment was auto-generated locally using llama.cpp version: 8456 (b31b30f) on NVIDIA DGX Spark

[tomoverlund]

This was due to an out of memory error. I was running in a virtual machine with only 4 gigs. I increased it to 12 gigs and the build finished correctly.

I suspected this at first, but I was watching the memory in a separate window and I always had 3+ gigs free. I guess there must be a spike at the point it failed.

Anyways, I can close this, but it might be good to add the error string for errno in the exception at ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp:

    pid_t pid = fork();
    if (pid < 0) {
        throw std::runtime_error("Failed to fork process" + strerror(errno));
    }

[tecosaur]

I'm running into this on a 32 GB system, with 20G free at the time I attempted building.

  [200/615] Generate vulkan shaders for mul_mm.comp
  Cannot allocate memory
  Error executing command for matmul_id_subgroup_iq4_xs_f16_f16acc: Failed to fork process
  Cannot allocate memory
  Error executing command for matmul_id_subgroup_iq4_xs_f16_aligned_f16acc: Failed to fork process
  Cannot allocate memory

This is with b8746.

[tomoverlund]

I'm running into this on a 32 GB system, with 20G free at the time I attempted building.

Are you building in parallel? That will eat more RAM.

[tecosaur]

No, this is with serial compilation, including patching

llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp

Line 319 in fb38d6f

uint32_t N = std::max(1u, std::min(16u, std::thread::hardware_concurrency()));

to be uint32_t N = 1;.

[0cc4m]

If you can't find a solution, please open a new issue for your case.