metal-whisper: ggml_metal_library_init fails to compile embedded Metal shader library on macOS 26.5.2 (Apple M4 Pro) — flash-attn kernel's threadgroup half4x4 array rejected

[kamilsa]

Summary

metal-whisper fails to load any model on macOS 26.5.2 with Apple M4 Pro. ggml_metal_library_init fails to compile the backend's entire embedded Metal shader library, because one kernel (flash-attention) declares a threadgroup array of half4x4 matrices that the current Metal shader compiler on this OS/toolchain rejects. Since ggml compiles its whole shader library up front rather than per-kernel-on-demand, this one bad kernel makes Metal unusable for whisper on this machine — even for models/requests that never use flash attention.

This is the same failure class as the companion reports #11529 (#11529) and #11530 (#11530) (stablediffusion-ggml, also on macOS 26.5.2): a Metal pipeline/library compile failure that surfaces to the operator only as exitCode=2 and rpc error: code = Unavailable desc = error reading from server: EOF, with the actual Metal compiler diagnostic invisible unless you manually redirect the backend's stderr.

Environment

• LocalAI v4.8.2 (5ff25d9d145e0a03a5b9a3559c620f1e1204ca6d)
• Backend metal-whisper, installed from quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-whisper, digest sha256:4f18b4b228d2a750e74cf9f006e3789bb3d99da5b9d651e4fc1d8e55121346fa
• Also reproduced on metal-whisper-development (:master-metal-darwin-arm64-whisper), same failure
• macOS 26.5.2 (build 25F84), Apple M4 Pro (Mac16,8)
• Model: ggml-large-v3-turbo.bin (whisper-large-v3-turbo), valid ggml magic, 1.62 GB

What LocalAI reports

Aug 20 15:18:54 WARN  Backend process exited unexpectedly id="whisper-large-turbo" address="127.0.0.1:50266" process="run.sh" exitCode="2"
Aug 20 15:18:54 ERROR Failed to load model modelID="whisper-large-turbo" error=failed to load model with internal loader: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF backend="whisper"

No indication a Metal shader ever failed to compile.

What is actually happening

Capturing the backend's own stderr (by running run.sh/the whisper binary directly instead of through LocalAI) shows the real error:

[INFO ] whisper_init_with_params_no_state: use gpu    = 1
[INFO ] whisper_init_with_params_no_state: flash attn = 1
[INFO ] ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
[INFO ] ggml_metal_library_init: using embedded metal library
[ERROR] ggml_metal_library_init: error: Error Domain=MTLLibraryErrorDomain Code=3
"program_source:14536:27: error: no matching constructor for initialization of
'threadgroup metal::half4x4[512]' (aka 'threadgroup matrix<half, 4, 4>[512]')
    threadgroup half4x4   sk4x4[NK*DK16];
                          ^

followed by ~450 lines of Metal compiler candidate-constructor diagnostics for metal::matrix, all stemming from the same threadgroup half4x4 sk4x4[NK*DK16] declaration (the flash-attention Metal kernel).

Because ggml/whisper.cpp compiles the entire embedded Metal shader source as one library (ggml_metal_library_init), this single kernel failing to compile takes down Metal initialization for the whole backend — even though flash_attention: "off" in the model's LocalAI YAML has no effect here (the library fails to compile before any per-request kernel selection happens).

Reproduction (bypassing LocalAI to isolate the backend)

cd ~/backends/metal-whisper
DYLD_LIBRARY_PATH="$(pwd)/lib" WHISPER_LIBRARY="$(pwd)/libgowhisper-fallback.so" \
  ./whisper -addr=127.0.0.1:59999 &
grpcurl -plaintext -proto backend.proto -d '{"Model":"ggml-large-v3-turbo.bin","ModelFile":"/path/to/ggml-large-v3-turbo.bin","Threads":14,"ContextSize":4096,"NBatch":512,"NGPULayers":99999999}' \
  127.0.0.1:59999 backend.Backend/LoadModel

→ ERROR: Code: Unavailable, Message: error reading from server: EOF, and the backend's own stderr shows the Metal compile error above.

What would help

1. Fix the ggml Metal shader source so the flash-attention kernel's threadgroup half4x4 sk4x4[NK*DK16] array-of-matrices declaration compiles under current Xcode/macOS 26.x Metal shader compilers (this is presumably a ggml-upstream fix, shared with ggml-org/whisper.cpp / ggml-org/llama.cpp, given sk4x4/NK*DK16 look like shared flash-attention kernel naming).
2. Same asks as stablediffusion-ggml backend segfaults instead of returning an error when a Metal pipeline fails to compile, and its stderr never reaches the LocalAI log #11529 (stablediffusion-ggml backend segfaults instead of returning an error when a Metal pipeline fails to compile, and its stderr never reaches the LocalAI log #11529): don't let a failed Metal library/pipeline compile surface only as exitCode=2 + EOF — propagate the real error, and get the backend's stderr into the LocalAI log so operators don't have to bypass LocalAI entirely to find the actual cause.
3. Given ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices is logged right before the failure, it's possible this shader path is only exercised/broken on non-M5 tensor-API-disabled devices — worth checking whether M5 Macs (where the tensor API is enabled) take a different code path and avoid this.

Related

• stablediffusion-ggml backend segfaults instead of returning an error when a Metal pipeline fails to compile, and its stderr never reaches the LocalAI log #11529 (stablediffusion-ggml backend segfaults instead of returning an error when a Metal pipeline fails to compile, and its stderr never reaches the LocalAI log #11529) (stablediffusion-ggml: Metal pipeline compile failure surfaces only as SIGSEGV/exitCode=2/EOF, same macOS 26.5.2)
• metal stablediffusion-ggml: shipped Metal library is missing kernel_mul_mv_ext_bf16_f32_r1_5, so every bf16 LTX-2 video model dies at pipeline compile #11530 (metal stablediffusion-ggml: shipped Metal library is missing kernel_mul_mv_ext_bf16_f32_r1_5, so every bf16 LTX-2 video model dies at pipeline compile #11530) (stablediffusion-ggml: missing Metal kernel for bf16, same root symptom pattern)