Eval bug: s390x GGML_NNPA=ON Generates Gibberish Tokens at Different Thread Counts

[taronaeo]

Name and Version

$ build/bin/llama-cli --version

version: 5985 (3f4fc97f)
built with gcc (GCC) 15.1.0 for s390x-redhat-linux

Operating systems

Linux

GGML backends

CPU

Hardware

IBM z17 40 IFLs / 128 GB Memory

Models

1. granite-3.3-2b-instruct-be.F32.gguf
2. DeepSeek-R1-0528-Qwen3-8B-BE.Q4_K_M.gguf

Problem description & steps to reproduce

-DGGML_NNPA=ON turns on the SIMD instruction set for the IBM Neural Network Processor Assist accelerator located within the Telum I/II chip. However we are noticing that at different thread counts, the generated token output is gibberish.

We are investigating but at this point in time, the best action will be to disable it by default and allow end-users to manually turn it out if they wish to try it out.

1. Compile with -DGGML_NNPA=ON
2. Test with llama-cli at different thread counts i.e., 1/2/4/8/16
3. Notice that at some thread counts, the generation becomes gibberish e.g.

Write me a dog walking business idea 1. 
     A   resources  A  tags       private  .

  A [end of text]

First Bad Commit

No response

Relevant log output

(thread = 1 OK) $ build/bin/llama-cli -m /devfield/XXX/hf_models/granite-3.3-2b-instruct-be.F32.gguf -t 1 -n 25 -p "Write me a dog walking business idea 1. " -no-cnv -ngl -1 --seed 1568795874 --ctx-size 16384

Write me a dog walking business idea 1. 
What is the name of the business?
2. What services does it offer?
3. Who are the target


(thread = 2 OK) $ build/bin/llama-cli -m /devfield/XXX/hf_models/granite-3.3-2b-instruct-be.F32.gguf -t 2 -n 25 -p "Write me a dog walking business idea 1. " -no-cnv -ngl -1 --seed 1568795874 --ctx-size 16384

Write me a dog walking business idea 1. 
What is the name of the business?
2. What services does it offer?
3. Who are the target

(thread = 4 OK) $ build/bin/llama-cli -m /devfield/XXX/hf_models/granite-3.3-2b-instruct-be.F32.gguf -t 4 -n 25 -p "Write me a dog walking business idea 1. " -no-cnv -ngl -1 --seed 1568795874 --ctx-size 16384

Write me a dog walking business idea 1. 
What is the name of the business?
2. What services does it offer?
3. Who are the target


(thread = 8 FAIL) $ build/bin/llama-cli -m /devfield/XXX/hf_models/granite-3.3-2b-instruct-be.F32.gguf -t 8 -n 25 -p "Write me a dog walking business idea 1. " -no-cnv -ngl -1 --seed 1568795874 --ctx-size 16384

Write me a dog walking business idea 1. riv (CNN,

##.subdonia:16:}}{ Aaron,

1. **


(thread = 16 FAIL) $ build/bin/llama-cli -m /devfield/XXX/hf_models/granite-3.3-2b-instruct-be.F32.gguf -t 16 -n 25 -p "Write me a dog walking business idea 1. " -no-cnv -ngl -1 --seed 1568795874 --ctx-size 16384

Write me a dog walking business idea 1. 50smithy2

Bank Of The 1
 [end of text]

[taronaeo]

I will create a PR to disable this compile flag by default, and update the documentation to reflect the same, soon. Logging it here for future reference.