metal : use F32 prec for K*Q in vec FA

[ggerganov]

Noticed that Qwen2-7B-Instruct with Metal FA produces garbage:

./llama-cli -m ./models/qwen2-7b-instruct/ggml-model-f16.gguf -p "I believe the meaning of life is" -n 16 -s 1 --temp 0.0 -fa

master

I believe the meaning of life is tobedtls_x509_crt_new_from_pem() create a new

PR

I believe the meaning of life is to be happy and to be healthy. I believe that the best way to achieve

Seems to be again insufficient precision in the K*Q multiplication, this time only in the vec FA kernel (it works OK with the non-vec kernel). Should make K*Q always use F32 precision - proves to be very sensitive operation.

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High