clBLAST support broken

[paralin]

Running commit 948ff13 the LLAMA_CLBLAST=1 support is broken.

After a Git Bisect I found that 4d98d9a is the first bad commit.

llm_load_tensors: ggml ctx size =    0.12 MiB
llm_load_tensors: using OpenCL for GPU acceleration
llm_load_tensors: mem required  =  205.19 MiB
llm_load_tensors: offloading 32 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 33/33 layers to GPU
llm_load_tensors: VRAM used: 5461.02 MiB
...................................................................................................
llama_new_context_with_model: n_ctx      = 512
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: KV self size  =   64.00 MiB, K (f16):   32.00 MiB, V (f16):   32.00 MiB
llama_build_graph: non-view tensors processed: 676/676
llama_new_context_with_model: compute buffer total size = 76.32 MiB
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
GGML_ASSERT: ggml.c:7765: ggml_are_same_shape(src0, src1)
llama.sh: line 17:  1461 Aborted                 (core dumped) ./main --threads ${NUM_CPUS} --n-gpu-layers ${MODEL_GPU_LAYERS} --model ./models/${MODEL} --temp ${MODEL_TEMP} --repeat_penalty 1.1764705882352942 --n-predict -1 --top_k 42 --top_p 0 -e ${MODEL_ARGS} --prompt "${MODEL_PROMPT}"

Originally reported in #4451 - but I realized this also applies to any model when I tried testing Intel Neural 7B v3-3.

Reverting commit 4d98d9a against commit 948ff1 fixes the issue.

cc @ggerganov

[ggerganov]

Does this patch fix it for non-Mixtral models?

diff --git a/ggml.c b/ggml.c
index 29e18a24..5b74bc4f 100644
--- a/ggml.c
+++ b/ggml.c
@@ -7762,7 +7762,7 @@ static void ggml_compute_forward_mul_f32(
 // TODO: OpenCL kernel support broadcast
 #ifdef GGML_USE_CLBLAST
     if (src1->backend == GGML_BACKEND_GPU) {
-        GGML_ASSERT(ggml_are_same_shape(src0, src1));
+        GGML_ASSERT(ggml_can_repeat_rows(b, a));
         if (ith == 0) {
             ggml_cl_mul(src0, src1, dst);
         }

[LostRuins]

ggml.c:7765:45: error: 'a' undeclared (first use in this function)
 7765 |         GGML_ASSERT(ggml_can_repeat_rows(b, a));

should a and b be src0 and src1?

[ggerganov]

Oops, yes:

GGML_ASSERT(ggml_can_repeat_rows(src1, src0));

[LostRuins]

// TODO: OpenCL kernel support broadcast
#ifdef GGML_USE_CLBLAST
    if (src1->backend == GGML_BACKEND_GPU) {
        GGML_ASSERT(ggml_can_repeat_rows(src1, src0));
        if (ith == 0) {
            ggml_cl_mul(src0, src1, dst);
        }
        return;
    }
#endif

Yup, this seems to work fine now.