Offload input layer for select models.

[pzdunows]

This change improves performance of LFM2 models.

Before:

D:\test>fix_disabled\llama-bench.exe -m LFM2-1.2B-Q8_0.gguf -ngl 100 -t 8
HIP Library Path: D:\test\fix_disabled\amdhip64_7.dll
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| lfm2 1.2B Q8_0                 |   1.16 GiB |     1.17 B | ROCm       | 100 |           pp512 |    13955.14 ± 240.79 |
| lfm2 1.2B Q8_0                 |   1.16 GiB |     1.17 B | ROCm       | 100 |           tg128 |        278.71 ± 5.62 |

build: 017eceed6 (7036)

After:

D:\test>fix_enabled\llama-bench.exe -m LFM2-1.2B-Q8_0.gguf -ngl 100 -t 8
HIP Library Path: D:\test\fix_enabled\amdhip64_7.dll
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| lfm2 1.2B Q8_0                 |   1.16 GiB |     1.17 B | ROCm       | 100 |           pp512 |    14617.30 ± 177.74 |
| lfm2 1.2B Q8_0                 |   1.16 GiB |     1.17 B | ROCm       | 100 |           tg128 |        328.27 ± 7.79 |

build: 017eceed6 (7036)

[pzdunows]

Hi @CISC, could you take a look at this change?
Thanks!

[CISC]

I think it's better if @ggerganov had a look at this, out of curiosity though do you think there are other models that would benefit from this?

[ggerganov]

There were a few issues with this model, think I fixed them here: #17548

[pzdunows]

I think it's better if @ggerganov had a look at this, out of curiosity though do you think there are other models that would benefit from this?

Hard to say, I was only looking into this specific model family.

[pzdunows]

There were a few issues with this model, think I fixed them here: #17548

Thanks, I'll check it out.