How to use different whisper models from huggingface?

[markuman]

I like to test this one https://huggingface.co/primeline/whisper-tiny-german-1224

But I've no idea how to load and use it using local-ai.

local-ai run huggingface://primeline/whisper-tiny-german-1224
4:01PM INF Setting logging to info
4:01PM INF Starting LocalAI using 8 threads, with models path: /home/m/git/lekker/markus.bergholz/tmp/gguf_convert/hugg/models
4:01PM INF LocalAI version: v2.26.0 (09941c0bfb9119bb01a04b2a0a16897ecf2cd087)
panic: runtime error: index out of range [2] with length 2

goroutine 1 [running]:
github.com/mudler/LocalAI/pkg/downloader.URI.ResolveURL({0xc00082ebd0, 0x30})
	/home/runner/_work/LocalAI/LocalAI/pkg/downloader/uri.go:190 +0x8d5
github.com/mudler/LocalAI/pkg/downloader.URI.DownloadFile({0xc00082ebd0, 0x30}, {0xc000b5f8c0, 0x58}, {0x0, 0x0}, 0x0, 0x0, 0x5b76f7a0)
	/home/runner/_work/LocalAI/LocalAI/pkg/downloader/uri.go:235 +0x73
github.com/mudler/LocalAI/pkg/startup.InstallModels({0xc0003ed600, 0x1, 0x1}, {0xc000839900, 0x3f}, 0x1, 0x0, {0xc00084b110, 0x1, 0x1})
	/home/runner/_work/LocalAI/LocalAI/pkg/startup/model_preload.go:74 +0x96a
github.com/mudler/LocalAI/core/application.New({0xc00075d0b0?, 0x5d00d4c0?, 0xc000846b88?})
	/home/runner/_work/LocalAI/LocalAI/core/application/startup.go:65 +0x6e6
github.com/mudler/LocalAI/core/cli.(*RunCMD).Run(0x5cabd198, 0x5cabd180)
	/home/runner/_work/LocalAI/LocalAI/core/cli/run.go:193 +0x1ad1
reflect.Value.call({0x1b89b40?, 0x5cabd198?, 0x1eb58e0?}, {0x1ebfd8e, 0x4}, {0xc0002fc6a8, 0x1, 0x1?})
	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/reflect/value.go:581 +0xca6
reflect.Value.Call({0x1b89b40?, 0x5cabd198?, 0xc00075d8e0?}, {0xc0002fc6a8?, 0x1eb58e0?, 0xc00075d820?})
	/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/reflect/value.go:365 +0xb9
github.com/alecthomas/kong.callFunction({0x1b89b40?, 0x5cabd198?, 0xc00075da38?}, 0xc0008a60f0)
	/home/runner/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/callbacks.go:98 +0x45a
github.com/alecthomas/kong.(*Context).RunNode(0xc000505700, 0xc0006c0960, {0xc00075dd50, 0x1, 0xc000884d90?})
	/home/runner/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/context.go:774 +0x7ee
github.com/alecthomas/kong.(*Context).Run(0xc000884d90?, {0xc00075dd50?, 0x17?, 0x7?})
	/home/runner/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/context.go:799 +0x132
main.main()
	/home/runner/_work/LocalAI/LocalAI/main.go:119 +0xaa5

Any ideas?

[markuman]

Ok, I can answer this question by myself now.

1. download the model to a local directory

from huggingface_hub import snapshot_download

model_id = "primeline/whisper-large-v3-german"  # Replace with the ID of the model you want to download
snapshot_download(repo_id=model_id, local_dir="whisper-large-v3-german")

2. clone whisper and whisper.cpp

3. Apply this patch to your whisper.cpp clone: Update convert-h5-to-ggml.py ggerganov/whisper.cpp#2840 if it is not merged

4. install python dependencies

pip install huggingface_hub  torch numpy transformers

5. convert to ggml .bin file using

python whisper.cpp/models/convert-h5-to-ggml.py <model_path_from_1>/ <path_to_whisper_clone>/ output_directory/