add close_ggml_native feature

[cyberdoors]

Adding the feature to disable GGML_NATIVE and enabling the close_ggml_native feature may result in a performance decrease, but it allows the executable to run on other computers.

[MarcusDunn]

I'm not sure if the feature is the way to go. I think hooking into cargo's --target-cpu is better. (this likely changes the default from native to something else)

[cyberdoors]

I'm not sure if the feature is the way to go. I think hooking into cargo's --target-cpu is better. (this likely changes the default from native to something else)

I tried running the executable on Windows using $env:RUSTFLAGS="-C target-cpu=x86-64"; cargo build --release, then copied the executable to another Windows machine, but it still failed to run. Disabling GGML_NATIVE in build.rs, however, allowed it to run successfully.

[AsbjornOlling]

I'm not sure if the feature is the way to go. I think hooking into cargo's --target-cpu is better. (this likely changes the default from native to something else)

I tried running the executable on Windows using $env:RUSTFLAGS="-C target-cpu=x86-64"; cargo build --release, then copied the executable to another Windows machine, but it still failed to run. Disabling GGML_NATIVE in build.rs, however, allowed it to run successfully.

I don't think the build.rs script can pick up on the target-cpu config being passed to cargo. If I understand correctly, what Marcus is suggesting is that we modify the build.rs script to pick up on the target-cpu setting (from cargo or rustc or whatever) and ask cmake to build for that specific architecture, such that the command you ran will work.

EDIT:
For the record, I think that's a great idea, and it would be very useful to me.
I'd like to work on it, but I don't expect to have a lot of extra time, so no promises.

[AsbjornOlling]

A different, but related task is to support the GGML_CPU_ALL_VARIANTS and GGML_BACKEND_DL configuration variables. Setting those makes llama.cpp compile dynamic libraries for multiple cpu variants, and then allows selecting the appropriate one at runtime.

This is the best of both worlds: maximum portability and performance.

[MarcusDunn]

I don't think the build.rs script can pick up on the target-cpu config being passed to cargo. If I understand correctly, what Marcus is suggesting is that we modify the build.rs script to pick up on the target-cpu setting (from cargo or rustc or whatever) and ask cmake to build for that specific architecture, such that the command you ran will work.

This is correct. Apologies for not being clear. I'd like GGML_NATIVE to only be set if rustc is run with --target-cpu=native (I'm unsure if cargo exposes this to build.rs)

A different, but related task is to support the GGML_CPU_ALL_VARIANTS and GGML_BACKEND_DL configuration variables. Setting those makes llama.cpp compile dynamic libraries for multiple cpu variants, and then allows selecting the appropriate one at runtime.

This would be interesting for sure. Given these exist, I wonder what the appropriate way to expose them would be. I'd be interested to see what consumers of this library (both apps and libs) would prefer.