Library not loaded: @rpath/libllama.dylib

[constantm]

Name and Version

I just downloaded the latest binary (b4519) to play around with on my Mac, however it looks like the binary didn't quite compile correctly. I tried downloading an earlier version (b4514) with the same result. Running llamba-cli yields:

➜  llama.cpp ./llama-cli --version
dyld[90496]: Library not loaded: @rpath/libllama.dylib
  Referenced from: <653E6B29-4AFF-3485-B031-B4F65747F8CF> /Users/constantmeiring/Downloads/build/llama.cpp/llama-cli

Operating systems

MacOS 15.1.1 (24B91)

GGML backends

Metal

Hardware

Macbook - M3 Max

Models

No response

Problem description & steps to reproduce

Download the latest build and try and run it on Mac.

First Bad Commit

No response

Relevant log output

dyld[90496]: Library not loaded: @rpath/libllama.dylib
  Referenced from: <653E6B29-4AFF-3485-B031-B4F65747F8CF> /Users/constantmeiring/Downloads/build/llama.cpp/llama-cli
  Reason: tried: '/Users/runner/work/llama.cpp/llama.cpp/build/src/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/src/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-blas/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-blas/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-metal/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-metal/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-rpc/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-rpc/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/src/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/src/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-blas/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-blas/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-metal/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-metal/libllama.dylib' (no such file), '/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-rpc/libllama.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/llama.cpp/llama.cpp/build/ggml/src/ggml-rpc/libllama.dylib' (no such file)
[1]    90496 abort      ./llama-cli --version

[psymonryan]

I can confirm this, it seems that the binaries no longer are statically linked:

output:

$ ./bin-arm64.new/llama-cli --version
dyld[38894]: Library not loaded: @rpath/libllama.dylib
  Referenced from: <ACB8BE98-DF3F-3422-8F2B-8015DBAB58DE> /Users/simon/models/bin-arm64.new/llama-cli

expected:

$ ./bin-arm64.old/llama-cli --version
version: 4400 (6e1531ac)
built with Apple clang version 15.0.0 (clang-1500.3.9.4) for arm64-apple-darwin23.6.0

[artemdevel]

I can confirm this as well. Looks like the last working version is 4406 (0da5d860). The size of the release zip file for macOS arm64 is 61Mb for 4406 but all further releases have smaller zip files (~13Mb).

[Satertek]

Looks like #10902 caused it by removing -DBUILD_SHARED_LIBS=OFF from the cmake build arguments at https://github.com/ggerganov/llama.cpp/blob/0da5d860266c6928b8c9408efbd264ae59fedda6/.github/workflows/build.yml#L64 .

Building the current master (5845661) locally:

without DBUILD_SHARED_LIBS=OFF

> otool -l llama-server
...
Load command 23
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 1351.0.0
compatibility version 1.0.0
Load command 24
          cmd LC_RPATH
      cmdsize 56
         path /Users/brian/git/llama.cpp/build/src (offset 12)
Load command 25
          cmd LC_RPATH
      cmdsize 56
         path /Users/brian/git/llama.cpp/build/ggml/src (offset 12)
Load command 26
          cmd LC_RPATH
      cmdsize 64
         path /Users/brian/git/llama.cpp/build/ggml/src/ggml-blas (offset 12)
Load command 27
          cmd LC_RPATH
      cmdsize 72
         path /Users/brian/git/llama.cpp/build/ggml/src/ggml-metal (offset 12)
Load command 28
          cmd LC_RPATH
      cmdsize 64
         path /Users/brian/git/llama.cpp/build/ggml/src/ggml-rpc (offset 12)
Load command 29
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 2789696
 datasize 4296
...

adding DBUILD_SHARED_LIBS=OFF back...

> otool -l llama-server
...
Load command 22
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libobjc.A.dylib (offset 24)
   time stamp 2 Wed Dec 31 18:00:02 1969
      current version 228.0.0
compatibility version 1.0.0
Load command 23
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 4984560
 datasize 7584
Load command 24
...

[bpotchik]

Assuming you want to used shared libraries. Add these options when generating your build files in order to have cmake perform the install_name_tool -add_rpath for each executable.

-DCMAKE_INSTALL_PREFIX=/path/to/install/location \
      -DCMAKE_INSTALL_RPATH=/path/to/install/location/lib \
      -DCMAKE_BUILD_WITH_INSTALL_RPATH=TRUE

Or update the cmake file with something like:

set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)

[ggerganov]

@constantm The latest releases should have fixed the issue.