Skip to content
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.

Bump ggml version to fix compilation bug on aarch64 #105

Closed
wants to merge 1 commit into from

Conversation

fmeef
Copy link

@fmeef fmeef commented Apr 4, 2023

On aarch64 (tested on m1 MacBook), compilation fails with the below message. It looks like this problem was fixed in ggml recently in ggerganov/ggml@652c0c0. This PR syncs the local ggml.c with upstream.

error: failed to run custom build command for `ggml-sys v0.1.0 (/llama-rs/ggml-sys)`

Caused by:
  process didn't exit successfully: `/llama-rs/target/debug/build/ggml-sys-796dc0784457db96/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=ggml
  OPT_LEVEL = Some("0")
  TARGET = Some("aarch64-unknown-linux-gnu")
  HOST = Some("aarch64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_aarch64-unknown-linux-gnu
  CC_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_aarch64_unknown_linux_gnu
  CC_aarch64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
  CFLAGS_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
  CFLAGS_aarch64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("neon")
  running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-I" "include" "-mcpu=native" "-pthread" "-o" "/llama-rs/target/debug/build/ggml-sys-809c6ec9c698a44e/out/ggml/ggml.o" "-c" "ggml/ggml.c"
  cargo:warning=ggml/ggml.c: In function 'dequantize_row_q4_1':
  cargo:warning=ggml/ggml.c:1041:13: note: use '-flax-vector-conversions' to permit conversions between vectors with differing element types or numbers of subparts
  cargo:warning= 1041 |             const uint16x8_t vi_0 = vmovl_s8(vget_low_u8 (vq));
  cargo:warning=      |             ^~~~~
  cargo:warning=ggml/ggml.c:1041:46: error: incompatible type for argument 1 of 'vmovl_s8'
  cargo:warning= 1041 |             const uint16x8_t vi_0 = vmovl_s8(vget_low_u8 (vq));
  cargo:warning=      |                                              ^~~~~~~~~~~~~~~~
  cargo:warning=      |                                              |
  cargo:warning=      |                                              uint8x8_t
  cargo:warning=In file included from ggml/ggml.c:164:
  cargo:warning=/usr/lib/gcc/aarch64-linux-gnu/10/include/arm_neon.h:8830:20: note: expected 'int8x8_t' but argument is of type 'uint8x8_t'
  cargo:warning= 8830 | vmovl_s8 (int8x8_t __a)
  cargo:warning=      |           ~~~~~~~~~^~~
  cargo:warning=ggml/ggml.c:1042:46: error: incompatible type for argument 1 of 'vmovl_s8'
  cargo:warning= 1042 |             const uint16x8_t vi_1 = vmovl_s8(vget_high_u8(vq));
  cargo:warning=      |                                              ^~~~~~~~~~~~~~~~
  cargo:warning=      |                                              |
  cargo:warning=      |                                              uint8x8_t
  cargo:warning=In file included from ggml/ggml.c:164:
  cargo:warning=/usr/lib/gcc/aarch64-linux-gnu/10/include/arm_neon.h:8830:20: note: expected 'int8x8_t' but argument is of type 'uint8x8_t'
  cargo:warning= 8830 | vmovl_s8 (int8x8_t __a)
  cargo:warning=      |           ~~~~~~~~~^~~
  exit status: 1

  --- stderr


  error occurred: Command "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-I" "include" "-mcpu=native" "-pthread" "-o" "/llama-rs/target/debug/build/ggml-sys-809c6ec9c698a44e/out/ggml/ggml.o" "-c" "ggml/ggml.c" with args "cc" did not execute successfully (status code exit status: 1).


warning: build failed, waiting for other jobs to finish...

This fixes a compilation bug on some arm cpus
@philpax
Copy link
Collaborator

philpax commented Apr 4, 2023

Interesting, I've been testing on my work M1 MBP and haven't run into this compilation error there. I wonder what the difference is?

In any case, we're trying to track ggml.c versions directly from upstream llama.cpp, so we're trying to avoid partial patches except when they're necessary.

I'll leave this PR open until we can decide the best solution - ideally we'd update to the latest upstream ggml.c, but that was segfaulting when I tested it.

@philpax
Copy link
Collaborator

philpax commented Apr 4, 2023

Oh - you're building for aarch64-unknown-linux-gnu, that's why it's different for you. I guess we'll have to add that to our test suite rather than later 😅

setzer22 added a commit that referenced this pull request Apr 4, 2023
@philpax
Copy link
Collaborator

philpax commented Apr 4, 2023

#107 fixes this and updates to the latest ggml - thanks for reporting!

@philpax philpax closed this Apr 4, 2023
philpax added a commit that referenced this pull request Apr 5, 2023
The latest version of GGML from llama.cpp uses s8(s8), which works.
This fix changed it to u8(s8), which doesn't.

This reverts commit 861ca8e.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants