Jetson Xavier NX 8GB Armv8-linux aarch compile error

[hitmachineWu]

Dear All,
I was tring to compile the llama.cpp on Nvidia Jetson Xavier NX 8GB
using

cmake -B build -DGGML_CUDA=ON

cmake --build build --config Release

g++ gcc version: 7.5.0
llama.cpp : tag: b4174
uname -m : aarch64

during the compile , error ouccurs:

'1 error detected in the compilation of "/tmp/tmpxft_00006a23_00000000-12_acc.compute_75.cpp1.ii".
ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:75: recipe for target 'ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/acc.cu.o' failed
make[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/acc.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[  9%] Building C object ggml/src/ggml-cpu/CMakeFiles/ggml-cpu.dir/ggml-cpu-quants.c.o
In file included from /home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:7:0:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c: In function ‘ggml_vec_dot_q2_K_q8_K’:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:290:27: error: implicit declaration of function ‘vld1q_s16_x2’; did you mean ‘vld1q_s16’? [-Werror=implicit-function-declaration]
 #define ggml_vld1q_s16_x2 vld1q_s16_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4217:41: note: in expansion of macro ‘ggml_vld1q_s16_x2’
         const ggml_int16x8x2_t q8sums = ggml_vld1q_s16_x2(y[i].bsums);
                                         ^~~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:290:27: error: invalid initializer
 #define ggml_vld1q_s16_x2 vld1q_s16_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4217:41: note: in expansion of macro ‘ggml_vld1q_s16_x2’
         const ggml_int16x8x2_t q8sums = ggml_vld1q_s16_x2(y[i].bsums);
                                         ^~~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:291:27: error: implicit declaration of function ‘vld1q_u8_x2’; did you mean ‘vld1q_u32’? [-Werror=implicit-function-declaration]
 #define ggml_vld1q_u8_x2  vld1q_u8_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4241:46: note: in expansion of macro ‘ggml_vld1q_u8_x2’
             const ggml_uint8x16x2_t q2bits = ggml_vld1q_u8_x2(q2); q2 += 32;
                                              ^~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:291:27: error: invalid initializer
 #define ggml_vld1q_u8_x2  vld1q_u8_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4241:46: note: in expansion of macro ‘ggml_vld1q_u8_x2’
             const ggml_uint8x16x2_t q2bits = ggml_vld1q_u8_x2(q2); q2 += 32;
                                              ^~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:293:27: error: implicit declaration of function ‘vld1q_s8_x2’; did you mean ‘vld1q_s32’? [-Werror=implicit-function-declaration]
 #define ggml_vld1q_s8_x2  vld1q_s8_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4243:40: note: in expansion of macro ‘ggml_vld1q_s8_x2’
             ggml_int8x16x2_t q8bytes = ggml_vld1q_s8_x2(q8); q8 += 32;
                                        ^~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:293:27: error: invalid initializer
 #define ggml_vld1q_s8_x2  vld1q_s8_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4243:40: note: in expansion of macro ‘ggml_vld1q_s8_x2’
             ggml_int8x16x2_t q8bytes = ggml_vld1q_s8_x2(q8); q8 += 32;
                                        ^~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4235:17: error: incompatible types when assigning to type ‘int8x16x2_t {aka struct int8x16x2_t}’ from type ‘int’
         q8bytes = ggml_vld1q_s8_x2(q8); q8 += 32;\
                 ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4249:13: note: in expansion of macro ‘SHIFT_MULTIPLY_ACCUM_WITH_SCALE’
             SHIFT_MULTIPLY_ACCUM_WITH_SCALE(2, 2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4235:17: error: incompatible types when assigning to type ‘int8x16x2_t {aka struct int8x16x2_t}’ from type ‘int’
         q8bytes = ggml_vld1q_s8_x2(q8); q8 += 32;\
                 ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4250:13: note: in expansion of macro ‘SHIFT_MULTIPLY_ACCUM_WITH_SCALE’
             SHIFT_MULTIPLY_ACCUM_WITH_SCALE(4, 4);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4235:17: error: incompatible types when assigning to type ‘int8x16x2_t {aka struct int8x16x2_t}’ from type ‘int’
         q8bytes = ggml_vld1q_s8_x2(q8); q8 += 32;\
                 ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4251:13: note: in expansion of macro ‘SHIFT_MULTIPLY_ACCUM_WITH_SCALE’
             SHIFT_MULTIPLY_ACCUM_WITH_SCALE(6, 6);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:7:0:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c: In function ‘ggml_vec_dot_q3_K_q8_K’:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:291:27: error: invalid initializer
 #define ggml_vld1q_u8_x2  vld1q_u8_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4801:36: note: in expansion of macro ‘ggml_vld1q_u8_x2’
         ggml_uint8x16x2_t qhbits = ggml_vld1q_u8_x2(qh);
                                    ^~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:291:27: error: invalid initializer
 #define ggml_vld1q_u8_x2  vld1q_u8_x2
                           ^

ggml/src/ggml-cpu/CMakeFiles/ggml-cpu.dir/build.make:117: recipe for target 'ggml/src/ggml-cpu/CMakeFiles/ggml-cpu.dir/ggml-cpu-quants.c.o' failed
make[2]: *** [ggml/src/ggml-cpu/CMakeFiles/ggml-cpu.dir/ggml-cpu-quants.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cuda/common.cuh(337): error: A __device__ variable cannot be marked constexpr

/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cuda/common.cuh(337): error: A __device__ variable cannot be marked constexpr

1 error detected in the compilation of "/tmp/tmpxft_00006a3e_00000000-12_arange.compute_75.cpp1.ii".
ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:89: recipe for target 'ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/arange.cu.o' failed
make[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/arange.cu.o] Error 1
1 error detected in the compilation of "/tmp/tmpxft_00006a47_00000000-12_argmax.compute_75.cpp1.ii".
ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:103: recipe for target 'ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/argmax.cu.o' failed
make[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/argmax.cu.o] Error 1
CMakeFiles/Makefile2:1759: recipe for target 'ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all' failed
make[1]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:1733: recipe for target 'ggml/src/ggml-cpu/CMakeFiles/ggml-cpu.dir/all' failed
make[1]: *** [ggml/src/ggml-cpu/CMakeFiles/ggml-cpu.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2''

I'm wondering if this problem was caused by the version of g++/gcc (7.5.0 maybe to low). Right now I'm tring to update the g++/gcc to 11.0.1

Are there any collaborators who can help me identify feasible solutions? Much appreciate.

[hitmachineWu]

I checked the cpu info of the Jetson Xavier NX 8GB using cat /proc/cpuinfo , the cpu is ARMv8, It seems like Xavier should support ARM_NEON, But these sentence indicates these variable wasn't found , if there is any compile config I can do to adjust the cmake file to compile llama.cpp on Jetson Xavier NX?

[hitmachineWu]

#10963