ocloc missing symbols from libigc.so

[stef]

howdy,

i'm trying to compile this fine project for alpine linux, during that i ran into a bunch of roadblocks with this (and following issues) i'll try to document and validate my fixes to these. first issue:

Error loading the Generic builtin resource
Build failed with error code: -11
Command was: /home/s/tasks/aports/ugly/compute-runtime/src/compute-runtime-20.08.15750/build/bin/ocloc -q -file scheduler.cl -device bdw -cl-intel-greater-than-4GB-buffer-required -64 -out_dir /home/s/tasks/aports/ugly/compute-runtime/src/compute-runtime-20.08.15750/build/bin/scheduler/x64/gen8 -cpp_file -options -I/usr/include/igc -I/usr/include/igc/cif -I/usr/include/igc/ocl_igc_shared/executable_format -I/usr/include/igc/ocl_igc_shared/device_enqueue -I ../gen8 -cl-kernel-arg-info -cl-std=CL2.0 -cl-intel-disable-a64WA
make[2]: *** [igdrcl_lib_release/scheduler/CMakeFiles/scheduler_Gen8core.dir/build.make:62: bin/scheduler/x64/gen8/scheduler_Gen8core.bin] Error 245
make[2]: Leaving directory '/home/s/tasks/aports/ugly/compute-runtime/src/compute-runtime-20.08.15750/build'
make[1]: *** [CMakeFiles/Makefile2:6914: igdrcl_lib_release/scheduler/CMakeFiles/scheduler_Gen8core.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

fixed by adding igc to the target_link_libraries of the offline_compiler (this patch also adds the libs for the symbols - like backtrace et al - that musl does not provide)

diff -Nurw compute-runtime-20.08.15750/offline_compiler/CMakeLists.txt src/compute-runtime-20.08.15750/offline_compiler/CMakeLists.txt
--- compute-runtime-20.08.15750/offline_compiler/CMakeLists.txt   2020-02-29 00:33:03.068525017 +0000
+++ src/compute-runtime-20.08.15750/offline_compiler/CMakeLists.txt  2020-02-29 00:41:59.361882810 +0000
@@ -140,7 +140,7 @@
 endif()

 if(UNIX)
-  target_link_libraries(ocloc dl pthread)
+  target_link_libraries(ocloc dl unwind execinfo igc)
 endif()

 set_target_properties(ocloc PROPERTIES FOLDER "offline_compiler")

it seems that by linking libigc.so this problem is fixed, but i wonder why is libigc missing at all, is it dynamically loaded, but something goes wrong during that? or is it ok to just at libigc as a target_link_library?

thanks for any insights.

[JacekDanecki]

IGC libraries are being loaded by ocloc using dlopen, see

• line 332
• line 16
• line 37
• line 15
• IGC line 2060](https://github.com/intel/intel-graphics-compiler/blob/master/IGC/CMakeLists.txt#L2060)
• line 373

Can you run ocloc manually with LD_DEBUG=libs environment variable set?
As you link ocloc with igc library I suppose there could be problem with symbols conflicts. You can also run ocloc with LD_DEBUG=all environment variable set, to see how symbols are resolved.

[stef]

musl does not implement LD_DEBUG. i'll strace ocloc instead and see what libs it loads.

[stef]

what's loaded:

"/usr/lib/libunwind.so.8"
"/usr/lib/libexecinfo.so.1"
"/usr/lib/libstdc++.so.6"
"/usr/lib/libgcc_s.so.1"
"/usr/lib/libigdfcl.so.1"
"/usr/lib/libLLVM-9.so"
"/usr/lib/libopencl-clang.so.9"
"/usr/lib/libffi.so.6"
"/lib/libz.so.1"
"/usr/lib/libxml2.so.2"
"/usr/lib/liblzma.so.5"
"/usr/lib/libigc.so.1"

strangely libgic gets loaded, but still i get the same error as in the issue indicated above. however if i add an LD_PRELOAD=/usr/lib/libigc.so.1 to the ocloc invocation ocloc succeeds and returns without output.

[stef]

it looks like libigc.so is mapped correctly, as seen in strace:

open("/usr/lib/libigc.so.1", O_RDONLY|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)     = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=28370792, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\241\r\0\0\0\0\0"..., 960) = 960
mmap(NULL, 28577792, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f3031a96000
mmap(0x7f3031b6b000, 7450624, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0xd5000) = 0x7f3031b6b000
mmap(0x7f3032286000, 1400832, PROT_READ, MAP_PRIVATE|MAP_FIXED, 3, 0x7f0000) = 0x7f3032286000
mmap(0x7f30323dc000, 18853888, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x945000) = 0x7f30323dc000
mmap(0x7f30335a5000, 204800, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f30335a5000
close(3)                          = 0

[JacekDanecki]

"Error loading the Generic builtin resource" this message came from IGC, see line 951

[stef]

when i run ocloc with LD_PRELOAD=/usr/lib/libigc.so and strace it, then i get this:

open("/usr/lib/libigc.so.1", O_RDONLY|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)     = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=28370792, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\241\r\0\0\0\0\0"..., 960) = 960
mmap(NULL, 28577792, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff64758a000
mmap(0x7ff64765f000, 7450624, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0xd5000) = 0x7ff64765f000
mmap(0x7ff647d7a000, 1400832, PROT_READ, MAP_PRIVATE|MAP_FIXED, 3, 0x7f0000) = 0x7ff647d7a000
mmap(0x7ff647ed0000, 18853888, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x945000) = 0x7ff647ed0000
mmap(0x7ff649099000, 204800, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff649099000
close(3)                          = 0

looks the same as in #265 (comment)

only it is loaded first, not last

[stef]

"Error loading the Generic builtin resource" this message came from IGC, see line 951

yes, and it is fixed by linking/preloading libigc to ocloc, i traced that.

[JacekDanecki]

Can you verify IGC was built correctly, linked with builtins? I've observed successful IGC builds although there were problems with builtins.

[stef]

"Error loading the Generic builtin resource" this message came from IGC, see line 951

llvm::LoadBufferFromResource is failing and returning NULL here: https://github.com/intel/intel-graphics-compiler/blob/67351f4e52f52eb2e4d68a9a40599a77733b5603/IGC/AdaptorOCL/OCL/LoadBuffer.cpp#L59

when libigc is not linked/preloaded by only dlopened

[stef]

Can you verify IGC was built correctly, linked with builtins? I've observed successful IGC builds although there were problems with builtins.

sure, how do i verify builtins?

[JacekDanecki]

On Ubuntu 20.04 system I've such builtins symbols exported by igc 1.0.3390:

nm -DC /usr/lib/x86_64-linux-gnu/libigc.so | grep _igc_bif_
00000000018dc020 D _igc_bif_BC_120
00000000018dc000 D _igc_bif_BC_120_size
0000000001957680 D _igc_bif_BC_121
000000000195766c D _igc_bif_BC_121_size
00000000019d3be0 D _igc_bif_BC_122
00000000019d3bdc D _igc_bif_BC_122_size
$ dpkg -l libigc | grep libigc
ii  libigc         1.0.3390-1~ppa1~focal1 amd64        Intel(R) Graphics Compiler

[stef]

confirmed, i have this:

% nm -DC /usr/lib/libigc.so | grep _igc_bif_
0000000000aa7020 D _igc_bif_BC_120
0000000000aa7000 D _igc_bif_BC_120_size
0000000000b226e0 D _igc_bif_BC_121
0000000000b226d8 D _igc_bif_BC_121_size
0000000000b9ecc0 D _igc_bif_BC_122
0000000000b9eca8 D _igc_bif_BC_122_size