Cant generate julia-1.8.0-beta3 on powerpc (ppc64le) in LLVM generation

[jdadavid]

Trying to make julia-1.8.0-beta3 from source (julia-1.8.0-beta3-full.tar.gz) on PowerPC (ppc64le), I get the following error

Sysimage built. Summary:
Total ─────── 103.591980 seconds
Base: ───────  41.754413 seconds 40.3066%
Stdlibs: ────  61.835644 seconds 59.6915%
Warning: git information unavailable; versioning information limited
    JULIA usr/lib/julia/sys-o.a
Generating REPL precompile statements... 40/40
Executing precompile statements... 2016/2061
Precompilation complete. Summary:
Total ─────── 180.627899 seconds
Generation ── 133.111079 seconds 73.6935%
Execution ───  47.516820 seconds 26.3065%
Intrinsic has incorrect argument type!
void ({} addrspace(10)*)* @llvm.ppc.cfence.p10sl_s
Intrinsic has incorrect argument type!
void ({} addrspace(10)*)* @llvm.ppc.cfence.p10sl_s
in function julia_put_buffered_59286
LLVM ERROR: Broken function found, compilation aborted!

signal (6): Aborted
in expression starting at none:0
gsignal at /lib64/power9/libc.so.6 (unknown line)
abort at /lib64/power9/libc.so.6 (unknown line)
_ZN4llvm18report_fatal_errorERKNS_5TwineEb at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm18report_fatal_errorEPKcb at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN12_GLOBAL__N_118VerifierLegacyPass13runOnFunctionERN4llvm8FunctionE at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin/../lib/libLLVM-13jl.so (unknown line)
operator() at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/src/aotcompile.cpp:541 [inlined]
jl_dump_native_impl at /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/src/aotcompile.cpp:552
Allocations: 163850041 (Pool: 163771637; Big: 78404); GC: 133
/bin/sh: line 1: 1532746 Aborted                 (core dumped) JULIA_BINDIR=/m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin WINEPATH="/m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin;$WINEPATH" JULIA_NUM_THREADS=1 /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/bin/julia -O3 -C "native" --output-o /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/lib/julia/sys-o.a.tmp --startup-file=no --warn-overwrite=yes --sysimage /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/lib/julia/sys.ji /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/contrib/generate_precompile.jl 1
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
make[1]: *** [sysimage.mk:89: /m100/home/userexternal/jdavid00/j180b3/julia-1.8.0-beta3/usr/lib/julia/sys-o.a] Error 1
make: *** [Makefile:88: julia-sysimg-release] Error 2

real    15m42.891s
user    12m46.399s
sys     2m27.137s

Seeing *** This error is usually fixed by running make clean. If the error persists, try make cleanall. *** , I tried it both without any success, same error each time.

I am stuck, as I could generate previous version without trouble, cf.

julia> versioninfo()
Julia Version 1.7.2
Commit bf53498635* (2022-02-06 15:21 UTC)
Platform Info:
  OS: Linux (powerpc64le-unknown-linux-gnu)
  CPU: POWER9, altivec supported
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, pwr9)
Environment:
  JULIA_HDF5_LIBRARY_PATH = /cineca/prod/opt/libraries/hdf5/1.10.6/gnu--8.4.0/lib
  JULIA_EDITOR = vi

I also tried with latest (julia-1.9.0-dev) version (github cloned), it also errors in LLVM, but slightly differently

Sysimage built. Summary:
Total ───────  98.499731 seconds
Base: ───────  38.779036 seconds 39.3697%
Stdlibs: ────  59.718697 seconds 60.6283%
    JULIA usr/lib/julia/sys-o.a
Generating REPL precompile statements... 40/40
Executing precompile statements... 1888/1919
Precompilation complete. Summary:
Total ─────── 162.544161 seconds
Generation ── 120.498513 seconds 74.1328%
Execution ───  42.045648 seconds 25.8672%
Intrinsic has incorrect argument type!
void ({} addrspace(10)*)* @llvm.ppc.cfence.p10sl_s
Intrinsic has incorrect argument type!
void ({} addrspace(10)*)* @llvm.ppc.cfence.p10sl_s
in function julia_put_buffered_32809
LLVM ERROR: Broken function found, compilation aborted!

signal (6): Aborted
in expression starting at none:0
gsignal at /lib64/power9/libc.so.6 (unknown line)
abort at /lib64/power9/libc.so.6 (unknown line)
_ZN4llvm18report_fatal_errorERKNS_5TwineEb at /m100/home/userexternal/jdavid00/jlatest/julia/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm18report_fatal_errorEPKcb at /m100/home/userexternal/jdavid00/jlatest/julia/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN12_GLOBAL__N_118VerifierLegacyPass13runOnFunctionERN4llvm8FunctionE at /m100/home/userexternal/jdavid00/jlatest/julia/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /m100/home/userexternal/jdavid00/jlatest/julia/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /m100/home/userexternal/jdavid00/jlatest/julia/usr/bin/../lib/libLLVM-13jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /m100/home/userexternal/jdavid00/jlatest/julia/usr/bin/../lib/libLLVM-13jl.so (unknown line)
operator() at /m100/home/userexternal/jdavid00/jlatest/julia/src/aotcompile.cpp:545 [inlined]
jl_dump_native_impl at /m100/home/userexternal/jdavid00/jlatest/julia/src/aotcompile.cpp:556
Allocations: 115905935 (Pool: 115818693; Big: 87242); GC: 153
/bin/sh: line 1: 86592 Aborted                 (core dumped) JULIA_BINDIR=/m100/home/userexternal/jdavid00/jlatest/julia/usr/bin WINEPATH="/m100/home/userexternal/jdavid00/jlatest/julia/usr/bin;$WINEPATH" JULIA_NUM_THREADS=1 /m100/home/userexternal/jdavid00/jlatest/julia/usr/bin/julia -O3 -C "native" --output-o /m100/home/userexternal/jdavid00/jlatest/julia/usr/lib/julia/sys-o.a.tmp --startup-file=no --warn-overwrite=yes --sysimage /m100/home/userexternal/jdavid00/jlatest/julia/usr/lib/julia/sys.ji /m100/home/userexternal/jdavid00/jlatest/julia/contrib/generate_precompile.jl 1
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
make[1]: *** [sysimage.mk:89: /m100/home/userexternal/jdavid00/jlatest/julia/usr/lib/julia/sys-o.a] Error 1
make: *** [Makefile:88: julia-sysimg-release] Error 2

real    19m59.439s
user    12m45.828s
sys     2m25.043s

[vchuravy]

@nemanjai do you have any insight here? This looks like something PPC internal that doesn't like the Julia address-spaces

[nemanjai]

I can look into what the requirements are for overloaded intrinsics. I suspect that LLVM doesn't like something about the encoding of the type in the name or something similar.
Can you please provide the LLVM IR coming out of Julia that I can try to compile statically to reproduce this?

[jdadavid]

Files

j180b3/julia-1.8.0-beta3/usr/lib/julia/corecompiler.ji
j180b3/julia-1.8.0-beta3/usr/lib/julia/sys.ji

are uploaded at https://filetransfer.io/data-package/IlZ1w9k4#link. Hope that is what you needed ?

[jdadavid]

FWIW, here are same for 1.9.0-dev

jlatest/julia/usr/lib/julia/corecompiler.ji
jlatest/julia/usr/lib/julia/sys.ji

jifiles-190.tar.xz is uploaded to https://filetransfer.io/data-package/KLGcgoiV

[vchuravy]

The .ji sadly don't contain bitcode or machine code so they are not what nemanjai is looking for.

Can you do make julia-sysimage-bc? That should generate a bitcode archive (hopefully before we hit the invalid IR)

[jdadavid]

Here are the files generated by make julia-sysimg-bc for both 1.8.0b3 and 1.9.0dev

https://filetransfer.io/data-package/WXX2zleZ

Hope that help !
many thanks,

[vchuravy]

Thanks this allows to reproduce independently.

xz -d sys-bc-180b3.a.xz
ar -x sys-bc-180b3.a
llc text.bc

llc text.bc 
Intrinsic has incorrect argument type!
void ({} addrspace(10)*)* @llvm.ppc.cfence.p10sl_s
Intrinsic has incorrect argument type!
void ({} addrspace(10)*)* @llvm.ppc.cfence.p10sl_s
Intrinsic has incorrect argument type!
void ({} addrspace(10)*)* @llvm.ppc.cfence.p10sl_s
in function julia_YY.9_25528
LLVM ERROR: Broken function found, compilation aborted!
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.	Program arguments: llc text.bc

[vchuravy]

@nemanjai used bugpoint to reduce to

; ModuleID = 'bugpoint-reduced-simplified.bc'
source_filename = "text"
target datalayout = "e-m:e-i64:64-n32:64-S128-v256:256:256-v512:512:512"
target triple = "powerpc64le-unknown-linux-gnu"

define hidden void @julia_YY.9_25528() #0 {
top:
  br i1 undef, label %err86, label %pass9

pass9:                                            ; preds = %top
  %0 = load atomic {} addrspace(10)*, {} addrspace(10)* addrspace(11)* undef acquire, align 8
  unreachable

err86:                                            ; preds = %top
  unreachable
}

attributes #0 = { "target-features" }

!llvm.module.flags = !{!0}

!0 = !{i32 2, !"Debug Info Version", i32 3}

[jdadavid]

Just tried to compile julia-1.8.0-rc1, the bug is not yet fixed ?

[vchuravy]

the bug is not yet fixed ?

No and there is no time-frame for when it will be.

[jdadavid]

Anyway, thanks for the update and the fine work.