`Instruction does not dominate all uses!`, `call token (...) @llvm.julia.gc_preserve_begin` when testing KiteModels

[KristofferC]

Happens on master and backports-release-1.9.

pkg> add KiteModels; test KiteModels
...
Instruction does not dominate all uses!
  %501 = call token (...) @llvm.julia.gc_preserve_begin({} addrspace(10)* %853), !dbg !619
  call void @llvm.julia.gc_preserve_end(token %501), !dbg !619
Instruction does not dominate all uses!
  %501 = call token (...) @llvm.julia.gc_preserve_begin({} addrspace(10)* %853), !dbg !619
  call void @llvm.julia.gc_preserve_end(token %501), !dbg !619
Assertion failed: (!verifyFunction(F, &errs())), function runOnFunction, file llvm-alloc-opt.cpp, line 1185.

[36030] signal (6): Abort trap: 6
in expression starting at /Users/kristoffercarlsson/.julia/packages/KiteModels/yqvYU/test/test_kps4.jl:7
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 31124682 (Pool: 31106186; Big: 18496); GC: 41

[staticfloat]

I can't reproduce this on linux64; I rebuilt the tip of release-1.9 (now that backports-release-1.9 is merged) and master and both pass the KiteMotels tests. I will re-try on macOS; were you testing on x86_64 or aarch64?

[gbaraldi]

This needs LLVM_ASSERTIONs I imagine. Do you have them on?

[gbaraldi]

I can reproduce, while bisecting the other issue I found that at least e50b32f is still bad.

[gbaraldi]

Instruction does not dominate all uses!
  %380 = call token (...) @llvm.julia.gc_preserve_begin({} addrspace(10)* %629), !dbg !506
  call void @llvm.julia.gc_preserve_end(token %380), !dbg !506
julia-debug: /workspace/srcdir/llvm-project/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp:10702: bool llvm::LoopVectorizePass::processLoop(llvm::Loop*): Assertion `!verifyFunction(*L->getHeader()->getParent(), &dbgs())' failed.

Loop Vec throws the assertion

[vtjnash]

I have trouble getting a stable reproducer, but may at least have one for opt here. The context is:

Instruction does not dominate all uses!
  %501 = call token (...) @llvm.julia.gc_preserve_begin({} addrspace(10)* %853), !dbg !619
  call void @llvm.julia.gc_preserve_end(token %501), !dbg !619
Instruction does not dominate all uses!
  %501 = call token (...) @llvm.julia.gc_preserve_begin({} addrspace(10)* %853), !dbg !619
  call void @llvm.julia.gc_preserve_end(token %501), !dbg !619

define void @"julia_clear!_2667"({} addrspace(10)* noundef nonnull align 8 dereferenceable(640) %0) #0 !dbg !4 {

KiteModels.clear!(KiteModels.KPS4{Float64, StaticArraysCore.MArray{Tuple{3}, Float64, 1, 3}, 11, 15, KiteModels.Spring{Int16, Float64}},)

pre48918.txt before some pass (after JuliaLICM)
post48918.txt after some pass (before AllocOpt)
diff48918.txt partial diff, excluding the lines with the bug

Note how call void @llvm.julia.gc_preserve_end(token %473) from L867 is present in L893, L936.loopexit, and oob105 before and after. It seems that some pass mucked up the CFG without taking this instruction into account (since putting it in a PhiNode, to make it legal to change the CFG, would have itself been illegal). You can see the change with opt -view-cfg between these inputs.

[gbaraldi]

Do we need to add something to gc_preserve_begin to tell it not move it like that?

[vtjnash]

Note that I am suspicious of the LCSSA pass, since it explicitly handles tokens by violating their SSA-ness property, and is the pass that was running when our IR got corrupted, but I cannot get it to fail in isolation with opt -lcssa (it runs inside the irce pass)
(https://github.com/llvm/llvm-project/blob/bcebadeba75f4c05e9879607a89ac30d892fdcad/llvm/lib/Transforms/Utils/LCSSA.cpp#L384-L389)

[gbaraldi]

For me this needs --check-bounds=yes to reproduce.

using KiteModels
s = KPS4{Float64, MVector{3, Float64}, 11,15, KiteModels.Spring{Int16, Float64}}()
clear!(s)

[gbaraldi]

Reduced even futher to

using StaticArrays
forces = zeros(SVector{2,MVector{1, Float64}})
function myclear2!(forces, segments)
    for i in 1:segments
        forces[i] .= zeros(1)
    end
end