Wrong return for supertype() in functions for compiled sysimage with option--strip-ir

[kruxigt]

I compile a package to a sysimage with the options --strip-ir --compile=all
I have the two following functions in my package

function test_supertype(t::Type)
    return supertype(t)
end

function test_supertype_math(t::Type{T}) where T <: Number
    return supertype(t)
end

when using the sysimage I get

test_supertype(Int64) -> Signed (correct)
test_supertype_math(Int64) -> Integer (incorrect)

This only happens with the --strip-ir option. It seems like it could be a bug, or am I doing something wrong?

PackageCompiler v2.1.5
versioninfo()
Julia Version 1.10.0-beta3
Commit 404750f (2023-10-03 12:53 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (x86_64-apple-darwin22.4.0)
CPU: 8 × Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, icelake-client)

[vtjnash]

It sounds similar to #47476, but the fix #47667 (16d3b92) should have been in v1.8.3 and later

[vtjnash]

I tried to make a reproducer on master, but this seems to work. To help aid my understanding of what you are encountering, do you know what the contents of these caches look like?

typeof(test_supertype_math).name.mt.cache
typeof(test_supertype_math).name.mt.leafcache
only(methods(test_supertype_math)).specializations
only(methods(test_supertype_math)).unspecialized

---

JL_DLLEXPORT void jl_force_compile_unspec(jl_method_t *m)
{
    jl_method_instance_t *unspec = jl_get_unspecialized(m);
    jl_code_instance_t *ucache = jl_get_method_inferred(unspec, (jl_value_t*)jl_any_type, 1, ~(size_t)0);
    if (jl_atomic_load_relaxed(&ucache->invoke) == NULL)
        jl_generate_fptr_for_unspecialized(ucache);
}

$ julia -q --compile=no
julia> function test_supertype_math(t::Type{T}) where T <: Number
           return supertype(t)
       end

julia> ccall(:jl_force_compile_unspec, Cvoid, (Any,), which(test_supertype_math, (Type{Int},)))

julia> @which(test_supertype_math(Int)).source = nothing

julia> test_supertype_math(Int)
Signed

[kruxigt]

typeof(TFire.test_supertype_math).name.mt.cache returns nothing
typeof(TFire.test_supertype_math).name.mt.leafcache returns:

32-element Vector{Any}:
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
    Tuple{typeof(TFire.test_supertype_math), Type{Int64}}
    Core.TypeMapEntry(nothing, Tuple{typeof(TFire.test_supertype_math), Type{Int64}}, Tuple{typeof(TFire.test_supertype_math), DataType}, svec(), 0x0000000000007b44, 0xffffffffffffffff, MethodInstance for TFire.test_supertype_math(::Type{Int64}), false, true, false)

only(methods(TFire.test_supertype_math)).specializations returns (after having tried the function for Int64)
MethodInstance for TFire.test_supertype_math(::Type{Int64})

only(methods(TFire.test_supertype_math)).unspecialized returns

test_supertype_math(t::Type{T}) where T<:Number
     @ TFire ~/.julia/packages/TFire/itd5h/src/Core/AddLayers.jl:119

[vtjnash]

Hmm, those are the values I had expected. Do you get the same result without --strip-ir and with --compile=no instead at runtime?
It might also be helpful to print the contents of the .cache field of the two MethodInstance objects (specialized and unspecialized):

julia> only(methods(test_supertype_math)).specializations.cache
Core.CodeInstance(MethodInstance for test_supertype_math(::Type{Int64}), #undef, 0x0000000000001c14, 0xffffffffffffffff, Type{Signed}, Signed, nothing, 0x000010e0, 0x000010e0, nothing, false, true, 0x01, Ptr{Nothing} @0x00007f2a1c625670, Ptr{Nothing} @0x0000000000000000)

julia> Base._uncompressed_ir(unspecialized, unspecialized.inferred)

And if you have gdb, to look at disassemble 0x00007f2a1c625670 on whatever pointers you have in those caches too.

[kruxigt]

I tried them on the sysimage I had where I had used --compile=all but not --strip-ir. This sysimage seems to work as intended.

The only one where I notice a difference is only(methods(TFire.test_supertype_math)).unspecialized which in this case returns MethodInstance for TFire.test_supertype_math(::Type{T}) where T<:Number

Further. For the case of --compile=all (which works as intended) I get

julia> only(methods(TFire.test_supertype_math)).specializations.cache
Core.CodeInstance(MethodInstance for TFire.test_supertype_math(::Type{Int64}), #undef, 0x0000000000001b72, 0xffffffffffffffff, Type{Signed}, Signed, nothing, 0x00000ce0, 0x00000ce0, nothing, false, true, 0x01, Ptr{Nothing} @0x0000000102c1ead0, Ptr{Nothing} @0x0000000000000000)

And for the case of --compile=all --strip-ir (which gives the incorrect supertype) I get

julia> only(methods(TFire.test_supertype_math)).specializations.cache
Core.CodeInstance(MethodInstance for TFire.test_supertype_math(::Type{Int64}), #undef, 0x0000000000000001, 0xffffffffffffffff, Any, #undef, #undef, 0x00000000, 0x00000000, nothing, false, false, 0x00, Ptr{Nothing} @0x000000010e623ec0, Ptr{Nothing} @0x000000013041d430)

I don't have gdb and I'm not used to that kind of binary debugging but if you consider it essential and can guide me a bit I could give it a try.

[vtjnash]

Yeah, a gdb disassembly might be useful, since I can't quite guess what 2 pointers it put there, as I didn't quite expect these pair: Ptr{Nothing} @0x000000010e623ec0, Ptr{Nothing} @0x000000013041d430

[kruxigt]

I think debugging this problem with gdb is above my current "paygrade". I should probably try to narrow down the causes even more in terms of what in the Julia code makes the problem appear..?

Or if it's really simple using gdb for this and someone can guide me through it.

[kruxigt]

I can reproduce the issue on a clean project on Julia v1.10 rc1
Steps:
1, Generate new package "TestBuild"

2, Add the following functions

function test_supertype(t::Type)
    return supertype(t)
end

function test_supertype_math(t::Type{T}) where T <: Number
    return supertype(t)
end

3, Build systemimage with
a = Cmd(["--strip-ir", "--compile=all", "--strip-metadata"])
create_sysimage(["TestBuild"]; sysimage_path="TestBuild.so", sysimage_build_args=a)

4 Start Julia with sysimage: Julia -JTestBuild.so --startup-file=no

5, Try the functions:

julia> using .TestBuild
julia> TestBuild.test_supertype(Int64)
Signed
julia> TestBuild.test_supertype_math(Int64)
Integer

This seems like a fairly serious issue?

julia> versioninfo()
Julia Version 1.10.0-rc1
Commit 5aaa948 (2023-11-03 07:44 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: macOS (x86_64-apple-darwin22.4.0)
CPU: 8 × Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, icelake-client)
Threads: 11 on 8 virtual cores

[kruxigt]

I haven't seen this problem yet in Julia 1.11.1. Maybe it is fixed?

[vtjnash]

I think Jeff and/or Keno found an OOB read recently that might have caused this