Illegal instruction during sort (related to package precompilation)

[maleadt]

The following MWE is reduced from a 1.9-only failure with the package SurrealNumbers.jl, as seen on PkgEval: https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2023-01/22/SurrealNumbers.primary.log. It relies on package precompilation, and can easily be reproduced using a single package:

SurrealNumbers.jl

module SurrealNumbers

struct SurrealFinite <: Real end

sort(SurrealFinite[])
Base.:(<)(::SurrealFinite, ::SurrealFinite) = 0

end

The definition of < after the toplevel sort invocation is key to reproducing this issue.

mwe.jl

using SurrealNumbers
x0 = x1 = SurrealNumbers.SurrealFinite()
sort([x1, x0])

JULIA_LOAD_PATH=. julia mwe.jl

Unreachable reached at 0x7f53440c5175

[4193087] signal (4.2): Illegal instruction
in expression starting at /home/tim/Julia/tools/creduce/main.jl:3
_sort! at ./sort.jl:747 [inlined]
_sort! at ./sort.jl:712 [inlined]
_sort! at ./sort.jl:659 [inlined]
_sort! at ./sort.jl:595 [inlined]
#sort!#28 at ./sort.jl:1373 [inlined]
sort! at ./sort.jl:1366 [inlined]
#sort#29 at ./sort.jl:1399
sort at ./sort.jl:1399

Bisected to #47924 (cc @timholy):

1f0700a29a4e0250c5c31cbc02e624009d1ed741 is the first bad commit
commit 1f0700a29a4e0250c5c31cbc02e624009d1ed741
Author: Tim Holy <tim.holy@gmail.com>
Date:   Tue Dec 20 08:30:20 2022 -0600

    Precompile cache: always add worklist CIs (#47924)

    We cache only those external CodeInstances that link back to the
    package being precompiled. Formerly we required a backedge;
    this PRs adds any whose `specTypes` could only link back to the
    package. This scoops up a few runtime-dispatched CodeInstances
    and their callees.

 src/staticdata_utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[gbaraldi]

It seems this does work on master. So maybe we could backport the fix?
We do get a couple different errors though, so master working might be a red herring.
1.8

ERROR: LoadError: TypeError: non-boolean (Int64) used in boolean context
Stacktrace:
 [1] sort!
   @ ./sort.jl:505 [inlined]
 [2] sort!(v::Vector{SurrealNumbers.SurrealFinite}, lo::Int64, hi::Int64, a::Base.Sort.QuickSortAlg, o::Base.Order.ForwardOrdering)
   @ Base.Sort ./sort.jl:571
 [3] sort!
   @ ./sort.jl:661 [inlined]
 [4] #sort!#8
   @ ./sort.jl:722 [inlined]
 [5] sort!
   @ ./sort.jl:704 [inlined]
 [6] #sort#9
   @ ./sort.jl:769 [inlined]
 [7] sort(v::Vector{SurrealNumbers.SurrealFinite})
   @ Base.Sort ./sort.jl:769
 [8] top-level scope
   @ ~/Issues/48391/mwe.jl:3
in expression starting at /Users/gabrielbaraldi/Issues/48391/mwe.jl:3

master

ERROR: LoadError: < not defined for SurrealNumbers.SurrealFinite
Stacktrace:
 [1] error(::String, ::String, ::Type)
   @ Base ./error.jl:44
 [2] no_op_err(name::String, T::Type)
   @ Base ./promotion.jl:485
in expression starting at /Users/gabrielbaraldi/Issues/48391/mwe.jl:3

[maleadt]

Let me bisect the fix then.

EDIT: so #48309 "fixed" this, but I don't think this is the complete story, as the lastest PkgEval report still has SurrealNumbers.jl crashing. Probably that PR broke my MWE, so I'll try reducing again on latest master.

EDIT 2: Hmm, I can't reproduce this on master, so we should definitely backport the fix (but it's labeled as such already). Very strange how the issue still happens on Nanosoldier though, I'll have to investigate that.

[maleadt]

Regardless of this not reproducing exactly on master, this pattern did work on 1.8, so it still seems like a regression of some sorts. @vtjnash can probably elaborate whether this should work, but it does seem like a reasonable pattern.

[maleadt]

I was told that it was confusing that the reduced example errored on 1.8, so here's a version of the MWE that doesn't:

SurrealNumbers.jl

module SurrealNumbers
struct SurrealFinite <: Real end
sort(SurrealFinite[])
Base.:(<)(x::SurrealFinite, y::SurrealFinite) = false
end

mwe.jl

using SurrealNumbers
x = SurrealNumbers.SurrealFinite()
sort([x,x])

❯ JULIA_LOAD_PATH=. julia +1.8 main.jl 
❯ echo $?
0

❯ JULIA_LOAD_PATH=. julia +1.9 main.jl
Unreachable reached at 0x7f28442ff1a5
[3917234] signal (4.2): Illegal instruction

❯ JULIA_LOAD_PATH=. julia +dev main.jl
ERROR: LoadError: < not defined for SurrealNumbers.SurrealFinite

@vtjnash I hope everything is clear now?