`Internal error: during type inference` on v1.12

[nickrobinson251]

On a private package [edit: repro in comment below], with Julia v1.12 (7 day old master, at 1b0fba7) i see the following:

 Internal error: during type inference of
 (::TaskGroups.var"#_maybe_spawn_worker##0#_maybe_spawn_worker##1"{TaskGroups.TaskGroup})()
 Encountered unexpected error in runtime:
 MethodError(f=Base.string, args=(Expr(:call, :<=, :id, Expr(:call, :length, Expr(:., :compact, :(:pending_nodes)))),), world=0x0000000000001961)
 thread (4) jl_method_error_bare at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/gf.c:2507
 thread (4) jl_method_error at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/gf.c:2525
 thread (4) jl_f_throw_methoderror at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/builtins.c:587 [inlined]
 thread (4) jl_f_throw_methoderror at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/builtins.c:583
 thread (4) _assert_tostring at ./error.jl:249
 thread (4) unknown function (ip: 0x7fffbce66dd2) at (unknown file)
 thread (4) did_already_insert at ./../usr/share/julia/Compiler/src/ssair/passes.jl:373
 thread (4) process_phinode_values at ./../usr/share/julia/Compiler/src/ssair/ir.jl:1281
 thread (4) jfptr_process_phinode_values_98514.1 at /nix/store/72g8yzry2c5vfsa161p2j4xbigrwpgbm-julia-1.12.0-DEV/lib/julia/sys.so (unknown line)
 thread (4) process_node! at ./../usr/share/julia/Compiler/src/ssair/ir.jl:1629
 thread (4) iterate_compact at ./../usr/share/julia/Compiler/src/ssair/ir.jl:1937
 thread (4) iterate at ./../usr/share/julia/Compiler/src/ssair/ir.jl:1859 [inlined]
 thread (4) compact! at ./../usr/share/julia/Compiler/src/ssair/ir.jl:2136
 thread (4) run_passes_ipo_safe at ./../usr/share/julia/Compiler/src/optimize.jl:1013
 thread (4) run_passes_ipo_safe at ./../usr/share/julia/Compiler/src/optimize.jl:1027 [inlined]
 thread (4) optimize at ./../usr/share/julia/Compiler/src/optimize.jl:1002
 thread (4) jfptr_optimize_118359.1 at /nix/store/72g8yzry2c5vfsa161p2j4xbigrwpgbm-julia-1.12.0-DEV/lib/julia/sys.so (unknown line)
 thread (4) finish_nocycle at ./../usr/share/julia/Compiler/src/typeinfer.jl:203
 thread (4) jfptr_finish_nocycle_119415.1 at /nix/store/72g8yzry2c5vfsa161p2j4xbigrwpgbm-julia-1.12.0-DEV/lib/julia/sys.so (unknown line)
 thread (4) typeinf at ./../usr/share/julia/Compiler/src/abstractinterpretation.jl:4275
 thread (4) typeinf_ext at ./../usr/share/julia/Compiler/src/typeinfer.jl:1154
 thread (4) typeinf_ext_toplevel at ./../usr/share/julia/Compiler/src/typeinfer.jl:1228
 thread (4) jfptr_typeinf_ext_toplevel_119358.1 at /nix/store/72g8yzry2c5vfsa161p2j4xbigrwpgbm-julia-1.12.0-DEV/lib/julia/sys.so (unknown line)
 thread (4) jl_apply at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/julia.h:2248 [inlined]
 thread (4) jl_type_infer at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/gf.c:452
 thread (4) jl_compile_method_internal at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/gf.c:2971
 thread (4) _jl_invoke at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/gf.c:3465 [inlined]
 thread (4) ijl_apply_generic at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/gf.c:3673
 thread (4) jl_apply at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/julia.h:2248 [inlined]
 thread (4) start_task at /build/julia-v1.12.0-DEV+RAI-d8e121836bcd203fd304b278e19b5a76db4b811f/src/task.c:1277

which points at

julia/Compiler/src/ssair/passes.jl

Line 373 in 1b0fba7

@assert id <= length(compact.pending_nodes)

[nickrobinson251]

Reproduction is to run this file repro_traskgroups_internal_error.jl

e.g.

> julia +nightly --startup-file=no repro_traskgroups_internal_error.jl
VERSION = v"1.12.0-DEV.1893"
Internal error: during type inference of
(::TaskGroups.var"#_maybe_spawn_worker##0#_maybe_spawn_worker##1"{TaskGroups.TaskGroup})()
Encountered unexpected error in runtime:
...

[giordano]

Do you have a reference to a previous commit where this worked, to help with bisection?

[nickrobinson251]

It works on v1.11.3 (and v1.10.8)

[giordano]

Something closer to master would be most helpful 🙂

[nickrobinson251]

Yeah, sorry, i don't have anything closer 😞

[nickrobinson251]

bisect points to 6de6b46 (#55876)

[giordano]

CC @topolarity

[nickrobinson251]

Please can this be added to the v1.12 milestone?

[Keno]

I'm gonna assign @topolarity to this one since his change was pointed to in the rebase, but if @serenity4 has some time, maybe he can help out.

[mlechu]

Somewhat creduced program producing the same error (mind the undefined variables)

module TaskGroups
using Base.Threads: Condition

macro lock1(l, expr)
    quote
        temp = $(esc(l))
        lock(temp)
        try
            $(esc(expr))
        finally
            unlock(temp)
        end
    end
end

struct TaskGroup
    cond::Threads.Condition
    pq
    function TaskGroup()
        new(Condition(), [])
    end
end

function _maybe_spawn_worker(g)
    schedule(Task(_worker_task(g)))
end

function _try_run(tg_was_full::Bool = false)
end

function _worker_task(tg::TaskGroup)
    while true
        tg_was_full, task = @lock1 tg.cond begin
            if isempty(tg.pq)
                return nothing
            end
            q, m
        end
        _try_run(tg_was_full)
    end
end

_maybe_spawn_worker(TaskGroup())

end

[vtjnash]

slightly more reduced by hand

julia> function _worker_task()                                                                                                  
           while true                                                                                                           
               r = let                                                                                                          
               try                                                                                                              
                   if @noinline rand(Bool)                                                                                      
                       return nothing                                                                                           
                   end                                                                                                          
                   q, m                                                                                                                                                                                                                                          
               finally                                                                                                          
                   missing                                                                                                      
               end                                                                                                              
               end                                                                                                              
               r[1]::Bool                                                                                                       
           end                                                                                                                  
       end; code_typed(_worker_task, ())                                                                                        

fails with

ERROR: AssertionError: id <= length(compact.pending_nodes)
2 <= 0

with IncrementalCompact state

2  1 ── %1  = φ (#11 => Compiler.OldSSAValue(17))::Core.Const(nothing)      │
   │    %2  = φ (#11 => Compiler.OldSSAValue(18))::Bool                     │                                                   
   └───       φ (#11 => Compiler.OldSSAValue(21))::Bool                     │                                                   
   2 ──       nothing::Nothing                                              │                                                   
4  3 ── %5  = enter #12                                                     │                                                   
5  4 ── %6  =    invoke Main.rand(Main.Bool::Type{Bool})::Bool              │
   └───       goto #7 if not %6                                             │                                                                                                                                                                                    
6  5 ── %8  = $(Expr(:leave, :(%5)))                                        │
   6 ──       goto #9                                                       │                                                                                                                                                                                    
8  7 ──       Main.q::Any                                                   │
   │          Main.m::Any                                                   │
   └─── %12 = $(Expr(:leave, :(%5)))                                        │                                                                                                                                                                                    
   8 ──       nothing::Nothing                                              │
10 9 ┄─       φ (#6 => 1, #8 => -1)::Int64                                  │                                                                                                                                                                                    
   │          φ (#6 => Main.nothing, #8 => %1)::Core.Const(nothing)         │
   │          φ (#6 => true, #8 => %2)::Bool                                │
─────────────────────────────────────────────────────────────────────────────
   │    %19 = φ (#6 => Compiler.OldSSAValue(32), #8 => %11)::Any            │
   │          nothing::Tuple{Any, Any}                                      │
   │    %21 = φ (#6 => %4, #8 => true)::Bool                                │
   │    %22 =   builtin (%16 === 1)::Bool                                   │
   └───       goto #11 if not %22                                           │                                                                                                                                                                                    
   10 ─       $(Expr(:throw_undef_if_not, Symbol(""), :(%18)))::Any         │
   └───       return %17                                                    │
   11 ─       $(Expr(:throw_undef_if_not, Symbol(""), :(%21)))::Any         │
   │            builtin Core.typeassert(%19, Main.Bool)::Bool               │
   └───       goto #1                                                       │
   12 ┄          invoke Base.rethrow()::Union{}                             │                        
   └───       unreachable                                                   │

note that this is perhaps is a bit confusing, since inference thinks there is a loop-carried dependency on the value and nullstate of the return value slot, both of which start initialized to #undef, but dynamically this should be okay

the failing phinode was the forwarded-constructor-field that got assigned to r[1], so the bad Compiler.OldSSAValue(32) here probably should have been referencing this same particular phivalue, via another dominating phi in block 1 (similar to the others there)