The per-thread empty!() call in compile() is broken on julia master. #429
Description
Julia master recently merged JuliaLang/julia#49094, which moved the interactive threadpool to come before the default threadpool.
This means that the threadids you get from iterating @threads :static no longer match up with the indexes.
For example, here is julia started with JULIA_NUM_THREADS=4,1 julia --proj:
julia> VERSION
v"1.10.0-DEV.1118"
julia> Threads.nthreads()
4
julia> Threads.maxthreadid()
5
julia> Threads.@threads :static for i in 1:Threads.nthreads()
@info i, Threads.threadid()
@assert Threads.threadid() === i "TimeZones.TZData.compile() must be called from the main, top-level Task."
end
[ Info: (1, 2)
[ Info: (3, 4)
[ Info: (4, 5)
[ Info: (2, 3)
ERROR: TaskFailedException
nested task error: AssertionError: TimeZones.TZData.compile() must be called from the main, top-level Task.
Stacktrace:
[1] macro expansion
@ ./REPL[5]:3 [inlined]
[2] (::var"#18#threadsfor_fun#4"{var"#18#threadsfor_fun#3#5"{UnitRange{Int64}}})(tid::Int64; onethread::Bool)
@ Main ./threadingconstructs.jl:163
[3] #18#threadsfor_fun
@ ./threadingconstructs.jl:130 [inlined]
[4] (::Base.Threads.var"#1#2"{var"#18#threadsfor_fun#4"{var"#18#threadsfor_fun#3#5"{UnitRange{Int64}}}, Int64})()
@ Base.Threads ./threadingconstructs.jl:108
...and 3 more exceptions.Using for i in 1:Threads.maxthreadid() doesn't work either, because it only iterates in the current thread pool:
julia> Threads.@threads :static for i in 1:Threads.maxthreadid()
@info i, Threads.threadid()
end
[ Info: (4, 4)
[ Info: (5, 5)
[ Info: (1, 2)
[ Info: (3, 3)
[ Info: (2, 2)(note there are two iterations with threadid() = 2.)
.........
It doesn't seem like there's any way to run a Threads.@threads over the interactive threads, from what I can see. Not 100% sure about that.
It might be that we need to either change the way this works so that instead of eagerly emptying all the caches, we do something more cooperative, like setting some "time to reset" bit for each thread, which is lazily checked on the first access on each thread and the cache is cleared before doing anything else? Or maybe triggering a channel that each thread has a task waiting on to clear its cache.
Or maybe we need to make a change in Julia to provide another possible schedule to @threads, like :interactive to force it to schedule over the interactive threadpool? :/
CC: @omus, @JeffBezanson