Threads.nthreads() not a compile-time constant.

[MasonProtter]

Since the number of threads can't be changed without restarting julia, I find this behavior quite surprising:

julia> @code_llvm Threads.nthreads()
;  @ threadingconstructs.jl:19 within `nthreads'
define i64 @julia_nthreads_20335() {
top:
; ┌ @ pointer.jl:105 within `unsafe_load' @ pointer.jl:105
   %0 = load i32, i32* inttoptr (i64 140333758273848 to i32*), align 8
; └
; ┌ @ boot.jl:707 within `Int64'
; │┌ @ boot.jl:626 within `toInt64'
    %1 = sext i32 %0 to i64
; └└
  ret i64 %1
}

In general, I think it would be useful to do something like

const NUM_THREADS = Int(unsafe_load(cglobal(:jl_n_threads, Cint)))
nthreads() = NUM_THREADS

so that branches / dispatches which depend on the number of threads can be eliminated.

[KristofferC]

Code doesn't have to be recompiled just because you restart julia.

[MasonProtter]

I'm not sure I understand. Are you saying this causes problems with static compilation?

[yuyichao]

Making nthread a constant is going in the wrong direction so we should not have this. And yes this cause problems with compilation.

I also don't believe there are many valuable cases that benefits from a static dispatch on nthread.

[MasonProtter]

using @spawn or @threads induces a lot of overhead, so often one wants to do things like

if nthreads() > 1
    f(x)
else
    threaded_f(x)
end 

There are cases where even the presence of a @spawn in a branch that doesn't get hit prevents optimizations of a function body, so in a tight kernel you often want to separate those out into separate function calls. Because nthreads() gets recomputed each call, the above if block won't get elided.

If we eventually do make machinery for changing the number of threads at runtime, it should behave like function redefinition and require you to hit the global scope for it to take effect.

[yuyichao]

Assuming you if is not in a loop this won't matter. If it is then you should move it out.
The single thread overhead for @thread pretty much all come from #15276 apart from the cases that is fixed in a PR aready. Even if it is defined in your way it won't help anything.

If we eventually do make machinery for changing the number of threads at runtime, it should behave like function redefinition and require you to hit the global scope for it to take effect.

Absolutely NOT.

[JeffBezanson]

We really do not need more reasons to incur large latencies recompiling everything.

[sbromberger]

It would be nice to have a way to dispatch single/multithreaded functions without incurring a runtime penalty. Is there a suggested way to do this? (How are other packages managing it?)

[DilumAluthge]

Instead of dispatching on the exact value of Threads.nthreads(), would it be easier on the compiler to allow us to just dispatch on Threads.nthreads() == 1 versus Threads.nthreads() != 1?

[sbromberger]

@DilumAluthge that would be sufficient, I think. Basically, I'd like a pain-free way to do

foo(a, b) = nthreads > 1 ? foo(a, b, Val{true}()) : foo(a, b, Val{false}())
foo(a, b, ::Val{true}) = # threaded version
foo(a, b, ::Val{false}) = # unthreaded version

[JeffBezanson]

would it be easier on the compiler to allow us to just dispatch on Threads.nthreads() == 1 versus Threads.nthreads() != 1

No.

Is if nthreads() > 1 really so slow that it's a deal-breaker even for potentially-multithreaded code? I know @threads currently has too much overhead, but it's never going to have less overhead than simply checking nthreads(), so I don't see why that would be the limiting factor.

From a practical standpoint, and for your own planning purposes, I can tell you we are not going to implement this any time soon. If you really need to configure some code for threaded/non-threaded cases with zero overhead, I would use const threaded = true # or false.

[sbromberger]

Sorry for prolonging the thread, but in thinking about this, would a SimpleTrait be a good way to handle things?

@traitdef IsThreaded
@traitimpl IsThreaded <- nthreads() > 1

[yuyichao]

In terms of a easy to write way to do things that you shouldn't do, sure. In terms of a long term, acceptable and well defined solution, no.

edit: and this is assume there's some compile time things involved. If there's none, than it might be well defined but it will strictly be slower than a branch.

[sbromberger]

what's the most correct way to handle this (to dispatch based on threading capability)?