Regression in workload where task is too busy to GC, from 1.10 to 1.11

[jakobnissen]

I have some example code which I use to demonstrate how a simple multithreaded workload scales. I found that it inexplicably scales much worse than expected in Julia 1.11 and today's master compared to Julia 1.10.

Example code (click to expand)

function copy_odds_branches!(dst, src)
    write_index = 0
    @inbounds for i in eachindex(src)
        v = src[i]
        if isodd(v)
            write_index += 1
            dst[write_index] = v
        end
    end
    return write_index
end

function frequent_cache_misses()
    src = rand(UInt8, 10_000_000)
    dst = similar(src)
    x = 0
    for i in 1:40
        x += copy_odds_branches!(dst, src)
    end
    x
end

function parallel_run(n_jobs)
    jobs = Task[]
    for _ in 1:n_jobs
        push!(jobs, Threads.@spawn frequent_cache_misses())
    end
    return sum(i -> fetch(i)::Int, jobs)
end

let
    parallel_run(1);
    for njobs in (1, 4, 8)
        @time parallel_run(njobs);
    end
end

This should be run in a Julia session with 8 threads available.
Here are the timings for difference versions of Julia:

1.10:
  0.972577 seconds (11 allocations: 19.074 MiB)
  1.063923 seconds (38 allocations: 76.297 MiB, 0.56% gc time)
  1.249448 seconds (74 allocations: 152.594 MiB)

1.11:
  1.023349 seconds (12 allocations: 19.074 MiB)
  2.079489 seconds (42 allocations: 76.296 MiB, 1.40% gc time)
  4.187955 seconds (82 allocations: 152.593 MiB, 0.67% gc time)

1.12 (a few day's old master)
  0.982807 seconds (12 allocations: 19.076 MiB, 1.53% gc time)
  1.999636 seconds (42 allocations: 76.296 MiB, 1.06% gc time)
  4.007706 seconds (82 allocations: 152.591 MiB, 1.55% gc time)

[gbaraldi]

This is a stop the world regression. We run a couple more GCs, but we can't run a GC in this workload until you finish the task.

[jakobnissen]

Adding a yield() point in frequent_cache_misses indeed does fix the issue. I suppose this is user error then - a multithreaded allocating workload without yield points will just be slow as long as Julia has a stop the world GC.

[oscardssmith]

technically we could and probably should add safepoints on backward jumps, but should so would have an unfortunately large performance impact unless we go the GraalVM approach and teach LLVM about how to safely delete them.

[bbrehm]

The appropriate thing is GC.safepoint(), not yield().

Unfortunately llvm fails to optimize that right -- consider

julia> function copy_odds_branches!(dst, src)
           #GC.safepoint() # this one is cheap
           write_index = 0
           @inbounds for i in eachindex(src)
             (i-1) & ((1<<24)-1) == 0 && GC.safepoint() #t his one slows down the loop, presumably due to bad codegen :(
               v = src[i]
               if isodd(v)
                   write_index += 1
                   dst[write_index] = v
               end
           end
           return write_index
       end

Whether the safepoint is placed on top of the loop or inside makes a timing difference from 1.0 to 1.3 seconds for the single job. Either way, only a single safepoint is triggered since we only have 1e7 < (1<<24) many array elements.

So obviously codegen failed to properly split the loop (absolutely no understanding of safepoints is required).

That is pretty annoying, because ultimately we must have a safepoint every so-and-so many loop iterations!

[gbaraldi]

What we want is to detect a loop like this and mark ourselves as safe to run GC concurrently to.

[jakobnissen]

In going to close this, because it's not really actionable.

• I'm not convinced this situation is worth optimizing for. The user should either make their code yield, or mark it as GC safe. The GC should, IMO, be tuned to optimize for other situations.
• Marking it GC safe was enabled in Allow for :foreigncall to transition to GC safe automatically #49933
• The proposal in Regression in workload where task is too busy to GC, from 1.10 to 1.11 #57118 (comment) is interesting (it would also allow something like autovectorizing short-circuiting functions like any and all), but out of scope for this issue.