Why?

Data-parallel functions like pmap and an a hypothetical tmap are not answers to everything, but they are answers to a lot of things. If we could at least support these kinds of primitives everywhere, including source-to-source AD like Zygote, then the parallelism needs of the 90% are solved and we can figure out later how to do the last 10%.

What?

There are a few things to do:

• Agree on how to support tmap, and possibly ptmap, i.e. EnsembleSplitThreads of DiffEq where it mixes multithreading on each process.
• Make it easier to put one process per node, i.e. a smart addprocs
• Have all of the ecosystem like Zygote support the whole set of primitives. This has started in support parallel maps (tmap, pmap, vmap, etc.) FluxML/Zygote.jl#728 .

Maybe if they could be linked better this could be easier? Like map(f,Parallel(),args...) or something, but it's probably too late for that so the low hanging fruit is just to do a different function for each.

For tmap, there's currently:

• https://github.com/mohamed82008/KissThreading.jl
• https://github.com/jw3126/ThreadingTools.jl

but both are not registered. There's nothing for ptmap.

@tkf might have some ideas.