A thread-safe rand wrapper that relies on a Threaded Array #27950

fisiognomico · 2018-07-05T22:01:05Z

This is an attempt to implement an additional utility in Random that generate random number safely in multi thread code. ( #10441 )
I choose to keep it as an external utility to avoid interfering in rand() performance for mono-thread programs. The independence of generators is guaranteed by randjump.
Something strange that I noticed is that testing this solution on commit fe4337613f0da442cf01c553c0a0a8b9dc8b66a2 (~25 days old master) gives on the trand() benchmarks

julia>@benchmark trand()
    memory estimate:  0 bytes
    allocs estimate:  0
    --------------
    minimum time:     5.395 ns (0.00% GC)
    median time:      6.527 ns (0.00% GC)
    mean time:        6.661 ns (0.00% GC)
    maximum time:     29.691 ns (0.00% GC)
    --------------
    samples:          10000
    evals/sample:     1000

Instead on the more recent commit 80db579cf938a75cbb1edf0dbe90fc27d85210ac (~2 days master)

julia>@benchmark trand()
    memory estimate:  16 bytes
    allocs estimate:  1
    --------------
    minimum time:     22.683 ns (0.00% GC)
    median time:      23.654 ns (0.00% GC)
    mean time:        31.057 ns (17.71% GC)
    maximum time:     38.199 μs (99.92% GC)
    --------------
    samples:          10000
    evals/sample:     996

I do not know what can justify such a degradation in performance.

EDIT : performance restored after 2nd commit.

vchuravy · 2018-07-05T23:43:52Z

stdlib/Random/src/RNGs.jl

@@ -582,3 +582,13 @@ function _randjump(mt::MersenneTwister, jumppoly::DSFMT.GF2X, len::Integer)
    end
    return mts
 end
+
+const TRNG = Array{MersenneTwister}


You probably want Vector or Array{MersenneTwister, 1} here.

vchuravy · 2018-07-05T23:45:55Z

stdlib/Random/src/RNGs.jl

+
+TRNG(r::MersenneTwister=GLOBAL_RNG) = TRNG(randjump(r, big(10)^20, Threads.nthreads()))
+
+function trand(r::Array{MersenneTwister}=ThreadRNG[])


vchuravy · 2018-07-05T23:49:27Z

stdlib/Random/src/RNGs.jl

+const TRNG = Array{MersenneTwister}
+const ThreadRNG = Ref{TRNG}()
+
+TRNG(r::MersenneTwister=GLOBAL_RNG) = TRNG(randjump(r, big(10)^20, Threads.nthreads()))


It confused me a bit that you wrote TRNG(...) since randjump returns Vector{MersenneTwister}. It might be good to remove the typealias (e.g. const TRNG = ...)

fisiognomico · 2018-07-08T18:17:08Z

After the changes the benchmarks are restored :

julia> versioninfo()
Julia Version 0.7.0-beta.118
Commit 8a6d62d6b4* (2018-07-05 21:52 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge)

julia> @benchmark trand()
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     5.534 ns (0.00% GC)
  median time:      6.363 ns (0.00% GC)
  mean time:        6.453 ns (0.00% GC)
  maximum time:     35.026 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

The test added is just to check up #10441.

yuyichao · 2018-07-08T21:07:42Z

stdlib/Random/src/Random.jl

@@ -248,6 +249,7 @@ function __init__()
        Base.showerror_nostdio(ex,
            "WARNING: Error during initialization of module Random")
    end
+    ThreadRNG[] = randjump(GLOBAL_RNG, big(10)^20, Threads.nthreads()) 


Note that this must be done on multiple threads instead of only one one thread.

Do you mean implementing something like resize_nthreads!() but that assigns to each buffer a different generator?

No I mean the engine has to be allocated on different threads. See the recent discourse thread about this. (On my phone, hard to paste link)

Yeah I had this discussion with @stevengj about placing TLS generators.
The problem is that

Using task_local_storage is impractical as they often get dereferenced.

Writing a interface to pthread_setspecifc() can be useless, as retrieving variables there can be slower than accessing an array. Also it would bring the problem of managing the keys at runtime.

Where also it would not probably benefit us for performance, as we have an overhead ~1ns with respect to rand, and not in entropy generation as the independence between streams is guaranteed by randjump.
The only remaining problem is false sharing which I will test today with perf.

No it has NOTHING to do with task_local_storage or pthread tls API. You just need to "allocated on different threads", i.e. let the thread allocate, that's it.

Where also it would not probably benefit us for performance

Oh, it'll absolutely kill you. Again, see the benchmarks in the discourse thread.

Ehm the discussion above was started by myself, and I did those benchmark (sorry I'm Gaetano :D ). If you refer to another one, please link it to me.

The problem is that the definition of trand() (that present in the benchmarks linked above) is done at runtime, and the ThreadRNG const array was also instantiated at runtime.
That's why there I had the opportunity to let a thread allocate the array of MersenneTwister.
Instead here, at stdlib, I have to pass through a reference to instantiate a global object at init.

Ehm the discussion above was started by myself

I never mention "discussion above", I said "discourse thread" which I didn't link since I was on my phone as explained above.
I don't think that thread is that hard to find. It's currently the second result when searching for "rand" and "thread" (sorted with date). https://discourse.julialang.org/search?q=rand%20thread%20order%3Alatest, the direct link is https://discourse.julialang.org/t/poor-performance-on-cluster-multithreading

ThreadRNG const array was also instantiated at runtime.

And speaking of that, I don't really understand why it has to be a ref of vector. Vector is mutable already.

Sorry, I thought about a previous thread.
OK I'm going to write an helper function to do that. Thanks for your support!

yuyichao · 2018-07-11T14:03:30Z

stdlib/Random/src/RNGs.jl

@@ -582,3 +582,19 @@ function _randjump(mt::MersenneTwister, jumppoly::DSFMT.GF2X, len::Integer)
    end
    return mts
 end
+
+const ThreadRNG = Vector{MersenneTwister}(undef, Threads.nthreads())


Resize in __init__

…readed array of MersenneTwister

fisiognomico · 2018-07-12T11:01:49Z

Hi @yuyichao ! Thanks again for you comments! I made the resize call at set_nthreads!() to mimic the behavior of resize_nthreads!(). Just to appoint some changes :
After rebase randjump(MT, steps, len) has been dismissed, so I had to change a bit the logic of generators instantiation. We still get independent streams (at least it seems from the naive test added), and the performance seems at the usual level.
The rebase did not went very well, so I have to squash the commits in about 2.
Anyway I had to pull out of Future the randjump(::MT, steps::Int), let me know if is not too early!

…t into Random. Implemented set_nthreads!() to allocate ThreadRNG at init.

stevengj · 2021-03-12T15:00:11Z

Is this closed by #32407?

vchuravy reviewed Jul 5, 2018

View reviewed changes

yuyichao reviewed Jul 8, 2018

View reviewed changes

kshyatt added the multithreading Base.Threads and related functionality label Jul 10, 2018

yuyichao requested changes Jul 11, 2018

View reviewed changes

fisiognomico added 2 commits July 12, 2018 11:22

Implemented trand(), a thread-safe rand() wrapper that relies on a th…

362ee25

…readed array of MersenneTwister

trand : got a better perfomance on bechmarks, added docs and tests

1f8b9c0

fisiognomico force-pushed the df/TRNG branch from 510d8d9 to f2652ab Compare July 12, 2018 09:38

Moved randjump(::MersenneTwister, ::steps) out of Future and placed i…

3e6756b

…t into Random. Implemented set_nthreads!() to allocate ThreadRNG at init.

fisiognomico force-pushed the df/TRNG branch from d80b25b to 3e6756b Compare July 12, 2018 23:34

rfourquet added the randomness Random number generation and the Random stdlib label Aug 27, 2018

vtjnash closed this Mar 12, 2021


		TRNG(r::MersenneTwister=GLOBAL_RNG) = TRNG(randjump(r, big(10)^20, Threads.nthreads()))

		function trand(r::Array{MersenneTwister}=ThreadRNG[])

Uh oh!

A thread-safe rand wrapper that relies on a Threaded Array #27950

A thread-safe rand wrapper that relies on a Threaded Array #27950

Uh oh!

Conversation

fisiognomico commented Jul 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fisiognomico commented Jul 8, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yuyichao Jul 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fisiognomico Jul 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fisiognomico Jul 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fisiognomico commented Jul 12, 2018

Uh oh!

stevengj commented Mar 12, 2021

Uh oh!

Uh oh!

fisiognomico commented Jul 5, 2018 •

edited

Loading

fisiognomico commented Jul 8, 2018 •

edited

Loading

yuyichao Jul 9, 2018 •

edited

Loading

fisiognomico Jul 9, 2018 •

edited

Loading

fisiognomico Jul 9, 2018 •

edited

Loading