Fix cong implementation to be properly random and not just cycling. #55509
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vtjnash
reviewed
Aug 16, 2024
Comment on lines
+1314
to
1315
| if (max < 2) | ||
| return 0; |
There was a problem hiding this comment.
This seems biased against returning 1
There was a problem hiding this comment.
The old rng is also broken here, which is fun. But basically the issue is that it's a [0, n) interval. So to get 0 and 1 you pass in 2.
There was a problem hiding this comment.
Though I'll see if anyone that's using this is actually checking that.
…pen interval and not a closed one
|
@d-netto could you check the change I made to the GC code there. It looked to me that it was assuming that cong return 0 to max in a closed interval and that's not exactly correct. |
|
What matters here is that we want to generate a random number in 0, 1, ... markthreads - 1. It seems like the previous code was generating random numbers in [0, max], so that's why we have the markthreads - 1 in there. If the new implementation is generating random numbers in [0, max) then your change is fine. |
|
The old code was also open :) |
|
What about the: do {...} while (x > max)? |
|
Ah OK, there is a --max in there. |
|
The first line in the old implementation does max--. I think when I ported it from blogpost it came from it was there and wrong. |
oscardssmith
added
multithreading
|
I believe we should backport this as the consequences of the scheduler being non-random seems worse than the scheduler being a little slower? |
IanButterworth
added
backport 1.10
|
Do we have actual examples where the non-randomness causes things to break? |
|
Something along these lines? Maybe there's more to attribute the difference to though % julia +1.10.5 -t6
julia> foo() = @sync for _ in 1:Threads.nthreads()
Threads.@spawn for _ in 1:100000
yield()
end
end
foo (generic function with 1 method)
julia> using Chairmarks
julia> @b foo()
466.335 ms (33 allocs: 3.234 KiB, without a warmup)% julia +nightly -t6
julia> versioninfo()
Julia Version 1.12.0-DEV.1122
Commit 1e217279439 (2024-08-30 13:37 UTC)
...
julia> @b foo()
408.224 ms (33 allocs: 2.016 KiB, without a warmup) |
33 tasks
KristofferC
pushed a commit
that referenced
this pull request
Sep 12, 2024
…55509) This was found by @IanButterworth. It unfortunately has a small performance regression due to actually using all the rng bits
KristofferC
added a commit
that referenced
this pull request
Sep 17, 2024
Backported PRs: - [x] #55480 <!-- Fix push! for OffsetVectors, add tests for push! and append! on AbstractVector --> - [x] #55443 <!-- Add test for upper/lower/titlecase and fix call --> - [x] #55524 <!-- Set `.jl` sources as read-only during installation --> - [x] #55500 <!-- make jl_thread_suspend_and_get_state safe --> - [x] #55506 <!-- Fix indexing in _mapreducedim for OffsetArrays --> - [x] #55564 <!-- Empty out loaded_precompiles dict instead of asserting it's empty. --> - [x] #55567 <!-- Initialize threadpools correctly during sysimg build --> - [x] #55596 <!-- Fast bounds-check for CartesianIndex ranges --> - [x] #55605 <!-- Reroute Symmetric/Hermitian + Diagonal through triangular --> - [x] #55640 <!-- win: move stack_overflow_warning to the backtrace fiber --> - [x] #55715 <!-- Add precompile signatures to Markdown to reduce latency. --> - [x] #55593 <!-- Fix invalidations for FileIO --> - [x] #55555 <!-- Revert "Don't expose guard pages to malloc_stack API consumers" --> - [x] #55720 <!-- Fix `pkgdir` for extensions --> - [x] #55729 <!-- Avoid confounding compilation side effects of `@time_imports` --> - [x] #55718 <!-- Fix `@time_imports` extension recognition --> - [x] #55522 <!-- Fix tr for Symmetric/Hermitian block matrices --> Contains multiple commits, manual intervention needed: - [ ] #55509 <!-- Fix cong implementation to be properly random and not just cycling. --> Non-merged PRs with backport label: - [ ] #55641 <!-- fall back to slower stat filesize if optimized filesize fails --> - [ ] #55534 <!-- Set stdlib sources as read-only during installation --> - [ ] #55499 <!-- propagate the terminal's `displaysize` to the `IOContext` used by the REPL --> - [ ] #55458 <!-- Allow for generically extracting unannotated string --> - [ ] #55457 <!-- Make AnnotateChar equality consider annotations --> - [ ] #55453 <!-- Privatise the annotations API, for StyledStrings --> - [ ] #55355 <!-- relocation: account for trailing path separator in depot paths --> - [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows --> - [ ] #55169 <!-- `propertynames` for SVD respects private argument --> - [ ] #54457 <!-- Make `String(::Memory)` copy --> - [ ] #53957 <!-- tweak how filtering is done for what packages should be precompiled --> - [ ] #51479 <!-- prevent code loading from lookin in the versioned environment when building Julia --> - [ ] #50813 <!-- More doctests for Sockets and capitalization fix --> - [ ] #50157 <!-- improve docs for `@inbounds` and `Base.@propagate_inbounds` --> - [ ] #41244 <!-- Fix shell `cd` error when working dir has been deleted -->
KristofferC
added a commit
that referenced
this pull request
Sep 25, 2024
Backported PRs: - [x] #55773 <!-- Add compat entry for `Base.donotdelete` --> - [x] #41244 <!-- Fix shell `cd` error when working dir has been deleted --> - [x] #55795 <!-- fix #52986, regression in `@doc` of macro without REPL loaded --> - [x] #55829 <!-- [Dates] Make test more robust against non-UTC timezones --> - [x] #55641 <!-- fall back to slower stat filesize if optimized filesize fails --> - [x] #55744 <!-- fix #45494, error in ssa conversion with complex type decl --> - [x] #55783 <!-- use `inferencebarrier` instead of `invokelatest` for 1-arg `@assert` --> - [x] #55739 <!-- Add `invokelatest` barrier to `string(...)` in `@assert` --> Need manual backport: - [ ] #55798 <!-- Broadcast binary ops involving strided triangular --> Contains multiple commits, manual intervention needed: - [ ] #55509 <!-- Fix cong implementation to be properly random and not just cycling. --> - [ ] #55569 <!-- Add a docs section about loading/precomp/ttfx time tuning --> - [ ] #55824 <!-- Replace regex package module checks with actual code checks --> Non-merged PRs with backport label: - [ ] #55845 <!-- privatize annotated string API, take two --> - [ ] #55828 <!-- Fix some corner cases of `isapprox` with unsigned integers --> - [ ] #55813 <!-- Check for conflicting `@ccallable` name before JIT registration --> - [ ] #55743 <!-- doc: heap snapshot viewing --> - [ ] #55741 <!-- Change annotations to use a NamedTuple --> - [ ] #55534 <!-- Set stdlib sources as read-only during installation --> - [ ] #55499 <!-- propagate the terminal's `displaysize` to the `IOContext` used by the REPL --> - [ ] #55458 <!-- Allow for generically extracting unannotated string --> - [ ] #55457 <!-- Make AnnotateChar equality consider annotations --> - [ ] #55453 <!-- Privatise the annotations API, for StyledStrings --> - [ ] #55355 <!-- relocation: account for trailing path separator in depot paths --> - [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows --> - [ ] #55169 <!-- `propertynames` for SVD respects private argument --> - [ ] #54457 <!-- Make `String(::Memory)` copy --> - [ ] #53957 <!-- tweak how filtering is done for what packages should be precompiled --> - [ ] #51479 <!-- prevent code loading from lookin in the versioned environment when building Julia --> - [ ] #50813 <!-- More doctests for Sockets and capitalization fix --> - [ ] #50157 <!-- improve docs for `@inbounds` and `Base.@propagate_inbounds` -->
KristofferC
added a commit
that referenced
this pull request
Oct 1, 2024
Backported PRs: - [x] #55849 <!-- Mmap: fix grow! for non file IOs --> - [x] #55863 <!-- Update TaskLocalRNG docstring according to #49110 --> - [x] #54433 <!-- Root globals in toplevel exprs --> - [x] #55828 <!-- Fix some corner cases of `isapprox` with unsigned integers --> - [x] #55890 <!-- Profile: fix order of fields in heapsnapshot & improve formatting --> - [x] #55884 <!-- inference: add missing `TypeVar` handling for `instanceof_tfunc` --> - [x] #55881 <!-- Install terminfo data under /usr/share/julia --> - [x] #55909 <!-- do not intentionally suppress errors in precompile script from being reported or failing the result --> - [x] #55355 <!-- relocation: account for trailing path separator in depot paths --> - [x] #55917 <!-- fix rawbigints OOB issues --> - [x] #55892 <!-- TOML: Avoid type-pirating `Base.TOML.Parser` --> - [x] #55798 <!-- Broadcast binary ops involving strided triangular --> - [x] #55919 <!-- Limit `@inbounds` to indexing in the dual-iterator branch in `copyto_unaliased!` --> Contains multiple commits, manual intervention needed: - [ ] #54009 <!-- allow extensions to trigger from packages in [deps] --> - [ ] #55509 <!-- Fix cong implementation to be properly random and not just cycling. --> - [ ] #55569 <!-- Add a docs section about loading/precomp/ttfx time tuning --> - [ ] #55824 <!-- Replace regex package module checks with actual code checks --> Non-merged PRs with backport label: - [ ] #55932 <!-- REPL: make UndefVarError aware of imported modules --> - [ ] #55910 <!-- Prevent extensions from blocking parallel pre-compilation --> - [ ] #55908 <!-- add logic to prefer loading modules that are already loaded --> - [ ] #55886 <!-- irrationals: restrict assume effects annotations to known types --> - [ ] #55871 <!-- lowering: don't reverse handler order in `(pop-handler-list ...)` --> - [ ] #55870 <!-- fix infinite recursion in `promote_type` for `Irrational` --> - [ ] #55867 <!-- update `hash` doc string: `widen` not required any more --> - [ ] #55851 <!-- [REPL] Fix #55850 by using `safe_realpath` instead of `abspath` in `projname` --> - [ ] #55813 <!-- Check for conflicting `@ccallable` name before JIT registration --> - [ ] #55743 <!-- doc: heap snapshot viewing --> - [ ] #55741 <!-- Change annotations to use a NamedTuple --> - [ ] #55534 <!-- Set stdlib sources as read-only during installation --> - [ ] #55499 <!-- propagate the terminal's `displaysize` to the `IOContext` used by the REPL --> - [ ] #55458 <!-- Allow for generically extracting unannotated string --> - [ ] #55457 <!-- Make AnnotateChar equality consider annotations --> - [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows --> - [ ] #55169 <!-- `propertynames` for SVD respects private argument --> - [ ] #54457 <!-- Make `String(::Memory)` copy --> - [ ] #53957 <!-- tweak how filtering is done for what packages should be precompiled --> - [ ] #51479 <!-- prevent code loading from lookin in the versioned environment when building Julia --> - [ ] #50813 <!-- More doctests for Sockets and capitalization fix --> - [ ] #50157 <!-- improve docs for `@inbounds` and `Base.@propagate_inbounds` -->
KristofferC
removed
backport 1.10
|
Not clear why the backport labels were removed here. |
|
Also, the backport PR commits including references to PRs that weren't included makes it much harder to figure out whether something was actually backported. @KristofferC |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was found by @IanButterworth. It unfortunately has a small performance regression due to actually using all the rng bits