Faster AbstractArray hashing #39950
Merged
Faster AbstractArray hashing #39950
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…stractArray`: ``` julia> using StaticArrays, BenchmarkTools julia> a = @svector [1,2,3,4,5]; julia> @Btime hash($a, UInt(0)) 77.935 ns (0 allocations: 0 bytes) 0xdeb6d0657a261f74 julia> @Btime hash(AbstractArray, UInt(0)) 58.643 ns (0 allocations: 0 bytes) 0xc03f1dbe32103a9e ``` This replaces the hash of the objectid with a static randomly-generated number. Now: ``` julia> @Btime hash($a, UInt(0)) 18.580 ns (0 allocations: 0 bytes) 0x5e77b8bf73067ebd ``` and for a random `Float64` vector ``` julia> @Btime hash($a, UInt(0)) 29.031 ns (0 allocations: 0 bytes) 0x9a574d69612587eb ```
Member
|
No, I actually didn't realize that |
mbauman
approved these changes
Mar 8, 2021
Member
Author
|
Actually I might be wrong about that, I assumed it came from the pointer. |
Member
Not sure what's happening, but it seems to be preserved within a given precompile/sysimg. |
nalimilan
reviewed
Mar 8, 2021
Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>
nalimilan
approved these changes
Mar 8, 2021
Member
|
While we're speeding this up, would it make sense to change the behavior of hashing small arrays (<25 elements say) to hash all elements rather than |
Member
Author
|
If that's viable I'd be fine with it, but it's a different kind of change than the one here and should be a separate PR. |
ElOceanografo
pushed a commit
to ElOceanografo/julia
that referenced
this pull request
May 4, 2021
Previously, the `object_id` lookup for `hash(AbstractArray, h)` dominated the hashing time for `AbstractArray`: ``` julia> using StaticArrays, BenchmarkTools julia> a = @svector [1,2,3,4,5]; julia> @Btime hash($a, UInt(0)) 77.935 ns (0 allocations: 0 bytes) 0xdeb6d0657a261f74 julia> @Btime hash(AbstractArray, UInt(0)) 58.643 ns (0 allocations: 0 bytes) 0xc03f1dbe32103a9e ``` This replaces the hash of the objectid with a static randomly-generated number. Now: ``` julia> @Btime hash($a, UInt(0)) 18.580 ns (0 allocations: 0 bytes) 0x5e77b8bf73067ebd ``` and for a random `Float64` vector ``` julia> @Btime hash($a, UInt(0)) 29.031 ns (0 allocations: 0 bytes) 0x9a574d69612587eb ``` Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>
antoine-levitt
pushed a commit
to antoine-levitt/julia
that referenced
this pull request
May 9, 2021
Previously, the `object_id` lookup for `hash(AbstractArray, h)` dominated the hashing time for `AbstractArray`: ``` julia> using StaticArrays, BenchmarkTools julia> a = @svector [1,2,3,4,5]; julia> @Btime hash($a, UInt(0)) 77.935 ns (0 allocations: 0 bytes) 0xdeb6d0657a261f74 julia> @Btime hash(AbstractArray, UInt(0)) 58.643 ns (0 allocations: 0 bytes) 0xc03f1dbe32103a9e ``` This replaces the hash of the objectid with a static randomly-generated number. Now: ``` julia> @Btime hash($a, UInt(0)) 18.580 ns (0 allocations: 0 bytes) 0x5e77b8bf73067ebd ``` and for a random `Float64` vector ``` julia> @Btime hash($a, UInt(0)) 29.031 ns (0 allocations: 0 bytes) 0x9a574d69612587eb ``` Co-authored-by: Milan Bouchet-Valat <nalimilan@club.fr>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, hashing AbstractArrays begins with hashing the type itself:
hash(AbstractArray, h)This ends up using the
object_idfallback, and it turns out to dominate the hashing time for smallAbstractArrays:This replaces the hash of the objectid with a static randomly-generated
number. Now:
and for a random
Float64vectorHowever, I'm unsure whether in #26022 this design was deliberate in making the hashing vary across sessions. If so, perhaps we could generate a random number on Julia startup? I'd be grateful for feedback from @mbauman.