-
Notifications
You must be signed in to change notification settings - Fork 103
Add HashMap.insertWithKey. #177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add HashMap.insertWithKey. #177
Conversation
This function is present in the container's Map interface, this brings more consistency to the HashMap interface so that it can more easily be used as a drop in replacement for Data.Map if desired. This partially addresses haskell-unordered-containers#172.
|
Currently working on benchmarks, there aren't any existing benchmarks for |
| pInsertWithKey :: Key -> [(Key, Int)] -> Bool | ||
| pInsertWithKey k = M.insertWithKey f k 1 `eq_` HM.insertWithKey f k 1 | ||
| where f (K k') a b = if k' >= 0 then a else b | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neither this nor the pre-existing pInsertWith is particularly nice. They both need a lot of thinking to decide if they test enough, and pInsertWith definitely does not. I think we want something like
pInsertWithKey :: Fun (Key, A, A) A -> Key -> A -> [(Key, A)] -> BoolThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated both of them to use Fun.
tests/Strictness.hs
Outdated
| | otherwise = isBottom $ HM.insertWith f k bottom m | ||
|
|
||
| pInsertWithKeyKeyStrict :: (Key -> Int -> Int -> Int) -> Int -> HashMap Key Int -> Bool | ||
| pInsertWithKeyKeyStrict f v m = isBottom $ HM.insertWithKey f bottom v m |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use Fun (Key, Int, Int) Int.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, although it complained about instances for Key so I used Int as the first param and used unK to pull out the int. I can change to Key and add necessary instances if you'd like.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops, thought these comments were for HashMapProperties, changing here as well.
|
|
||
| pInsertWithKeyValueStrict :: (Key -> Int -> Int -> Int) -> Key -> Int -> HashMap Key Int | ||
| -> Bool | ||
| pInsertWithKeyValueStrict f k v m |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same deal here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
We use apply instead of apply2 and apply3 because the latter two are only available in recent QuickCheck versions. We also use (Int, Int, Int) -> Int for the function for insertWithKey so we don't need to add additional instances to Key to work with `Fun`.
treeowl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you should probably use A for the values instead of Int, for clarity.
|
Missing what instances? Are those instances somewhere else? |
I may be using |
|
Oh, yeah, add the instance for sure. You may need a CoArbitrary one too.
On Jan 22, 2018 11:42 PM, "Matt Renaud" <notifications@github.com> wrote:
/home/mrenaud/projects/unordered-containers/tests/HashMapProperties.hs:321:9:
error:
• No instance for (Test.QuickCheck.Function.Function Key)
arising from a use of ‘testProperty’
• In the expression: testProperty "insertWithKey" pInsertWithKey
In the second argument of ‘testGroup’, namely
‘[testProperty "size" pSize, testProperty "member" pMember,
testProperty "lookup" pLookup, testProperty "insert" pInsert,
....]’
In the expression:
testGroup
"basic interface"
[testProperty "size" pSize, testProperty "member" pMember,
testProperty "lookup" pLookup, testProperty "insert" pInsert, ....]
I may be using Fun wrong, but that's the error when changing the first
tuple element from Int to Key. I'll keep playing around with it.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_S4brMtt-9DJTx3zidHvX_00SMK0ks5tNWMogaJpZM4RpAxf>
.
|
|
Actually, the CoArbitrary one is probably there already.
…On Jan 22, 2018 11:43 PM, "David Feuer" ***@***.***> wrote:
Oh, yeah, add the instance for sure. You may need a CoArbitrary one too.
On Jan 22, 2018 11:42 PM, "Matt Renaud" ***@***.***> wrote:
/home/mrenaud/projects/unordered-containers/tests/HashMapProperties.hs:321:9: error:
• No instance for (Test.QuickCheck.Function.Function Key)
arising from a use of ‘testProperty’
• In the expression: testProperty "insertWithKey" pInsertWithKey
In the second argument of ‘testGroup’, namely
‘[testProperty "size" pSize, testProperty "member" pMember,
testProperty "lookup" pLookup, testProperty "insert" pInsert,
....]’
In the expression:
testGroup
"basic interface"
[testProperty "size" pSize, testProperty "member" pMember,
testProperty "lookup" pLookup, testProperty "insert" pInsert, ....]
I may be using Fun wrong, but that's the error when changing the first
tuple element from Int to Key. I'll keep playing around with it.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_S4brMtt-9DJTx3zidHvX_00SMK0ks5tNWMogaJpZM4RpAxf>
.
|
|
Instance issue fixed, the insertWithKey value strictness check is failing though, I'll have to look more closely. |
|
You can't double curry like that--this is Haskell, not Idris!
…On Jan 23, 2018 12:09 AM, "Matt Renaud" ***@***.***> wrote:
Instance issue fixed, the insertWithKey value strictness check is failing
though, I'll have to look more closely.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_SJndnW9Z5J3WmN2hbzMuu0B9g4Rks5tNWlsgaJpZM4RpAxf>
.
|
|
Recent QuickCheck has apply2 and apply3 or some such; you can upgrade as
required.
…On Jan 23, 2018 12:10 AM, "David Feuer" ***@***.***> wrote:
You can't double curry like that--this is Haskell, not Idris!
On Jan 23, 2018 12:09 AM, "Matt Renaud" ***@***.***> wrote:
> Instance issue fixed, the insertWithKey value strictness check is failing
> though, I'll have to look more closely.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#177 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABzi_SJndnW9Z5J3WmN2hbzMuu0B9g4Rks5tNWlsgaJpZM4RpAxf>
> .
>
|
TODO: Fix pInsertWithKeyStrict. [ci skip]
1b1e73d to
824cca1
Compare
:( I'm having issues with dependency solving when changing the lower bound. I'll try again tomorrow. |
|
You can also just write the functions yourself. They're trivial. You'll
find them somewhere in the containers test suite. (I believe QuickCheck
snagged them from me!)
…On Jan 23, 2018 12:24 AM, "Matt Renaud" ***@***.***> wrote:
You can't double curry like that--this is Haskell, not Idris!
:(
I'm having issues with dependency solving when changing the lower bound.
I'll try again tomorrow.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_ZugsHrieXpcMOv8jv5W2YYJ7JxFks5tNWz9gaJpZM4RpAxf>
.
|
This function is only available in new versions of QuickCheck which we may not have for old GHC versions. This inlines the function here for portability. Skipping CI since there's still a known failure (pInsertWithKeyValueStrict). [ci skip]
insertWith is implemented using insertWithKey so other than noise they should
have similar results.
benchmarking HashMap/insertWith/String
time 1.932 ms (1.771 ms .. 2.124 ms)
0.954 R² (0.923 R² .. 0.983 R²)
mean 1.750 ms (1.664 ms .. 1.872 ms)
std dev 301.9 μs (227.4 μs .. 410.5 μs)
variance introduced by outliers: 87% (severely inflated)
benchmarking HashMap/insertWith/ByteString
time 2.272 ms (2.194 ms .. 2.347 ms)
0.978 R² (0.951 R² .. 0.994 R²)
mean 1.840 ms (1.744 ms .. 1.962 ms)
std dev 331.1 μs (266.5 μs .. 486.0 μs)
variance introduced by outliers: 88% (severely inflated)
benchmarking HashMap/insertWith/Int
time 1.321 ms (1.241 ms .. 1.454 ms)
0.955 R² (0.932 R² .. 0.976 R²)
mean 1.365 ms (1.316 ms .. 1.420 ms)
std dev 178.3 μs (150.3 μs .. 230.6 μs)
variance introduced by outliers: 81% (severely inflated)
benchmarking HashMap/insertWithKey/String
time 1.783 ms (1.714 ms .. 1.900 ms)
0.959 R² (0.934 R² .. 0.983 R²)
mean 1.614 ms (1.526 ms .. 1.747 ms)
std dev 289.4 μs (217.9 μs .. 372.9 μs)
variance introduced by outliers: 89% (severely inflated)
benchmarking HashMap/insertWithKey/ByteString
time 1.647 ms (1.605 ms .. 1.688 ms)
0.992 R² (0.983 R² .. 0.997 R²)
mean 1.563 ms (1.524 ms .. 1.644 ms)
std dev 164.4 μs (85.16 μs .. 308.3 μs)
variance introduced by outliers: 71% (severely inflated)
benchmarking HashMap/insertWithKey/Int
time 1.313 ms (1.289 ms .. 1.335 ms)
0.994 R² (0.990 R² .. 0.997 R²)
mean 1.289 ms (1.270 ms .. 1.338 ms)
std dev 83.74 μs (50.75 μs .. 157.2 μs)
variance introduced by outliers: 49% (moderately inflated)
Skipping CI, known test failure being investigated. [ci skip]
4c20c88 to
e7f949a
Compare
|
Still getting this strictness property failure: It's weird because |
|
Oh duh, the Strict module has its own implementations of insertWithKey, it doesn't re-export the Base implementation. |
|
You still need to benchmark old vs. new implementations of |
|
But benchmarking is definitely preferred. |
|
Old implementation (code duplication): New implementation (in terms of insertWithKey): The differences appear to be due to noise. |
|
They do indeed. Can you try applying less noise?
On Jan 23, 2018 2:01 PM, "Matt Renaud" <notifications@github.com> wrote:
Old implementation (code duplication):
benchmarking HashMap/insertWith/String
time 2.124 ms (2.065 ms .. 2.187 ms)
0.990 R² (0.984 R² .. 0.994 R²)
mean 1.852 ms (1.778 ms .. 1.924 ms)
std dev 222.5 μs (191.1 μs .. 266.3 μs)
variance introduced by outliers: 75% (severely inflated)
benchmarking HashMap/insertWith/ByteString
time 2.396 ms (2.237 ms .. 2.554 ms)
0.975 R² (0.966 R² .. 0.985 R²)
mean 2.031 ms (1.922 ms .. 2.140 ms)
std dev 331.9 μs (264.6 μs .. 469.7 μs)
variance introduced by outliers: 84% (severely inflated)
benchmarking HashMap/insertWith/Int
time 1.252 ms (1.162 ms .. 1.347 ms)
0.976 R² (0.963 R² .. 0.993 R²)
mean 1.137 ms (1.108 ms .. 1.178 ms)
std dev 103.3 μs (73.73 μs .. 165.2 μs)
variance introduced by outliers: 67% (severely inflated)
New implementation (in terms of insertWithKey):
benchmarking HashMap/insertWith/String
time 1.932 ms (1.771 ms .. 2.124 ms)
0.954 R² (0.923 R² .. 0.983 R²)
mean 1.750 ms (1.664 ms .. 1.872 ms)
std dev 301.9 μs (227.4 μs .. 410.5 μs)
variance introduced by outliers: 87% (severely inflated)
benchmarking HashMap/insertWith/ByteString
time 2.272 ms (2.194 ms .. 2.347 ms)
0.978 R² (0.951 R² .. 0.994 R²)
mean 1.840 ms (1.744 ms .. 1.962 ms)
std dev 331.1 μs (266.5 μs .. 486.0 μs)
variance introduced by outliers: 88% (severely inflated)
benchmarking HashMap/insertWith/Int
time 1.321 ms (1.241 ms .. 1.454 ms)
0.955 R² (0.932 R² .. 0.976 R²)
mean 1.365 ms (1.316 ms .. 1.420 ms)
std dev 178.3 μs (150.3 μs .. 230.6 μs)
variance introduced by outliers: 81% (severely inflated)
The differences appear to be due to noise.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_RuvFp8YfjjVD4JGGcvDflukh10nks5tNix-gaJpZM4RpAxf>
.
|
Could you elaborate? I'm not sure exactly how I would do that, I've only used criterion a few times in the past. Is there an option to increase the number of runs? I went through the tutorial and some docs but didn't find anything. |
|
I mostly meant using a quieter system. It helps to use kill -STOP on your
web browser-related processes.
…On Jan 23, 2018 3:24 PM, "Matt Renaud" ***@***.***> wrote:
Can you try applying less noise?
Could you elaborate? I'm not sure exactly how I would do that, I've only
used criterion a few times in the past. Is there an option to increase the
number of runs? I went through the tutorial and some docs but didn't find
anything.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_bLkgPzIwC8RKAl7xwexIu7ipWOOks5tNj_6gaJpZM4RpAxf>
.
|
|
(You can kill -CONT afterwards to get them going again.)
…On Jan 23, 2018 3:28 PM, "David Feuer" ***@***.***> wrote:
I mostly meant using a quieter system. It helps to use kill -STOP on your
web browser-related processes.
On Jan 23, 2018 3:24 PM, "Matt Renaud" ***@***.***> wrote:
> Can you try applying less noise?
>
> Could you elaborate? I'm not sure exactly how I would do that, I've only
> used criterion a few times in the past. Is there an option to increase the
> number of runs? I went through the tutorial and some docs but didn't find
> anything.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#177 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABzi_bLkgPzIwC8RKAl7xwexIu7ipWOOks5tNj_6gaJpZM4RpAxf>
> .
>
|
Haha, gotcha :P |
|
Updated benchmark results. TL;DR; no difference. Old (code duplication): New (implemented via insertWithKey): |
|
Um... I'm sorry to do this to you after all that work, but I realized this
is another function that doesn't deserve to exist in containers either. :-(
…On Jan 23, 2018 6:31 PM, "Matt Renaud" ***@***.***> wrote:
Updated benchmark results:
*Old (code duplication):*
benchmarking HashMap/insertWith/String
time 1.635 ms (1.604 ms .. 1.665 ms)
0.996 R² (0.994 R² .. 0.998 R²)
mean 1.663 ms (1.636 ms .. 1.695 ms)
std dev 102.2 μs (81.08 μs .. 130.7 μs)
variance introduced by outliers: 46% (moderately inflated)
benchmarking HashMap/insertWith/ByteString
time 1.634 ms (1.610 ms .. 1.662 ms)
0.997 R² (0.995 R² .. 0.998 R²)
mean 1.643 ms (1.614 ms .. 1.666 ms)
std dev 90.85 μs (73.72 μs .. 110.7 μs)
variance introduced by outliers: 41% (moderately inflated)
benchmarking HashMap/insertWith/Int
time 1.060 ms (1.052 ms .. 1.069 ms)
0.999 R² (0.998 R² .. 0.999 R²)
mean 1.063 ms (1.054 ms .. 1.076 ms)
std dev 38.48 μs (29.12 μs .. 54.26 μs)
variance introduced by outliers: 25% (moderately inflated)
*New (implemented via insertWithKey):*
benchmarking HashMap/insertWith/String
time 1.687 ms (1.641 ms .. 1.729 ms)
0.994 R² (0.990 R² .. 0.997 R²)
mean 1.716 ms (1.681 ms .. 1.748 ms)
std dev 112.6 μs (91.79 μs .. 141.6 μs)
variance introduced by outliers: 49% (moderately inflated)
benchmarking HashMap/insertWith/ByteString
time 1.649 ms (1.621 ms .. 1.679 ms)
0.997 R² (0.996 R² .. 0.998 R²)
mean 1.650 ms (1.628 ms .. 1.672 ms)
std dev 69.74 μs (57.69 μs .. 87.52 μs)
variance introduced by outliers: 28% (moderately inflated)
benchmarking HashMap/insertWith/Int
time 1.081 ms (1.072 ms .. 1.090 ms)
0.999 R² (0.998 R² .. 0.999 R²)
mean 1.084 ms (1.073 ms .. 1.095 ms)
std dev 37.65 μs (30.82 μs .. 47.29 μs)
variance introduced by outliers: 23% (moderately inflated)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_RRGkMk0d0_kbjihRmbOQH6f0i4Eks5tNmn5gaJpZM4RpAxf>
.
|
Haha, no worries :)
I'm assuming because you can just write your function and partially apply the key? Map.insertWith (f k) k v m
where f k a b = ...I was kinda wondering why this was its own function since it didn't seem to do any internal optimization. |
|
Well, it avoids building a closure. But I really don't think that one
closure is worth the API clutter!
…On Jan 24, 2018 9:04 PM, "Matt Renaud" ***@***.***> wrote:
sorry to do this to you after all that work
Haha, no worries :)
doesn't deserve to exist in containers either
I'm assuming because you can just write your function and partially apply
the key?
Map.insert (f k) k v m
where f k a b = ...
I was kinda wondering why this was its own function since it didn't seem
to do any internal optimization.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_VR-MRIIzStgzvNe_OHu8mSmp4arks5tN-EkgaJpZM4RpAxf>
.
|
|
I completely agree. Can we deprecate and remove the function from the containers API?! |
|
That'll need a libraries proposal.
…On Jan 24, 2018 9:27 PM, "Matt Renaud" ***@***.***> wrote:
I completely agree. Can we deprecate and remove the function from the
containers API?!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#177 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABzi_YphNbS8fG8_TaOS9EWNtzZvXjXfks5tN-aRgaJpZM4RpAxf>
.
|
|
Closing this since we instead plan to deprecate this function in containers. |
This function is present in the container's Map interface, Map.insertWithKey, this brings more consistency to the
HashMapinterface so that it can more easily be used as a drop in replacement forData.Mapif desired.This partially addresses #172.