Skip to content

Use map[string]struct{} for compression map in Len #820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 27, 2018

Conversation

tmthrgd
Copy link
Collaborator

@tmthrgd tmthrgd commented Nov 26, 2018

map[string]int requires 8 bytes per entry to store the unused position information.

I couldn't get any clean benchmarks from this, but map[string]struct{} definitely uses less memory than map[string]int.

This does unfortunately make the related test cases slightly weaker as they no longer test that the Len code is finding compression points in the correct spot.

Updates #709.

Edit: I was able to get some better benchmarks that show a very notable improvement in memory use.

$ benchstat {old,new}.bench
name                       old time/op    new time/op    delta
MsgLength-12                  329ns ± 0%     328ns ± 0%   -0.27%  (p=0.040 n=9+9)
MsgLengthNoCompression-12    9.37ns ± 1%    9.34ns ± 2%     ~     (p=0.099 n=10+10)
MsgLengthPack-12             2.26µs ± 9%    2.36µs ±23%     ~     (p=0.780 n=9+10)
MsgLengthMassive-12          63.1µs ± 8%    53.0µs ± 3%  -15.89%  (p=0.000 n=10+10)

name                       old alloc/op   new alloc/op   delta
MsgLength-12                  32.0B ± 0%     32.0B ± 0%     ~     (all equal)
MsgLengthNoCompression-12     0.00B          0.00B          ~     (all equal)
MsgLengthPack-12               896B ± 0%      896B ± 0%     ~     (all equal)
MsgLengthMassive-12          20.2kB ± 0%    14.8kB ± 0%  -26.74%  (p=0.000 n=8+9)

name                       old allocs/op  new allocs/op  delta
MsgLength-12                   1.00 ± 0%      1.00 ± 0%     ~     (all equal)
MsgLengthNoCompression-12      0.00           0.00          ~     (all equal)
MsgLengthPack-12               8.00 ± 0%      8.00 ± 0%     ~     (all equal)
MsgLengthMassive-12             138 ± 0%       138 ± 0%     ~     (all equal)

@tmthrgd tmthrgd requested a review from miekg November 26, 2018 22:20
@codecov-io
Copy link

codecov-io commented Nov 26, 2018

Codecov Report

Merging #820 into master will decrease coverage by 0.04%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #820      +/-   ##
==========================================
- Coverage   57.88%   57.84%   -0.05%     
==========================================
  Files          42       42              
  Lines       10824    10824              
==========================================
- Hits         6266     6261       -5     
- Misses       3473     3477       +4     
- Partials     1085     1086       +1
Impacted Files Coverage Δ
msg.go 77.93% <100%> (-0.65%) ⬇️
zcompress.go 26.24% <100%> (ø) ⬆️
server.go 68.44% <0%> (-0.25%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7e7488...1451211. Read the comment docs.

@tmthrgd tmthrgd mentioned this pull request Nov 26, 2018
@miekg
Copy link
Owner

miekg commented Nov 27, 2018

I would have sworn this length was somehow needed, guess not.

@miekg
Copy link
Owner

miekg commented Nov 27, 2018

sorry, went to the wrong way through the PRs, created a merge conflict /o\

@miekg
Copy link
Owner

miekg commented Nov 27, 2018

lgtm

map[string]int requires 8 bytes per entry to store the unused position
information.
@tmthrgd
Copy link
Collaborator Author

tmthrgd commented Nov 27, 2018

These were going to create a merge conflict regardless of ordering. I've manually rebased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants