Remove parsing allocs #55
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WIP: "Mesh" and "Numbers" regress significantly. Seems like a runtime sideeffect, but I will investigate. * Re-use the channel between runs. * Fix temp slice escaping in findStructuralIndices. Cleanup: Don't send params already in internalParsedJson. ``` benchmark old ns/op new ns/op delta BenchmarkParseSmall/copy-32 1095 753 -31.20% BenchmarkParseSmall/nocopy-32 1014 684 -32.53% BenchmarkParseMedium/copy-32 4359 3981 -8.67% BenchmarkParseMedium/nocopy-32 3672 3203 -12.77% BenchmarkParseLarge/copy-32 57643 56759 -1.53% BenchmarkParseLarge/nocopy-32 47546 45712 -3.86% BenchmarkParseApache_builds/copy-32 140467 131633 -6.29% BenchmarkParseApache_builds/nocopy-32 105274 96613 -8.23% BenchmarkParseCanada/copy-32 11454566 12444019 +8.64% BenchmarkParseCanada/nocopy-32 11440363 12444615 +8.78% BenchmarkParseCitm_catalog/copy-32 1499697 1388706 -7.40% BenchmarkParseCitm_catalog/nocopy-32 1209008 1176916 -2.65% BenchmarkParseGithub_events/copy-32 71298 69322 -2.77% BenchmarkParseGithub_events/nocopy-32 57038 54787 -3.95% BenchmarkParseGsoc_2018/copy-32 1281316 1272766 -0.67% BenchmarkParseGsoc_2018/nocopy-32 1026481 1011825 -1.43% BenchmarkParseInstruments/copy-32 302662 289939 -4.20% BenchmarkParseInstruments/nocopy-32 257394 244188 -5.13% BenchmarkParseMarine_ik/copy-32 12879480 13697518 +6.35% BenchmarkParseMarine_ik/nocopy-32 12551823 13322868 +6.14% BenchmarkParseMesh/copy-32 3825973 4120152 +7.69% BenchmarkParseMesh/nocopy-32 3838699 4116863 +7.25% BenchmarkParseMesh_pretty/copy-32 4310342 4594306 +6.59% BenchmarkParseMesh_pretty/nocopy-32 4290558 4595993 +7.12% BenchmarkParseNumbers/copy-32 741692 863226 +16.39% BenchmarkParseNumbers/nocopy-32 741221 864157 +16.59% BenchmarkParseRandom/copy-32 1044593 1019031 -2.45% BenchmarkParseRandom/nocopy-32 728212 727910 -0.04% BenchmarkParseTwitter/copy-32 604714 595169 -1.58% BenchmarkParseTwitter/nocopy-32 456703 448870 -1.72% BenchmarkParseTwitterEscaped/copy-32 994717 991081 -0.37% BenchmarkParseTwitterEscaped/nocopy-32 865551 865523 -0.00% BenchmarkParseUpdate_center/copy-32 741393 724639 -2.26% BenchmarkParseUpdate_center/nocopy-32 515773 499503 -3.15% BenchmarkJsonParserLarge/nocopy-32 59050 57776 -2.16% BenchmarkParseNumber/Pos/63bit-32 82.6 86.5 +4.68% BenchmarkParseNumber/Neg/63bit-32 84.4 88.7 +5.11% BenchmarkParseNumberFloat-32 61.5 65.8 +6.94% BenchmarkParseAtof64FloatGolang-32 32.4 32.7 +0.96% BenchmarkParseNumberFloatExp-32 70.4 71.9 +2.09% BenchmarkParseNumberBig-32 131 147 +11.73% BenchmarkParseNumberRandomBits-32 140 153 +9.67% BenchmarkParseNumberRandomFloats-32 122 123 +1.15% BenchmarkParseIntGolang/Pos/63bit-32 29.8 29.9 +0.60% BenchmarkParseIntGolang/Neg/63bit-32 30.1 30.4 +1.23% benchmark old MB/s new MB/s speedup BenchmarkParseSmall/copy-32 172.58 250.88 1.45x BenchmarkParseSmall/nocopy-32 186.38 276.26 1.48x BenchmarkParseMedium/copy-32 534.13 584.79 1.09x BenchmarkParseMedium/nocopy-32 634.04 726.72 1.15x BenchmarkParseLarge/copy-32 487.78 495.38 1.02x BenchmarkParseLarge/nocopy-32 591.36 615.09 1.04x BenchmarkParseApache_builds/copy-32 906.08 966.89 1.07x BenchmarkParseApache_builds/nocopy-32 1208.99 1317.37 1.09x BenchmarkParseCanada/copy-32 196.52 180.89 0.92x BenchmarkParseCanada/nocopy-32 196.76 180.89 0.92x BenchmarkParseCitm_catalog/copy-32 1151.70 1243.75 1.08x BenchmarkParseCitm_catalog/nocopy-32 1428.61 1467.57 1.03x BenchmarkParseGithub_events/copy-32 913.52 939.55 1.03x BenchmarkParseGithub_events/nocopy-32 1141.90 1188.83 1.04x BenchmarkParseGsoc_2018/copy-32 2597.20 2614.64 1.01x BenchmarkParseGsoc_2018/nocopy-32 3241.98 3288.94 1.01x BenchmarkParseInstruments/copy-32 728.03 759.97 1.04x BenchmarkParseInstruments/nocopy-32 856.06 902.36 1.05x BenchmarkParseMarine_ik/copy-32 231.64 217.81 0.94x BenchmarkParseMarine_ik/nocopy-32 237.69 223.94 0.94x BenchmarkParseMesh/copy-32 189.13 175.62 0.93x BenchmarkParseMesh/nocopy-32 188.50 175.76 0.93x BenchmarkParseMesh_pretty/copy-32 365.95 343.33 0.94x BenchmarkParseMesh_pretty/nocopy-32 367.63 343.20 0.93x BenchmarkParseNumbers/copy-32 202.41 173.91 0.86x BenchmarkParseNumbers/nocopy-32 202.54 173.72 0.86x BenchmarkParseRandom/copy-32 488.68 500.94 1.03x BenchmarkParseRandom/nocopy-32 701.00 701.29 1.00x BenchmarkParseTwitter/copy-32 1044.32 1061.07 1.02x BenchmarkParseTwitter/nocopy-32 1382.77 1406.90 1.02x BenchmarkParseTwitterEscaped/copy-32 565.40 567.47 1.00x BenchmarkParseTwitterEscaped/nocopy-32 649.77 649.79 1.00x BenchmarkParseUpdate_center/copy-32 719.16 735.78 1.02x BenchmarkParseUpdate_center/nocopy-32 1033.75 1067.42 1.03x BenchmarkJsonParserLarge/nocopy-32 476.16 486.66 1.02x benchmark old allocs new allocs delta BenchmarkParseSmall/copy-32 11 1 -90.91% BenchmarkParseSmall/nocopy-32 11 1 -90.91% BenchmarkParseMedium/copy-32 11 1 -90.91% BenchmarkParseMedium/nocopy-32 11 1 -90.91% BenchmarkParseLarge/copy-32 16 3 -81.25% BenchmarkParseLarge/nocopy-32 16 3 -81.25% BenchmarkParseApache_builds/copy-32 21 3 -85.71% BenchmarkParseApache_builds/nocopy-32 21 3 -85.71% BenchmarkParseCanada/copy-32 249 3 -98.80% BenchmarkParseCanada/nocopy-32 249 3 -98.80% BenchmarkParseCitm_catalog/copy-32 109 3 -97.25% BenchmarkParseCitm_catalog/nocopy-32 109 3 -97.25% BenchmarkParseGithub_events/copy-32 16 3 -81.25% BenchmarkParseGithub_events/nocopy-32 16 3 -81.25% BenchmarkParseGsoc_2018/copy-32 66 3 -95.45% BenchmarkParseGsoc_2018/nocopy-32 66 3 -95.45% BenchmarkParseInstruments/copy-32 32 3 -90.62% BenchmarkParseInstruments/nocopy-32 32 3 -90.62% BenchmarkParseMarine_ik/copy-32 466 3 -99.36% BenchmarkParseMarine_ik/nocopy-32 466 3 -99.36% BenchmarkParseMesh/copy-32 121 3 -97.52% BenchmarkParseMesh/nocopy-32 121 3 -97.52% BenchmarkParseMesh_pretty/copy-32 121 3 -97.52% BenchmarkParseMesh_pretty/nocopy-32 121 3 -97.52% BenchmarkParseNumbers/copy-32 27 3 -88.89% BenchmarkParseNumbers/nocopy-32 27 3 -88.89% BenchmarkParseRandom/copy-32 75 3 -96.00% BenchmarkParseRandom/nocopy-32 75 3 -96.00% BenchmarkParseTwitter/copy-32 52 3 -94.23% BenchmarkParseTwitter/nocopy-32 52 3 -94.23% BenchmarkParseTwitterEscaped/copy-32 52 3 -94.23% BenchmarkParseTwitterEscaped/nocopy-32 52 3 -94.23% BenchmarkParseUpdate_center/copy-32 57 3 -94.74% BenchmarkParseUpdate_center/nocopy-32 57 3 -94.74% BenchmarkJsonParserLarge/nocopy-32 50 37 -26.00% benchmark old bytes new bytes delta BenchmarkParseSmall/copy-32 664 16 -97.59% BenchmarkParseSmall/nocopy-32 664 16 -97.59% BenchmarkParseMedium/copy-32 664 16 -97.59% BenchmarkParseMedium/nocopy-32 664 16 -97.59% BenchmarkParseLarge/copy-32 796 75 -90.58% BenchmarkParseLarge/nocopy-32 792 71 -91.04% BenchmarkParseApache_builds/copy-32 959 115 -88.01% BenchmarkParseApache_builds/nocopy-32 931 86 -90.76% BenchmarkParseCanada/copy-32 37405 32413 -13.35% BenchmarkParseCanada/nocopy-32 37403 30778 -17.71% BenchmarkParseCitm_catalog/copy-32 6426 3189 -50.37% BenchmarkParseCitm_catalog/nocopy-32 5404 2402 -55.55% BenchmarkParseGithub_events/copy-32 801 81 -89.89% BenchmarkParseGithub_events/nocopy-32 793 73 -90.79% BenchmarkParseGsoc_2018/copy-32 17642 15540 -11.91% BenchmarkParseGsoc_2018/nocopy-32 9844 7733 -21.44% BenchmarkParseInstruments/copy-32 1303 196 -84.96% BenchmarkParseInstruments/nocopy-32 1256 143 -88.61% BenchmarkParseMarine_ik/copy-32 177273 176118 -0.65% BenchmarkParseMarine_ik/nocopy-32 173463 165752 -4.45% BenchmarkParseMesh/copy-32 14516 12118 -16.52% BenchmarkParseMesh/nocopy-32 14545 12117 -16.69% BenchmarkParseMesh_pretty/copy-32 11148 8389 -24.75% BenchmarkParseMesh_pretty/nocopy-32 11088 8421 -24.05% BenchmarkParseNumbers/copy-32 1238 286 -76.90% BenchmarkParseNumbers/nocopy-32 1237 283 -77.12% BenchmarkParseRandom/copy-32 4159 1966 -52.73% BenchmarkParseRandom/nocopy-32 3143 1013 -67.77% BenchmarkParseTwitter/copy-32 2585 973 -62.36% BenchmarkParseTwitter/nocopy-32 2016 420 -79.17% BenchmarkParseTwitterEscaped/copy-32 3005 1424 -52.61% BenchmarkParseTwitterEscaped/nocopy-32 2506 918 -63.37% BenchmarkParseUpdate_center/copy-32 3298 1541 -53.27% BenchmarkParseUpdate_center/nocopy-32 2122 404 -80.96% BenchmarkJsonParserLarge/nocopy-32 1353 633 -53.22% ```
harshavardhana
approved these changes
Jan 25, 2022
vcabbage
reviewed
Feb 24, 2022
| @@ -28,7 +28,7 @@ import ( | |||
| func (pj *internalParsedJson) initialize(size int) { | |||
| // Estimate the tape size to be about 15% of the length of the JSON message | |||
| avgTapeSize := size * 15 / 100 | |||
| if cap(pj.Tape) < avgTapeSize { | |||
| if cap(pj.Tape) <= avgTapeSize { | |||
There was a problem hiding this comment.
@klauspost Was this an intentional change? Allocating a new tape with a cap of avgTapeSize when the existing tape has a cap of avgTapeSize doesn't seem desirable.
There was a problem hiding this comment.
@vcabbage You are right. Let me test it out.
klauspost
added a commit
to klauspost/simdjson-go-1
that referenced
this pull request
Feb 24, 2022
Regression from minio#55
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cleanup: Don't send params already in internalParsedJson.