receive: use s2 when decompressing snappy data, reduce CPU usage for incoming remote write requests by ~5-7% #4354
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use https://github.com/klauspost/compress/tree/master/s2#s2-compression for decompressing Snappy data. I think if this pays off then we could also try using this in Prometheus for encoding? Benchmarks show these numbers: ``` benchmark old ns/op new ns/op delta BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/OK-16 1139713 1071864 -5.95% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/conflict_errors-16 1294286 1207937 -6.67% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/OK-16 11613568 10849901 -6.58% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/conflict_errors-16 13100752 12478380 -4.75% BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/OK-16 96128949 89515761 -6.88% BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/conflict_errors-16 96769948 90741611 -6.23% benchmark old allocs new allocs delta BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/OK-16 4578 4578 +0.00% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/conflict_errors-16 7618 7617 -0.01% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/OK-16 45325 45324 -0.00% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/conflict_errors-16 75133 75133 +0.00% BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/OK-16 81 82 +1.23% BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/conflict_errors-16 203 202 -0.49% benchmark old bytes new bytes delta BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/OK-16 1254474 1254280 -0.02% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_500_of_them/conflict_errors-16 1427791 1427618 -0.01% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/OK-16 13826832 13806752 -0.15% BenchmarkHandlerReceiveHTTP/typical_labels_under_1KB,_5000_of_them/conflict_errors-16 15412336 15412232 -0.00% BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/OK-16 120041647 120018331 -0.02% BenchmarkHandlerReceiveHTTP/extremely_large_label_value_10MB,_10_of_them/conflict_errors-16 119704515 119701011 -0.00% ``` Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
|
The ASM code seems to roughly follow the Go code. So, it is easier to compare https://github.com/klauspost/compress/blob/master/s2/decode_other.go with https://github.com/golang/snappy/blob/master/decode_other.go. The code seems to be quite similar and copied/pasted in some places which means that it had been battle-tested even before S2, with some tweaks at the beginning of the decoding routine and at the end. |
Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use https://github.com/klauspost/compress/tree/master/s2#s2-compression
for decompressing Snappy data. I think if this pays off then we could
also try using this in Prometheus for encoding?
Benchmarks show these numbers:
Signed-off-by: Giedrius Statkevičius giedrius.statkevicius@vinted.com