Skip to content

Return request data to the pool in Ingester.Push() #1638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 11, 2019

Conversation

bboreham
Copy link
Contributor

@bboreham bboreham commented Sep 1, 2019

After #1526 the unmarshall code is trying to get memory out of the pool, but in ingesters it is never returned so the pool.Get() call just slows things down.

After making this change it still wasn't that great, because the pool was boxing TimeSeries objects. So change those to *TimeSeries.

before:

BenchmarkIngesterPush-2   	     100	  23525939 ns/op	 3881766 B/op	   42317 allocs/op

after:

BenchmarkIngesterPush-2   	     100	  22937648 ns/op	 3273382 B/op	   32482 allocs/op

(note out of the 3-4MB per iteration, 1MB is overhead from the FingerprintLocker)

@bboreham bboreham force-pushed the ingester-timeseries-pool branch 5 times, most recently from ba78f89 to 062bccc Compare September 5, 2019 09:45
Signed-off-by: Bryan Boreham <bryan@weave.works>
Signed-off-by: Bryan Boreham <bryan@weave.works>
Casting a struct to {}interface has an overhead, because the runtime
needs something more like a pointer so it creates a small descriptor
object.  Changing the data structure to use a pointer avoids this
overhead.

Signed-off-by: Bryan Boreham <bryan@weave.works>
@bboreham bboreham force-pushed the ingester-timeseries-pool branch from 062bccc to fab5ced Compare September 6, 2019 11:33
Copy link
Contributor

@codesome codesome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pool is better used with pointers 👍. And I assume the BenchmarkIngesterPush code was consistent for the before and after comparison? (Because client.ToWriteRequest is additional in the new test)

@bboreham
Copy link
Contributor Author

I'm not so sure now what those benchmarks were run against. Here's a new set with more permutations, on a different machine (go version go1.12 linux/amd64):

Current master 7447f75e9a97072e0b7b07d51174e55974d1baa4:
BenchmarkIngesterPush 	     100	  19752857 ns/op	 3280511 B/op	   31703 allocs/op
Before this PR db09d07e546e9c558154cfcf7d6766e06b0466cf: 
BenchmarkIngesterPush 	     100	  21412348 ns/op	 3291262 B/op	   31719 allocs/op
After changing benchmark f35f374c84991f7501da80ff550209f8f5118a9a:
BenchmarkIngesterPush 	      50	  33123151 ns/op	12155937 B/op	   62149 allocs/op
This PR fab5ced5e3c071a930966781b6eb786d8d34cc83:
BenchmarkIngesterPush 	     100	  26417682 ns/op	 3178133 B/op	   32073 allocs/op
Rebased against master d886f6e85a0356e4abbfd76cb685fddb69a8b626:
BenchmarkIngesterPush 	     100	  24700828 ns/op	 3174894 B/op	   32078 allocs/op

I'll merge since it's approved and check we get the expected drop in allocations in my staging environment.

@bboreham bboreham merged commit b367cd8 into master Sep 11, 2019
@bboreham bboreham deleted the ingester-timeseries-pool branch September 11, 2019 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants