objstore/azure: Only create an http client once #4928
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`getContainerURL` was creating a new `http.Client` within `HTTPSender`. This increased memory pressure on components making requests to Azure. Move `http.Client` creation out of `HTTPSender`, only creating it once per call to `getContainerURL`. Signed-off-by: Andrew Seigner <andrew@sig.gy>
GiedriusS
approved these changes
Dec 7, 2021
3 tasks
squat
pushed a commit
to squat/thanos
that referenced
this pull request
Dec 8, 2021
`getContainerURL` was creating a new `http.Client` within `HTTPSender`. This increased memory pressure on components making requests to Azure. Move `http.Client` creation out of `HTTPSender`, only creating it once per call to `getContainerURL`. Signed-off-by: Andrew Seigner <andrew@sig.gy>
squat
pushed a commit
to squat/thanos
that referenced
this pull request
Dec 8, 2021
`getContainerURL` was creating a new `http.Client` within `HTTPSender`. This increased memory pressure on components making requests to Azure. Move `http.Client` creation out of `HTTPSender`, only creating it once per call to `getContainerURL`. Signed-off-by: Andrew Seigner <andrew@sig.gy>
Merged
squat
pushed a commit
that referenced
this pull request
Dec 9, 2021
`getContainerURL` was creating a new `http.Client` within `HTTPSender`. This increased memory pressure on components making requests to Azure. Move `http.Client` creation out of `HTTPSender`, only creating it once per call to `getContainerURL`. Signed-off-by: Andrew Seigner <andrew@sig.gy>
3 tasks
jordy1024
pushed a commit
to jordy1024/thanos
that referenced
this pull request
Dec 12, 2021
`getContainerURL` was creating a new `http.Client` within `HTTPSender`. This increased memory pressure on components making requests to Azure. Move `http.Client` creation out of `HTTPSender`, only creating it once per call to `getContainerURL`. Signed-off-by: Andrew Seigner <andrew@sig.gy>
jordy1024
pushed a commit
to jordy1024/thanos
that referenced
this pull request
Dec 12, 2021
`getContainerURL` was creating a new `http.Client` within `HTTPSender`. This increased memory pressure on components making requests to Azure. Move `http.Client` creation out of `HTTPSender`, only creating it once per call to `getContainerURL`. Signed-off-by: Andrew Seigner <andrew@sig.gy> Signed-off-by: wenmaoba <wenmaoba@tencent.com>
pracucci
pushed a commit
to cortexproject/cortex
that referenced
this pull request
Dec 14, 2021
* Update Thanos to latest main Update Thanos dependency to include thanos-io/thanos#4928, to conserve memory. Signed-off-by: Andrew Seigner <andrew@sig.gy> * Update changelog to summarize user-facing changes Signed-off-by: Andrew Seigner <andrew@sig.gy>
alvinlin123
pushed a commit
to ac1214/cortex
that referenced
this pull request
Jan 14, 2022
* Update Thanos to latest main Update Thanos dependency to include thanos-io/thanos#4928, to conserve memory. Signed-off-by: Andrew Seigner <andrew@sig.gy> * Update changelog to summarize user-facing changes Signed-off-by: Andrew Seigner <andrew@sig.gy> Signed-off-by: Alvin Lin <alvinlin@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
getContainerURLwas creating a newhttp.ClientwithinHTTPSender.This increased memory pressure on components making requests to Azure.
Move
http.Clientcreation out ofHTTPSender, only creating it onceper call to
getContainerURL.Signed-off-by: Andrew Seigner andrew@sig.gy
Changes
In objstore/azure, move
http.Clientcreation out ofHTTPSender, only creating it onceper call to
getContainerURL.Verification
We observed a 2x~4x memory increase in our alertmanager, compactor, ruler during an upgrade from Cortex v1.10.0 to v1.11.0, and tracked it down to a change in
getContainerURL.Memory usage of alertmanager, compactor, ruler with Cortex v1.10.0, v1.11.0, and this PR: