Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flag to white list container labels for prometheus metrics #2113

Merged
merged 2 commits into from
Dec 5, 2018

Conversation

mikkeloscar
Copy link
Contributor

This adds a flag white_listed_container_labels which allows specifying
a comma separated list of container labels that are allowed to be
attached to prometheus metrics. This helps limit the label space of
each data point and limits the memory used by cadvisor compared to just
add all labels from all containers which is the default.

This is somewhat similar to the store_container_labels flag but a bit
more flexible because it allows white listing some common labels e.g.
io.kubernetes.pod.name, io.kubernetes.pod.namespace etc.

Without this white listing in place we saw the memory footprint of
cadvisor explode in our production environment because users add a lot
of data in pod labels, data which is not useful for metrics but may be
useful for the individual applications.

This adds a flag `white_listed_container_labels` which allows specifying
a comma separated list of container labels that are allowed to be
attached to prometheus metrics. This helps limit the label space of
each data point and limits the memory used by cadvisor compared to just
add all labels from all containers which is the default.

This is somewhat similar to the `store_container_labels` flag but a bit
more flexible because it allows white listing some common labels e.g.
`io.kubernetes.pod.name`, `io.kubernetes.pod.namespace` etc.

Without this white listing in place we saw the memory footprint of
cadvisor explode in our production environment because users add a lot
of data in pod labels, data which is not useful for metrics but may be
useful for the individual applications.

Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
@k8s-ci-robot
Copy link
Collaborator

Hi @mikkeloscar. Thanks for your PR.

I'm waiting for a google or kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@dashpole
Copy link
Collaborator

dashpole commented Dec 3, 2018

/ok-to-test

Copy link
Collaborator

@dashpole dashpole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple nits. I think this change is good overall.

Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
@mikkeloscar
Copy link
Contributor Author

@dashpole Thanks for the review, I addressed your comments!

Copy link
Collaborator

@dashpole dashpole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@dashpole dashpole merged commit c097b20 into google:master Dec 5, 2018
@mikkeloscar mikkeloscar deleted the white-list-labels branch December 5, 2018 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants