Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change default oom victim container name #2160

Merged
merged 1 commit into from
Feb 5, 2019

Conversation

gaorong
Copy link
Contributor

@gaorong gaorong commented Feb 1, 2019

background

In kubernetes, kubelet subscribes to cadvisor and follow the OOM event, the request is as below,

	request := events.Request{
		EventType: map[cadvisorapi.EventType]bool{
			cadvisorapi.EventOom: true,
		},
		ContainerName:        "/",
		IncludeSubcontainers: false,
	}

kubelet's related code is here, then cadvisor will return all system oom event against this request.

what happened to me

However, when I used this request as kubernetes with the only difference is the eventType, which is EventOomKill in mine, it seems not to work to me, I expect that cadvisor return all system oom kill event. (aside: The difference of EventOomKill and EventOom is described here).

cadvisor's behavior

When some process indeed been killed because of oom, cadvisor check if event satisfies request. In a situation described as above, cadvisor checks satisfaction and invokes checkIfIsSubcontainer, but checkIfIsSubcontainer always fails, which shouldn't happen.

this PR fixes this bug by initializing OomInstance.VictimContainerName as '/', which is the same as what OomInstance.ContainerName does. when oom parser parses kmsg's log and doesn't find a killed container, it will stay this name. and then if someone request as above, it will match this oom kill event and return this event.

@k8s-ci-robot
Copy link
Collaborator

Hi @gaorong. Thanks for your PR.

I'm waiting for a google or kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@gaorong
Copy link
Contributor Author

gaorong commented Feb 1, 2019

/assign @vmarmol

@dashpole
Copy link
Collaborator

dashpole commented Feb 1, 2019

/ok-to-test

@gaorong gaorong force-pushed the victimContainerName branch from 5a8b292 to 4072895 Compare February 2, 2019 02:01
@gaorong
Copy link
Contributor Author

gaorong commented Feb 2, 2019

the CI is green now

@dashpole
Copy link
Collaborator

dashpole commented Feb 5, 2019

/lgtm

Copy link
Collaborator

@dashpole dashpole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@dashpole dashpole merged commit 0995283 into google:master Feb 5, 2019
@gaorong gaorong deleted the victimContainerName branch February 6, 2019 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants