issue: high cpu usage #844
Conversation
|
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project, in which case you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed, please reply here (e.g.
|
|
Can one of the admins verify this patch? |
|
I signed it! 2015년 8월 7일 (금) 오후 3:41, cadvisorJenkinsBot notifications@github.com님이 작성:
|
|
CLAs look good, thanks! |
|
ok to test |
| @@ -63,6 +63,7 @@ type containerData struct { | |||
| loadAvg float64 // smoothed load average seen so far. | |||
| housekeepingInterval time.Duration | |||
| maxHousekeepingInterval time.Duration | |||
| allowedHouseKeepingGap time.Duration | |||
There was a problem hiding this comment.
Looks like we never set a default for this one?
There was a problem hiding this comment.
It was dropped while testing via cli argument.
At the moment, it would be better to set as a constant.
I set 1 minute as a default in housekeeping().
|
Actually, what if in housekeeping we just set next to now when next is not "before" now? Should keep this from happening without needed a configurable jump. Nice find btw :) |
|
@vmarmol Yes, Your suggestion is the simplest way to stop the issue, but I want to distinguish between two cases, 1) system time change(by suspend/resume), 2) actually host machine is very busy(really really busy), so I tried to fix nexthousekeeping() function. |
jhjeong-kr
force-pushed
the
master
branch
2 times, most recently
from
db86ff8 to
bcfeff4
Compare
|
Hey @jhjeong-kr I would prefer we make the change. I'm not sure we need to distinguish between the cases unless you strongly disagree. |
|
@vmarmol OK. May I update the source as you recommend? If you prefer a simpler way, I can update this PR within a couple of hours. |
|
Updating the source SGTM. |
…ng()" problem if system time has changed Signed-off-by: Jin-Hwan Jeong <jhjeong.kr@gmail.com>
|
I updated the source code. |
|
LGTM, thanks @jhjeong-kr! |
housekeeping function is invoked 1sec(default) even if system time changed greatly.
let's suppose that cadvisor is running on a vm and the vm is about to be suspended for 1 day,
and, after 1 day, vm is resumed.
At the moment, cadvisor runs at full speed, because nexthousekeeping is current housekeeping + 1sec.
Therefore, cadvisor collects stats 1day * 24hours * 60minutes * 60seconds times without sleep.
In my systems, cadvisor occupies 100% cpu time almost for 3 hours.
soluton:
If cadvisor notices time change(allowedHouseKeepingGap or higher), cadvisor has to adjust nexthousekeeping.
I implemented this patch with allowedHouseKeepingGap constant.
Signed-off-by: Jin-Hwan Jeong jhjeong.kr@gmail.com