-
Notifications
You must be signed in to change notification settings - Fork 375
sandbox: Stop and clean up containers that fail to create #2826
sandbox: Stop and clean up containers that fail to create #2826
Conversation
Thank you for raising your pull request. Please note that the main development of Kata Containers has moved to the 2.0-dev branch of https://github.com/kata-containers/kata-containers repository. The kata-containers/runtime repository is kept for 1.x release maintenance. Please check twice if your change should go to the 2.0-dev branch directly. If it is strongly required for adding the change to Kata Containers 1.x releases, please ping @kata-containers/runtime to assign a dedicated developer to be responsible for porting the change to 2.0-dev branch. Thanks! |
/test-ubuntu |
@evanfoster, thanks for the contribution! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice catch, lgtm!
@@ -1,4 +1,5 @@ | |||
// Copyright (c) 2016 Intel Corporation | |||
// Copyright (c) 2020 Adobe Inc. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cheeky =)
I'm seeing all of the sandbox mounts get cleaned up with this PR, but I do see this which is a bit concerning:
This is running with cri-o/cri-o#3924 EDIT: Some relevant CRI-O logs:
EDIT 2: I was testing with an older version where EDIT 3: I'm still having the same issue when forcing deletion. EDIT 4: The issue was actually coming from CRI-O. PR is cri-o/cri-o#3949 |
While the removing the container info of the failed container from the containers' map seems harmless, it'll end up causing an error when actually cleaning up / deleting the failed container. The reason for this is that CreateContainer, on Kata side, will remove the failed container, wich will result on DeleteContainer, on the CRI-O side, being called and then failing to actually remove the container, as its info has already been removed from the containers' map. This issue has been reported as part of a PR opened by Evan Foster: kata-containers/runtime#2826 (comment) Signed-off-by: Fabiano Fidêncio <fidencio@redhat.com> Tested-by: Evan Foster <efoster@adobe.com>
@evanfoster, I ended up closing that CRI-O PR in favour of cri-o/cri-o#3956 |
00e0ada
to
d7e3957
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@evanfoster, I've added one more simple comment in line.
A container that is created and added to a sandbox can still fail the final creation steps. In this case, the container must be stopped and have its resources cleaned up to prevent leaking sandbox mounts. Fixes kata-containers#2816 Signed-off-by: Evan Foster <efoster@adobe.com>
d7e3957
to
337f2e0
Compare
/test-ubuntu |
Codecov Report
@@ Coverage Diff @@
## master #2826 +/- ##
=======================================
Coverage 50.51% 50.51%
=======================================
Files 118 118
Lines 17331 17336 +5
=======================================
+ Hits 8754 8757 +3
- Misses 7503 7504 +1
- Partials 1074 1075 +1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @evanfoster.
@evanfoster, may I ask you two favours? git cherry-pick will not work for the second case, but tweaking the patch should be trivial. Thanks a lot for the collaboration! :-) |
A container that is created and added to a sandbox can still fail the final creation steps. In this case, the container must be stopped and have its resources cleaned up to prevent leaking sandbox mounts. Forward port of kata-containers/runtime#2826 Fixes kata-containers#2816 Signed-off-by: Evan Foster <efoster@adobe.com>
A container that is created and added to a sandbox can still fail the final creation steps. In this case, the container must be stopped and have its resources cleaned up to prevent leaking sandbox mounts. Forward port of kata-containers/runtime#2826 Fixes kata-containers#2816 Signed-off-by: Evan Foster <efoster@adobe.com>
A container that is created and added to a sandbox can still fail the final creation steps. In this case, the container must be stopped and have its resources cleaned up to prevent leaking sandbox mounts. Forward port of kata-containers/runtime#2826 Fixes kata-containers#2816 Signed-off-by: Evan Foster <efoster@adobe.com>
A container that is created and added to a sandbox can still fail
the final creation steps. In this case, the container must be stopped
and have its resources cleaned up to prevent leaking sandbox mounts.
Fixes #2816
Signed-off-by: Evan Foster efoster@adobe.com