Skip to content

🐛 add finalizer to kcp in first reconcile to avoid race condition with delete #3199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 17, 2020
Merged

🐛 add finalizer to kcp in first reconcile to avoid race condition with delete #3199

merged 1 commit into from
Jun 17, 2020
+32 −10

Conversation

jzhoucliqr
Copy link
Contributor

This fix add finalizer to kcp and force requeue for the first reconciliation, before kcp init, to avoid race condition between kcp init and kcp delete.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):

Fixes #3198

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 16, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @jzhoucliqr. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 16, 2020
@jzhoucliqr jzhoucliqr changed the title (:bug:, patch and bugfixes) add finalizer to kcp in first reconcile to avoid race condition with delete (:bug:) add finalizer to kcp in first reconcile to avoid race condition with delete Jun 16, 2020
@jzhoucliqr jzhoucliqr changed the title (:bug:) add finalizer to kcp in first reconcile to avoid race condition with delete 🐛 add finalizer to kcp in first reconcile to avoid race condition with delete Jun 16, 2020
vincepri
vincepri reviewed Jun 16, 2020
View reviewed changes
@jzhoucliqr jzhoucliqr force-pushed the add-kcp-finalizer-first branch from 525ce8a to fb4d786 Compare June 16, 2020 19:01
@vincepri
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 16, 2020
vincepri
vincepri approved these changes Jun 16, 2020
View reviewed changes
Copy link
Member

@vincepri vincepri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/assign @detiber

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 16, 2020
@vincepri
Copy link
Member

We should probably remove https://github.com/kubernetes-sigs/cluster-api/pull/3199/files#diff-b9435e3ca04d94ed2d00d8754ddae768R218 now that we're setting the finalizer here

detiber
detiber reviewed Jun 16, 2020
View reviewed changes
util/util.go Outdated
Comment on lines 648 to 657
// FinalizerExists check if the given finalizer exists in the object o
func FinalizerExists(o metav1.Object, finalizer string) bool {
f := o.GetFinalizers()
for _, e := range f {
if e == finalizer {
return true
}
}
return false
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really wish we could leverage controllerutil.ContainsFinalizer here rather than having to add a helper for this here. @vincepri is it worth backporting kubernetes-sigs/controller-runtime@b999e72#diff-4c146d38c03cf55e321de9ad53af4452 to controller-runtime v0.5.x for that purpose?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can, I can cut a release today if you're able to backport it to release-0.5

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kubernetes-sigs/controller-runtime#1000

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.5.6

@jzhoucliqr jzhoucliqr force-pushed the add-kcp-finalizer-first branch from fb4d786 to 7b2d4ab Compare June 16, 2020 21:42
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 16, 2020
vincepri
vincepri approved these changes Jun 16, 2020
View reviewed changes
Copy link
Member

@vincepri vincepri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/milestone v0.3.7

@k8s-ci-robot k8s-ci-robot added this to the v0.3.7 milestone Jun 16, 2020
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 16, 2020
@jzhoucliqr jzhoucliqr force-pushed the add-kcp-finalizer-first branch from 7b2d4ab to 0996e49 Compare June 16, 2020 21:59
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 16, 2020
@jzhoucliqr
Copy link
Contributor Author

/test pull-cluster-api-verify

@jzhoucliqr
Copy link
Contributor Author

/test pull-cluster-api-test

@jzhoucliqr jzhoucliqr force-pushed the add-kcp-finalizer-first branch from 0996e49 to e84a8f6 Compare June 16, 2020 22:26
@vincepri
Copy link
Member

/test pull-cluster-api-verify

@vincepri
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 17, 2020
@fabriziopandini
Copy link
Member

/approve
@vincepri @detiber
IMO setting finalizers as a first thing is potentially a good design pattern. Should we open an issue to align all the other controllers to this approach?

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini, jzhoucliqr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 17, 2020
@k8s-ci-robot k8s-ci-robot merged commit 3de5009 into kubernetes-sigs:master Jun 17, 2020
@detiber
Copy link
Member

detiber commented Jun 17, 2020

IMO setting finalizers as a first thing is potentially a good design pattern. Should we open an issue to align all the other controllers to this approach?

+1 from me, surprised there haven't been more complaints about similar race conditions with the other controllers yet :)

@vincepri
Copy link
Member

Sounds fair, if there is enough repetition everywhere, we should provide a generic utility? It can definitely come later though, no reason to do this in v0.3.7

@fabriziopandini fabriziopandini mentioned this pull request Jun 17, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@detiber detiber detiber left review comments

@vincepri vincepri vincepri approved these changes

@CecileRobertMichon CecileRobertMichon Awaiting requested review from CecileRobertMichon

@ncdc ncdc Awaiting requested review from ncdc

Assignees

@detiber detiber

@vincepri vincepri

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
v0.3.7
Development

Successfully merging this pull request may close these issues.

KCP deleted without deleting the CP Machine
5 participants
@jzhoucliqr @k8s-ci-robot @vincepri @fabriziopandini @detiber