Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for uncordoning nodes #84

Merged
merged 1 commit into from
Sep 11, 2020
Merged

Conversation

cezarsa
Copy link
Contributor

@cezarsa cezarsa commented Aug 19, 2020

Following discussions in #27 (comment) it can be useful to draino to detect that a condition that triggered a node being cordoned is no longer present.

This PR introduces the ability for draino to track which conditions triggered the cordon+drain process in an annotation named draino.planet.com/conditions.
Whenever this annotation is present draino will check if the conditions are still present in the node, if they are not present anymore draino will try to uncordon the node and possibly skip draining the node if it wasn't scheduled yet.

I'm marking this as a draft because I'm still going to write a few unit tests, but the functionality is mostly ready and I've been able to test it on a real cluster.

One question, would the maintainers like for me to put this feature behind a flag (eg: --allow-uncordon)? I don't think it's dangerous to allow uncordoning but it can be unexpected for users upgrading draino.

Fixes #27

@codecov
Copy link

codecov bot commented Aug 19, 2020

Codecov Report

Merging #84 into master will increase coverage by 2.49%.
The diff coverage is 78.57%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #84      +/-   ##
==========================================
+ Coverage   74.18%   76.67%   +2.49%     
==========================================
  Files           7        7              
  Lines         488      553      +65     
==========================================
+ Hits          362      424      +62     
+ Misses        116      115       -1     
- Partials       10       14       +4     
Impacted Files Coverage Δ
internal/kubernetes/nodefilters.go 92.45% <ø> (-1.49%) ⬇️
internal/kubernetes/eventhandler.go 67.83% <76.11%> (+24.08%) ⬆️
internal/kubernetes/drainer.go 82.77% <88.23%> (+0.35%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46799e2...0f4b2e1. Read the comment docs.

Copy link
Contributor

@jacobstr jacobstr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really lovely @cezarsa. I've merged a large MR ahead of yours. Would you be able to rebase and resolve the conflicts here?

expected: &core.Node{ObjectMeta: meta.ObjectMeta{Name: nodeName}},
},
{
name: "UncordonUnschedulableNodeWithMutator",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking for a test case where a node remains unschedulable even if draino has no reason to uncordon it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cezarsa
Copy link
Contributor Author

cezarsa commented Sep 11, 2020

Thanks for taking a look at this. Rebased and conflicts solved.

@jacobstr jacobstr merged commit d92f02b into planetlabs:master Sep 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for uncordon?
2 participants