[KEP-2568] Production Readiness Review.

[vinayakankugoyal]

KEP-2568 proposes that we run the kubeadm control-plane as non-root and follow other security best practices.

[k8s-ci-robot]

Hi @vinayakankugoyal. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vinayakankugoyal]

/cc @neolit123

[vinayakankugoyal]

/assign prod-readiness-approvers

[k8s-ci-robot]

@vinayakankugoyal: GitHub didn't allow me to assign the following users: prod-readiness-approvers.

Note that only kubernetes members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign prod-readiness-approvers

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vinayakankugoyal]

/cc prod-readiness-approvers

[vinayakankugoyal]

Adding @ehashman for the PRR review.

[vinayakankugoyal]

/cc @ehashman

[ehashman]

/assign
/ok-to-test

[vinayakankugoyal]

@ehashman - I am new to the PRR process, so not sure how it goes. Does the PRR questionnaire seem ok? I am happy to provide more details. Thank you so much for looking at this KEP.

[ehashman]

Hi @vinayakankugoyal, not sure why the verify script isn't catching this. You will need to add a file to the keps/prod-readiness directory for this KEP marking me as the approver. Because the KEP already merged I can't comment line-by-line. I'll do a review in my next comment.

[ehashman]

Alpha -> Beta Graduation

Are these graduation criteria for alpha or beta? From the kep.yaml this appears to be a net new feature. I'm curious what the roadmap is supposed to look like for each of Alpha, Beta, and Graduated. (the latter two stages can be TBD/tentative)

(Version Skew Strategy) Nothing in the design of this feature is tied to the version of the control-plane.

Below I see there's a feature gate for all the control plane components to enable them to run as non-root, so I don't think this is correct. I don't think it's supported to run as non-root in version <1.22. If you have a mixed deployment X and X-1 (e.g. 1.22 and 1.21), it's possible you end up with some control plane components still running as root.

Can you update this answer to clarify how that works?

Does enabling the feature change any default behavior?

Yes it will change the default behavior of kubeadm from running the control-plane components as root to non-root.

The feature flags control the behaviour of each component. A feature flag in kube-apiserver won't affect kubeadm. How do the feature flags on each component affect their behaviour? What impact could it have on a cluster that wasn't deployed by kubeadm?

Additionally, can you talk a little bit more about how enabling the feature flag will change kubeadm's behaviour, and what users should expect?

Are there any tests for feature enablement/disablement?

This question is missing an answer.

Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?

Yes, in kubeadm control-plane bootstrap process when we create files and directories we would have to change the permissions and the owners of these files. So there will be a minute increase in bootstrap time for control-plane.

Is it possible to block or fail on the creation of users/groups? Should this be called out as a possible delay?

[vinayakankugoyal]

Hi @vinayakankugoyal, not sure why the verify script isn't catching this. You will need to add a file to the keps/prod-readiness directory for this KEP marking me as the approver. Because the KEP already merged I can't comment line-by-line. I'll do a review in my next comment.

Done. Thanks!

[vinayakankugoyal]

Alpha -> Beta Graduation

Are these graduation criteria for alpha or beta? From the kep.yaml this appears to be a net new feature. I'm curious what the roadmap is supposed to look like for each of Alpha, Beta, and Graduated. (the latter two stages can be TBD/tentative)

These are graduation criteria for alpha. I think we'd come up with the Beta and Graduated criteria later, once we have hit alpha. (There wasn't a section for dev -> Alpha graduation criteria, so I put it under this section.)

(Version Skew Strategy) Nothing in the design of this feature is tied to the version of the control-plane.

Below I see there's a feature gate for all the control plane components to enable them to run as non-root, so I don't think this is correct. I don't think it's supported to run as non-root in version <1.22. If you have a mixed deployment X and X-1 (e.g. 1.22 and 1.21), it's possible you end up with some control plane components still running as root.

Can you update this answer to clarify how that works?

The featuregate mentioned below is not a featuregate for the control-plane components but is a featuregate passed to kubeadm itself. It will control how kubeadm deploys the control-plane. Nothing in this design updates the feturegate flags passed to the control-plane binaries, the featuregate flag will be passed to the kubeadm binary itself and it will control how kubeadm updates the Pod config/manifest of the control-plane components. Sorry if this was not clear from my documentation.

Does enabling the feature change any default behavior?
Yes it will change the default behavior of kubeadm from running the control-plane components as root to non-root.

The feature flags control the behaviour of each component. A feature flag in kube-apiserver won't affect kubeadm. How do the feature flags on each component affect their behaviour? What impact could it have on a cluster that wasn't deployed by kubeadm?

Additionally, can you talk a little bit more about how enabling the feature flag will change kubeadm's behaviour, and what users should expect?

The feature flag will not change the behavior of the control-plane components, it will change the behavior of kubeadm. kubeadm will configure the Pods of the components differently when this featuregate is enabled.

Are there any tests for feature enablement/disablement?

This question is missing an answer.

Added. Thanks!

Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?
Yes, in kubeadm control-plane bootstrap process when we create files and directories we would have to change the permissions and the owners of these files. So there will be a minute increase in bootstrap time for control-plane.

Is it possible to block or fail on the creation of users/groups? Should this be called out as a possible delay?

Yes, it is possible to fail on the creation of users and groups. I'll call this out.

[vinayakankugoyal]

@ehashman - Are there any more questions/concerns? I am happy to provide more details. Thank you so much for looking at this KEP.

[ehashman]

Because it appears that this enhancement is 100% out of tree (i.e. all changes made in k/kubeadm, no changes to k/kubernetes) I do not believe PRR approval is applicable. Pending confirmation from PRR team.

[ehashman]

Per @johnbelamaric, please add a note that the PRR fields are not applicable, as this KEP is out of tree.

/approve
for PRR

[vinayakankugoyal]

Per @johnbelamaric, please add a note that the PRR fields are not applicable, as this KEP is out of tree.

/approve
for PRR

Added the note in the check list and the PRR questionnaire section.

[vinayakankugoyal]

/assign @neolit123

[neolit123]

/lgtm
/approve
after PRR

lets move the discussion to the other PR that sets this KEP to implementable.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ehashman, neolit123, vinayakankugoyal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• keps/prod-readiness/OWNERS [ehashman]
• keps/sig-cluster-lifecycle/OWNERS [neolit123]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment