Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API for SELinuxRelabelPolicy #91838

Closed
wants to merge 6 commits into from
Closed

Conversation

jsafrane
Copy link
Member

@jsafrane jsafrane commented Jun 5, 2020

SELinuxRelabelPolicy allows users to optionally skip recursive relabeling of all files on pod on volumes on systems with SELinux support.

KEP: https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1710-selinux-relabeling

When the feature is enabled + supported by volume plugin / CSI driver + enabled in Pod.SecurityContext + kubelet knows SELinux context of pod volumes, kubelet mounts pod's volumes with -o context=<xyz>. No relabeling is then necessary.

What type of PR is this?
/kind api-change

Changes from KEP:
type SELinuxRelabelPolicy -> PodSELinuxRelabelPolicy
Enum values OnVolumeMount, AlwaysRelabel -> SELinuxRelabel*

Does this PR introduce a user-facing change?:

Added SELinuxRelabelPolicy alpha feature gate, which allows pods to opt-in to skip recursive relabeling of pod's volumes when SELinux is enabled and mount the volume with the correct SELinux label directly.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]:  https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1710-selinux-relabeling

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 5, 2020
@jsafrane
Copy link
Member Author

jsafrane commented Jun 5, 2020

Preview of implementation of this feature is available in #91839 (All kubelet changes are there + AWS EBS and CSI volume plugin updates).

@fejta-bot
Copy link

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@jsafrane
Copy link
Member Author

jsafrane commented Jun 5, 2020

cc @msau42 @tallclair @gnufied
/label api-review

@k8s-ci-robot k8s-ci-robot added the api-review Categorizes an issue or PR as actively needing an API review. label Jun 5, 2020
@gnufied
Copy link
Member

gnufied commented Jun 10, 2020

/assign

@smarterclayton
Copy link
Contributor

/assign @smarterclayton

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 12, 2020
// "OnVolumeMount" tries to mount volumes used by the Pod with the right context and skip recursive ownership
// change. Kubernetes may fall back to policy "Always" if a storage backed does not support this policy.
// + optional
SELinuxRelabelPolicy *PodSELinuxRelabelPolicy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't remember if this came up on the KEP, but should this be per-volume rather than per-pod?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See below, I'm following FSGroupChangePolicy.

// "OnVolumeMount" tries to mount volumes used by the Pod with the right context and skip recursive ownership
// change. Kubernetes may fall back to policy "Always" if a storage backed does not support this policy.
// + optional
SELinuxRelabelPolicy *PodSELinuxRelabelPolicy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we group this & FSGroupChangePolicy together in a way that makes it clear both pertain to volumes? I'm thinking a sub-struct along the lines of VolumeSecurityContext? See above comment, it may make sense to move this to the volume struct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just followed fsGroupChangePolicy, which, after the same discussion, ended up in SecurityContext. @gnufied, do you remember why we kept it there?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to know why this wouldn't be in the volume struct too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The discussion from fsGroupChangePolicy - kubernetes/enhancements#1478 (comment) . tldr; of that discussion is -

  1. Having this setting on per-volume basis is more complicated for the end user and its puts this burden of deciding which policy to choose for which volume type.
  2. Also, if there were an automated tool which injected such policies in pods, the tool also has to be aware of volumes it should target.
  3. On the other hand - having this setting on per-volume basis does not really buy much at the cost of added complexity.

// SELinuxRelabelOnVolumeMount indicates that, if supported by the storage backend, SELinux
// label will be assigned during mount of pod's volumes as a mount option. This can skip
// potentially long recursive SELinux label change performed by container runtime when starting
// a pod.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you not always set this? Add the drawbacks to the doc comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's... complicated. I added more description there, however, I was not able to avoid mentioning Linux kernel.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the tradeoff when you skip this? I.e. you talk about a benefit (performance) but you don't present what you lose (tim's question). I would guess that you are not secure. If you're not secure, why. Is there a link somewhere to the presentation of why relabel is not desirable that we can copy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I updated only storage/v1, forgetting core/v1.

Added "As tradeoff, all pods that simultaneously access the volumes must run with the same SELinux context to be able to access data on them, even if they use just a subPath of them."

The goal is to say it's safe, unless you use a single volume with two pods with different SELinux contexts, each using a different subpath of the volume. This was working before and it is working even now with policy "Always".

pkg/apis/core/types.go Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jun 17, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 24, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jsafrane
To complete the pull request process, please assign smarterclayton
You can assign the PR to them by writing /assign @smarterclayton in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
Copy link
Contributor

@jsafrane: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
pull-kubernetes-e2e-gce-storage-snapshot 6a41f9fa4466ac3c7d83b18037fb5e282bdc93fc link /test pull-kubernetes-e2e-gce-storage-snapshot
pull-kubernetes-kubemark-e2e-gce-big 1eaa4cc link /test pull-kubernetes-kubemark-e2e-gce-big
pull-kubernetes-verify 1eaa4cc link /test pull-kubernetes-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jsafrane
Copy link
Member Author

/close

mount -o context=XYZ is hard from a CSI driver container, see kubernetes/enhancements#1879.
Reworking to merge with fsGroupChangePolicy in the linked PR.

@k8s-ci-robot
Copy link
Contributor

@jsafrane: Closed this PR.

In response to this:

/close

mount -o context=XYZ is hard from a CSI driver container, see kubernetes/enhancements#1879.
Reworking to merge with fsGroupChangePolicy in the linked PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-review Categorizes an issue or PR as actively needing an API review. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants