Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update kubelet authentication/authorization docs #1684

Merged
merged 5 commits into from
Dec 4, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion _data/reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,15 @@ toc:
- title: kube-scheduler
path: /docs/admin/kube-scheduler/
- title: kubelet
path: /docs/admin/kubelet/
section:
- title: Overview
path: /docs/admin/kubelet/
- title: Master-Node communication
path: /docs/admin/master-node-communication/
- title: TLS bootstrapping
path: /docs/admin/kubelet-tls-bootstrapping/
- title: Kubelet authentication/authorization
path: /docs/admin/kubelet-authentication-authorization/

- title: Glossary
section:
Expand Down
1 change: 1 addition & 0 deletions docs/admin/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ assignees:
- lavalamp
- ericchiang
- deads2k
- liggitt

---

Expand Down
10 changes: 6 additions & 4 deletions docs/admin/authorization.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
assignees:
- erictune
- lavalamp
- deads2k
- liggitt

---

Expand Down Expand Up @@ -565,10 +567,10 @@ Access to non-resource paths are sent as:

Non-resource paths include: `/api`, `/apis`, `/metrics`, `/resetMetrics`,
`/logs`, `/debug`, `/healthz`, `/swagger-ui/`, `/swaggerapi/`, `/ui`, and
`/version.` Clients require access to `/api`, `/api/*/`, `/apis/`, `/apis/*`,
`/apis/*/*`, and `/version` to discover what resources and versions are present
on the server. Access to other non-resource paths can be disallowed without
restricting access to the REST api.
`/version.` Clients require access to `/api`, `/api/*`, `/apis`, `/apis/*`,
and `/version` to discover what resources and versions are present on the server.
Access to other non-resource paths can be disallowed without restricting access
to the REST api.

For further documentation refer to the authorization.v1beta1 API objects and
plugin/pkg/auth/authorizer/webhook/webhook.go.
Expand Down
5 changes: 5 additions & 0 deletions docs/admin/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,3 +84,8 @@ project](/docs/admin/salt).
* **Sysctls** [sysctls](/docs/admin/sysctls.md)

* **Audit** [audit](/docs/admin/audit)

* **Securing the kubelet**
* [Master-Node communication](/docs/admin/master-node-communication/)
* [TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/)
* [Kubelet authentication/authorization](/docs/admin/kubelet-authentication-authorization/)
81 changes: 81 additions & 0 deletions docs/admin/kubelet-authentication-authorization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
assignees:
- liggitt

---

* TOC
{:toc}

## Overview

A kubelet's HTTPS endpoint exposes APIs which give access to data of varying sensitivity,
and allow you to perform operations with varying levels of power on the node and within containers.

This document describes how to authenticate and authorize access to the kubelet's HTTPS endpoint.

## Kubelet authentication

By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured
authentication methods are treated as anonymous requests, and given a username of `system:anonymous`
and a group of `system:unauthenticated`.

To disable anonymous access and send `401 Unauthorized` responses to unauthenticated requests:
* start the kubelet with the `--anonymous-auth=false` flag
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does disabling this result in getting a user named "" or a 401?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

401


To enable X509 client certificate authentication to the kubelet's HTTPS endpoint:
* start the kubelet with the `--client-ca-file` flag, providing a CA bundle to verify client certificates with
* start the apiserver with `--kubelet-client-certificate` and `--kubelet-client-key` flags
* see the [apiserver authentication documentation](/docs/admin/authentication/#x509-client-certs) for more details

To enable API bearer tokens (including service account tokens) to be used to authenticate to the kubelet's HTTPS endpoint:
* ensure the `authentication.k8s.io/v1beta1` API group is enabled in the API server
* start the kubelet with the `--authentication-token-webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
* the kubelet calls the `TokenReview` API on the configured API server to determine user information from bearer tokens

## Kubelet authorization

Any request that is successfully authenticated (including an anonymous request) is then authorized. The default authorization mode is `AlwaysAllow`, which allows all requests.

There are many possible reasons to subdivide access to the kubelet API:
* anonymous auth is enabled, but anonymous users' ability to call the kubelet API should be limited
* bearer token auth is enabled, but arbitrary API users' (like service accounts) ability to call the kubelet API should be limited
* client certificate auth is enabled, but only some of the client certificates signed by the configured CA should be allowed to use the kubelet API

To subdivide access to the kubelet API, delegate authorization to the API server:
* ensure the `authorization.k8s.io/v1beta1` API group is enabled in the API server
* start the kubelet with the `--authorization-mode=Webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
* the kubelet calls the `SubjectAccessReview` API on the configured API server to determine whether each request is authorized

The kubelet authorizes API requests using the same [request attributes](/docs/admin/authorization/#request-attributes) approach as the apiserver.

The verb is determined from the incoming request's HTTP verb:

HTTP verb | request verb
----------|---------------
POST | create
GET, HEAD | get
PUT | update
PATCH | patch
DELETE | delete

The resource and subresource is determined from the incoming request's path:

Kubelet API | resource | subresource
-------------|----------|------------
/stats/* | nodes | stats
/metrics/* | nodes | metrics
/logs/* | nodes | log
/spec/* | nodes | spec
*all others* | nodes | proxy

The namespace and API group attributes are always an empty string, and
the resource name is always the name of the kubelet's `Node` API object.

When running in this mode, ensure the user identified by the `--kubelet-client-certificate` and `--kubelet-client-key`
flags passed to the apiserver is authorized for the following attributes:
* verb=*, resource=nodes, subresource=proxy
* verb=*, resource=nodes, subresource=stats
* verb=*, resource=nodes, subresource=log
* verb=*, resource=nodes, subresource=spec
* verb=*, resource=nodes, subresource=metrics
96 changes: 96 additions & 0 deletions docs/admin/kubelet-tls-bootstrapping.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
assignees:
- mikedanese

---

* TOC
{:toc}

## Overview

This document describes how to set up TLS client certificate boostrapping for kubelets.
Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this text be updated for version 1.5?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave that as a follow up for @mikedanese, this PR just moved the content to its own file

Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client
certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439)
and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43).

## apiserver configuration

You must provide a token file which specifies at least one "bootstrap token" assigned to a kubelet boostrap-specific group.
This group will later be used in the controller-manager configuration to scope approvals in the default approval
controller. As this feature matures, you should ensure tokens are bound to an RBAC policy which limits requests
using the bootstrap token to only be able to make requests related to certificate provisioning. When RBAC policy
is in place, scoping the tokens to a group will allow great flexibility (e.g. you could disable a particular
bootstrap group's access when you are done provisioning the nodes).

### Token auth file
Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number
generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example:

`head -c 16 /dev/urandom | od -An -t x | tr -d ' '`

will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8`

The token file will look like the following example, where the first three values can be anything and the quoted group
name should be as depicted:

```
02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a set of authorization rules for this user group? I know that this requires the bootstrap token to only have access to specific sub resources (or it can auto approve its own request).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was just a move of the existing doc, and this is just an example group... there's nothing in-tree that actually uses that group name

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay nvm. I've opened #1703

```

Add the `--token-auth-file=FILENAME` flag to the apiserver command to enable the token file.
See docs at http://kubernetes.io/docs/admin/authentication/#static-token-file for further details.

### Client certificate CA bundle

Add the `--client-ca-file=FILENAME` flag to the apiserver command to enable client certificate authentication,
referencing a certificate authority bundle containing the signing certificate.

## controller-manager configuration
The API for requesting certificates adds a certificate-issuing control loop to the KCM. This takes the form of a
[cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using assets on disk.
Currently, all certificates issued have one year validity and a default set of key usages.

### Signing assets
You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates.
This CA should be trusted by the apiserver for authentication with the `--client-ca-file=SOMEFILE` flag. The management
of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes.
Both certificate and key are assumed to be PEM-encoded.

The new controller-manager flags are:
```
--cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
```

### Auto-approval
To ease deployment and testing, the alpha version of the certificate request API includes a flag to approve all certificate
requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
token in the token file above. Use of this flag circumvents makes the "approval" process described below and is not recommended
for production use.

The flag is:
```
--insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
```

## kubelet configuration
To use request a client cert from the certificate request API, the kubelet needs a path to a kubeconfig file that contains the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you perhaps mean:

To request a client cert from the certificate request API, the kubelet needs a path to the kubeconfig file that contains the ...

The phrase To use request seems a bit odd.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was a straight move of the content in kubelet-tls-bootstrapping.md, I'd rather save content changes in that file for a follow-up

bootstrap auth token. If the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a
client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate
is written to the path specified by `--kubeconfig`. The certificate and key file will be stored in the directory pointed
by `--cert-dir`. The new flag is:

```
--experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
```

## kubectl approval
The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an
"Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external
approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl.
An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe <name>`. There are
[currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update
the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available
at [https://github.com/gtank/csrctl](https://github.com/gtank/csrctl).

Loading