From 0ffd2a77e7a6625755a9e000af59f5305946e1df Mon Sep 17 00:00:00 2001 From: Jordan Liggitt Date: Mon, 14 Nov 2016 21:14:33 -0500 Subject: [PATCH] split out TLS bootstrapping topic --- docs/admin/index.md | 4 ++ docs/admin/kubelet-tls-bootstrapping.md | 96 +++++++++++++++++++++++++ docs/admin/master-node-communication.md | 89 ++--------------------- 3 files changed, 106 insertions(+), 83 deletions(-) create mode 100644 docs/admin/kubelet-tls-bootstrapping.md diff --git a/docs/admin/index.md b/docs/admin/index.md index 47f47f8116a4c..f6bde390f92f4 100644 --- a/docs/admin/index.md +++ b/docs/admin/index.md @@ -84,3 +84,7 @@ project](/docs/admin/salt). * **Sysctls** [sysctls](/docs/admin/sysctls.md) * **Audit** [audit](/docs/admin/audit) + +* **Securing the kubelet** + * [Master-Node communication](/docs/admin/master-node-communication/) + * [TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/) diff --git a/docs/admin/kubelet-tls-bootstrapping.md b/docs/admin/kubelet-tls-bootstrapping.md new file mode 100644 index 0000000000000..6d93c8e46b17a --- /dev/null +++ b/docs/admin/kubelet-tls-bootstrapping.md @@ -0,0 +1,96 @@ +--- +assignees: +- mikedanese + +--- + +* TOC +{:toc} + +## Summary + +This document describes setting up TLS client certificate bootstrapping for kubelets. +Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level +Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client +certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439) +and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43). + +## apiserver configuration + +You must provide a token file which specifies at least one "bootstrap token" assigned to a kubelet boostrap-specific group. +This group will later be used in the controller-manager configuration to scope approvals in the default approval +controller. As this feature matures, you should ensure tokens are bound to an RBAC policy which limits requests +using the bootstrap token to only be able to make requests related to certificate provisioning. When RBAC policy +is in place, scoping the tokens to a group will allow great flexibility (e.g. you could disable a particular +bootstrap group's access when you are done provisioning the nodes). + +### Token auth file +Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number +generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example: + +`head -c 16 /dev/urandom | od -An -t x | tr -d ' '` + +will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8` + +The token file will look like the following example, where the first three values can be anything and the quoted group +name should be as depicted: + +``` +02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:kubelet-bootstrap" +``` + +Add the `--token-auth-file=FILENAME` flag to the apiserver command to enable the token file. +See docs at http://kubernetes.io/docs/admin/authentication/#static-token-file for further details. + +### Client certificate CA bundle + +Add the `--client-ca-file=FILENAME` flag to the apiserver command to enable client certificate authentication, +referencing a certificate authority bundle containing the signing certificate. + +## controller-manager configuration +The API for requesting certificates adds a certificate-issuing control loop to the KCM. This takes the form of a +[cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using assets on disk. +Currently, all certificates issued have one year validity and a default set of key usages. + +### Signing assets +You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates. +This CA should be trusted by the apiserver for authentication with the `--client-ca-file=SOMEFILE` flag. The management +of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes. +Both certificate and key are assumed to be PEM-encoded. + +The new controller-manager flags are: +``` +--cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key" +``` + +### Auto-approval +To ease deployment and testing, the alpha version of the certificate request API includes a flag to approve all certificate +requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap +token in the token file above. Use of this flag circumvents makes the "approval" process described below and is not recommended +for production use. + +The flag is: +``` +--insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap" +``` + +## kubelet configuration +To use request a client cert from the certificate request API, the kubelet needs a path to a kubeconfig file that contains the +bootstrap auth token. If the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a +client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate +is written to the path specified by `--kubeconfig`. The certificate and key file will be stored in the directory pointed +by `--cert-dir`. The new flag is: + +``` +--experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig" +``` + +## kubectl approval +The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an +"Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external +approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl. +An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe `. There are +[currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update +the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available +at https://github.com/gtank/csrctl. + diff --git a/docs/admin/master-node-communication.md b/docs/admin/master-node-communication.md index 7fe5fd1a91869..d7424467de6c6 100644 --- a/docs/admin/master-node-communication.md +++ b/docs/admin/master-node-communication.md @@ -28,9 +28,12 @@ client [authentication](/docs/admin/authentication/) enabled. Nodes should be provisioned with the public root certificate for the cluster such that they can connect securely to the apiserver along with valid client credentials. For example, on a default GCE deployment, the client credentials -provided to the kubelet are in the form of a client certificate. Pods that -wish to connect to the apiserver can do so securely by leveraging a service -account so that Kubernetes will automatically inject the public root +provided to the kubelet are in the form of a client certificate. See +[kubelet TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/) for +automated provisioning of kubelet client certificates. + +Pods that wish to connect to the apiserver can do so securely by leveraging a +service account so that Kubernetes will automatically inject the public root certificate and a valid bearer token into the pod when it is instantiated. The `kubernetes` service (in all namespaces) is configured with a virtual IP address that is redirected (via kube-proxy) to the HTTPS endpoint on the @@ -84,83 +87,3 @@ cluster (connecting to the ssh server listening on port 22) and passes all traffic destined for a kubelet, node, pod, or service through the tunnel. This tunnel ensures that the traffic is not exposed outside of the private GCE network in which the cluster is running. - -### Kubelet TLS Bootstrap - -Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level -Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client -certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439) -and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43). - -##### apiserver configuration -You must provide a token file which specifies at least one "bootstrap token" assigned to a kubelet boostrap-specific group. -This group will later be used in the controller-manager configuration to scope approvals in the default approval -controller. As this feature matures, you should ensure tokens are bound to an RBAC policy which limits requests -using the bootstrap token to only be able to make requests related to certificate provisioning. When RBAC policy -is in place, scoping the tokens to a group will allow great flexibility (e.g. you could disable a particular -bootstrap group's access when you are done provisioning the nodes). - -##### Token auth file -Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number -generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example: - -`head -c 16 /dev/urandom | od -An -t x | tr -d ' '` - -will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8` - -The token file will look like the following example, where the first three values can be anything and the quoted group -name should be as depicted: - -``` -02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:kubelet-bootstrap" -``` - -Add the `--token-auth-file=FILENAME` flag to the apiserver command to enable the token file. -See docs at http://kubernetes.io/docs/admin/authentication/#static-token-file for further details. - -#### controller-manager configuration -The API for requesting certificates adds a certificate-issuing control loop to the KCM. This takes the form of a -[cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using assets on disk. -Currently, all certificates issued have one year validity and a default set of key usages. - -##### Signing assets -You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates. -This CA should be trusted by the apiserver for authentication with the `--client-ca-file=SOMEFILE` flag. The management -of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes. -Both certificate and key are assumed to be PEM-encoded. - -The new controller-manager flags are: -``` ---cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key" -``` - -##### Auto-approval -To ease deployment and testing, the alpha version of the certificate request API includes a flag to approve all certificate -requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap -token in the token file above. Use of this flag circumvents makes the "approval" process described below and is not recommended -for production use. - -The flag is: -``` ---insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap" -``` - -#### kubelet configuration -To use request a client cert from the certificate request API, the kubelet needs a path to a kubeconfig file that contains the -bootstrap auth token. If the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a -client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate -is written to the path specified by `--kubeconfig`. The certificate and key file will be stored in the directory pointed -by `--cert-dir`. The new flag is: - -``` ---experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig" -``` - -#### kubectl approval -The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an -"Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external -approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl. -An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe `. There are -[currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update -the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available -at https://github.com/gtank/csrctl.