kubernetes · tengqm · Nov 1, 2018 · Nov 1, 2018 · Nov 2, 2018 · Nov 6, 2018
diff --git a/content/en/docs/concepts/architecture/nodes.md b/content/en/docs/concepts/architecture/nodes.md
@@ -159,6 +159,20 @@ to be unreachable. (The default timeouts are 40s to start reporting
 ConditionUnknown and 5m after that to start evicting pods.) The node controller
 checks the state of each node every `--node-monitor-period` seconds.
 
+In versions of Kubernetes prior to 1.13, NodeStatus is the heartbeat from the
+node. Starting from Kubernetes 1.13, node lease feature is introduced as an
+alpha feature (feature gate `NodeLease`,
+[KEP-0009](https://github.com/kubernetes/community/blob/master/keps/sig-node/0009-node-heartbeat.md)).
+When node lease feature is enabled, each node has an associated `Lease` object in
+`kube-node-lease` namespace that is renewed by the node periodically, and both
+NodeStatus and node lease are treated as heartbeats from the node. Node leases
+are renewed frequently while NodeStatus is reported from node to master only
+when there is some change or enough time has passed (default is 1 minute, which
+is longer than the default timeout of 40 seconds for unreachable nodes). Since
+node lease is much more lightweight than NodeStatus, this feature makes node
+heartbeat significantly cheaper from both scalability and performance
+perspectives.
+
 In Kubernetes 1.4, we updated the logic of the node controller to better handle
 cases when a large number of nodes have problems with reaching the master
 (e.g. because the master has networking problem). Starting with 1.4, the node
@@ -213,11 +227,12 @@ For self-registration, the kubelet is started with the following options:
   - `--register-node` - Automatically register with the API server.
   - `--register-with-taints` - Register the node with the given list of taints (comma separated `<key>=<value>:<effect>`). No-op if `register-node` is false.
   - `--node-ip` - IP address of the node.
-  - `--node-labels` - Labels to add when registering the node in the cluster.
+  - `--node-labels` - Labels to add when registering the node in the cluster (see label restrictions enforced by the [NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction) in 1.13+).
   - `--node-status-update-frequency` - Specifies how often kubelet posts node status to master.
 
-Currently, any kubelet is authorized to create/modify any node resource, but in practice it only creates/modifies
-its own. (In the future, we plan to only allow a kubelet to modify its own node resource.)
+When the [Node authorization mode](/docs/reference/access-authn-authz/node/) and 
+[NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction) are enabled,
+kubelets are only authorized to create/modify their own Node resource.
 
 #### Manual Node Administration
 

diff --git a/content/en/docs/concepts/cluster-administration/cloud-providers.md b/content/en/docs/concepts/cluster-administration/cloud-providers.md
@@ -17,30 +17,32 @@ kubeadm has configuration options to specify configuration information for cloud
 in-tree cloud provider can be configured using kubeadm as shown below:
 
 ```yaml
-apiVersion: kubeadm.k8s.io/v1alpha3
+apiVersion: kubeadm.k8s.io/v1beta1
 kind: InitConfiguration
 nodeRegistration:
   kubeletExtraArgs:
     cloud-provider: "openstack"
     cloud-config: "/etc/kubernetes/cloud.conf"
 ---
+apiVersion: kubeadm.k8s.io/v1beta1
 kind: ClusterConfiguration
-apiVersion: kubeadm.k8s.io/v1alpha3
-kubernetesVersion: v1.12.0
-apiServerExtraArgs:
-  cloud-provider: "openstack"
-  cloud-config: "/etc/kubernetes/cloud.conf"
-apiServerExtraVolumes:
-- name: cloud
-  hostPath: "/etc/kubernetes/cloud.conf"
-  mountPath: "/etc/kubernetes/cloud.conf"
-controllerManagerExtraArgs:
-  cloud-provider: "openstack"
-  cloud-config: "/etc/kubernetes/cloud.conf"
-controllerManagerExtraVolumes:
-- name: cloud
-  hostPath: "/etc/kubernetes/cloud.conf"
-  mountPath: "/etc/kubernetes/cloud.conf"
+kubernetesVersion: v1.13.0
+apiServer:
+  extraArgs:
+    cloud-provider: "openstack"
+    cloud-config: "/etc/kubernetes/cloud.conf"
+  extraVolumes:
+  - name: cloud
+    hostPath: "/etc/kubernetes/cloud.conf"
+    mountPath: "/etc/kubernetes/cloud.conf"
+controllerManager:
+  extraArgs:
+    cloud-provider: "openstack"
+    cloud-config: "/etc/kubernetes/cloud.conf"
+  extraVolumes:
+  - name: cloud
+    hostPath: "/etc/kubernetes/cloud.conf"
+    mountPath: "/etc/kubernetes/cloud.conf"
 ```
 
 The in-tree cloud providers typically need both `--cloud-provider` and `--cloud-config` specified in the command lines

diff --git a/content/en/docs/concepts/configuration/assign-pod-node.md b/content/en/docs/concepts/configuration/assign-pod-node.md
@@ -92,6 +92,21 @@ For example, the value of `kubernetes.io/hostname` may be the same as the Node n
 and a different value in other environments.
 {{< /note >}}
 
+## Node isolation/restriction
+
+Adding labels to Node objects allows targeting pods to specific nodes or groups of nodes.
+This can be used to ensure specific pods only run on nodes with certain isolation, security, or regulatory properties.
+When using labels for this purpose, choosing label keys that cannot be modified by the kubelet process on the node is strongly recommended.
+This prevents a compromised node from using its kubelet credential to set those labels on its own Node object,
+and influencing the scheduler to schedule workloads to the compromised node.
+
+The `NodeRestriction` admission plugin prevents kubelets from setting or modifying labels with a `node-restriction.kubernetes.io/` prefix.
+To make use of that label prefix for node isolation:
+
+1. Ensure you are using the [Node authorizer](/docs/reference/access-authn-authz/node/) and have enabled the [NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction).
+2. Add labels under the `node-restriction.kubernetes.io/` prefix to your Node objects, and use those labels in your node selectors.
+For example, `example.com.node-restriction.kubernetes.io/fips=true` or `example.com.node-restriction.kubernetes.io/pci-dss=true`.
+
 ## Affinity and anti-affinity
 
 `nodeSelector` provides a very simple way to constrain pods to nodes with particular labels. The affinity/anti-affinity

diff --git a/content/en/docs/concepts/configuration/taint-and-toleration.md b/content/en/docs/concepts/configuration/taint-and-toleration.md
@@ -223,9 +223,7 @@ certain condition is true. The following taints are built in:
     as unusable. After a controller from the cloud-controller-manager initializes
     this node, the kubelet removes this taint.
 
-When the `TaintBasedEvictions` alpha feature is enabled (you can do this by
-including `TaintBasedEvictions=true` in `--feature-gates` for Kubernetes controller manager,
-such as `--feature-gates=FooBar=true,TaintBasedEvictions=true`), the taints are automatically
+In version 1.13, the `TaintBasedEvictions` feature is promoted to beta and enabled by default, hence the taints are automatically
 added by the NodeController (or kubelet) and the normal logic for evicting pods from nodes
 based on the Ready NodeCondition is disabled.
 
@@ -236,7 +234,7 @@ in a rate-limited way. This prevents massive pod evictions in scenarios such
 as the master becoming partitioned from the nodes.
 {{< /note >}}
 
-This alpha feature, in combination with `tolerationSeconds`, allows a pod
+This beta feature, in combination with `tolerationSeconds`, allows a pod
 to specify how long it should stay bound to a node that has one or both of these problems.
 
 For example, an application with a lot of local state might want to stay
@@ -246,7 +244,7 @@ The toleration the pod would use in that case would look like
 
 ```yaml
 tolerations:
-- key: "node.alpha.kubernetes.io/unreachable"
+- key: "node.kubernetes.io/unreachable"
   operator: "Exists"
   effect: "NoExecute"
   tolerationSeconds: 6000
@@ -257,9 +255,9 @@ Note that Kubernetes automatically adds a toleration for
 unless the pod configuration provided
 by the user already has a toleration for `node.kubernetes.io/not-ready`.
 Likewise it adds a toleration for
-`node.alpha.kubernetes.io/unreachable` with `tolerationSeconds=300`
+`node.kubernetes.io/unreachable` with `tolerationSeconds=300`
 unless the pod configuration provided
-by the user already has a toleration for `node.alpha.kubernetes.io/unreachable`.
+by the user already has a toleration for `node.kubernetes.io/unreachable`.
 
 These automatically-added tolerations ensure that
 the default pod behavior of remaining bound for 5 minutes after one of these
@@ -270,7 +268,7 @@ admission controller](https://git.k8s.io/kubernetes/plugin/pkg/admission/default
 [DaemonSet](/docs/concepts/workloads/controllers/daemonset/) pods are created with
 `NoExecute` tolerations for the following taints with no `tolerationSeconds`:
 
-  * `node.alpha.kubernetes.io/unreachable`
+  * `node.kubernetes.io/unreachable`
   * `node.kubernetes.io/not-ready`
 
 This ensures that DaemonSet pods are never evicted due to these problems,

diff --git a/content/en/docs/concepts/storage/storage-classes.md b/content/en/docs/concepts/storage/storage-classes.md
@@ -121,11 +121,6 @@ the class or PV, so mount of the PV will simply fail if one is invalid.
 
 ### Volume Binding Mode
 
-{{< feature-state for_k8s_version="v1.12" state="beta" >}}
-
-**Note:** This feature requires the `VolumeScheduling` feature gate to be
-enabled.
-
 The `volumeBindingMode` field controls when [volume binding and dynamic
 provisioning](/docs/concepts/storage/persistent-volumes/#provisioning) should occur.
 
@@ -157,10 +152,6 @@ The following plugins support `WaitForFirstConsumer` with pre-created Persistent
 * [Local](#local)
 
 ### Allowed Topologies
-{{< feature-state for_k8s_version="v1.12" state="beta" >}}
-
-**Note:** This feature requires the `VolumeScheduling` feature gate to be
-enabled.
 
 When a cluster operactor specifies the `WaitForFirstConsumer` volume binding mode, it is no longer necessary
 to restrict provisioning to specific topologies in most situations. However,

diff --git a/content/en/docs/concepts/workloads/controllers/daemonset.md b/content/en/docs/concepts/workloads/controllers/daemonset.md
@@ -161,14 +161,14 @@ Although Daemon Pods respect
 the following tolerations are added to DaemonSet Pods automatically according to
 the related features.
 
-| Toleration Key                           | Effect     | Alpha Features                                               | Version | Description                                                  |
-| ---------------------------------------- | ---------- | ------------------------------------------------------------ | ------- | ------------------------------------------------------------ |
-| `node.kubernetes.io/not-ready`           | NoExecute  | `TaintBasedEvictions`                                        | 1.8+    | When `TaintBasedEvictions` is enabled, they will not be evicted when there are node problems such as a network partition. |
-| `node.kubernetes.io/unreachable`         | NoExecute  | `TaintBasedEvictions`                                        | 1.8+    | When `TaintBasedEvictions` is enabled, they will not be evicted when there are node problems such as a network partition. |
-| `node.kubernetes.io/disk-pressure`       | NoSchedule |                                                              | 1.8+    |                                                              |
-| `node.kubernetes.io/memory-pressure`     | NoSchedule |                                                              | 1.8+    |                                                              |
-| `node.kubernetes.io/unschedulable`       | NoSchedule |                                                              | 1.12+   | DaemonSet pods tolerate unschedulable attributes by default scheduler.                                                    |
-| `node.kubernetes.io/network-unavailable` | NoSchedule |                                                              | 1.12+   | DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler.                      |
+| Toleration Key                           | Effect     | Version | Description                                                  |
+| ---------------------------------------- | ---------- | ------- | ------------------------------------------------------------ |
+| `node.kubernetes.io/not-ready`           | NoExecute  | 1.13+    | DaemonSet pods will not be evicted when there are node problems such as a network partition. |
+| `node.kubernetes.io/unreachable`         | NoExecute  | 1.13+    | DaemonSet pods will not be evicted when there are node problems such as a network partition. |
+| `node.kubernetes.io/disk-pressure`       | NoSchedule | 1.8+    |                                                              |
+| `node.kubernetes.io/memory-pressure`     | NoSchedule | 1.8+    |                                                              |
+| `node.kubernetes.io/unschedulable`       | NoSchedule | 1.12+   | DaemonSet pods tolerate unschedulable attributes by default scheduler.                                                    |
+| `node.kubernetes.io/network-unavailable` | NoSchedule | 1.12+   | DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler.                      |
 
 
 

diff --git a/content/en/docs/reference/_index.md b/content/en/docs/reference/_index.md
@@ -20,6 +20,7 @@ This section of the Kubernetes documentation contains references.
 
 * [Kubernetes API Overview](/docs/reference/using-api/api-overview/) - Overview of the API for Kubernetes.
 * Kubernetes API Versions
+  * [1.13](/docs/reference/generated/kubernetes-api/v1.13/)
   * [1.12](/docs/reference/generated/kubernetes-api/v1.12/)
   * [1.11](/docs/reference/generated/kubernetes-api/v1.11/)
   * [1.10](https://v1-10.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/)

diff --git a/content/en/docs/reference/access-authn-authz/admission-controllers.md b/content/en/docs/reference/access-authn-authz/admission-controllers.md
@@ -415,6 +415,25 @@ This admission controller limits the `Node` and `Pod` objects a kubelet can modi
 kubelets must use credentials in the `system:nodes` group, with a username in the form `system:node:<nodeName>`.
 Such kubelets will only be allowed to modify their own `Node` API object, and only modify `Pod` API objects that are bound to their node.
 In Kubernetes 1.11+, kubelets are not allowed to update or remove taints from their `Node` API object.
+
+In Kubernetes 1.13+, the `NodeRestriction` admission plugin prevents kubelets from deleting their `Node` API object,
+and enforces kubelet modification of labels under the `kubernetes.io/` or `k8s.io/` prefixes as follows:
+
+* **Prevents** kubelets from adding/removing/updating labels with a `node-restriction.kubernetes.io/` prefix.
+This label prefix is reserved for administrators to label their `Node` objects for workload isolation purposes,
+and kubelets will not be allowed to modify labels with that prefix.
+* **Allows** kubelets to add/remove/update these labels and label prefixes:
+  * `kubernetes.io/hostname`
+  * `beta.kubernetes.io/arch`
+  * `beta.kubernetes.io/instance-type`
+  * `beta.kubernetes.io/os`
+  * `failure-domain.beta.kubernetes.io/region`
+  * `failure-domain.beta.kubernetes.io/zone`
+  * `kubelet.kubernetes.io/`-prefixed labels
+  * `node.kubernetes.io/`-prefixed labels
+
+Use of any other labels under the `kubernetes.io` or `k8s.io` prefixes by kubelets is reserved, and may be disallowed or allowed by the `NodeRestriction` admission plugin in the future.
+
 Future versions may add additional restrictions to ensure kubelets have the minimal set of permissions required to operate correctly.
 
 ### OwnerReferencesPermissionEnforcement {#ownerreferencespermissionenforcement}

diff --git a/content/en/docs/reference/access-authn-authz/rbac.md b/content/en/docs/reference/access-authn-authz/rbac.md
@@ -644,11 +644,13 @@ These roles include:
 
 * system:controller:attachdetach-controller
 * system:controller:certificate-controller
+* system:controller:clusterrole-aggregation-controller
 * system:controller:cronjob-controller
 * system:controller:daemon-set-controller
 * system:controller:deployment-controller
 * system:controller:disruption-controller
 * system:controller:endpoint-controller
+* system:controller:expand-controller
 * system:controller:generic-garbage-collector
 * system:controller:horizontal-pod-autoscaler
 * system:controller:job-controller
@@ -661,6 +663,7 @@ These roles include:
 * system:controller:replicaset-controller
 * system:controller:replication-controller
 * system:controller:resourcequota-controller
+* system:controller:root-ca-cert-publisher
 * system:controller:route-controller
 * system:controller:service-account-controller
 * system:controller:service-controller