Merge pull request kubernetes#33230 from nate-double-u/merged-main-de…

…v-1.24

Merged main into dev-1.24

content/en/blog/_posts/2021-04-19-introducing-indexed-jobs.md

@@ -62,7 +62,7 @@ spec:
 Note that completion mode is an alpha feature in the 1.21 release. To be able to
 use it in your cluster, make sure to enable the `IndexedJob` [feature
 gate](/docs/reference/command-line-tools-reference/feature-gates/) on the
-[API server](docs/reference/command-line-tools-reference/kube-apiserver/) and
+[API server](/docs/reference/command-line-tools-reference/kube-apiserver/) and
 the [controller manager](/docs/reference/command-line-tools-reference/kube-controller-manager/).
 
 When you run the example, you will see that each of the three created Pods gets a

content/en/blog/_posts/2021-12-16-StatefulSet-PVC-Auto-Deletion.md

@@ -8,8 +8,8 @@ slug: kubernetes-1-23-statefulset-pvc-auto-deletion
 **Author:** Matthew Cary (Google)
 
 Kubernetes v1.23 introduced a new, alpha-level policy for
-[StatefulSets](docs/concepts/workloads/controllers/statefulset/) that controls the lifetime of
-[PersistentVolumeClaims](docs/concepts/storage/persistent-volumes/) (PVCs) generated from the
+[StatefulSets](/docs/concepts/workloads/controllers/statefulset/) that controls the lifetime of
+[PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/) (PVCs) generated from the
 StatefulSet spec template for cases when they should be deleted automatically when the StatefulSet
 is deleted or pods in the StatefulSet are scaled down.
 
@@ -82,7 +82,7 @@ This policy forms a matrix with four cases. I’ll walk through and give an exam
     new replicas will automatically use them.
 
 Visit the
-[documentation](docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-policies) to
+[documentation](/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-policies) to
 see all the details.
 
 ## What’s next?

content/en/blog/_posts/2022-05-03-dockershim-historical-context.md

@@ -18,7 +18,7 @@ The [Container Runtime Interface](/blog/2016/12/container-runtime-interface-cri-
 
 However, this little software shim was never intended to be a permanent solution. Over the course of years, its existence has introduced a lot of unnecessary complexity to the kubelet itself. Some integrations are inconsistently implemented for Docker because of this shim, resulting in an increased burden on maintainers, and maintaining vendor-specific code is not in line with our open source philosophy. To reduce this maintenance burden and move towards a more collaborative community in support of open standards, [KEP-2221 was introduced](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2221-remove-dockershim), proposing the removal of the dockershim. With the release of Kubernetes v1.20, the deprecation was official.
 
-We didn’t do a great job communicating this, and unfortunately, the deprecation announcement led to some panic within the community. Confusion around what this meant for Docker as a company, if container images built by Docker would still run, and what Docker Engine actually is led to a conflagration on social media. This was our fault; we should have more clearly communicated what was happening and why at the time. To combat this, we released [a blog](/blog/2020/12/02/dont-panic-kubernetes-and-docker/) and [accompanying FAQ](/blog/2020/12/02/dockershim-faq/) to allay the community’s fears and correct some misconceptions about what Docker is and how containers work within Kubernetes. As a result of the community’s concerns, Docker and Mirantis jointly agreed to continue supporting the dockershim code in the form of [cri-dockerd](https://www.mirantis.com/blog/the-future-of-dockershim-is-cri-dockerd/), allowing you to continue using Docker Engine as your container runtime if need be. For the interest of users who want to try other runtimes, like containerd or cri-o, [migration documentation was written](docs/tasks/administer-cluster/migrating-from-dockershim/change-runtime-containerd/).
+We didn’t do a great job communicating this, and unfortunately, the deprecation announcement led to some panic within the community. Confusion around what this meant for Docker as a company, if container images built by Docker would still run, and what Docker Engine actually is led to a conflagration on social media. This was our fault; we should have more clearly communicated what was happening and why at the time. To combat this, we released [a blog](/blog/2020/12/02/dont-panic-kubernetes-and-docker/) and [accompanying FAQ](/blog/2020/12/02/dockershim-faq/) to allay the community’s fears and correct some misconceptions about what Docker is and how containers work within Kubernetes. As a result of the community’s concerns, Docker and Mirantis jointly agreed to continue supporting the dockershim code in the form of [cri-dockerd](https://www.mirantis.com/blog/the-future-of-dockershim-is-cri-dockerd/), allowing you to continue using Docker Engine as your container runtime if need be. For the interest of users who want to try other runtimes, like containerd or cri-o, [migration documentation was written](/docs/tasks/administer-cluster/migrating-from-dockershim/change-runtime-containerd/).
 
 We later [surveyed the community](https://kubernetes.io/blog/2021/11/12/are-you-ready-for-dockershim-removal/) and [discovered that there are still many users with questions and concerns](/blog/2022/01/07/kubernetes-is-moving-on-from-dockershim). In response, Kubernetes maintainers and the CNCF committed to addressing these concerns by extending documentation and other programs. In fact, this blog post is a part of this program. With so many end users successfully migrated to other runtimes, and improved documentation, we believe that everyone has a paved way to migration now.
 

content/en/docs/concepts/architecture/nodes.md

@@ -312,16 +312,18 @@ controller deletes the node from its list of nodes.
 The third is monitoring the nodes' health. The node controller is
 responsible for:
 
-- In the case that a node becomes unreachable, updating the NodeReady condition
-  of within the Node's `.status`. In this case the node controller sets the
-  NodeReady condition to `ConditionUnknown`.
+- In the case that a node becomes unreachable, updating the `Ready` condition
+  in the Node's `.status` field. In this case the node controller sets the
+  `Ready` condition to `Unknown`.
 - If a node remains unreachable: triggering
   [API-initiated eviction](/docs/concepts/scheduling-eviction/api-eviction/)
   for all of the Pods on the unreachable node. By default, the node controller
-  waits 5 minutes between marking the node as `ConditionUnknown` and submitting
+  waits 5 minutes between marking the node as `Unknown` and submitting
   the first eviction request.
 
-The node controller checks the state of each node every `--node-monitor-period` seconds.
+By default, the node controller checks the state of each node every 5 seconds.
+This period can be configured using the `--node-monitor-period` flag on the
+`kube-controller-manager` component.
 
 ### Rate limits on eviction
 
@@ -331,7 +333,7 @@ from more than 1 node per 10 seconds.
 
 The node eviction behavior changes when a node in a given availability zone
 becomes unhealthy. The node controller checks what percentage of nodes in the zone
-are unhealthy (NodeReady condition is `ConditionUnknown` or `ConditionFalse`) at
+are unhealthy (the `Ready` condition is `Unknown` or `False`) at
 the same time:
 
 - If the fraction of unhealthy nodes is at least `--unhealthy-zone-threshold`
@@ -384,7 +386,7 @@ If you want to explicitly reserve resources for non-Pod processes, see
 
 ## Node topology
 
-{{< feature-state state="alpha" for_k8s_version="v1.16" >}}
+{{< feature-state state="beta" for_k8s_version="v1.18" >}}
 
 If you have enabled the `TopologyManager`
 [feature gate](/docs/reference/command-line-tools-reference/feature-gates/), then
@@ -412,7 +414,7 @@ enabled by default in 1.21.
 
 Note that by default, both configuration options described below,
 `shutdownGracePeriod` and `shutdownGracePeriodCriticalPods` are set to zero,
-thus not activating Graceful node shutdown functionality.
+thus not activating the graceful node shutdown functionality.
 To activate the feature, the two kubelet config settings should be configured appropriately and
 set to non-zero values.
 

content/en/docs/concepts/scheduling-eviction/pod-overhead.md

@@ -108,7 +108,7 @@ Once a Pod is scheduled to a node, the kubelet on that node creates a new {{< gl
 text="cgroup" term_id="cgroup" >}} for the Pod. It is within this pod that the underlying
 container runtime will create containers.
 
-If the resource has a limit defined for each container (Guaranteed QoS or Bustrable QoS with limits defined),
+If the resource has a limit defined for each container (Guaranteed QoS or Burstable QoS with limits defined),
 the kubelet will set an upper limit for the pod cgroup associated with that resource (cpu.cfs_quota_us for CPU
 and memory.limit_in_bytes memory). This upper limit is based on the sum of the container limits plus the `overhead`
 defined in the PodSpec.

content/en/docs/concepts/services-networking/ingress.md

@@ -74,7 +74,7 @@ A minimal Ingress resource example:
 
 {{< codenew file="service/networking/minimal-ingress.yaml" >}}
 
-As with all other Kubernetes resources, an Ingress needs `apiVersion`, `kind`, and `metadata` fields.
+An Ingress needs `apiVersion`, `kind`, `metadata` and `spec` fields.
 The name of an Ingress object must be a valid
 [DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
 For general information about working with config files, see [deploying applications](/docs/tasks/run-application/run-stateless-application-deployment/), [configuring containers](/docs/tasks/configure-pod-container/configure-pod-configmap/), [managing resources](/docs/concepts/cluster-administration/manage-deployment/).

content/en/docs/concepts/workloads/controllers/daemonset.md

@@ -76,9 +76,9 @@ A Pod Template in a DaemonSet must have a [`RestartPolicy`](/docs/concepts/workl
 The `.spec.selector` field is a pod selector.  It works the same as the `.spec.selector` of
 a [Job](/docs/concepts/workloads/controllers/job/).
 
-As of Kubernetes 1.8, you must specify a pod selector that matches the labels of the
-`.spec.template`. The pod selector will no longer be defaulted when left empty. Selector
-defaulting was not compatible with `kubectl apply`. Also, once a DaemonSet is created,
+You must specify a pod selector that matches the labels of the
+`.spec.template`.
+Also, once a DaemonSet is created,
 its `.spec.selector` can not be mutated. Mutating the pod selector can lead to the
 unintentional orphaning of Pods, and it was found to be confusing to users.
 
@@ -91,8 +91,8 @@ The `.spec.selector` is an object consisting of two fields:
 
 When the two are specified the result is ANDed.
 
-If the `.spec.selector` is specified, it must match the `.spec.template.metadata.labels`.
-Config with these not matching will be rejected by the API.
+The `.spec.selector` must match the `.spec.template.metadata.labels`.
+Config with these two not matching will be rejected by the API.
 
 ### Running Pods on select Nodes
 
@@ -107,7 +107,7 @@ If you do not specify either, then the DaemonSet controller will create Pods on
 
 ### Scheduled by default scheduler
 
-{{< feature-state for_kubernetes_version="1.17" state="stable" >}}
+{{< feature-state for_k8s_version="1.17" state="stable" >}}
 
 A DaemonSet ensures that all eligible nodes run a copy of a Pod. Normally, the
 node that a Pod runs on is selected by the Kubernetes scheduler. However,

content/en/docs/contribute/style/diagram-guide.md

@@ -8,7 +8,7 @@ weight: 15
 <!--Overview-->
 
 This guide shows you how to create, edit and share diagrams using the Mermaid
-Javascript library. Mermaid.js allows you to generate diagrams using a simple
+JavaScript library. Mermaid.js allows you to generate diagrams using a simple
 markdown-like syntax inside Markdown files. You can also use Mermaid to
 generate `.svg` or `.png` image files that you can add to your documentation.
 

content/en/docs/reference/access-authn-authz/abac.md

@@ -33,13 +33,13 @@ properties:
       - `group`, type string; if you specify `group`, it must match one of the groups of the authenticated user. `system:authenticated` matches all authenticated requests. `system:unauthenticated` matches all unauthenticated requests.
     - Resource-matching properties:
       - `apiGroup`, type string; an API group.
-        - Ex: `extensions`
+        - Ex: `apps`, `networking.k8s.io`
         - Wildcard: `*` matches all API groups.
       - `namespace`, type string; a namespace.
         - Ex: `kube-system`
         - Wildcard: `*` matches all resource requests.
       - `resource`, type string; a resource type
-        - Ex: `pods`
+        - Ex: `pods`, `deployments`
         - Wildcard: `*` matches all resource requests.
     - Non-resource-matching properties:
       - `nonResourcePath`, type string; non-resource request paths.

content/en/docs/reference/access-authn-authz/rbac.md

@@ -384,11 +384,11 @@ rules:
 ```
 
 Allow reading/writing Deployments (at the HTTP level: objects with `"deployments"`
-in the resource part of their URL) in both the `"extensions"` and `"apps"` API groups:
+in the resource part of their URL) in the `"apps"` API groups:
 
 ```yaml
 rules:
-- apiGroups: ["extensions", "apps"]
+- apiGroups: ["apps"]
   #
   # at the HTTP level, the name of the resource for accessing Deployment
   # objects is "deployments"
@@ -397,7 +397,7 @@ rules:
 ```
 
 Allow reading Pods in the core API group, as well as reading or writing Job
-resources in the `"batch"` or `"extensions"` API groups:
+resources in the `"batch"` API group:
 
 ```yaml
 rules:
@@ -407,7 +407,7 @@ rules:
   # objects is "pods"
   resources: ["pods"]
   verbs: ["get", "list", "watch"]
-- apiGroups: ["batch", "extensions"]
+- apiGroups: ["batch"]
   #
   # at the HTTP level, the name of the resource for accessing Job
   # objects is "jobs"
@@ -517,23 +517,14 @@ subjects:
   namespace: kube-system
 ```
 
-For all service accounts in the "qa" group in any namespace:
+For all service accounts in the "qa" namespace:
 
 ```yaml
 subjects:
 - kind: Group
   name: system:serviceaccounts:qa
   apiGroup: rbac.authorization.k8s.io
 ```
-For all service accounts in the "dev" group in the "development" namespace:
-
-```yaml
-subjects:
-- kind: Group
-  name: system:serviceaccounts:dev
-  apiGroup: rbac.authorization.k8s.io
-  namespace: development
-```
 
 For all service accounts in any namespace:
 

content/en/docs/reference/glossary/cadvisor.md

@@ -4,7 +4,7 @@ id: cadvisor
 date: 2021-12-09
 full_link: https://github.com/google/cadvisor/
 short_description: >
-  Tool that provides understanding of the resource usage and perfomance characteristics for containers 
+  Tool that provides understanding of the resource usage and performance characteristics for containers
 aka:
 tags:
 - tool

content/en/docs/reference/labels-annotations-taints/_index.md

@@ -168,6 +168,44 @@ Used on: Pod
 This annotation is used to set [Pod Deletion Cost](/docs/concepts/workloads/controllers/replicaset/#pod-deletion-cost)
 which allows users to influence ReplicaSet downscaling order. The annotation parses into an `int32` type.
 
+### kubernetes.io/ingress-bandwidth
+
+{{< note >}}
+Ingress traffic shaping annotation is an experimental feature.
+If you want to enable traffic shaping support, you must add the `bandwidth` plugin to your CNI configuration file (default `/etc/cni/net.d`) and
+ensure that the binary is included in your CNI bin dir (default `/opt/cni/bin`).
+{{< /note >}}
+
+Example: `kubernetes.io/ingress-bandwidth: 10M`
+
+Used on: Pod
+
+You can apply quality-of-service traffic shaping to a pod and effectively limit its available bandwidth.
+Ingress traffic (to the pod) is handled by shaping queued packets to effectively handle data.
+To limit the bandwidth on a pod, write an object definition JSON file and specify the data traffic
+speed using `kubernetes.io/ingress-bandwidth` annotation. The unit used for specifying ingress
+rate is bits per second, as a [Quantity](/docs/reference/kubernetes-api/common-definitions/quantity/).
+For example, `10M` means 10 megabits per second.
+
+### kubernetes.io/egress-bandwidth
+
+{{< note >}}
+Egress traffic shaping annotation is an experimental feature.
+If you want to enable traffic shaping support, you must add the `bandwidth` plugin to your CNI configuration file (default `/etc/cni/net.d`) and
+ensure that the binary is included in your CNI bin dir (default `/opt/cni/bin`).
+{{< /note >}}
+
+Example: `kubernetes.io/egress-bandwidth: 10M`
+
+Used on: Pod
+
+Egress traffic (from the pod) is handled by policing, which simply drops packets in excess of the configured rate.
+The limits you place on a pod do not affect the bandwidth of other pods.
+To limit the bandwidth on a pod, write an object definition JSON file and specify the data traffic
+speed using `kubernetes.io/egress-bandwidth` annotation. The unit used for specifying egress
+rate is bits per second, as a [Quantity](/docs/reference/kubernetes-api/common-definitions/quantity/).
+For example, `10M` means 10 megabits per second.
+
 ### beta.kubernetes.io/instance-type (deprecated)
 
 {{< note >}} Starting in v1.17, this label is deprecated in favor of [node.kubernetes.io/instance-type](#nodekubernetesioinstance-type). {{< /note >}}

content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md

@@ -14,7 +14,6 @@ card:
 This page shows how to install the `kubeadm` toolbox.
 For information on how to create a cluster with kubeadm once you have performed this installation process, see the [Using kubeadm to Create a Cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) page.
 
-{{% dockershim-removal %}}
 
 ## {{% heading "prerequisites" %}}
 

content/en/docs/setup/production-environment/tools/kubeadm/kubelet-integration.md

@@ -10,6 +10,8 @@ weight: 80
 
 {{% dockershim-removal %}}
 
+{{< feature-state for_k8s_version="v1.11" state="stable" >}}
+
 The lifecycle of the kubeadm CLI tool is decoupled from the
 [kubelet](/docs/reference/command-line-tools-reference/kubelet), which is a daemon that runs
 on each node within the Kubernetes cluster. The kubeadm CLI tool is executed by the user when Kubernetes is

content/en/docs/tasks/administer-cluster/access-cluster-api.md

@@ -150,7 +150,7 @@ describes how you can configure this as a cluster administrator.
 
 ### Programmatic access to the API
 
-Kubernetes officially supports client libraries for [Go](#go-client), [Python](#python-client), [Java](#java-client), [dotnet](#dotnet-client), [Javascript](#javascript-client), and [Haskell](#haskell-client). There are other client libraries that are provided and maintained by their authors, not the Kubernetes team. See [client libraries](/docs/reference/using-api/client-libraries/) for accessing the API from other languages and how they authenticate.
+Kubernetes officially supports client libraries for [Go](#go-client), [Python](#python-client), [Java](#java-client), [dotnet](#dotnet-client), [JavaScript](#javascript-client), and [Haskell](#haskell-client). There are other client libraries that are provided and maintained by their authors, not the Kubernetes team. See [client libraries](/docs/reference/using-api/client-libraries/) for accessing the API from other languages and how they authenticate.
 
 #### Go client
 

content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md

@@ -16,7 +16,6 @@ weight: 30
 
 You can use Kubernetes to run a mixture of Linux and Windows nodes, so you can mix Pods that run on Linux on with Pods that run on Windows. This page shows how to register Windows nodes to your cluster.
 
-{{% dockershim-removal %}}
 
 ## {{% heading "prerequisites" %}}
  {{< version-check >}}

content/en/docs/tasks/configure-pod-container/migrate-from-psp.md

@@ -312,7 +312,7 @@ appropriate Pod Security profile is applied to new namespaces.
 
 You can also statically configure the Pod Security admission controller to set a default enforce,
 audit, and/or warn level for unlabeled namespaces. See
-[Configure the Admission Controller](docs/tasks/configure-pod-container/enforce-standards-admission-controller/#configure-the-admission-controller)
+[Configure the Admission Controller](/docs/tasks/configure-pod-container/enforce-standards-admission-controller/#configure-the-admission-controller)
 for more information.
 
 ## 5. Disable PodSecurityPolicy {#disable-psp}