Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KEP-1880 alpha: IPAddresses #37620

Merged
merged 10 commits into from
Apr 4, 2023
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@ For a reference to old feature gates that are removed, please refer to
| `MinDomainsInPodTopologySpread` | `false` | Beta | 1.25 | |
| `MinimizeIPTablesRestore` | `false` | Alpha | 1.26 | - |
| `MultiCIDRRangeAllocator` | `false` | Alpha | 1.25 | |
| `MultiCIDRServiceAllocator` | `false` | Alpha | 1.27 | |
| `NetworkPolicyStatus` | `false` | Alpha | 1.24 | |
| `NodeInclusionPolicyInPodTopologySpread` | `false` | Alpha | 1.25 | 1.25 |
| `NodeInclusionPolicyInPodTopologySpread` | `true` | Beta | 1.26 | |
Expand Down Expand Up @@ -625,6 +626,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `MixedProtocolLBService`: Enable using different protocols in the same `LoadBalancer` type
Service instance.
- `MultiCIDRRangeAllocator`: Enables the MultiCIDR range allocator.
- `MultiCIDRServiceAllocator`: Track IP address allocations for Service cluster IPs using IPAddress objects.
- `NetworkPolicyEndPort`: Enable use of the field `endPort` in NetworkPolicy objects,
allowing the selection of a port range instead of a single port.
- `NetworkPolicyStatus`: Enable the `status` subresource for NetworkPolicy objects.
Expand Down
40 changes: 38 additions & 2 deletions content/en/docs/reference/networking/virtual-ips.md
Original file line number Diff line number Diff line change
Expand Up @@ -276,14 +276,16 @@ populated in terms of the Service's virtual IP address (and port).
One of the primary philosophies of Kubernetes is that you should not be
exposed to situations that could cause your actions to fail through no fault
of your own. For the design of the Service resource, this means not making
you choose your own port number if that choice might collide with
you choose your own IP address if that choice might collide with
someone else's choice. That is an isolation failure.

In order to allow you to choose a port number for your Services, we must
In order to allow you to choose an IP address for your Services, we must
ensure that no two Services can collide. Kubernetes does that by allocating each
Service its own IP address from within the `service-cluster-ip-range`
CIDR range that is configured for the {{< glossary_tooltip term_id="kube-apiserver" text="API Server" >}}.

#### IP address allocation tracking

To ensure each Service receives a unique IP, an internal allocator atomically
updates a global allocation map in {{< glossary_tooltip term_id="etcd" >}}
prior to creating each Service. The map object must exist in the registry for
Expand All @@ -296,6 +298,40 @@ in-memory locking). Kubernetes also uses controllers to check for invalid
assignments (e.g. due to administrator intervention) and for cleaning up allocated
IP addresses that are no longer used by any Services.

{{< feature-state for_k8s_version="v1.27" state="alpha" >}}
If you enable the `MultiCIDRServiceAllocator`
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/),
the control plane replaces the existing etcd allocator with a new one, using IPAddress
objects instead of an internal global allocation map. The ClusterIP address
associated to each `Service` will have a referenced IPAddress object.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you also need to enable an alpha API group for this to actually work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, good catch, it needs to enable networking.k8s.io/v1alpha1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is interesting, so if you only enable the feature gate it will fail to create any service

aojea marked this conversation as resolved.
Show resolved Hide resolved

The background controller is also replaced by a new one to handle the new IPAddress
objects and the migration from the old allocator model.

One of the main benefits of the new allocator is that it removes the size limitations
for the `service-cluster-ip-range`, there is no limitations for IPv4 and for IPv6
users can use masks equal or larger than /64 (previously it was /108).

Users now will be able to inspect the IP addresses assigned to their Services, and
new network APIs, like Gateway API, can use this new object to extend the Kubernetes
networking capabilities overcoming the limitations of current Services API.
aojea marked this conversation as resolved.
Show resolved Hide resolved

```shell
kubectl get services
```
```
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 2001:db8:1:2::1 <none> 443/TCP 3d1h
```
```shell
kubectl get ipaddresses
```
```
NAME PARENTREF
2001:db8:1:2::1 services/default/kubernetes
2001:db8:1:2::a services/kube-system/kube-dns
```

#### IP address ranges for Service virtual IP addresses {#service-ip-static-sub-range}

{{< feature-state for_k8s_version="v1.25" state="beta" >}}
Expand Down