Add support for dual-stack Pod/Service CIDRs and node IP addresses

[brandond]

Proposed Changes

• Add support for dual-stack cluster/service CIDRs and node addresses
• Begin transitioning the ip/externalip/hostname handoff (used by the K3s cloud provider) from labels to annotations, as labels do not allow the character-set necessary to pass through IPv6 addresses.
  • If a node has labels they will be maintained, but new nodes will use only annotations. The cloud provider will continue to check both, with annotations taking precedence.
• Dual-stack node addresses are not autodetected; if desired a dual-stack address list must be passed to --node-ip (and --node-external-ip as necessary). Kubeadm is taking the same approach: Dual-Stack support for Kubeadm kubernetes/kubeadm#1612 (comment)

This opens the door to dual-stack operation, even if flannel and the kube-router NPC don't support it yet.

Note: Cluster-DNS and the Kubernetes apiserver --advertise-address themselves do not support dual-stack operation yet: kubernetes/kubeadm#1612 (comment)

Types of Changes

Feature

Verification

• Confirm that K3s works as usual when specifying only IPv4 CIDRs, or when using defaults
• Confirm that specifying dual-stack CIDRs raises an error if not run with --disable-network-policy --flannel-backend=none

Example test:

• Add "ipv6": true, "fixed-cidr-v6": "fd7c:53a5:aef5:0::/64" to /etc/docker/daemon.json and restart docker
• Run K3s server with IPv6:

  docker run --name=k3s-server-1 --rm -it --privileged rancher/k3s:BUILD_TAG --debug server --cluster-cidr=10.42.0.0/16,fd42::/48 --service-cidr=10.43.0.0/16,fd43::/112 --disable-network-policy --flannel-backend=none --node-ip=172.17.0.2,fd7c:53a5:aef5::242:ac11:2
  

• Check for dual-stack CIDRs and node addresses:

  docker exec -it k3s-server-1 kubectl get nodes -o yaml | grep -EA6 'spec:|addresses:'
    spec:
      podCIDR: 10.42.0.0/24
      podCIDRs:
      - 10.42.0.0/24
      - fd42::/64
      providerID: k3s://7a78f12c7fb1
      taints:
  --
        addresses:
      - address: 172.17.0.2
        type: InternalIP
      - address: fd7c:53a5:aef5::242:ac11:2
        type: InternalIP
      - address: 7a78f12c7fb1
        type: Hostname
  

Linked Issues

For #3158

Further Comments

While the CLI has been directly changed to use StringSlices instead of strings, I have opted to add separate fields to the the Config types in order to retain backwards/forwards compatibility between builds with and without this change.

[briandowns]

The code itself seems fine however for this change, it'd probably be beneficial to include some basic tests. And link over to the associated rancher/docs update pr.

[unixfox]

I think #3049 can be closed right then?

[janeczku]

Should we close #3158 as well then?

[brandond]

Should we close #3158 as well then?

Issues for merged PRs don't get closed until QA validates the fix.

[janeczku]

Okay, i thought it had already been validated given that the issue's milestone (v1.21.0+k3s1) has already been released.

[brandond]

No, when QA validates it you'll see a comment from the QA team and the issue will be closed. Experimental features don't always see full validation from QA prior to release.