Change kube-proxy mode from IPTables to IPVS

[RobertLucian]

Description

The current mode of operation of kube-proxy is done through IPTables. The big limitation of IPTables is that it gets increasingly slow as more deployments/services/nodes are added to the cluster. IPTables has a complexity of O(n) where n is proportional to the size of the cluster, whereas IPVS has O(1) complexity (as it is based on hashes).

However, the way kube-proxy programs the iptables rules means that it is nominally an O(n) style algorithm, where n grows roughly in proportion to your cluster size (or more precisely the number of services and number of backend pods behind each service).

This has been a stable implementation on k8s since 1.11.

Motivation

It allows the cluster to grow to very big sizes while keeping the latencies and resource consumption of the control plane within nominal values. This is especially important to enterprise users who can see their clusters grow big.

Limitation

It doesn't seem like supporting this on EKS/GKE is a trivial task and that's because the IPVS dependencies are required on each node's image.

On the bright side, enabling this for an on-premises cluster is trivial.

Additional context

1. https://kubernetes.io/docs/concepts/services-networking/service/#proxy-mode-ipvs
2. https://www.objectif-libre.com/en/blog/2018/03/19/kubernetes-ipvs/
3. https://www.projectcalico.org/comparing-kube-proxy-modes-iptables-or-ipvs/
4. https://medium.com/@jeremy.i.cowan/the-problem-with-kube-proxy-enabling-ipvs-on-eks-169ac22e237e