EKS Cluster w/ Windows OS Nodes

This example shows how to provision an EKS cluster with Windows OS based nodes.

Prerequisites:

Ensure that you have the following tools installed locally:

1. aws cli
2. kubectl
3. terraform

Deploy

To provision this example:

terraform init
terraform apply

Enter yes at command prompt to apply

Validate

The following command will update the kubeconfig on your local machine and allow you to interact with your EKS Cluster using kubectl to validate the deployment.

1. Run update-kubeconfig command:

aws eks --region us-east-1 update-kubeconfig --name ipv4-prefix-delegation

2. List the nodes running currently

kubectl get nodes

# Output should look similar to below
NAME                         STATUS   ROLES    AGE     VERSION
ip-10-0-47-78.ec2.internal   Ready    <none>   7m12s   v1.24.7-eks-fb459a0

3. Inspect the nodes settings and check for the max allocatable pods - should be 110 in this scenario with m5.xlarge:

kubectl describe node ip-10-0-47-78.ec2.internal

# Output should look similar to below (truncated for brevity)
  Capacity:
    attachable-volumes-aws-ebs:  25
    cpu:                         4
    ephemeral-storage:           104845292Ki
    hugepages-1Gi:               0
    hugepages-2Mi:               0
    memory:                      15919124Ki
    pods:                        110 # <- this should be 110
  Allocatable:
    attachable-volumes-aws-ebs:  25
    cpu:                         3920m
    ephemeral-storage:           95551679124
    hugepages-1Gi:               0
    hugepages-2Mi:               0
    memory:                      14902292Ki
    pods:                        110 # <- this should be 110

4. List out the pods running currently:

kubectl get pods -A

# Output should look like below
NAMESPACE     NAME                       READY   STATUS        RESTARTS   AGE
kube-system   aws-node-77rwz             1/1     Running       0          6m5s
kube-system   coredns-657694c6f4-fdz4f   1/1     Running       0          5m12s
kube-system   coredns-657694c6f4-kvm92   1/1     Running       0          5m12s
kube-system   kube-proxy-plwlc           1/1     Running       0          6m5s

5. Inspect one of the aws-node-* (AWS VPC CNI) pods to ensure prefix delegation is enabled and warm prefix target is 1:

kubectl describe pod aws-node-77rwz -n kube-system

# Output should look like below (truncated for brevity)
  Environment:
    ADDITIONAL_ENI_TAGS:                    {}
    AWS_VPC_CNI_NODE_PORT_SUPPORT:          true
    AWS_VPC_ENI_MTU:                        9001
    AWS_VPC_K8S_CNI_CONFIGURE_RPFILTER:     false
    AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG:     false
    AWS_VPC_K8S_CNI_EXTERNALSNAT:           false
    AWS_VPC_K8S_CNI_LOGLEVEL:               DEBUG
    AWS_VPC_K8S_CNI_LOG_FILE:               /host/var/log/aws-routed-eni/ipamd.log
    AWS_VPC_K8S_CNI_RANDOMIZESNAT:          prng
    AWS_VPC_K8S_CNI_VETHPREFIX:             eni
    AWS_VPC_K8S_PLUGIN_LOG_FILE:            /var/log/aws-routed-eni/plugin.log
    AWS_VPC_K8S_PLUGIN_LOG_LEVEL:           DEBUG
    DISABLE_INTROSPECTION:                  false
    DISABLE_METRICS:                        false
    DISABLE_NETWORK_RESOURCE_PROVISIONING:  false
    ENABLE_IPv4:                            true
    ENABLE_IPv6:                            false
    ENABLE_POD_ENI:                         false
    ENABLE_PREFIX_DELEGATION:               true # <- this should be set to true
    MY_NODE_NAME:                            (v1:spec.nodeName)
    WARM_ENI_TARGET:                        1 # <- this should be set to 1
    WARM_PREFIX_TARGET:                     1
    ...

Destroy

To teardown and remove the resources created in this example:

terraform destroy -auto-approve