EKS Quickstart

This project is an example of how you can combine the AWS Cloud Development Kit (CDK) and the AWS Elastic Kubernetes Serivce (EKS) to quickly deploy a more complete and "production ready" Kubernetes environment on AWS.

What does this QuickStart create for you:

1. An appropriate VPC (/22 CDIR w/1024 IPs by default - though you can edit this in eks_cluster.py) with public and private subnets across three availabilty zones.
2. A new EKS cluster with:
   1. A dedicated new IAM role to create it from. The role that creates the cluster is a permanent, and rather hidden, full admin role that doesn't appear in nor is subject to the aws-auth config map. So, you want a dedicated role explicity for that purpose like CDK does for you here that you can then restrict access to assume unless you need it (e.g. you lock yourself out of the cluster with by making a mistake in the aws-auth configmap).
   2. A new Managed Node Group with 3 x m5.large instances spread across 3 Availability Zones.
3. The AWS Load Balancer Controller (https://kubernetes-sigs.github.io/aws-load-balancer-controller) to allow you to seamlessly use ALBs for Ingress and NLB for Services.
4. External DNS (https://github.com/kubernetes-sigs/external-dns) to allow you to automatically create/update Route53 entries to point your 'real' names at your Ingresses and Services.
5. A new managed Amazon Elasticsearch Domain and an aws-for-fluent-bit DaemonSet (https://github.com/aws/aws-for-fluent-bit) to ship all your container logs there - including enriching them with the Kubernetes metadata using the kubernetes fluent-bit filter.
6. (Temporarily until the AWS Managed Prometheus/Grafana are available) The kube-prometheus Operator (https://github.com/prometheus-operator/kube-prometheus) which gives you a Prometheus that will collect all your cluster metrics as well as a Grafana to visualise them.
   1. TODO: Add some initial alerts for sensible common items in the cluster via Prometheus/Alertmanager
7. The AWS EBS CSI Driver (https://github.com/kubernetes-sigs/aws-ebs-csi-driver)
8. The AWS EFS CSI Driver (https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html)
9. A OPA Gatekeeper to enforce prevenetative secruity and operational policies (https://github.com/open-policy-agent/gatekeeper)
   1. TODO: Add some sensible initial policies to make our cluster 'secure by default'
10. The cluster autoscaler (CA) (https://github.com/kubernetes/autoscaler)
11. The metrics-server (required for the Horizontal Pod Autoscaler (HPA)) (https://github.com/kubernetes-sigs/metrics-server)
12. TODO: A GitOps Pipeline based on CodeBuild doing another cdk deploy whenever eks_cluster.py changes

Why Cloud Development Kit (CDK)?

TODO: Explain the benefits of the CDK

Getting started

Install Local Prerequisites

There are some prerequsistes you likely will need to install on the machine doing your environment bootstrapping including Node, Python, the AWS CLI, the CDK, fluxctl and Helm

Ubuntu 20.04.2 LTS (including via Windows 10's WSL)

Run sudo ./ubuntu-prepreqs.sh

Mac

TODO: Make equivilent bootstrap script to get a Mac

Amazon Linux 2 / Cloud9

TODO: Make an equivilent boostrap script for Amazon Linux 2 including Cloud 9

Deploy the VPC and EKS cluster with frequntly used add-ons

1. Make sure that you have your AWS CLI configured with administrative access to the AWS account in question (e.g. an aws s3 ls works)
   1. This can be via setting your access key and secret in your .aws folder via aws configure or in your environment variables by copy and pasting from AWS SSO etc.
2. Run cd eks-quickstart/cluster-bootstrap
3. Run pip install -r requirements.txt to install the required Python bits of the CDK
4. Run export CDK_DEPLOY_REGION=ap-southeast-2 replacing ap-southeast-2 with your region of choice
5. Run export CDK_DEPLOY_ACCOUNT=123456789123 replacing 123456789123 with your AWS account number
6. (Only required the first time you use the CDK in this account) Run cdk bootstrap to create the S3 bucket where it puts the CDK puts its artifacts
7. (Only required the first time ES in VPC mode is used in this account) Run aws iam create-service-linked-role --aws-service-name es.amazonaws.com
8. Run cdk deploy --require-approval never
9. (Temporary until it is added to our Helm Chart - PR open) Run kubectl edit configmap fluentbit-0-1-6-aws-for-fluent-bit --namespace=cluster-addons and add the following to the bottom Replace_Dots On

Set up your Client VPN to access the environment

1. Create the necessary keys and upload them to ACM as per https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/client-authentication.html#mutual
2. Run cd client-vpn
3. Edit client_vpn.py and put the ARNs for your client and server certs as well as the client_cidr_block and target_network_cidr if required
4. Run pip install -r requirements.txt
5. Run cdk deploy --require-approval never
6. Go to the Client VPN Endpoints Service in the AWS Console
7. Go to the Associations Tab and click Associate
8. Pick the EKSClusterStack/VPC for the VPC
9. Pick any subnet in the Choose a subnet to associate dropdown box
10. Click the Associate button
11. Go to the Security Groups tab
12. Click the Apply Security Groups button
13. Tick the box next to the security group whose Group Name starts with eks-cluster-sg-clusterXXXXXXX... (this is the SG that has access to the EKS Control Plane private endpoints)
14. Click Apply Security Groups

TODO: Complete/improve the VPN instructions including how to set up the client

Allow access to the Elasticsearch and Kibana to query your logs

We put the Elasticsearch both in the VPC (i.e. not on the Internet) as well as in the same Security Group we use for controlling access to our EKS Control Plane.

We did this so that if we put the Client VPN in that security group as well then it will have access from a network perspective to both manage EKS and Elasticsearch/Kibana.

Since this ElasticSearch can only be reached from a network perspective if you are running within this VPC, or have private access to it via a VPN or DirectConnect, then it is not that risky to allow 'open access' to it - especially in a Proof of Concept (POC) environment.

In order to do this:

1. Go to the Amazon Elaticsearch Service within the AWS Console
2. Click on the Domain that starts with eksclus-
3. Click on the Actions button on top and choose Modify Access Policy
4. In the Domain access policy dropdown choose "Allow open access to the domain" and click Submit

TODO: Add instructions for the first-time Kibana Index setup

TODO: Walk through how to do a few basic things in Kibana with your cluster logs

Checking out Grafana and the out-of-the-box metrics dashboards

TODO: Walk through how to get to the out-of-the-box metrics dashboards in Grafana

Deploy some sample apps to explore our new Kubernetes environment and its features

TODO: Walk through deploying some apps that show off some of the cluster add-ons we've installed

Upgrading your cluster

TODO: Walk through how to do an EKS Cluster to a new Kubernetes version and/or the Managed Node Group to the latest AMI upgrade via CDK

Upgrading an add-on

TODO: Walk through how to upgrade an individual add-on manifest/chart via CDK