Feature/terraform (#39)

* Re-added terraform
* Added terraform to create a kind cluster with local registry
* Added docker network to kind through terraform
* Added k3s/k3d through terraform
* Made metrics optional
* Made tracing and metrics optional
* let traefik handle configurable node ips
* Added mkcert generation scripts

.gitignore

@@ -86,4 +86,6 @@ site/
 cacerts.yaml
 .terraform.lock.hcl
 charts/
-coredns-*.yaml
+coredns-*.yaml
+src/clusters/kind/devinfra-config
+src/clusters/k3d/k3s-devinfra.yaml

docs/installation-terraform.md

@@ -0,0 +1,54 @@
+# Installation
+
+## Prerequisites
+
+Check you have all the [cli tools](./preparation/cli.md) installed and that you can connect to a running cluster.
+
+```bash
+kubectl cluster-info --context <your cluster name>
+```
+
+> When you created your local cluster on windows and now running WSL, copy your windows context to your WSL installation.
+
+```bash
+cp /mnt/c/users/<your username>/.kube/config ~/.kube/config
+```
+
+## Update Local DNS
+
+Since we are running completely locally, we want to add some entries to the hosts file to forward domainnames to our locally running cluster.
+On Windows, open `C:\Windows\system32\drivers\etc\hosts` in an editor as `Administrator` and add the following entries.
+On Linux edit your `/etc/hosts` file as root.
+
+```text
+127.0.0.1 k8s.local infrastructure.k8s.local traefik.infrastructure.k8s.local
+127.0.0.1 jaeger.infrastructure.k8s.local prometheus.infrastructure.k8s.local grafana.infrastructure.k8s.local
+# 127.0.0.1 seq.infrastructure.k8s.local
+127.0.0.1 vault.infrastructure.k8s.local
+127.0.0.1 es.infrastructure.k8s.local kibana.infrastructure.k8s.local
+127.0.0.1 login.k8s.local admin.login.k8s.local api.login.k8s.local
+127.0.0.1 argo.k8s.local
+127.0.0.1 linkerd.infrastructure.k8s.local
+```
+
+## Installer
+
+### Linux/WSL
+
+Next run the installer in a Linux/WSL shell:
+
+```bash
+cd src/terraform
+./generate-certificate.sh
+terraform init
+terraform apply -auto-approve
+```
+
+### Windows
+
+```bash
+cd src/terraform
+./generate-certificate.ps1
+terraform init
+terraform apply -auto-approve
+```

docs/preparation/cli.md

@@ -12,6 +12,8 @@ choco install base64
 choco install mkcert
 choco install kubernetes-cli
 choco install kubernetes-helm
+choco install linkerd2
+choco install k3d
 ```
 
 ### Installing the argoCD CLI
@@ -39,12 +41,12 @@ Now move `argocd.exe` to a location in your PATH.
 sudo apt-get update
 sudo apt-get install -y apt-transport-https ca-certificates curl wget libnss3-tools cl-base64
 
-# kind
+# kind (if needed)
 curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.11.1/kind-linux-amd64
 chmod +x ./kind
 mv ./kind /usr/local/bin/kind
 
-# mkcert
+# mkcert (if needed)
 wget -O mkcert https://github.com/FiloSottile/mkcert/releases/download/v1.4.3/mkcert-v1.4.3-linux-amd64
 sudo mv mkcert-v1.4.3-linux-amd64 /usr/local/bin/mkcert
 sudo chmod +x /usr/local/bin/mkcert
@@ -62,11 +64,14 @@ echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt
 sudo apt-get update
 sudo apt-get install helm
 
-# argocd
+# argocd (if needed)
 sudo curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
 sudo chmod +x /usr/local/bin/argocd
 
 #linkerd
 curl -sL https://run.linkerd.io/install | sh
 export PATH=$PATH:~/.linkerd2/bin
+
+#k3d (if needed)
+wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash
 ```

docs/preparation/docker-desktop.md

@@ -1,6 +1,6 @@
 # Docker desktop
 
-To add WSL2 support to your k8s cluster of Docker desktop, read the details in the [official blog](https://kubernetes.io/blog/2020/05/21/wsl-docker-kubernetes-on-the-windows-desktop/)
+To add WSL2 support to your k8s cluster of `Docker Desktop`, read the details in the [official blog](https://kubernetes.io/blog/2020/05/21/wsl-docker-kubernetes-on-the-windows-desktop/)
 
 After installation, make sure you enable `WSL2` in the settings
 
@@ -9,3 +9,5 @@ After installation, make sure you enable `WSL2` in the settings
 and you enable `kubernetes` of course:
 
 ![enable k8s](../images/docker-desktop-kubernetes-settings.png)
+
+`Docker Desktop` creates a single-node cluster.

docs/preparation/k3d.md

@@ -0,0 +1,47 @@
+# K3D
+
+## K3D installation
+
+### K3D CLI installation
+
+On Windows:
+
+```Powershell
+choco install k3d
+```
+
+On linux:
+
+```bash
+wget -q -O - https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash
+```
+
+### Creating a cluster with terraform
+
+Edit `terraform.tfvars` to your needs. Then run `terraform apply` to create the cluster.
+
+```shell
+cd ./src/clusters/k3d
+terraform init
+terraform apply --auto-approve
+```
+
+When using WSL2 (Windows Subsystem for Linux), you need to copy the context configuration to your .kubeconfig file on the WSL2 home directory or the other way round, depending on which shell you use to run the terraform code.
+
+```powershell
+cp /mnt/c/users/$(whoami)/.kube/config ~/.kube/config
+```
+
+or
+
+```powershell
+cp ~/.kube/config /mnt/c/users/$(whoami)/.kube/config
+```
+
+The cluster creation automatically add configuration to connect to the new cluster
+
+```powershell
+kubectl cluster-info --context k3d-devinfra
+```
+
+To delete the cluster again, just run `terraform destroy --auto-approve`.

docs/preparation/kind.md

@@ -17,6 +17,16 @@ curl -sL https://run.linkerd.io/install | sh
 export PATH=$PATH:~/.linkerd2/bin
 ```
 
+### Creating a cluster with terraform
+
+Edit `terraform.tfvars` to your needs. Then run `terraform apply` to create the cluster.
+
+```shell
+cd ./src/clusters/kind
+terraform init
+terraform apply --auto-approve
+```
+
 ### Creating a 3-node k8s-cluster
 
 An example of running a multi-node cluster on docker can be done with `kind`. There are some restrictions with Windows. The provided config `./src/clusters/kind/config-devinfra.yaml` provides a 3-node cluster. There is also a `traefik` ingress test setup to verify your networking configuration.

mkdocs.yml

@@ -11,8 +11,10 @@ nav:
           - Docker desktop: './preparation/docker-desktop.md'
           - Minikube: './preparation/minikube.md'
           - Kind: './preparation/kind.md'
+          - K3D: './preparation/k3d.md'
         - Testing Ingress: './preparation/test-ingress-controller.md'
       - Install:
+        - Terraform: './installation-terraform.md'
         - Shell: './installation-shell.md'
         - ArgoCD: './installation-argocd.md'
     - About:

src/clusters/k3d/devinfra-template.yaml

@@ -0,0 +1,84 @@
+# k3d configuration file, saved as e.g. /home/me/myk3dcluster.yaml
+apiVersion: k3d.io/v1alpha3 # this will change in the future as we make everything more stable
+kind: Simple # internally, we also have a Cluster config, which is not yet available externally
+name:  ${cluster-name} # name that you want to give to your cluster (will still be prefixed with `k3d-`)
+servers: ${server-node-count} # same as `--servers 1`
+agents: ${worker-node-count} # same as `--agents 2`
+kubeAPI: # same as `--api-port myhost.my.domain:6445` (where the name would resolve to 127.0.0.1)
+  host: "${cluster-domain}" # important for the `server` setting in the kubeconfig
+  hostIP: "${cluster-ip}" # where the Kubernetes API will be listening on
+  hostPort: "${cluster-port}" # where the Kubernetes API listening port will be mapped to on your host system
+image: rancher/k3s:${k3s-version} # same as `--image rancher/k3s:v1.20.4-k3s1`
+network: ${network} # same as `--network my-custom-net`
+# subnet: "172.28.0.0/16" # same as `--subnet 172.28.0.0/16`
+token: superSecretToken # same as `--token superSecretToken`
+# volumes: # repeatable flags are represented as YAML lists
+#   - volume: /my/host/path:/path/in/node # same as `--volume '/my/host/path:/path/in/node@server:0;agent:*'`
+#     nodeFilters:
+#       - server:0
+#       - agent:*
+ports:
+  - port: 80:80 # same as `--port '8080:80@loadbalancer'`
+    nodeFilters:
+      - loadbalancer
+  - port: 443:443 # same as `--port '8080:80@loadbalancer'`
+    nodeFilters:
+      - loadbalancer
+  - port: 8100:8100 # same as `--port '8080:80@loadbalancer'`
+    nodeFilters:
+      - loadbalancer
+# env:
+#   - envVar: bar=baz # same as `--env 'bar=baz@server:0'`
+#     nodeFilters:
+#       - server:0
+registries: # define how registries should be created or used
+  create: # creates a default registry to be used with the cluster; same as `--registry-create registry.localhost`
+    name: registry.localhost
+    host: "0.0.0.0"
+    hostPort: "5000"
+#   use:
+#     - k3d-myotherregistry:5000 # some other k3d-managed registry; same as `--registry-use 'k3d-myotherregistry:5000'`
+#   config: | # define contents of the `registries.yaml` file (or reference a file); same as `--registry-config /path/to/config.yaml`
+#     mirrors:
+#       "my.company.registry":
+#         endpoint:
+#           - http://my.company.registry:5000
+options:
+  k3d: # k3d runtime settings
+    wait: true # wait for cluster to be usable before returining; same as `--wait` (default: true)
+    timeout: "60s" # wait timeout before aborting; same as `--timeout 60s`
+    disableLoadbalancer: false # same as `--no-lb`
+    disableImageVolume: false # same as `--no-image-volume`
+    disableRollback: false # same as `--no-Rollback`
+    loadbalancer:
+      configOverrides:
+        - settings.workerConnections=2048
+  k3s:
+    extraArgs:
+      # - arg: --cluster-cidr=172.18.0.0/16
+      #   nodeFilters:
+      #     - server:*
+      - arg: --disable=traefik
+        nodeFilters:
+          - server:*
+      - arg: --disable=metrics-server
+        nodeFilters:
+          - server:*
+  # k3s: # options passed on to K3s itself
+  #   extraArgs: # additional arguments passed to the `k3s server|agent` command; same as `--k3s-arg`
+  #     - arg: --tls-san=my.host.domain
+  #       nodeFilters:
+  #         - server:*
+  #   nodeLabels:
+  #     - label: foo=bar # same as `--k3s-node-label 'foo=bar@agent:1'` -> this results in a Kubernetes node label
+  #       nodeFilters:
+  #         - agent:1
+  # kubeconfig:
+  #   updateDefaultKubeconfig: true # add new cluster to your default Kubeconfig; same as `--kubeconfig-update-default` (default: true)
+  #   switchCurrentContext: true # also set current-context to the new cluster's context; same as `--kubeconfig-switch-context` (default: true)
+  # runtime: # runtime (docker) specific options
+  #   gpuRequest: all # same as `--gpus all`
+  #   labels:
+  #     - label: bar=baz # same as `--runtime-label 'bar=baz@agent:1'` -> this results in a runtime (docker) container label
+  #       nodeFilters:
+  #         - agent:1

src/clusters/k3d/devinfra.yaml

@@ -0,0 +1,84 @@
+# k3d configuration file, saved as e.g. /home/me/myk3dcluster.yaml
+apiVersion: k3d.io/v1alpha3 # this will change in the future as we make everything more stable
+kind: Simple # internally, we also have a Cluster config, which is not yet available externally
+name: devinfra # name that you want to give to your cluster (will still be prefixed with `k3d-`)
+servers: 1 # same as `--servers 1`
+agents: 2 # same as `--agents 2`
+kubeAPI: # same as `--api-port myhost.my.domain:6445` (where the name would resolve to 127.0.0.1)
+  host: "k8s.local" # important for the `server` setting in the kubeconfig
+  hostIP: "127.0.0.1" # where the Kubernetes API will be listening on
+  hostPort: "6445" # where the Kubernetes API listening port will be mapped to on your host system
+image: rancher/k3s:v1.21.6-k3s1 # same as `--image rancher/k3s:v1.20.4-k3s1`
+network: k3d # same as `--network my-custom-net`
+# subnet: "172.28.0.0/16" # same as `--subnet 172.28.0.0/16`
+token: superSecretToken # same as `--token superSecretToken`
+# volumes: # repeatable flags are represented as YAML lists
+#   - volume: /my/host/path:/path/in/node # same as `--volume '/my/host/path:/path/in/node@server:0;agent:*'`
+#     nodeFilters:
+#       - server:0
+#       - agent:*
+ports:
+  - port: 80:80 # same as `--port '8080:80@loadbalancer'`
+    nodeFilters:
+      - loadbalancer
+  - port: 443:443 # same as `--port '8080:80@loadbalancer'`
+    nodeFilters:
+      - loadbalancer
+  - port: 8100:8100 # same as `--port '8080:80@loadbalancer'`
+    nodeFilters:
+      - loadbalancer
+# env:
+#   - envVar: bar=baz # same as `--env 'bar=baz@server:0'`
+#     nodeFilters:
+#       - server:0
+registries: # define how registries should be created or used
+  create: # creates a default registry to be used with the cluster; same as `--registry-create registry.localhost`
+    name: registry.localhost
+    host: "0.0.0.0"
+    hostPort: "5000"
+#   use:
+#     - k3d-myotherregistry:5000 # some other k3d-managed registry; same as `--registry-use 'k3d-myotherregistry:5000'`
+#   config: | # define contents of the `registries.yaml` file (or reference a file); same as `--registry-config /path/to/config.yaml`
+#     mirrors:
+#       "my.company.registry":
+#         endpoint:
+#           - http://my.company.registry:5000
+options:
+  k3d: # k3d runtime settings
+    wait: true # wait for cluster to be usable before returining; same as `--wait` (default: true)
+    timeout: "60s" # wait timeout before aborting; same as `--timeout 60s`
+    disableLoadbalancer: false # same as `--no-lb`
+    disableImageVolume: false # same as `--no-image-volume`
+    disableRollback: false # same as `--no-Rollback`
+    loadbalancer:
+      configOverrides:
+        - settings.workerConnections=2048
+  k3s:
+    extraArgs:
+      # - arg: --cluster-cidr=172.18.0.0/16
+      #   nodeFilters:
+      #     - server:*
+      - arg: --disable=traefik
+        nodeFilters:
+          - server:*
+      - arg: --disable=metrics-server
+        nodeFilters:
+          - server:*
+  # k3s: # options passed on to K3s itself
+  #   extraArgs: # additional arguments passed to the `k3s server|agent` command; same as `--k3s-arg`
+  #     - arg: --tls-san=my.host.domain
+  #       nodeFilters:
+  #         - server:*
+  #   nodeLabels:
+  #     - label: foo=bar # same as `--k3s-node-label 'foo=bar@agent:1'` -> this results in a Kubernetes node label
+  #       nodeFilters:
+  #         - agent:1
+  # kubeconfig:
+  #   updateDefaultKubeconfig: true # add new cluster to your default Kubeconfig; same as `--kubeconfig-update-default` (default: true)
+  #   switchCurrentContext: true # also set current-context to the new cluster's context; same as `--kubeconfig-switch-context` (default: true)
+  # runtime: # runtime (docker) specific options
+  #   gpuRequest: all # same as `--gpus all`
+  #   labels:
+  #     - label: bar=baz # same as `--runtime-label 'bar=baz@agent:1'` -> this results in a runtime (docker) container label
+  #       nodeFilters:
+  #         - agent:1