A GitLab CI pipeline that provisions a K3s Kubernetes cluster, and configures GitOps-based application deployment with Flux.
This project consists of a GitLab CI pipeline that:
- Provisions a K3s Kubernetes cluster with K3s Ansible inside an Ansible execution environment
- Installs Flux Operator to the cluster
- Configures Flux to sync with an existing Git repository containing manifests for the cluster applications and services.
- Creates TLS secrets for HashiCorp Vault, which is deployed in the synced repository.
Previously, I had managed my homelab cluster with a single repository that contained Ansible playbooks for installing K3s, and Terraform modules for installing Kubernetes services with the Kubernetes Terraform provider.
As the homelab grew, I found that the large Terraform state made updating applications slow. Furthermore, this system involved manually applying cluster & application state instead of using GitOps.
So, this project aims to solve these problems by migrating the installation of my homelab cluster and its applications to a GitLab CI pipeline. Additionally, the application state is managed in a GitOps manner in its own repository, for better portability and separation of responsibilities.
- Remote machine/s suitable for running Kubernetes, accessible over SSH.
- A Git repository containing the Kubernetes manifests to be synced with the cluster (default path: clusters/homelab)
K3s Ansible is used to install Kubernetes.
ansible/inventory.yaml should be configured to include the addresses of the machines you intend to be the server and agent nodes of your cluster.
ansible/group_vars/k3s_cluster.yaml should contain your desired k3s-ansible and K3s cluster configurations.
Further details can be found in K3s Install README.
Instead of the default ServiceLB LoadBalancer controller K3s includes, MetalLB is installed here.
metallb/lb-config.yaml configures the IP pool that MetalLB can advertise as external IPs.
Flux is installed to the cluster via the Flux Operator, and syncs to an existing Git repository containing Kubernetes mainfests.
flux/flux-instance.yaml should be configured to contain the details of the existing repository.
Further details can be found in Flux Install README.
Valid Vault server certificates can be generated by following a guide like this: Cert Manager: Vault Issuer Docs.
Also take note of the name used to create the Vault server leaf certificate, as this is the SNI name required for some clients to properly verify the Vault server.
| Variable | Description |
|---|---|
| ANSIBLE_USER | Ansible User to login to the remote Kubernetes host with. |
| SSH_KEY_B64 | Base 64-encoded SSH key that is permitted to SSH into the remote Kubernetes node. |
| EXEC_ENV_TAG | Version of the Ansible execution environment image to use. |
| HELM_KUBECTL_IMAGE | Name and tag of the image with Helm and Kubectl used to install Flux. |
| FLUX_OPERATOR_VERSION | Version of the Flux Operator to use. |
| GITLAB_TOKEN | GitLab Personal Access Token for the Kubernetes manifests repository with Maintainer role and api, read_repository permissions. This allows Flux to authenticate to GitLab to sync the repository. |
| GITLAB_CERT_B64 | Base 64-encoded public certificate for the GitLab server, allowing Flux to verify the GitLab server certificate. |
| METALLB_VERSION | Version of MetalLB Helm chart to install. |
| VAULT_SERVER_CERT_B64 | Base 64-encoded public certificate for the Vault server to be deployed to the cluster. |
| VAULT_SERVER_KEY_B64 | Base 64-encoded key certificate for the Vault server to be deployed to the cluster. |
| VAULT_CA_B64 | Base 64-encoded CA certificate for the Vault server to be deployed to the cluster. |