NOTE: This repository is currently in a state of flux as I finalize details of my cluster and slowly both learn and also move to different architectural patterns. In particular, the Helm and Terraform files will likely be drastically updated later as I migrate files and (eventually) bring Atlantis online for applying Terraform changes.

GitOps

🔎 About

This repository contains ArgoCD, Helm, and Terraform files for declarative deployments with Kubernetes, specifically k3s.

You can use these files to stand up your own on-prem Kubernetes cluster. While this repository was built to be run on Raspberry Pi devices, it should be equally valid anywhere Kubernetes can run.

🎖️ Features

• App-of-apps: A root Argo CD Application deployment schema which recursively manages child apps
• Namespace deployments: argocd, cert-manager, kube-system, logging, longhorn-system, and monitoring
• Cert-manager: X.509 certificate management for Kubernetes
• Longhorn: Distributed on-prem file storage
• Metal LB: An on-prem native software load balancer
• Monitoring: Prometheus Operator with Grafana using storage PVC through Longhorn
• Dashboard UI for:
  • Argo CD: For controlling deployments and rollbacks
  • Grafana: For building dashboards against Prometheus data
  • Longhorn: For controlling the distributed block storage setup
  • Prometheus: For querying against raw data from pods/nodes/deployment resources

🧱 Project Management

Work for this repository is housed in this Trello board.

📁 Project Structure

├── argocd/                                      # ArgoCD application definitions
│   ├── apps/                                    # Application-level deployments
│   │   ├── argocd/                              #
│   │   ├── cert-manager/                        #
│   │   ├── kube-system/                         #
│   │   ├── logging/                             #
│   │   ├── longhorn-system/                     #
│   │   └── monitoring/                          #
│   ├── namespaces/                              # Namespace-level deployments
│   └── root/                                    # Root ArgoCD application deployment
├── helm/                                        # Helm charts
│   ├── argocd/                                  #
│   ├── cert-manager/                            #
│   ├── grafana/                                 #
│   ├── longhorn/                                #
│   ├── prometheus/                              #
│   ├── prometheus-operator/                     #
│   └── prometheus-service-monitors/             #
└── terraform/                                   # Terraform configurations (WIP)
    ├── namespaces.tf                            #
    └── provider.tf                              #

🛠️ Built With

Hardware

The cluster this repo's files runs on uses Raspberry Pi 5 devices, specifically the 16gb version.

Here's the hardware list of what each of the control/worker nodes is using:

1. Raspberry Pi 5
2. NVMe + POE+ Pi 5 Hat and Active Cooler
3. Samsung 2TB NVMe SSD
4. 256gb Micro SD Card

Software

• Argo CD
• Cert Manager
• Grafana
• Grafana Loki
• Grafana Promtail (soon to be removed for Grafana Alloy)
• Helm
• Kubernetes, specifically K3s
• Longhorn
• Metal LB
• OpenFaaS (coming soon)
• Prometheus (including Prometheus Operator)
• Sealed Secrets
• Terraform
• Traefik

🙇🏻‍♂️ Acknowledgements

• Edede Oiwoh for inspiring me to build a home cluster and for bouncing ideas around
• rpi4cluster.com for tips on GitOps with Raspberry Pi setups (even if the notes weren't current and Helm/Argo configurations weren't file-based)
• Twitter (now X), Loom, and Tesla for teaching me proper GitOps processes and giving me a chance to move mountains with them
• gitops-patterns repository for what will likely be ongoing sources of truth for modern architecture patterns