Modern-devops-pipeline

Description Tries to show the essence of a modern devops practices

• I have made the effort to deploy on AKS as its a more realistically true approach than deploying minikube and I also realised at starting that this is gonna be a big project
• I had to have around 12 gb of ram to run all the components.
• Used load balancer for access my application in real time.

Architecture

🔧 Tools Used

Tool	Description	Logo
Kubernetes	Open-source platform for automating deployment, scaling, and managing containerized applications.
Argo CD	Declarative, GitOps continuous delivery tool for Kubernetes applications.
Docker	Platform for developing, shipping, and running applications in containers.
GitHub Actions	Automates software workflows directly in your GitHub repository, such as CI/CD pipelines.
Argo Rollouts	Advanced deployment controller for Kubernetes providing features like canary releases and blue/green deployments.
Prometheus	Monitoring tool for collecting and querying time-series data.
Grafana	Analytics and visualization of performance metrics.
Elasticsearch	Search and analytics engine used to store, search, and analyze large volumes of data.
Fluentd	Open-source data collector for unified logging.
Kibana	Data visualization and exploration tool for Elasticsearch.

Website Image

Steps in brief

1. Creation of app and manual deployment

• Creating a Kubernetes cluster in azure eks

az aks create \
    --resource-group NetworkWatcherRG \
    --name AKS \
    --node-count 1 \
    --node-vm-size Standard_B2ls_v2 \
    --enable-addons monitoring \
    --generate-ssh-keys

• Getting the creds for azure

 az aks get-credentials --resource-group NetworkWatcherRG --name AKS

• Deploying a msql db with secret

kubectl create secret generic mysql-secret \
  --from-literal=mysql-user=yourusername \
  --from-literal=mysql-password=yourpassword

• Deploying a mysql server on K8s mysql.yaml

  • Choosing to go with 100Mi is intentional as we dont need much during testing

• Writing a simple node.js application with mysql in the back for storing data -- check server.js

  • Its a dynamic apps that takes data from /manhwa endpoint

• Using docker file to create a docker image -- check docker file

• Deploying the node app on k8s -- check k8s/yaml

2. Integration CI principles

• Using github actions we create CI for this

  • Steps taken in CI
  1. Implemented simple tests
  2. Pushing to docker
  3. In the second job
  4. ** Implemented that change of new docker image in the app.yaml file **

• Updating the changes in the app.yaml file allows us to have one push CI/CD with Argocd

For demonstration just look in actions of this repo

3. Installing argo cd

• Installing argo cd in the cluster with official doc
• Applyin the argocd.yaml in k8s folder for application to be applied

As its a public repo we dont need any secrets

4. Installing argo cd Rollout

• For this i have decided to a lot of stuff for it work seamlessly

  1. Created a new namespace for rollouts versions
  2. Connected to default service of mysql coz i dont want to have another pod running without any use and this also showcase the coredns concept of kubernetes
  3. Implemented rollouts

• Created app_rollout.yaml under argo_rollout folder to deploy the application

Argocd rollout cli outputs

Final output of Argo cd rollout dashboard

5. Observability and Monitoring

• Installing prometheus and grafana

helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --set nodeSelector."kubernetes\.io/hostname"="aks-temp-10754646-vmss000000"

• Had to specifically tell k8s to schedule on a new node as the previos node memory got 100%

Grafana dashboard ⬇️ Prometheus dashboard ⬇️

• Installing EFK stack
• Will like to tell it took a lot time as there many pre requisite to it
• Also the system requirements in the default YAML was too much
• Check out the EKF folder So had to do a lot of customisation and took help from youtube

Terminal view ⬇️ index in elastic serach ⬇️ logs of default namespace ⬇️