This is a personal work in progress. Keep in mind your suggestions are welcome! :)
These workflows are highly opinionated kubectl-apply or helm-upgrade pipelines.
- Config tour repository with these secrets.
- Create a directory
.github/workflow
in your repository and add this file. - Create a another directory
manifests
and add this file. - Commit and open your repository actions page! :)
- Versioning with Semantic-Release
- Linter:
- Hadolint for Dockerfiles
- ESLint for Javascript
- PyLint for Python
- Static analysis:
This pipeline assumes you have just 4
types of repositories:
Name | Short | Description | Result |
---|---|---|---|
Web Application | app | Front-end application with internet-facing ingress | language-based pod |
Mobile Application | mob | Mobile application | apk |
Service | svc | Microservice that may - or may not - have ingress | nginx-based pod |
Infrastructure as Code | iac | Code that generates cloud infrastructure | - |
Those repositories must obey a name pattern.
{ecosystem}-{type}-{name/client/integration}
Examples:
ff-svc-clients
: microservice that manages clients' dataff-app-budget
: application that organizes the company financesff-mob-auth
: 2FA mobile applicationff-iac-aws
: infra as code to manage the aws environment
Checkout the test repositories:
Type | Solution | Repository | Pipeline | Deploy |
---|---|---|---|---|
app | filipeforattini/ff-app-react | |||
app | filipeforattini/ff-app-vue | |||
svc | filipeforattini/ff-svc-express | |||
svc | filipeforattini/ff-svc-fastapi | |||
svc | filipeforattini/ff-svc-flask | |||
svc | filipeforattini/ff-svc-moleculer | |||
svc | filipeforattini/ff-svc-nestjs | |||
svc | filipeforattini/ff-svc-nextjs |
Name | Short | Description |
---|---|---|
Development | dev | Env for you and your team to test and explore |
Staging | stg | Stable env for code shipping |
Sandbox | sbx | Production-like env for external developers |
Production | prd | Where the magic happens |
Disaster Recovery | dry | Production copy |
flowchart
start[Start]
analysis[Analysis]
node-test[Node Tests]
python-test[Python Tests]
static-analysis[Static Analysis]
start --- analysis
analysis --- |event/push| static-analysis
analysis --- |event/pull_request| static-analysis
analysis --- |event/workflow_dispatch| trigger-manual
static-analysis --- |lang/javascript| node-test
static-analysis --- |lang/python| python-test
static-analysis --- |lang/go| go-test
subgraph node:
node-test --- node-release
node-release --- node-trigger
end
subgraph python:
python-test --- python-release
python-release --- python-trigger
end
subgraph go:
go-test --- go-release
go-release --- go-trigger
end
node-trigger --- |deployment/dev| finish
python-trigger --- |deployment/dev| finish
go-trigger --- |deployment/dev| finish
trigger-manual --- |env/xxx| finish
analysis --- |event/deployment| build
build --- |env/dev| env-dev
build --- |env/stg| env-stg
build --- |env/prd| env-prd
subgraph dev:
env-dev --- |app.dev.domain.io| DEV
env-dev --- |app-commit.dev.domain.io| DEV
end
subgraph stg:
env-stg --- |app.stg.domain.io| STG
end
subgraph prd:
env-prd --- |app.prd.domain.io| PRD
end
├─ .github
│ └─ workflows
│ │ └─ pipeline.yml
│ └─ dependabot.yml
├─ manifests
│ ├─ configs
│ │ └─ dev.env
│ ├─ dependencies
│ │ └─ dev.yml
│ ├─ secrets
│ │ └─ dev.gpg
│ ├─ k8s.yml
│ └─ helm.yml
├─ build
│ // distibuition version of our code
└─ src
// our code goes here
Name | Description |
---|---|
GPG_PASSPHRASE | |
KUBE_CONFIG | Your ~/.kube/config file as base64. |
PIPELINE_DEPLOY_SECRET | A GitHub token, see the permissions below. |
REGISTRY_USERNAME | Registry username. |
REGISTRY_PASSWORD | Registry password. |
Add this pipeline to your repository creating a file pipeline.yml
in your .github/workflows
directory.
name: pipeline
on:
push:
deployment:
release:
types: [created]
pull_request:
types: [opened, reopened]
workflow_dispatch:
inputs:
environment:
description: 'Environment'
required: true
type: choice
default: 'dev'
options:
- dev
- stg
- prd
jobs:
SVC:
uses: filipeforattini/ff-iac-github-actions/.github/workflows/svc.yml@main
secrets: inherit
with:
mainBranch: main
containerRegistry: ghcr.io
environmentsAsNamespaces: true
Name | Default | Description |
---|---|---|
containerRegistry | ghcr.io | Container registry host that you will use |
environmentsASnamespaces | false |
Create a file k8s.yml
in your manifests
directory.
#@data/values
---
port: 1234
env:
- name: TZ
value: America/Sao_Paulo
ingress:
enable: true
type: traefik
tls:
enable: true
domain: your.domain
Configure your k8s cluster and get your ~/.kube/config
.
git commit -m "action(scope): subject"
Where the actions:
feat
: new feature for the user, not a new feature for the build scriptfix
: bug fix for the user, not a fix for a build scriptdocs
: documentation changesstyle
: formatting, lack of semicolons, etc; no changes to the production coderefactor
: refactoring the production code, for example. renaming a variabletest
: adding missing tests, refactoring tests; no changes to the production codechore
:updating grunted tasks, etc; no changes to the production code
Adds BREAKING CHANGE
in the commit message and it will generate a new major version.
gpg -v \
--symmetric \
--cipher-algo AES256 \
--output ./manifests/secrets/dev.gpg \
./manifests/secrets/dev.env
Thanks to:
- https://evilmartians.com/chronicles/build-images-on-github-actions-with-docker-layer-caching
- https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/
- https://blog.gitguardian.com/github-actions-security-cheat-sheet/
This ecosystem generates few data per second as samples for our apis.
In this implementation, each service has its own resources.
flowchart
https---ingress
subgraph k8s
ingress---|ff-svc-nestjs.dev.forattini.app|nestjs
ingress---|ff-svc-nextjs.dev.forattini.app|nextjs
ingress---|ff-svc-fastapi.dev.forattini.app|fastapi
ingress---|ff-svc-moleculer.dev.forattini.app|moleculer
subgraph ff-svc-nextjs
nextjs---rabbitmq-nextjs[rabbit]
end
subgraph ff-svc-moleculer
moleculer---postgres-moleculer[postgres]
moleculer---mysql-moleculer[mysql]
moleculer---redis-moleculer[redis]
moleculer---rabbitmq-moleculer[rabbit]
moleculer---etcd-moleculer[etcd]
moleculer---nats-moleculer[nats]
moleculer---rabbitmq-nextjs
end
subgraph ff-svc-fastapi
fastapi---postgres-fastapi[postgres]
fastapi---rabbitmq-fastapi[rabbit]
fastapi---rabbitmq-nextjs
end
subgraph ff-svc-nestjs
nestjs---postgres-nestjs[postgres]
nestjs---rabbitmq-nestjs[rabbitmq]
nestjs---rabbitmq-nextjs
end
end
In this implementation, all services connects to a shared resource.
flowchart
https---ingress
subgraph k8s
ingress---|ff-svc-nestjs.dev.forattini.app|nestjs
ingress---|ff-svc-nextjs.dev.forattini.app|nextjs
ingress---|ff-svc-fastapi.dev.forattini.app|fastapi
ingress---|ff-svc-moleculer.dev.forattini.app|moleculer
subgraph ff-svc-moleculer
moleculer---postgres-moleculer[postgres]
moleculer---mysql-moleculer[mysql]
moleculer---redis-moleculer[redis]
moleculer---rabbitmq-moleculer[rabbitmq]
moleculer---etcd-moleculer[etcd]
moleculer---nats-moleculer[nats]
end
subgraph ff-svc-nextjs
nextjs---rabbitmq-moleculer
nextjs---postgres-moleculer
nextjs---mysql-moleculer
nextjs---redis-moleculer
end
subgraph ff-svc-fastapi
fastapi---postgres-moleculer
fastapi---rabbitmq-moleculer
end
subgraph ff-svc-nestjs
nestjs---postgres-moleculer
nestjs---rabbitmq-moleculer
end
end