(PoC) FastAPI CI/CD Pipeline with Terraform and GitHub Actions

1. Overview

This project demonstrates a CI/CD pipeline for a Python FastAPI application, featuring automated continuous integration and a manually-triggered deployment process. A key objective is to separate distinct deployment tasks —such as infrastructure provisioning (Terraform), database migration, and application deployment (CodeDeploy)—within the same source base. This separation enhances safety and control over the deployment process.

The entire infrastructure is managed as code using Terraform, and the integration/deployment process is orchestrated by GitHub Actions. The application is containerized with Docker and deployed to AWS using CodeDeploy.

The main goal is to showcase a robust and repeatable process for building, testing, provisioning, and deploying a modern web application.

Note: This project is a Proof of Concept (PoC) for demonstration purposes. For production use, it is crucial to enhance security measures and implement more robust application features. For example, We have to use AWS Secrets Manager for RDS db configuration.

---

2. Core Technologies

• Application: FastAPI, Uvicorn
• Containerization: Docker
• CI/CD: GitHub Actions
• Infrastructure as Code (IaC): Terraform
• Cloud Provider: AWS (EC2, RDS, S3, CodeDeploy, etc.)
• Database: Tortoise ORM (with PostgreSQL in mind)

---

3. CI/CD Pipeline Explained

The pipeline is divided into two main stages: Integration (CI) and Deployment (CD), which are triggered by events in the GitHub repository.

3.1. CI (Continuous Integration)

The CI process is triggered on every pull request to the main branch. Its purpose is to ensure that new changes are safe and meet quality standards before being merged.

• integration_01_app.yml:

  • Application Testing: Runs the application's unit and integration tests using pytest.
  • Linting & Formatting: Checks code quality and style with ruff.

• integration_02_terraform.yml:

  • Terraform Validate: Runs terraform validate to check for syntax errors in the Terraform configuration.
  • Terraform Plan: Runs terraform plan to create an execution plan and verify that the proposed infrastructure changes are as expected. This plan is posted as a comment in the pull request for review.

3.2. CD (Continuous Deployment)

The CD process is manually triggered from the main branch. This provides explicit control over the release process and handles the entire procedure of deploying the new version of the application.

• deployment_01_build_push.yml:

  • Build Docker Image: Builds the production Docker image for the FastAPI application using Dockerfile.deployment.
  • Push to ECR: Tags the image and pushes it to a container registry (e.g., Amazon ECR).

• deployment_02_terraform_apply.yml:

  • Apply Infrastructure: Runs terraform apply to provision or update the cloud infrastructure on AWS according to the definitions in the /terraform directory.

• deployment_03_db_migration.yml:

  • Database Migration: Executes database migrations (e.g., using Tortoise ORM's aerich) to update the database schema to the latest version. This is typically run as a task on the newly provisioned infrastructure.

• deployment_04_codedeploy.yml:

  • Trigger CodeDeploy: Starts a new deployment process with AWS CodeDeploy, which pulls the new Docker image and deploys it to the target environment (e.g., EC2 instances) according to the rules in appspec.yaml.

3.3. Destroying Infrastructure

• deployment_99_terraform_destroy.yml:
  • This is a manually triggered workflow.
  • It runs terraform destroy to safely tear down all cloud resources managed by Terraform. This is useful for development/staging environments to manage costs.

---

4. Infrastructure as Code (Terraform)

The /terraform directory contains all infrastructure definitions.

• main.tf: Defines the core AWS resources, such as:
  • VPC, Subnets, Security Groups
  • EC2 instances or an ECS cluster for running the application
  • RDS for the database
  • S3 buckets for storage
  • IAM roles and policies for permissions
  • CodeDeploy application and deployment groups
• variables.tf: Declares variables to make the infrastructure configuration reusable and configurable (e.g., region, instance types).
• outputs.tf: Defines outputs from the infrastructure, such as the application URL or load balancer DNS name.

---

5. Future Improvements

This project provides a solid foundation for a CI/CD pipeline. Here are some ways it could be extended and improved:

• Separate Environments (Staging/Production):
  • Use Terraform Workspaces or separate state files to manage distinct environments (e.g., staging, production).
  • Create different GitHub Actions workflows for deploying to each environment, potentially with manual approval steps for production.