Awesome ECS

A curated list of guides, development tools, and resources for:

Amazon Elastic Container Service (ECS)

AWS Fargate

This list includes both community created content as well as content created by AWS.

Are you looking for infrastructure as code templates and tutorials for ECS? Containers on AWS is the home for infrastructure patterns and sample code to get you started fast.

Do you prefer video instead? If so check out Containers from the Couch for videos on all things AWS + containers.

Want to add something? Open a PR! 🙂

First steps

• Amazon ECS Core Concepts - Introduction to core concepts of Amazon ECS, including what are containers, why it is important to decouple applications from their underlying compute infrastructure, how ECS works, and what integrations that ECS has with the rest of AWS.
• Building Blocks of Amazon ECS - Learn about the basic building blocks of ECS and how they fit together to fully understand how it works and how you can use it!
• ECS Workshop - A detailed workshop that guides you through creating a simple microservice deployment, load testing it, and monitoring it.
• AWS Copilot - The getting started guide for AWS Copilot helps you deploy your first simple static website container on Fargate.

Pick your container hosting strategy:

• Amazon EC2 or AWS Fargate? - Which compute capacity makes more sense for your workload: serverless AWS Fargate, or EC2 instances?
  • If you pick EC2 capacity learn about ECS capacity providers linked to EC2 auto scaling groups and running containers on EC2 spot capacity and consider Bottlerocket as the container focused operating system for your Amazon ECS cluster.

Pick a tool for deploying your application

• AWS Copilot - The easiest starting experience for launching your local container on Fargate. This commandline tool helps you build and deploy your application, as well as deploy CI/CD pipelines that automatically rebuild and redeploy your application on Git push. It creates infrastructure as code templates for you behind the scenes.
• AWS Cloud Development Kit - AWS CDK is an SDK that lets developers define and deploy AWS infrastructure using familiar programming languages, often the same language that the application itself is coded in. CDK creates CloudFormation automatically behind the scenes.
  • aws-ecs - This module provides simple low level constructs for creating ECS and Fargate services. It gets about 100k downloads per week on NPM, so it is quite popular as a choice.
  • aws-ecs-patterns - A more beginner friendly interface to CDK. These patterns help you setup simple things like a "load balanced service" or a "scheduled task"
  • ecs-service-extensions - This CDK module provides the most extendable interface for ECS services. It lets you deploy an ECS service and then optionally attach extensions to it, which do things like add the service to a service mesh, or add an observability sidecar, etc. Read more about how to build custom extensions for ECS deployments
• CloudFormation - You can choose to write CloudFormation templates to describe your deployment directly, in which case these sample templates will help. Check out prebuilt CloudFormation patterns for AWS CloudFormation and Amazon ECS + AWS Fargate
• Terraform ECS Blueprints - Production ready AWS ECS infrastructure as code with Terraform
• Troposphere + ECS - For Python users Troposphere can help create CloudFormation templates. This example shows how to create an ECS deployment using Troposphere

Older tools

The following tools may not be as up-to-date or maintained, but are retained here for reference:

• ECS CLI v1 - The original CLI for ECS is designed to be mostly compatible with Docker Compose. It turns a local Docker Compose file into a remote deployment.
• fargate - Command line tool for interacting with AWS Fargate. With just a single command you can build, push, and launch your container in Fargate, orchestrated by ECS.
• fargate-create - A CLI tool for creating new projects based on Terraform templates and Fargate CLI. Supported stacks:
  • Web Application (ALB - HTTP/HTTPS)
  • Network Application (NLB)
  • Background Worker (Service)
  • Scheduled Task (Job)
• mu - Automates everything relating to ECS devops and CI/CD. This framework lets you write a simple metadata file and it constructs all the infrastructure you need so that you can deploy to ECS by simply pushing to your Git repo.
• deployfish - Write a simplified deployfish.yml file describing your deployment and let this tool handle the heavy lifting of deploying your service.
• Airship Terraform for ECS
• CloudFormation reference architecture - An older CloudFormation reference architecture for ECS
• Cloudonaut CloudFormation templates
• • empire - Control layer on top of ECS that provides a Heroku like workflow
• broadside - Ruby based command line tool for deploying to ECS
• UFO - Ruby based tool for building containers and shipping them to ECS
• bash deployment script by Justin Kulesza
• pnzr - Go based tool for building and pushing to ECS, also has integraton with AWS KMS for secrets management.
• deplojo - Python based deployment tool using ECS
• convox - Easily build, deploy and scale applications on ECS
• ecsctl - Open source tool similar to Kubernetes kubectl for ECS.
• ecs-deploy - Simple but powerful tool for initiating automatic blue green deploys on ECS
• ecspresso - Minimalistic: JSON file goes in, service launches
• ecsrun - Easily run one-off tasks against an ECS Task Definition using a config file based approach.
• shipctl - Tool that supports deploying a task on ECS, rolling back, or just running a one-off task
• ecsdeploy - A client and simplified web interface for managing your ECS cluster, rolling out and rolling back application versions
• ecs-service - CLI tool for deploying to ECS using CloudFormation with support for .env files for environment specific configuration of your containers
• kms-env - CLI tool for managing secrets using AWS KMS in .env files which can be used in conjunction with ecs-service to supply secrets to your containers
• ecsq - A developer friendly tool for querying the state of an ECS cluster
• Wonqa is a tool for spinning up disposable QA environments in AWS Fargate, with SSL enabled by Let's Encrypt. More details about Wonqa on the Wonder Engineering blog.

Solutions

• Service Discovery
  • Using built-in AWS CloudMap service discovery
  • Using built-in ECS Service Connect
  • Using Consul
  • Using Weaveworks
  • Using linkerd
  • Using HAProxy
• Scaling
  • Amazon ECS Scalability Best Practices
  • Target tracking based application scaling
  • Step scaling policies for application scaling
• Service Mesh
  • Service Connect (Envoy Proxy based service mesh built-in to Amazon ECS)
  • AWS App Mesh
• Data Persistance
  • Using Amazon Elastic File System example in AWS Cloud Development Kit
  • Using Amazon Elastic File System example in CloudFormation
  • Using Amazon Elastic File System example in AWS Copilot
• Secrets Management
  • AWS Secrets Manager + Amazon ECS
  • Using Parameter Store and IAM Roles for Tasks
  • Using ECS Task roles for managing AWS credentials
  • The right way to store secrets using Parameter Store
  • Open source tool for automatically getting secrets from SSM into your docker environment
  • Scaling from 10 to 16k+ tasks in a single ECS cluster
• Administration
  • Monitor cluster state with Cloudwatch event stream
  • React to ECS events such as crashed containers using a serverless function
  • Run an ECS Task on every cluster instance
  • Accessing Rails console inside of an ECS task
• CI/CD
  • AWS CodePipeline
  • Atlassian Bamboo
  • JetBrains TeamCity
  • Lambda
  • CircleCI
  • Shippable
  • Gitlab
  • SemaphoreCI
• CI/CD mechanics:
  • Rolling blue/green deploy in place
  • Canary Deploy
  • Isolated Regression Testing
• Open source:
  • Watchbot - This tool by Mapbox helps you run data processing across an ECS cluster in response to external events
  • ecs-export - A tool for exporting ECS cluster metrics to Prometheus for advanced querying
  • docker-elk-ecs - Connecting Amazon ECS container logs to an ELK (Elasticsearch, Logstash, Kibana) stack
  • Sample task definitions - Sample task definitions for running applications like Nginx, Tomcat, Gunicorn, Wildfly, Kibana, and Jetty as containers under Amazon ECS

Reference Architectures

• Bun JavaScript container that uses AWS SDK to connect to DynamoDB - A tiny hit counter application. It demonstratres launching a Bun JavaScript container, with an IAM role that grants it access to use a DynamoDB table as it's state store.
• NGINX Reverse Proxy sidecar container on AWS ECS - A common architecture for serving a website or API from a container
• Mythical Misfits - Deploy a sample application to serverless AWS Fargate capacity
• Cats n' Dogs - A fun workshop that covers service and container-instance auto-scaling, spot-fleet integration, container placement strategies, service discovery, secrets management with AWS Systems Manager Parameter Store, among other things.
• Deploy Jupyter notebook container with Amazon ECS - High cost, high performance machine learning playground, orchestrated by Amazon ECS on specialized AWS Inferentia enabled hardware.

Courses

• Free
  • ECS Workshop - Learn how to deploy a 3 tier, polyglot, microservice based application to AWS Fargate
  • Cloudskills: The beginners guide to running Docker containers on AWS
  • Simplilearn: AWS ECS Tutorial
• Paid
  • Microservices with Docker, Flask, and React - Learn how to build, test, and deploy microservices powered by Docker, Flask, and React on Amazon ECS!
  • AWS Developer: Optimizing on AWS - Learn how you can optimize your applications on AWS with Amazon ECS!
  • Udemy: ECS Deep Dive
  • Cloud Academy: Introduction to ECS
  • Coursera: Building Containerized Applications on AWS

Blogs

• Run a PHP application on AWS Fargate - An in-depth guide to run a Laravel app on AWS Fargate with Github Actions for deployment
• Deploying a Rails app to Fargate - Step by step walkthrough of deploying a Ruby + RDS app, with helpful debugging tips
• How to use AWS Fargate and Lambda for long-running processes in a Serverless app - Great tutorial showing how to leverage the power of long running docker containers in Fargate alongside Lambda. Example application processes video files to extract thumbnails using just S3, Lambda, and Fargate... no EC2.
• Setting up service discovery for AWS Fargate using CloudFormation
• Using the Clair image scanner with Fargate, ECR and CodeBuild/CodePipeline
• A lighter way to deploy to ECS/Fargate - combining update-service/task data from several sources including metadata of the docker image itself to deploy new versions.
• Chaos testing with ECS - Chaos toolkit supports ECS as a target for chaos engineering
• Private subnets tutorial
• End-to-end TLS traffic - How to setup end-to-end TLS from client to container, as well as from container to container
• Docker for .NET Developers

Presentations

• Amazon ECS Scalability Best Practices
• Running your Dockerized application(s) on AWS EC2 Container Service by Marco Pas
• Microservices on AWS with Weaveworks
• Running a Virtual World via ECS - Linden Labs on their usage of ECS
• Advanced Task Scheduling with AWS ECS
• Instacart on running microservices on Amazon ECS
• Building Next-Generation Applications with Amazon ECS - How Meteor Built Galaxy on Amazon ECS
• Amazon ECS at Coursera: A General Purpose Microservice