mwaa_cli

Introduction

Amazon Managed Workflows for Apache Airflow (MWAA) is a managed orchestration service for Apache Airflow. It is a fully managed service that makes it easy to create, schedule, and monitor workflows.

This project is a command line interface for MWAA. It is a wrapper around the AWS CLI and curl commands.

Installation

Prerequisites

• jq
• curl
• aws-cli

Install

Just copy and add the mwaa_cli.sh script to your path.

For example:

curl -s https://raw.githubusercontent.com/raphaelmansuy/mwaa_cli/main/mwaa_cli.sh -o /usr/local/bin/mwaa_cli.sh
chmod +x /usr/local/bin/mwaa_cli.sh

How to use

Usage: mwaa_cli.sh <command> <args>(<options>)

Options:

  -h, --help: Display this help
  -e, --environment: Set the MWAA environment name (example: airflow_env_1)
  -r, --region: Set the AWS region (example: eu-west-1)
  -p, --profile: Set the AWS CLI profile (example: saml)

Examples:

mwaa_cli.sh dags list
mwaa_cli.sh dags list-runs -d <dag_id>
mwaa_cli.sh list_tasks <dag_id>
mwaa_cli.sh trigger_dag <dag_id>
mwaa_cli.sh dags pause <dag_id>
mwaa_cli.sh dags unpause <dag_id>

Frequently used commands

List DAGs

mwaa_cli.sh dags list

List DAG runs

mwaa_cli.sh dags list-runs -d <dag_id>

Trigger DAG

mwaa_cli.sh trigger_dag <dag_id>

Pause DAG

mwaa_cli.sh dags pause <dag_id>

Unpause DAG

mwaa_cli.sh dags unpause <dag_id>

List dags and display with a specific format (JSON, YAML, CSV, TSV, table)

mwaa_cli.sh dags list --output json | jq

List all dags that are not paused

./mwaa_cli.sh dags list --output json | jq '.[] | select(.paused == "False")'

Execute a command for each dag that is not paused

./mwaa_cli.sh dags list --output json  | jq '.[] | select(.paused == "False") .dag_id' | tr '\n' '\0' | xargs -0 -n1 echo

Pause all dags that are not paused

./mwaa_cli.sh dags list --output json | jq '.[] | select(.paused == "False") | .dag_id' | tr '\n' '\0' | xargs -0 -n1 ./mwaa_cli.sh dags pause 

Unpause all dags that are paused

./mwaa_cli.sh dags list --output json | jq '.[] | select(.paused == "True") | .dag_id' | tr '\n' '\0' | xargs -0 -n1 ./mwaa_cli.sh dags unpause 

How it works

The script uses the AWS CLI to get the MWAA environment details (endpoint, IAM role, etc.). It then uses curl to call the MWAA API.

Reference of AirFlow commands https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#cli-commands )

Created with love ❤️ by Raphaël MANSUY