Puppet module for installing Apache-Airflow v1.10.6
I use this module to create airflow cluster.
- Use packer to build the AMI
- Use puppet-masterless provisioner
- Create airflow master node AMI which runs webserver and scheduler services
- Create airflow worker node AMI which runs only worker service
- Make sure the master node doesn't scale, if it does, problems might occur because of multiple scheduler running
- Use the userdata in CFT template to execute the init scripts.
- Update the CFT template