An experimental workflow orchestration system built on top of HashiCorp Nomad. Nomad Pipeline allows users to define, schedule, and monitor multi-step workflows (flows) that are executed as Nomad jobs.
Note This project is the result of two hack week projects and is unsupported and unofficial.
The documentation for Nomad Pipeline can be found within the docs directory. The examples directory contains example flow definitions and Nomad job files.
- Run logs are not periodically removed from the data directory
- Run state objects are not periodically garbage collected
- Shared state across flow jobs, provided by Dynamic Host Volumes or suchlike
- Offer runners via the Nomad libvirt driver
- Identify and automatically upload end of pipeline artifacts
- Persistent pipeline runners that accept flow runs over a long-lived connection
- Storage backend HA locking