Add management functionality and job executor/targets #493
Description
Describe the enhancement
Ability to define a runner which can accept 1..N jobs and send them to a given target/executor. Where a target may be docker on the local instance, a docker engine in a cluster, or an instance in an autoscaling group.
At present a runner gets spun up in a container/instance and sits there polling for a job that it then runs locally. The only way I can see to guarantee a clean env is to flush the instance/container and start again following every single job. This causes a large number of unnecessary events to occur as the runner has to re-register on every startup. There is also no easy way of autoscaling. Similar to how gitlab have done it, I would like to see something where
- I can spin up a runner and register it
- the runner can be configured to accept, e.g. 10 concurrent jobs
- each job will be sent to a different instance in an autoscaling group
In this way, we get the following
- an always on instance (manager)
- existing functionality (if using local target)
- new functionality by specifying a different target (a docker engine or instances in an auto scaling group)
Additional information
For refs, see https://docs.gitlab.com/runner/configuration/runner_autoscale_aws/ and https://docs.gitlab.com/runner/executors/docker.html
I couldn't find anything like this mentioned in the community board nor issues. Apologies if this has already been addressed.