Add management functionality and job executor/targets

[halberom]

Describe the enhancement
Ability to define a runner which can accept 1..N jobs and send them to a given target/executor. Where a target may be docker on the local instance, a docker engine in a cluster, or an instance in an autoscaling group.

At present a runner gets spun up in a container/instance and sits there polling for a job that it then runs locally. The only way I can see to guarantee a clean env is to flush the instance/container and start again following every single job. This causes a large number of unnecessary events to occur as the runner has to re-register on every startup. There is also no easy way of autoscaling. Similar to how gitlab have done it, I would like to see something where

• I can spin up a runner and register it
• the runner can be configured to accept, e.g. 10 concurrent jobs
• each job will be sent to a different instance in an autoscaling group

In this way, we get the following

• an always on instance (manager)
• existing functionality (if using local target)
• new functionality by specifying a different target (a docker engine or instances in an auto scaling group)

Additional information
For refs, see https://docs.gitlab.com/runner/configuration/runner_autoscale_aws/ and https://docs.gitlab.com/runner/executors/docker.html

I couldn't find anything like this mentioned in the community board nor issues. Apologies if this has already been addressed.