Multiple targets per service (load balancing)

[kevinmcconnell]

We should add the ability to register multiple targets for a single service, and distribute requests amongst them round-robin. This will give us the ability to load balance the traffic for a service across multiple hosts. With this setup, Kamal can provide a complete solution for running an app over multiple hosts, by running a proxy instance on a different host.

Adding this also means:

• The semantics of deploying needs to be extended to handle the case where only some of the new targets become healthy. Probably the behaviour here should be that we health-check every host in the list, and if any of them fail to become healthy we return non-zero without having applied any of the changes. But if all become healthy, we apply the new set as an atomic change. This includes the case there there are overlaps between the old and new list.

  For example, if you're deployed to t1, t2, t3 and you attempt a new deployment for t1, t2, t5, t6 but t6 fails, we keep the original set of t1, t2, t3.

• Healthchecks should be ongoing, rather than stopping after success as they do now. We'll want to notice an unhealthy target after the fact and take it out of the routing until it becomes healthy again.