Rework concurrency

[nevillelyh]

• Add concurrency.{max,current} to /health-check endpoint
  • For regular models, max is from cog.yaml and current is len(pending) of the sole runner
  • For procedure mode, max == maxRunners and current is the sum of len(pending) across all runners

For procedure mode:

• Set maxRunners = # CPU x 4, this is also the Cog server max concurrency
• We respect concurrency.max in each runner's cog.yaml, which is 1 for non-async runners
• We will spin up multiple copies of the same runner if its concurrency.max has been reached, e.g. 4 copies for a async runner with concurrency.max = 2 for a total of 8 concurrent predictions
• However we will reject new predictions when # pending == maxRunners, to prevent context switching, even if any runner might claim a higher concurrency.max, e.g. with # CPU = 1, maxRunners = 4, 4 predictions on a single runner even if its concurrency.max = 16
• This way when server concurrency.current < concurrency.max, we're guaranteed to have an evict-able runner slot.