ECS Executor - Supporting Capacity Providers

[o-nikolas]

Description

From feedback here:

The ECS Executor does not currently support Capacity Providers. It currently requires that users specify a LaunchType, and if you're specifying a LaunchType then you cannot specify a Capacity Provider, they are mutually exclusive.

Using Capacity Providers can be essential for organizations that use EC2-hosted ECS Clusters. Capacity Providers are very cost efficient since you can use them to dynamically scale your EC2 instances based on demand.

Use case/motivation

No response

Related issues

#34381

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[ferruzzi]

Taking a look at this one now.

[ferruzzi]

@o-nikolas @mshober First big decision. According to the run_task API docs:

If a capacityProviderStrategy is specified, the launchType parameter must be omitted. If no capacityProviderStrategy or launchType is specified, the defaultCapacityProviderStrategy for the cluster is used.

Currently the executor defaults to a FARGATE launchtype is nothing is provided. We are still "experimental" so we CAN change that without notice or deprecation. Should we adjust and adopt the API's default? I think it makes sense to follow their lead here, but it's a digression from our current behavior so I wanted to run it by you.

[o-nikolas]

@o-nikolas @mshober First big decision. According to the run_task API docs:

If a capacityProviderStrategy is specified, the launchType parameter must be omitted. If no capacityProviderStrategy or launchType is specified, the defaultCapacityProviderStrategy for the cluster is used.

Currently the executor defaults to a FARGATE launchtype is nothing is provided. We are still "experimental" so we CAN change that without notice or deprecation. Should we adjust and adopt the API's default? I think it makes sense to follow their lead here, but it's a digression from our current behavior so I wanted to run it by you.

I don't think the cluster default capacity provider strategy is mandatory, right? So it could lead to a failure case. I'd vote to keep the default launchType set to fargate if we get no other indication (we could do a describe to see if there is a default capacity provider strategy set and use that of course).

[ferruzzi]

we could do a describe to see if there is a default capacity provider strategy set and use that of course

I think I like that.

Proposal:

If both are provided, fail early with an exception
If only one is provided, use it
If none are provided:
  - describe the cluster and see if there is a default CP to fall back on
      - if yes, provide nothing to the API and let it default there
      - if no, provide the FARGATE launch type.

[o-nikolas]

we could do a describe to see if there is a default capacity provider strategy set and use that of course

I think I like that.

Proposal:

If both are provided, fail early with an exception
If only one is provided, use it
If none are provided:
  - describe the cluster and see if there is a default CP to fall back on
      - if yes, provide nothing to the API and let it default there
      - if no, provide the FARGATE launch type.

Looks good to me!