Can the desired target change during deployments?

[josselinbuils]

Hello,

First thank you for this great tool, it is really helpful 😀

We have some timeout issues recently when using it to deploy our website.

After investigation, it seems to be because the "desired_target" changes during the deployment because our auto scaling mechanism but the tool does not seem to take it into account.

We could disable the auto scaling during our deployments but there is a risk for our infrastructure in case of traffic pics.

Is it a known bahavior? If yes, what would be the complexity to make the tool manage that?

Regards,
Josselin

[fabfuel]

Hi Josselin,

thanks for sharing, it's always great to hear that the tool is helpful - highly appreciated!

Normally what you describe should (🤞) not happen, as the ecs deploy command does not change the desired count configuration of the service.

Could you please share a few (simplified) examples of how you deploy (the commands you run) and describe how the "desired_target" (do you mean the "desired count" of the ECS service or another config option?) changes from before to after the deployment?

Thanks
Fabian

[josselinbuils]

Hello,

Here is the kind of command we launch:

ecs deploy ${ECS_CLUSTER_NAME} ${ECS_SERVICE_NAME} -i ${ECS_SERVICE_NAME} ${DOCKER_IMAGE}:${DOCKER_TAG} ${EXTRA_DOCKER_IMAGES} --region ${AWS_REGION} --profile assume-role-ecs --no-deregister --ignore-warnings --rollback --timeout ${ECS_DEPLOY_TIMEOUT} -e ${ECS_SERVICE_NAME} DD_VERSION ${DD_VERSION}"

We are deploying multiple versions of our website and one of them uses a lot of instances (~150).

We have issues with this one because our scaling mechanism seems to change the desired number of containers during the deployments and the tool does not seem to take it into account.

Or maybe there is another issue but we reach the desired number of containers in the AWS console but the ecs-deploy tool timeouts and then rollbacks.

[josselinbuils]

It seems to be a pagination issue because we use more than 100 instances, one of our SREs will propose a PR to fix it 😄

[josselinbuils]

By the way, is it wanted not to allow having more instances than necessary to deploy ?

Regarding this condition: https://github.com/fabfuel/ecs-deploy/blob/develop/ecs_deploy/ecs.py#L1182.