Updating the AutoScale/HPA causing the pods to redeploy #2585
Description
Component(s)
operator
What happened?
Description
When updating the AutoScale HPA settings the containers are redeployed.
Steps to Reproduce
Deploy Otel Collector with AutoScale set to min: 3
Validate that 3 replicas are running.
Update AutoScale to Min: 4
Observer that the deployment/statefulSet gets redeployed cause the running replicas to need to all be restarted.
Expected Result
HPA CRD would be updated without recycling the deployment.
Actual Result
All Replicas are restarted
Kubernetes Version
v1.27.7-gke.1121000
Operator version
v0.92.1
Collector version
v0.92.0
Environment information
Environment
GKE
Log output
No response
Additional context
Ideally an UPDATE to an existing HPA would only update the HPA so that running instances are not restarted.
When a system is under heavy load there may be a need to increase the Max Replicas.
Restarting all the containers causes a huge backlog as the replicas come back online.
If the goal was to mitigate load by increasing replicas this behavior makes it much worse before it can use the new HPA.