Fix initial deployment downtime

[miguelvalerio]

Fixes #1444.

After taking a look at the code, I think the current implementation could cause some issues: considering that if might be a bit of latency between scaling de Deployment/DaemonSet to 0 and updating the Service, the Service would be left with no Pods to communicate to for a bit. This would be even worse if, for some reason, updating the Service failed, leaving the apex service with no Pods for even longer.

This solution moves the ScaleToZero to be performed only after the apex service is already pointing to the primary target, and should therefore mitigate the issues described above.

[codecov-commenter]

Codecov Report

Patch coverage: 42.85% and project coverage change: +0.03% 🎉

Comparison is base (2944581) 54.53% compared to head (b25e12d) 54.56%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1451      +/-   ##
==========================================
+ Coverage   54.53%   54.56%   +0.03%     
==========================================
  Files          84       84              
  Lines       10146    10143       -3     
==========================================
+ Hits         5533     5535       +2     
+ Misses       3956     3953       -3     
+ Partials      657      655       -2     

Files Changed	Coverage Δ
pkg/canary/daemonset_controller.go	57.52% <ø> (+0.37%)	⬆️
pkg/canary/deployment_controller.go	44.38% <ø> (+0.05%)	⬆️
pkg/controller/scheduler.go	45.30% <42.85%> (-0.03%)	⬇️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[aryan9600]

left a couple of nits, but apart from that its looking good!

[aryan9600]

could you these commits into one commit and make sure you rebase against main? we should be ready to merge then :)

[miguelvalerio]

should be good now, thanks :)

[aryan9600]

lgtm! thanks @miguelvalerio 🙇