[BUG] ALB addon pods should include CriticalAddonsOnly op=Exists toleration

[ivanthelad]

Describe the bug
The Gateway for containers controller does not tolerate the system-node pools. It should include the toleration. currently it include the default tolerations from the scheduler

•    node.kubernetes.io/memory-pressure:NoSchedule op=Exists  
  

•    node.kubernetes.io/not-ready:NoExecute op=Exists for 300s   
  

•    node.kubernetes.io/unreachable:NoExecute op=Exists for 6000s
  

The ALB controller should include

•    CriticalAddonsOnly op=Exists
  

To Reproduce
None: Inspect the pod alb-controller under the ns azure-alb-system

Expected behavior
The ALB pods should have a preference for the system node pools. As they currently don't have that toleration they are repelled by the system pools

Environment (please complete the following information):

• CLI Version 2.65.0
• Kubernetes version 1.30.4
• CLI Extension version
  • alb 1.0.0

[ivanthelad]

helm upgrade alb-controller oci://mcr.microsoft.com/application-lb/charts/alb-controller --post-renderer yq --post-renderer-args 'select(.kind=="Deployment") |= .spec.template.spec.tolerations += [{"key": "CriticalAddonsOnly", "operator": "Exists"}]'

as a workaround to modify ALB controller generated by helm

[microsoft-github-policy-service]

@JackStromberg would you be able to assist?

[ivanthelad]

helm upgrade alb-controller oci://mcr.microsoft.com/application-lb/charts/alb-controller --post-renderer yq --post-renderer-args 'select(.kind=="Deployment") |= .spec.template.spec.tolerations += [{"key": "CriticalAddonsOnly", "operator": "Exists"}]'

as a workaround to modify ALB controller generated by helm

[microsoft-github-policy-service]

@JackStromberg would you be able to assist?