feat: extend environment variables for default load balancer configuration #1052
Conversation
|
Hey @M4t7e, just to confirm, the "8 Annotations" limit is from the I found this thread where the limit was discussed and added, it sounds like this is a "soft" limit that can be raised if a reasonable case is made for more than 8 annotations: kubernetes-sigs/gateway-api#1757 (comment) |
|
Hey @apricote, yes, that's the limit I was referring to. I began researching ways to preset annotations in GatewayAPI, since the concept is not to have a single Gateway (Load Balancer) for everything, like it is often the case for Ingress Controller, but to have the flexibility of creating multiple Gateways. To avoid repeating the same config, I was looking into options setting global annotation setting. That's when I came across this issue: kubernetes-sigs/gateway-api#2734 From what I understand, the annotation limit can only be increased through provider-specific implementations like Istio and Envoy already support. In my case, I’m planning to use Cilium GatewayAPI, which as far as I know doesn’t support adding annotations with their custom config. Btw, this is the actual issue where we want to add GatewayAPI support: hcloud-k8s/terraform-hcloud-kubernetes#216 |
|
I personally dislike the many annotations, and have often wondered how we could provide a better alternative. The Gateway API @lukasmetzner will be back next week to take a closer look at the MR. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1052 +/- ##
==========================================
- Coverage 68.43% 64.98% -3.46%
==========================================
Files 23 23
Lines 2531 2610 +79
==========================================
- Hits 1732 1696 -36
- Misses 629 746 +117
+ Partials 170 168 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
e2e test not passing is fine here. This is an expected permission issue. |
|
@lukasmetzner I made the changes you suggested. Could you please take another look? |
@M4t7e The code looks and the e2e tests succeed locally on my machine. They are failing here due to expected permission issues with forks. The only remaining thing is the Could you please fix them, then we can merge this PR. You can either run |
|
@lukasmetzner I updated the code formatting to comply with the latest ❯ golangci-lint run
internal/metrics/metrics.go:24:1: File is not properly formatted (goimports)
"k8s.io/component-base/metrics/legacyregistry"
^
1 issues:
* goimports: 1 |
golangci-lint seems to behave differently when |
lukasmetzner
changed the title
lukasmetzner
had a problem deploying
to
e2e-robot
GitHub Actions
Failure
### Watch-Based Route Reconciliation
Previously, route reconciliation is performed at a fixed interval of
30s. This leads to unnecessary API requests, as a `GET
/v1/networks/{id}` call is triggered every 30s, even when no changes
have occurred.
Upstream, we have contributed an event-driven approach, similar to the
mechanisms used by other controllers such as the Load Balancer
controller. With this new approach, route reconciliation is triggered by
node additions, node deletions, or changes to a node’s `PodCIDRs` or
`Addresses`. Additionally, to ensure consistency, reconciliation still
occurs periodically at a randomized interval between 12 and 24 hours.
#### Enabled by default
This feature is now **enabled by default**.
If you encounter any problems you can disable the feature by setting the
following Helm value:
`args.feature-gates=CloudControllerManagerWatchBasedRoutesReconciliation=false`
### Global Load Balancer Defaults
Configure cluster-wide defaults for Load Balancers via the extended
`HCLOUD_LOAD_BALANCERS_*` env vars. These values automatically apply
during Load Balancer creation and reconciliation whenever annotations
are omitted. Learn more about it in the [reference
documentation](docs/reference/load_balancer_envs.md)
### Features
- extend environment variables for default load balancer configuration
(#1052)
- enable watch based route reconciliation by default (#1112)
3 participants
This PR introduces additional environment variables for load balancer configuration. These variables are designed to be set globally as defaults and can be overridden using annotations.
The main motivation is to improve support for GatewayAPI, as the
Gatewayannotation limit of 8 is restrictive and many settings are commonly needed across all load balancers from the same or even differen GatewayAPI providers. Additionally, this change allows environment-specific presets such as the new subnet IP range to be set globally. This removes the need to configure these settings in each service or use templating/patching to use the same service manifest for different environments.New environment vars:
HCLOUD_LOAD_BALANCERS_ALGORITHM_TYPEHCLOUD_LOAD_BALANCERS_DISABLE_PUBLIC_NETWORKHCLOUD_LOAD_BALANCERS_HEALTH_CHECK_INTERVALHCLOUD_LOAD_BALANCERS_HEALTH_CHECK_RETRIESHCLOUD_LOAD_BALANCERS_HEALTH_CHECK_TIMEOUTHCLOUD_LOAD_BALANCERS_PRIVATE_SUBNET_IP_RANGEHCLOUD_LOAD_BALANCERS_TYPEHCLOUD_LOAD_BALANCERS_USES_PROXYPROTOCOL