Add Support for User-Defined Gateway Addresses

[danehans]

#352 adds support for managing gateway status addresses. However, this PR does not support user-defined Gateway addresses. Therefore, EG should compare the status address of the Gateway's associated service against the user-defined address and surface the proper Gateway status condition.

[shawnh2]

hi, is this issue still available? i'd like to working on it.

[Xunzhuo]

Great, thanks @shawnh2

[shawnh2]

hi guys, running through few things.

• EG should compare the status address of the Gateway's associated service against the user-defined address and surface the proper Gateway status condition.

  How to define proper?

• I couldn't see any operations on GatewaySpec.Addresses for now in EG, so what's troubling me most is WHAT is our implementation or purpose on this field? FYI, I checked out Contour, it only support a single address, and that address should also be appeared in statusAddresses.

[arkodg]

@shawnh2 the Gateway API allows the user to define a custom address at the Gateway level
https://github.com/kubernetes-sigs/gateway-api/blob/a78e09220feec7947eb1a08b658b45f3b4182d7f/apis/v1beta1/gateway_types.go#L142
we need to first figure out if we / Envoy Gateway can support this feature or not i.e. can we set these addresses on the K8s Service or not.

If that is possible, we will also need to read those values and update the status

gateway/internal/status/gateway.go

Line 25 in f81c011

func UpdateGatewayStatusProgrammedCondition(gw *gwapiv1b1.Gateway, svc *corev1.Service, deployment *appsv1.Deployment) {

[arkodg]

as @tommie suggested, the user defined gateway addresses might need to be set in service.Spec.ExternalIPs https://kubernetes.io/docs/concepts/services-networking/service/#external-ips in the infra mgr layer and also used to update Gateway Status

[tommie]

I codified a PoC in https://github.com/envoyproxy/gateway/compare/release/v0.3...tommie:envoyproxy-gateway:external-ips?expand=1

It only cares about IPAddress addresses, there are no tests, etc.

Based off of v0.3 because of #1307.

[shawnh2]

I codified a PoC in https://github.com/envoyproxy/gateway/compare/release/v0.3...tommie:envoyproxy-gateway:external-ips?expand=1

It only cares about IPAddress addresses, there are no tests, etc.

Based off of v0.3 because of #1307.

hi, looking great, mind i merge it into my branch first? and i will complete the rest.

[tommie]

@shawnh2 go for it!

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[LanceEa]

I see this issue is still open but was referenced by PR #1322
which implemented support for Service.spec.ExternalIPs. However, this doesn't work for all clouds, AFAIK.

I can confirm what was mentioned here: #1405 (comment)
is true for GKE. In GKE, it leverages the Service.spec.loadBalancerIP field and the GKE admission controller actually
fails to create the service if ExternalIPs is filled in. https://cloud.google.com/kubernetes-engine/docs/concepts/service-load-balancer-parameters#spd-static-ip

Looking at the K8s docs, it says the Service.spec.loadBalancerIP was deprecated in 1.24 https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer but it doesn't say to use the ExternalIPs but rather use Gateway which is weird because all cloud vendors don't have Gateway support. When I look at the docs for the ExternalIPs it has a note saying that K8s does nothing with it and it is up to cluster administrator.

So, my questions are:

1. Is this issue still open? If so what is missing?
2. Has there been any other discussions on how to handle the Cloud Vendors that currently support the loadBalancerIP field of a service?

On 2, I'm happy to create a new issue if it is off topic but curious to see if there was any thought put into this.

[tommie]

Looking at the description, it seems like this PR is about EG reading Service.status to update Gateway.status.addresses, but I took it to mean the full chain Gateway.spec.listeners.addresses -> Service.status -> Gateway.status.addresses, since only doing the latter wouldn't help anything. I feel this is done, but my bare-metal use-case is probably uncommon (and there is #1035).

If there's an issue with Service.status.ExternalIPs, then EG is probably setting it even for LoadBalancer services, which it probably shouldn't, as discussed: #1035 (comment). #1463 was related to that discussion. The question is if

gateway/internal/infrastructure/kubernetes/proxy/resource_provider.go

Line 100 in 634ccdb

serviceSpec.ExternalIPs = r.infra.Addresses

should only be for type not being LoadBalancer.

I think (2) would be a different question.

[arkodg]

@LanceEa Id prefer if we close this issue and create a new one, if we believe the feature has not been implemented correctly.
Here is the current implementation

• When a user sets Gateway.Spec.Addresses , EG sets that value to the generated managed Envoy Proxy Service's ExternalIPs field which also sets (not appends) the value in Gateway.Status.Addresses field