HTTP3 support on gateway

[kristiandrucker]

Description:
I'd like to ask about HTTP3 support in Envoy Gateway. Been looking in the official gateway-api sig as well as this repository but couldn't seem to find any mention of HTTP3.
Is it a feature that needs implementing, or is it just undocumented for now?

Relevant Links:

https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/http3
https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.ProtocolType

[youngnick]

Gateway API hasn't defined HTTP3 standards yet, the tracking issue is kubernetes-sigs/gateway-api#687. This is probably blocked until there's more progress there.

[tanujd11]

I think there is some progress on the gateway API issue for HTTP3 support stating no API changes are required. This to be supported by controller implementations.

[tanujd11]

Hey @arkodg I would like to work on this.

[zirain]

thanks @tanujd11

[arkodg]

looks like the implementation in Envoy has been highlighted by
kubernetes-sigs/gateway-api#687 (comment) (thanks @alibo)

what is probably needed is a way for a user to opt in at the Gateway Listener (downstream) level to enable HTTP3 at the xds listener level (alt-svc header, udp listener, quic filter ... ) or should we enable this by default ?

@LanceEa ccing you since you've built this out in Emissary, any suggestion on how we can add this opt in knob using Gateway API ?

an interim solution could be to use EnvoyPatchPolicy post v0.5.0

[tanujd11]

@arkodg Istio creates mirror listener to all HTTP ports. It is behind a feature flag however. The connection is created by client and they can decide based on alt-svc header received whether to upgrade the connection to HTTP3. In the infra a UDP port needs to be opened for the same.

[arkodg]

yeah @tanujd11 , I see 3 options here for native support

• add a feature flag in the EnvoyProxy API that enables the mirror listener by default
• we use some information from the Gateway API fields such as Gateway protocol to specifically set a HTTP3 Listener
• request for new field to be introduced in the Gateway API to fulfill this case.

[LanceEa]

@arkodg @tanujd11 - I haven't put a lot of thought into how it would fit in the Gateway-API world but I did read through #687 and here are just things to think about.

Adding HTTP/3 Downstream support has the following considerations:

1. How should the Service of type LoadBalancer be handled?

   • As the issue outlined, K8s support for MixedProtocolLBService didn't reach beta until 1.26 and will GA in 1.28
   • Cloud support for MixedProtocolLBService is pretty much not existent, AFAIK but I haven't checked in awhile
   • Since Envoy Gateway creates the LB Service we will need to account for the fact that the MixedProtocolLBService is not well supported by cloud vendors and I don't have any insight into if/when this would happen.
     • In Emissary-ingress, the controller doesn't generate these Services like EG does and when we implemented it a year ago we recommended users to manually create two Services for the different protocols you can see some examples here: https://www.getambassador.io/docs/emissary/latest/topics/running/http3#cloud-service-provider-setup. Note, there might be better ways now.

2. Clients (typically browser) in most cases need to be told to use HTTP/3 since QUIC is over UDP

• TIL about HTTPS DNS RR which sounds promising but not sure when it will become mainstream or whether it should factor into current implementation.
• Wide support for alt-svc exists so ensuring the TCP Listener is returning the alt-svc with the correct information is a safe bet.
  • Emissary-ingress uses strict conventions to auto-magically add this to the response headers for the TCP listener but truthfully I'm not in love with the design (I'm the one that designed it 😄 ) because it isn't very flexible and made a lot of assumptions.

3. Configuring Envoy xDS correctly
   • this is fairly straight forward but some little things like QUIC needs a TLS cert to be valid.
   • if listening to multiple protocols on same IP and Port then enable port re-use needs to be set
   • determine the API for allowing end-users to customize alt-svc and/or http3 connection manager options and how it flows through into here.
     • In Emissary-ingress, we didn't add these customizations partially because we try not to expose Envoy internals unless necessary.
     • As @arkodg, pointed out this is the one that needs the design and TBH I haven't thought much about it yet.

I'm happy to chat more about this at one of the community meetings.

[arkodg]

we now have an API / ClientTrafficPolicy https://gateway.envoyproxy.io/latest/api/extension_types.html#clienttrafficpolicy which can be used as a home for this feature field

[tanujd11]

@arkodg I will take a look, Thanks.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.