Skip to content

docs: replace Ingress NGINX with Gateway API (Envoy Gateway) #5160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
heruan wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

docs: replace Ingress NGINX with Gateway API (Envoy Gateway) #5160

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions articles/tools/azure-cloud/tls.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ order: 30

= Transport Layer Security (TLS)

.Ingress NGINX Retirement
[IMPORTANT]
====
The Kubernetes community has https://kubernetes.io/blog/2025/11/11/ingress-nginx-retirement/[announced] that the Ingress NGINX controller is being retired, with best-effort maintenance only until March 2026. The https://gateway-api.sigs.k8s.io/[Gateway API] is the recommended replacement. The instructions below reference NGINX Ingress and may need to be adapted if you're using the Gateway API.
====

To help with using `letsencrypt`, there's an option in the Terraform variables to enable `letsencrypt` and `certmanager` in the cluster.

After `certmanager` is installed, you'll still need to https://learn.microsoft.com/en-us/azure/aks/ingress-tls?tabs=azure-cli#create-a-ca-cluster-issuer[create the cluster issuer].
Expand Down
97 changes: 67 additions & 30 deletions articles/tools/kubernetes/getting-started.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,59 +229,96 @@ my-app-v1-f87bfcbb4-rxvjb 1/1 Running 0 22s
----


== Ingress Rules
== Gateway

To access the application, you need to provide some ingress rules. If you don't already have `ingress-nginx` installed in your cluster, install it with the following command:
To access the application from outside the cluster, you need to set up a gateway. This tutorial uses https://gateway.envoyproxy.io/[Envoy Gateway] as the https://gateway-api.sigs.k8s.io/[Gateway API] implementation, which provides built-in support for session persistence -- a requirement for Vaadin applications running with multiple replicas.

.Ingress NGINX Retirement
[IMPORTANT]
====
The Kubernetes community has https://kubernetes.io/blog/2025/11/11/ingress-nginx-retirement/[announced] that the Ingress NGINX controller is being retired, with best-effort maintenance only until March 2026. The Gateway API is the recommended replacement. This tutorial uses Envoy Gateway as the Gateway API implementation, but other implementations that support session persistence should also work.
====

Install Envoy Gateway in your cluster using Helm:

[source,terminal]
helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.6.0 -n envoy-gateway-system --create-namespace

Then create a gateway manifest:

.`gateway.yaml`
[source,yaml]
----
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: public-gateway
spec:
gatewayClassName: eg
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: All
----

Deploy it to your cluster:

[source,terminal]
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.4.0/deploy/static/provider/cloud/deploy.yaml
kubectl apply -f gateway.yaml

Then create an ingress rule manifest file like so:
Next, create an HTTP route manifest to direct traffic to the application with cookie-based session persistence:

.`ingress-v1.yaml`
.`route-v1.yaml`
[source,yaml]
----
apiVersion: networking.k8s.io/v1
kind: Ingress
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: my-app
annotations:
kubernetes.io/ingress.class: "nginx"
# --- Optional ---
# If server Push is enabled in the application and uses Websocket for transport,
# these settings replace the default Websocket connection timeouts in Nginx.
nginx.ingress.kubernetes.io/proxy-send-timeout: "86400"
nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
# ---
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/affinity-mode: "persistent"
spec:
parentRefs:
- name: public-gateway
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-app-v1
port:
number: 80
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: my-app-v1
port: 80
sessionPersistence:
type: Cookie
----

Deploy the manifest to your cluster with the following command:
Deploy the manifest to your cluster:

[source,terminal]
kubectl apply -f ingress-v1.yaml
kubectl apply -f route-v1.yaml

The application should now be available at `http://localhost`.

.Accessing Application Locally
[NOTE]
====
To access the application from your local machine, it may be necessary to use the `port-forward` utility. In this case use the following command:
To access the application from your local machine, it may be necessary to use the `port-forward` utility. First, find the Envoy proxy service:

[source,terminal]
export ENVOY_SERVICE=$(kubectl get svc -l gateway.envoyproxy.io/owning-gateway-name=public-gateway -o jsonpath='{.items[0].metadata.name}')

Then forward a local port:

[source,terminal]
kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 8080:80
kubectl port-forward service/${ENVOY_SERVICE} 8080:80

The application should now be available at `http://localhost:8080`.
====
Expand Down
177 changes: 73 additions & 104 deletions articles/tools/kubernetes/update-version.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,155 +96,124 @@ my-app-v1-f87bfcbb4-rxvjb 1/1 Running 0 10m
----


== Deploy Canary Ingress Rules
== Route New Traffic to the New Version

Before switching permanently to the new version, you can test access by deploying "canary" ingress rules. This routes new sessions to the new version, while keeping existing sessions on the previous version.
Before switching permanently to the new version, you can route only new sessions to the new version, while keeping existing sessions on the previous version. This is achieved using the Gateway API's weighted routing and cookie-based session persistence.

Create the ingress rule manifest like so:
Update the HTTP route manifest to include both versions:

.ingress-v2-canary.yaml
.route-v1-v2.yaml
[source,yaml]
----
apiVersion: networking.k8s.io/v1
kind: Ingress
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: my-app-canary
annotations:
kubernetes.io/ingress.class: "nginx"
# --- Optional ---
# If server Push is enabled in the application and uses Websocket for transport,
# these settings replace the default Websocket connection timeouts in Nginx.
nginx.ingress.kubernetes.io/proxy-send-timeout: "86400"
nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
# ---
nginx.ingress.kubernetes.io/affinity: "cookie"
# Redirects all the requests to the new version of the application
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "100"
name: my-app
spec:
parentRefs:
- name: public-gateway
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-app-v2
port:
number: 80
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: my-app-v1
port: 80
weight: 0
- name: my-app-v2
port: 80
weight: 100
sessionPersistence:
type: Cookie
----

Then deploy it to your cluster:
Deploy it to your cluster:

[source,terminal]
kubectl apply -f ingress-v2-canary.yaml


== Notify Existing Users

[IMPORTANT]
.Configuration Snippets
====
As of version 1.9.0 of Ingress NGINX (helm chart version 4.8.0), the `nginx.ingress.kubernetes.io/configuration-snippet` annotation is no longer supported by default. It is considered a security risk, as it allows arbitrary code execution in the NGINX configuration. To use this annotation, you must enable it explicitly in the NGINX Ingress Controller configuration.
kubectl apply -f route-v1-v2.yaml

Create a patch file:
New sessions are now routed to the new version, while existing sessions remain connected to the previous version through cookie-based session persistence.

.ingress-controller-patch.yaml
[source,yaml]
----
data:
annotations-risk-level: "Critical"
allow-snippet-annotations: "true"
----

Next, apply the patch to your cluster:
== Notify Existing Users

[source,terminal]
kubectl -n ingress-nginx patch configmap ingress-nginx-controller --type merge --patch-file ingress-controller-patch.yaml
====
Next, you may want to notify existing users that a new version is available. To do this, add an HTTP header filter that injects the `X-AppUpdate` header into all requests. The Kubernetes Kit uses this header to show a notification to users on the previous version, prompting them to switch.

Next, you may want to notify existing users. To do this, create the ingress rule manifest:
Update the HTTP route manifest to include a request header modifier:

.ingress-v1-notify.yaml
.route-v1-v2-notify.yaml
[source,yaml]
----
apiVersion: networking.k8s.io/v1
kind: Ingress
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: my-app
annotations:
kubernetes.io/ingress.class: "nginx"
# --- Optional ---
nginx.ingress.kubernetes.io/proxy-send-timeout: "86400"
nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
# ---
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/affinity-mode: "persistent"
# Adds the X-AppUpdate new version header to the requests for the current
# application which is used to trigger the version update notification popup
nginx.ingress.kubernetes.io/configuration-snippet: proxy_set_header X-AppUpdate "2.0.0";
spec:
parentRefs:
- name: public-gateway
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-app-v1
port:
number: 80
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: my-app-v1
port: 80
weight: 0
- name: my-app-v2
port: 80
weight: 100
sessionPersistence:
type: Cookie
filters:
- type: RequestHeaderModifier
requestHeaderModifier:
add:
- name: X-AppUpdate
value: "2.0.0"
----

Next, deploy it to your cluster:
Deploy it to your cluster:

[source,terminal]
kubectl apply -f ingress-v1-notify.yaml
kubectl apply -f route-v1-v2-notify.yaml


== Remove Previous Version

Once you're sure of the new version deployment, you can remove the previous version and make the ingress rules point permanently to the new version.
Once you're satisfied with the new version deployment, you can remove the previous version and route all traffic to the new version.

First, create the ingress rule manifest like this:
First, update the HTTP route manifest to point only to the new version:

.ingress-v2.yaml
.route-v2.yaml
[source,yaml]
----
apiVersion: networking.k8s.io/v1
kind: Ingress
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: my-app
annotations:
kubernetes.io/ingress.class: "nginx"
# --- Optional ---
nginx.ingress.kubernetes.io/proxy-send-timeout: "86400"
nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"
# ---
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/affinity-mode: "persistent"
spec:
parentRefs:
- name: public-gateway
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-app-v2
port:
number: 80
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: my-app-v2
port: 80
sessionPersistence:
type: Cookie
----

Then deploy it to your cluster like so:
Deploy it to your cluster:

[source,terminal]
kubectl apply -f ingress-v2.yaml
kubectl apply -f route-v2.yaml

Now delete the previous version and the canary ingress rules:
Now delete the previous version deployment and service:

[source,terminal]
----
kubectl delete -f app-v1.yaml
kubectl delete -f ingress-v2-canary.yaml
----
Loading