Adding a priority field to a backend

[arkodg]

Creating this discussion as a first step before formally creating a GEP for #1802

WHAT

Add the ability to add a priority to a backend ref, allowing the application owner to build backend tiers. i.e. when backends with the highest priority are healthy, all traffic will be routed to them. However when those backends are unhealthy (as detected by the control plane + data plane), the traffic is shifted to the next lower priority.

Taking an example config similar to the ones described in GEP: Gateway API Interaction with Multi-Cluster Services ( https://gateway-api.sigs.k8s.io/geps/gep-1748/)

kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: store
spec:
  parentRefs:
  - name: external-http
  rules:
  - backendRefs:
    - kind: Service
      name: store
      port: 8080
      priority: 0
    - group: multicluster.x-k8s.io
      kind: ServiceImport
      name: store-global
      port: 8080
      priority: 1

In this example all traffic is routed to Service store (which has the highest priority 0) in the common case. When the store Service endpoints are unhealthy, then traffic is routed to ServiceImport store-global (at a lower priority 1) and these endpoints may be outside this cluster.

WHY

• For this specific case, the priority knob helps keep traffic within the cluster, eliminating any egress costs, until the local service is unhealthy and then traffic needs to be egressed outside the cluster.
• The priority field provides a fine tuned way to deal with High Availability outcomes, which is not possible today.

[howardjohn]

This makes me a bit uneasy. I feel like this type of routing is more of a property of the backend then the route. For example, Service already has a field internalTrafficPolicy that is exactly like this (but less feature rich)

[robscott]

I've seen a similar use case for something like active/standby or active/active systems. I agree with @howardjohn that we have to be careful to avoid overlapping with existing config like internalTrafficPolicy, but I think the specific example you shared of spilling over to another cluster, or just generally spilling over to another Service feels somewhat distinct. Would be helpful to collect interest and use cases here so we have a bit more clarity. I'm curious if the use cases could be solved by a waterfall type LB algorithm where the first set of backends are filled before spilling over to the next set of backends.

[arkodg]

looking at the spec for internalTrafficPolicy https://kubernetes.io/docs/concepts/services-networking/service-traffic-policy/, it is specific to creating a hard preference for

• constraint 1 - only applies to endpoints within the same service ( same backendRef )
• constraint 2 - hard preference limited to - local node and same cluster

above API presented, alleviates both these constraints

[howardjohn]

Yeah I know, its quite the same. Just suggesting there is some prior art to represent this at the backend level vs route level. K8s core cannot really express anything but those constraints since it is pretty limited

[youngnick]

I share the concerns from @robscott and @howardjohn here, this is one of those things that seems straightforward but may have big ramifications, because now it's another routing dimension.

I'd also encourage everyone to think about - do we tell a user (in Status or similar) which backend is taking the traffic at the moment and why? If not, that seems not ideal, but if so, then we have another possible rapid-update, apiserver overload problem (since it seems likely that backends may fluctuate reasonably frequently between health and unhealthy).

[arkodg]

Usecase 2. - A business unit may want to route all the ingressed application traffic through a middleware security service before sending it to the application service. However they would also like to address the scenario of, if the middleware is down, they would be okay if the application traffic went directly to the app service, for such a failure scenario.
Here's what the HTTPRoute would look like

kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: store
spec:
  parentRefs:
  - name: external-http
  rules:
  - backendRefs:
    - kind: Service
      name: enterprise-security-middleware-svc
      port: 8080
      priority: 0
    - kind: Service
      name: app-svc
      port: 8080
      priority: 1

[youngnick]

I understand and can see the usecase, but I'm very worried about adding more dimensions to route rules. What happens when you specify a priority and a weight? Now we need to consider that every time we consider any interactions.

I'm not saying "no", I'm saying that I think we all need to take some time to consider the implications - I don't see this being added before GA, tbh.

[arkodg]

I'm just adding use cases as requested above :)

Reg interaction of weights and priorities, weights are applicable ( from a traffic split perspective) within backbends having the same priority