Backend CRD with multi-port Service selects wrong endpoint port

[iamdempa]

Backend CRD with multi-port Service selects wrong endpoint port

Environment

Component	Version
HAProxy Ingress Controller	3.2.6
Helm Chart	kubernetes-ingress-1.49.0
HAProxy	3.2.12-6011f448e
Backend CRD API	ingress.v3.haproxy.org/v3
Kubernetes	1.30 (AKS)

Description

When using a Backend CRD (haproxy.org/cr-backend annotation) with a Service that has multiple ports, the controller selects the wrong port for the backend servers. The Ingress specifies port 8080, but HAProxy routes to port 1337 (the second port in the Service).

This issue persists even after:

• Deleting and recreating the application pod
• Modifying the Backend CRD (adding server_timeout, connect_timeout)
• Waiting for cache resync

Backends without a Backend CRD (using only Ingress annotations) work correctly.

Steps to Reproduce

1. Create a Service with multiple ports:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  ports:
  - name: my-app-http        # First port - application
    port: 8080
    targetPort: 8080
  - name: my-app-metrics     # Second port - metrics
    port: 1337
    targetPort: 1337

2. Create a Backend CRD:

apiVersion: ingress.v3.haproxy.org/v3
kind: Backend
metadata:
  name: my-service-backend
  namespace: default
spec:
  name: my-service-backend
  server_timeout: 30000
  connect_timeout: 5000

3. Create an Ingress referencing port 8080 with the Backend CRD:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    haproxy.org/cr-backend: default/my-service-backend
spec:
  ingressClassName: haproxy
  rules:
  - host: my-app.example.com
    http:
      paths:
      - backend:
          service:
            name: my-service
            port:
              number: 8080      # Explicitly requesting port 8080
        path: /
        pathType: Prefix

4. Check the generated HAProxy config:

kubectl exec <haproxy-pod> -- grep -A10 "backend.*my-service" /etc/haproxy/haproxy.cfg

Expected:

server SRV_1 10.x.x.x:8080 enabled   # Port 8080 as specified in Ingress

Actual:

server SRV_1 10.x.x.x:1337 enabled   # WRONG - using port 1337 (metrics)

Observed Behavior

Comparing a working backend vs broken backend:

Backend	Uses Backend CRD	Server Slot	Port	Status
service-a	Yes (generation: 1, never modified)	SRV_2	8080	Correct
service-b	Yes (generation: 71, modified multiple times)	SRV_1	1337	Wrong
Services without Backend CRD	No	SRV_1 or SRV_2	Correct	Correct

The broken backend persistently uses port 1337 regardless of:

• Pod deletion/recreation (endpoint update)
• Backend CRD modifications (adding server_timeout, connect_timeout)
• HAProxy reloads

Configuration Details

Ingress correctly specifies port 8080:

{
  "service": {
    "name": "my-service",
    "port": { "number": 8080 }
  }
}

Service has both ports correctly defined:

[
  { "name": "app-http", "port": 8080, "targetPort": 8080 },
  { "name": "app-metrics", "port": 1337, "targetPort": 1337 }
]

EndpointSlice has correct port mapping:

[
  { "name": "app-http", "port": 8080 },
  { "name": "app-metrics", "port": 1337 }
]

But HAProxy config uses wrong port:

backend ns_svc_my-service_app-http
  server SRV_1 10.x.x.x:1337 enabled    # WRONG - should be 8080

Root Cause Hypothesis

When a Backend CRD is used, the controller's endpoint-to-port resolution logic does not correctly match the Ingress service port reference to the corresponding EndpointSlice port. Instead, it appears to select the wrong port from the multi-port EndpointSlice.

This may be related to the fix in v1.7.4/v1.10.13 ("Fix an incorrect lookup of the Endpoints corresponding to a Service Port") not covering the Backend CRD code path.

Workaround

None found. The following do NOT fix the issue:

• Deleting/recreating the application pod
• Modifying the Backend CRD
• Adding server_timeout and connect_timeout to the Backend CRD
• HAProxy controller restarts

Potential workaround (not tested): Use port name instead of port number in the Ingress, but Kubernetes limits port names to 15 characters which may not be practical.

Impact

• Traffic routed to wrong port (e.g., metrics port instead of application port)
• Results in HTTP 404 or unexpected responses (unless the app happens to serve on both ports)
• Only affects services with multiple ports when Backend CRD is used
• Issue persists indefinitely once triggered

Related Issues

• Wrong port in haproxy.cfg #39 — Wrong port in haproxy.cfg (fixed in v1.1.5). Note: The workaround from Wrong port in haproxy.cfg #39 ("set servicePort to container port") does not apply here because our Service already has port == targetPort (8080). This is a different bug where the controller selects the wrong port from a multi-port Service entirely.
• Http response rules (http_response_rule_list) are not created when using backend crd v3 #761 — http_response_rule_list not created with Backend CRD v3 (confirms Backend CRD v3 has multiple bugs)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.