nginx with istio

[slehlib]

Hello,

I have some question about nginx controler with istio when mtls is set enforced
kubernetes v1.27.4
istioctl version
client version: 1.19.0
control plane version: 1.19.0
data plane version: 1.19.0 (21 proxies)
nginx-ingress:3.2.1
in my kubernetes cluster i have 3 namespaces: istio-system, prometheus and nginx-ingress, il try to run all internal traffic are MTLS but when a active peerauthentificat STRIC beetwen prometheus and nginx

            > GET / HTTP/1.1
            > Host: prometheus-qualif.infra
            > User-Agent: curl/7.50.3
            > Accept: */*
            >
            < HTTP/1.1 502 Bad Gateway
            < Server: nginx/1.25.2
            < Date: Thu, 12 Oct 2023 16:54:56 GMT
            < Content-Type: text/html
            < Content-Length: 157
            < Connection: keep-alive
            < Strict-Transport-Security: max-age=157680000

i add this annotation on my nginx daemonset
sidecar.istio.io/inject: "true"
traffic.sidecar.istio.io/excludeInboundPorts: 80,443
traffic.sidecar.istio.io/excludeOutboundIPRanges: 10.233.0.1/16
traffic.sidecar.istio.io/includeInboundPorts: ""

and in my ingress for prometheus
nginx.ingress.kubernetes.io/service-upstream: "true"
nginx.ingress.kubernetes.io/upstream-vhost: prometheus-server.prometheus.svc.cluster.local
if i remove mtls stric on prometheus namespaces :
> GET / HTTP/1.1
> Host: prometheus-qualif.infra
> User-Agent: curl/7.50.3
> Accept: /
>
< HTTP/1.1 302 Found
< Server: nginx/1.25.2
< Date: Thu, 12 Oct 2023 17:02:36 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 29
< Connection: keep-alive
< location: /graph
< x-envoy-upstream-service-time: 0
< x-envoy-peer-metadata-id: sidecar10.233.119.8prometheus-server-58f4d79c6-rwt4b.prometheus~prometheus.svc.cluster.local
< x-envoy-peer-metadata:

Any solution to use mtls enforced
thanks

[jasonwilliams14]

@slehlib Hello and welcome!

The annotations you are using for your Ingress resource:

nginx.ingress.kubernetes.io/service-upstream: "true"
nginx.ingress.kubernetes.io/upstream-vhost: prometheus-server.prometheus.svc.cluster.local

Are for a different NGINX Ingress controller. That is the community project.
This particular project is bulit and managed by NGINX.

Take a look at our document on using NGINX Ingress controller with Istio service mesh

https://docs.nginx.com/nginx-ingress-controller/tutorials/nginx-ingress-istio/

HTH

[slehlib]

thanks you for your response
i already read this document and i have the same issue
kubectl get virtualservers.k8s.nginx.org -n nginx-ingress
NAME STATE HOST IP PORTS AGE
prometheus Valid prometheus-qualif.infra 9m6s

kubectl get virtualservers.k8s.nginx.org -n nginx-ingress -oyaml
apiVersion: v1
items:
- apiVersion: k8s.nginx.org/v1
  kind: VirtualServer
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"k8s.nginx.org/v1","kind":"VirtualServer","metadata":{"annotations":{},"name":"prometheus","namespace":"nginx-ingress"},"spec":{"host":"prometheus-qualif.infra","routes":[{"action":{"proxy":{"requestHeaders":{"set":[{"name":"Host","value":"prometheus-server.prometheus.svc.cluster.local"}]},"upstream":"prometheus"}},"path":"/"}],"upstreams":[{"name":"prometheus","port":80,"service":"prometheus-server","use-cluster-ip":true}]}}
    creationTimestamp: "2023-10-12T17:05:22Z"
    generation: 3
    name: prometheus
    namespace: nginx-ingress
    resourceVersion: "17914757"
    uid: d8e2c89c-e5a0-4ba6-822a-704d4fbd6799
  spec:
    host: prometheus-qualif.infra
    routes:
    - action:
        proxy:
          requestHeaders:
            set:
            - name: Host
              value: prometheus-server.prometheus.svc.cluster.local
          upstream: prometheus
      path: /
    upstreams:
    - name: prometheus
      port: 80
      service: prometheus-server
      use-cluster-ip: true
  status:
    message: 'Configuration for nginx-ingress/prometheus was added or updated '
    reason: AddedOrUpdated
    state: Valid
kind: List
metadata:
  resourceVersion: ""

curl -vcurl -kI http://prometheus-qualif.infra

• Rebuilt URL to: http://prometheus-qualif.infra/
• Trying 192.168.1.54...
• TCP_NODELAY set
• Connected to prometheus-qualif.infra (192.168.1.54) port 80 (#0)

HEAD / HTTP/1.1
Host: prometheus-qualif.infra
User-Agent: curl/7.50.3
Accept: /

< HTTP/1.1 502 Bad Gateway
HTTP/1.1 502 Bad Gateway
< Server: nginx/1.25.2
Server: nginx/1.25.2
< Date: Thu, 12 Oct 2023 17:16:16 GMT
Date: Thu, 12 Oct 2023 17:16:16 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 157
Content-Length: 157
< Connection: keep-alive
Connection: keep-alive
< Strict-Transport-Security: max-age=157680000
Strict-Transport-Security: max-age=157680000

<

• Curl_http_done: called premature == 0
• Connection #0 to host prometheus-qualif.infra left intact

[jasonwilliams14]

This looks to be a problem with NGINX Ingress connecting to the upstream.
Did you inject both NGINX Ingress controller and your upstream with the istio sidecar proxy?

[jasonwilliams14]

@slehlib were you able to resolve your issue?

[vepatel]

closing as stale