Add Support to Enable Proxy Protocol

[hgranillo]

Hello Dear APISIX Helm Chart maintainers!

This PR adds the option to enable Proxy Protocol (pp) in the APISIX configuration file and allows to add the proxy protocol listeners to the gateway Kubernetes service.

Made all ports configurable in a fashion similar to apisix-gateway and apisix-gateway-tls

I left enable_tcp_pp_to_upstream out because at this moment I have no way to test it. I can add the toggle if needed to merge this PR.

I tested these changes with the following configuration in one of my EKS Clusters.
I'm using the aws-load-balancer-controller to provision a NBL with SSL/TLS Offloading and Proxy Protocol enabled (see loadbalancer annotations below)

This allows me to send HTTP(80) -> ProxyProtocol HTTP(9181) and HTTPS 443 -> ProxyProtocol HTTP (9181)

values.yaml

gateway:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: service_name=ingress-apisix
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: 3600
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: 10
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: proxy_protocol_v2.enabled=true
    # https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/service/nlb/#configuration
    # The AWS in-tree controller ignores those services resources that have the service.beta.kubernetes.io/aws-load-balancer-type annotation as external
    service.beta.kubernetes.io/aws-load-balancer-type: "external"
    #Env Specific Annotations
    service.beta.kubernetes.io/aws-load-balancer-name: "my-ingress-apisix"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xx-xxxx-x:xxxxxxxxxxxxxx:certificate/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

  http:
    enabled: true
    servicePort: 9080
    containerPort: 9080
  tls:
    enabled: true
    servicePort: 9443
    containerPort: 9443
    existingCASecret: ""
    certCAFilename: ""
    http2:
      enabled: true
    sslProtocols: "TLSv1.2 TLSv1.3"
  proxyProtocol:
    http:
      enabled: true
      servicePort: 80
      containerPort: 9181
    https:
      enabled: true
      servicePort: 443
      containerPort: 9181

I've also tested sending HTTPs traffic to the HTTPs Proxy Protocol port with a regular non SSL/TLS offloading NLB and seemed to work fine.

Solves: #325

[heresie]

Hi @hgranillo !

I'm also looking to enable properly the Proxy Protocol on APISIX instances deployed with the Helm Chart. Your changes would be welcomed.

Can somebody review this PR ?

[hgranillo]

Hi @heresie I've forgot about this PR, it has been a while... and I've moved away from apisix since then.

Anyway, my changes are now conflicting with main, please feel free to adapt them.

[heresie]

Thank you @hgranillo for your help. I made a new rebased PR with some minor changes.

[heresie]

#528 was merged, thank you again @hgranillo

[hgranillo]

@heresie thank you for taking care of it!
I guess I'm closing this PR now, thanks again.

[renanramonh]

Those changes are not on the latest chart 2.10.0. Can it be added there by any chance? I see it's just on 1.x branch, otherwise I'll need to keep using an outdated chart :/

[heresie]

@renanramonh

The latest try to implement this is available on #756