Proxy protocol support for X-Forwarded-Port

[djboris9]

NGINX Ingress controller version: 0.27.1 and master
Kubernetes version: 1.17.1

Environment:

• Cloud provider or hardware configuration: on premise
• OS: RedHat Enterprise Linux 7.7
• Kernel: 3.10.0-1062.4.1.el7.x86_64
• Install tools: kubeadm
• Others: F5 loadbalancer with proxy protocol in front of Ingress. Ingresses deployed as DaemonSet using HostPort

What happened:
The port is inconsistent in the provided http headers (443 vs 8443):

X-Forwarded-For: 10.1.2.3
X-Forwarded-Host: kubenurse.example.com:8443
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Real-Ip: 10.1.2.3
X-Scheme: https

What you expected to happen:
The X-Forwareded-Host and X-Forwareded-Port should have the same port, which should be the frontend port on the loadbalancer, which is also passed by the proxy protocol:

X-Forwarded-For: 10.1.2.3
X-Forwarded-Host: kubenurse.example.com:8443
X-Forwarded-Port: 8443
X-Forwarded-Proto: https
X-Real-Ip: 10.1.2.3
X-Scheme: https

How to reproduce it:

1. Install ingress-nginx Version 0.27.1 on an existing Kubernetes cluster
2. Configure a F5 loadbalancer (AWS ELB should also work) with proxy protocol enabled and the frontend port must be different than the backend port. The backend should point on the Ingress NodePort or HostPort
3. Update the following values in thenginx-configuration ConfigMap: real-ip-header: proxy_protocol, use-proxy-protocol: "true"
4. Make a request to a dummy site where you can see the incoming request headers (e.g. a container with nc -l -p 8080 running, kuard or in our case kubenurse
5. Look at the X-Forwarded-Host and X-Forwarded-Port headers

Anything else we need to know:
If we edit the nginx.tmpl value of

ingress-nginx/rootfs/etc/nginx/template/nginx.tmpl

Line 1155 in 12aa726

{{ $proxySetHeader }} X-Forwarded-Port $pass_port;

to $proxy_protocol_server_port, the correct port (according to proxy protocol) is used.
And if we disable the proxy protocol, the value of X-Forwarded-Port will be the server port.

I created a fork and patched it, but I'm not sure about $pass_server_port and $pass_port, which are used by the lua_ingress.lua script:

ingress-nginx/rootfs/etc/nginx/template/nginx.tmpl

Lines 1035 to 1037 in 12aa726

	set $pass_server_port $server_port;
	set $best_http_host $http_host;
	set $pass_port $pass_server_port;

Maybe this mechanism of using the proxy protocol port should be implemented in the lua script, so I'm asking about your opinion which way to go and what exactly is the purpose of this port rewriting in the script.

/kind bug

[aledbf]

@djboris9 this should be using the same approach you see here

ingress-nginx/rootfs/etc/nginx/template/nginx.tmpl

Line 322 in 12aa726

{{ if $all.Cfg.UseProxyProtocol }}

(pass_server_port variable should be set in a map with server_port or proxy_protocol_server_port if proxy protocol is enabled)

[djboris9]

Thanks for your fast response @aledbf

Do you mean something like this:

map $pass_server_port $proxy_protocol_server_port {
    {{ if $all.Cfg.UseProxyProtocol }}
    default          $proxy_protocol_server_port;
    ''               $server_port;
    {{ else }}
    default          $server_port;
    {{ end}}
}

or maybe a more direct way like this:

{{ if $all.Cfg.UseProxyProtocol }}
set $pass_server_port    $proxy_protocol_server_port;
{{ else }}
set $pass_server_port    $server_port;
{{ end }}

Currently, I don't see a reason for a map but I'm also not used to write Nginx configurations.

So I can update my fork, test it and create a PR.

[aledbf]

or maybe a more direct way like this:

Yes, that is a trivial change. Please open a PR with such a change.