buildResolvers should not use IPv6 on hosts that do not support IPv6

[kinghrothgar]

Kubernetes version (use kubectl version):
1.9.1-gke.0

Ingress nginx version
quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.10.1

Environment:

• Cloud provider or hardware configuration:
  GKE
• OS (e.g. from /etc/os-release):
  Container-Optimized OS from Google

What happened:
I am trying to setup external authentication with oauth2_proxy. I configured the ingress with the following annotations:

nginx.ingress.kubernetes.io/auth-signin: https://hello.levenlabs.com/oauth2/sign_in
nginx.ingress.kubernetes.io/auth-url: https://hello.levenlabs.com/oauth2/auth

When I got to https://hello.levenlabs.com, it 403's with these logs:

128.227.42.67 - [128.227.42.67] - - [25/Jan/2018:20:43:43 +0000] "GET / HTTP/1.1" 403 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 0 0.120 [external-authentication] 104.27.183.226:443 0 0.057 403
128.227.42.67 - [128.227.42.67] - - [25/Jan/2018:20:43:43 +0000] "GET / HTTP/1.1" 403 197 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 728 0.120 [external-authentication]
- - - -
2018/01/25 20:43:43 [error] 257#257: *4927 connect() to [2400:cb00:2048:1::681b:b7e2]:443 failed (101: Network is unreachable) while connecting to upstream, client: 128.227.42.67, server: hello.levenlabs.com, request: "GET /favicon.ico HTTP/1.1", subrequest: "/_external-auth-Lw", upstream: "https://[2400:cb00:2048:1::681b:b7e2]:443/oauth2/auth", host: "hello.levenlabs.com", referrer: "https://hello.levenlabs.com/"
2018/01/25 20:43:43 [warn] 257#257: *4927 upstream server temporarily disabled while connecting to upstream, client: 128.227.42.67, server: hello.levenlabs.com, request: "GET /favicon.ico HTTP/1.1", subrequest: "/_external-auth-Lw", upstream: "https://[2400:cb00:2048:1::681b:b7e2]:443/oauth2/auth", host: "hello.levenlabs.com", referrer: "https://hello.levenlabs.com/"
2018/01/25 20:43:43 [error] 257#257: *4927 connect() to [2400:cb00:2048:1::681b:b6e2]:443 failed (101: Network is unreachable) while connecting to upstream, client: 128.227.42.67, server: hello.levenlabs.com, request: "GET /favicon.ico HTTP/1.1", subrequest: "/_external-auth-Lw", upstream: "https://[2400:cb00:2048:1::681b:b6e2]:443/oauth2/auth", host: "hello.levenlabs.com", referrer: "https://hello.levenlabs.com/"
2018/01/25 20:43:43 [warn] 257#257: *4927 upstream server temporarily disabled while connecting to upstream, client: 128.227.42.67, server: hello.levenlabs.com, request: "GET /favicon.ico HTTP/1.1", subrequest: "/_external-auth-Lw", upstream: "https://[2400:cb00:2048:1::681b:b6e2]:443/oauth2/auth", host: "hello.levenlabs.com", referrer: "https://hello.levenlabs.com/"
128.227.42.67 - [128.227.42.67] - - [25/Jan/2018:20:43:43 +0000] "GET /favicon.ico HTTP/1.1" 403 0 "https://hello.levenlabs.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
0 0.062 [external-authentication] [2400:cb00:2048:1::681b:b7e2]:443, [2400:cb00:2048:1::681b:b6e2]:443, 104.27.183.226:443 0, 0, 0 0.000, 0.000, 0.062 502, 502, 403
128.227.42.67 - [128.227.42.67] - - [25/Jan/2018:20:43:43 +0000] "GET /favicon.ico HTTP/1.1" 403 197 "https://hello.levenlabs.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" 702 0.062 [external-authentication] - - - -

It attempted to use an IPv6 address on a host that doesn't support it (GCE). If the host does not support it, the buildResolvers func should set it to off I believe.

What you expected to happen:
I expected nginx to successfully connect to https://hello.levenlabs.com/oauth2/auth for external authentication

How to reproduce it (as minimally and precisely as possible):
Setup an nginx ingress with the following annotations:

nginx.ingress.kubernetes.io/auth-signin: https://DOMAINm/oauth2/sign_in
nginx.ingress.kubernetes.io/auth-url: https://DOMAIN/oauth2/auth

Where DOMAIN is a domain that resolves to an IPv4 and IPv6 address.

[aledbf]

It attempted to use an IPv6 address on a host that doesn't support it (GCE). If the host does not support it, the buildResolvers func should set it to off I believe.

The ingress controller doesn't know where it's running. There's no difference between cloud providers, baremetal or minikube. That said we can add an option to disable IPV6 from the dns resolver
http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver

[kinghrothgar]

I'm gonna try to take a crack at this. To clarify, you are suggesting adding a CLI option to the nginx-ingress-controller, say, enableDNSResolverIPv6 / --enable-dns-resolver-ipv6?

[aledbf]

@kinghrothgar no, I was thinking of adding an additional entry in the configuration configmap, like enable-ipv6-dns and use as default the helper https://github.com/kubernetes/ingress-nginx/blob/master/internal/net/net.go#L41
Then using an if in the buildresolver function you can enable/disable IPV6