authorization for https connections?

[legale]

Hi. Thank you for your work.

Is there any way to enable authorization for https connections?

[chobits]

hi @legale

could you give more details of how to reproduce this issue?

[toadzhou]

Hi @chobits
Auth_basic is invalid for https

curl -x 127.0.0.1:3128 http://www.test.com 401
curl -u admin:admin -x 127.0.0.1:3128 http://www.test.com 200
curl -x 127.0.0.1:3128 https://www.test.com 200
curl -u admin:admin -x 127.0.0.1:3128 https://www.test.com 200

server {
    listen                         3128;

    # dns resolver used by forward proxying
    resolver                       8.8.8.8;


    # forward proxy for CONNECT request
    proxy_connect;
    proxy_connect_allow            all;
    #proxy_connect_allow            80 443 563;
    proxy_connect_connect_timeout  100s;
    proxy_connect_read_timeout     100s;
    proxy_connect_send_timeout     100s;

    # forward proxy for non-CONNECT request
    location / {
        # auth
        auth_basic "test auth";        #This is invalid for https
        auth_basic_user_file htpasswd;

        autoindex on;
        proxy_pass http://$host;
        proxy_set_header Host $host;
    }
}

[gamelton]

It seems there is issue with HTTPS basic authentication.

I get 401 Unauthorized.

Here is nginx config

server {
    listen                         3129 ssl;
    ssl_certificate                /etc/letsencrypt/live/proxy.address/fullchain.pem;
    ssl_certificate_key            /etc/letsencrypt/live/proxy.address/privkey.pem;
    resolver                       8.8.8.8;
    auth_pam                       "nginxProxyAuth";
    auth_pam_service_name          "nginx";
    proxy_connect;
    proxy_connect_allow            all;
    location / {
        proxy_pass                 http://$host;
        proxy_set_header           Host $host;
    }
}

Here is curl trace

curl --proxy https://proxy.address:3129 http://example.com/ --proxy-user user:password --include --verbose --proxy-cacert "C:\curl\bin\curl-ca-bundle.crt"

* Connected to proxy.address (1.2.3.4) port 3129 (#0)
* successfully set certificate verify locations:
*   CAfile: C:\curl\bin\curl-ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* Proxy certificate:
*  subject: CN=proxy.address
*  start date: Sep 27 17:32:01 2018 GMT
*  expire date: Dec 26 17:32:01 2018 GMT
*  subjectAltName: host "proxy.address" matched cert's "proxy.address"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Proxy auth using Basic with user 'user'
> GET http://example.com/ HTTP/1.1
> Host: example.com
> Proxy-Authorization: Basic cm9vdDpTTHNza29yZA==
> User-Agent: curl/7.61.1
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 401 Unauthorized

I happen to have squid configured on the same machine. HTTPS basic auth works on it.

* Connected to proxy.address (1.2.3.4) port 3129 (#0)
* successfully set certificate verify locations:
*   CAfile: C:\curl\bin\curl-ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / AES256-GCM-SHA384
* Proxy certificate:
*  subject: CN=proxy.address
*  start date: Sep 27 17:32:01 2018 GMT
*  expire date: Dec 26 17:32:01 2018 GMT
*  subjectAltName: host "proxy.address" matched cert's "proxy.address"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Proxy auth using Basic with user 'root'
> GET http://example.com/ HTTP/1.1
> Host: example.com
> Proxy-Authorization: Basic cm9vdDpTTHNza29yZA==
> User-Agent: curl/7.61.1
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK

[gamelton]

Actually it seems nginx PAM authentication breaks when used with this module.
HTTP basic authentication also fails.

curl --proxy http://proxy.address:3128 http://example.com/ --proxy-user user:password --include --verbose

* Connected to proxy.address (1.2.3.4) port 3128 (#0)
* Proxy auth using Basic with user 'user'
> GET http://example.com/ HTTP/1.1
> Host: example.com
> Proxy-Authorization: Basic cm9vdDpTTHNza29yZA==
> User-Agent: curl/7.61.1
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 401 Unauthorized

nginx configuration

server {
    listen                         3128;
    resolver                       8.8.8.8;
    auth_pam                       "nginxProxyAuth";
    auth_pam_service_name          "nginx";
    proxy_connect;
    proxy_connect_allow            all;
    location / {
        proxy_pass                 http://$host;
        proxy_set_header           Host $host;
    }
}

I used nginx PAM auth module. I checked that basic auth on simple page works. Please @chobits advise on authentication options for this module.

[chobits]

hi，

let me check.

(I only check nginx basic auth module before)

[toadzhou]

nginx的auth模块也不起作用吧

[moralrebuild]

any progress about this critical bug? From my point of view, it provides little value if this module insists to break/deny the existing authentication mechanism.

[chobits]

any progress about this critical bug? From my point of view, it provides little value if this module insists to break/deny the existing authentication mechanism.

This module can work with lua-nginx-module. During REWRITE phase (directive rewrite_by_lua_*), you can check request and detect whether to break/deny the request.

But you should know that the request which you can check is only the CONNECT request, not any data proxied under CONNECT tunnel.

[moralrebuild]

@chobits Thanks for your quick reply. But we all want is nothing else but basic auth to prevent illegal traffic pass through forward proxy. Do you think it is not important?

[chobits]

hi @moralrebuild

@gamelton 's case has two problem, which are not associated with proxy connect module.
(#42 (comment))

1. His test case does not generate CONNECT request, please note that proxy connect module can only handle CONNECT request, other requests (GET/POST) is handled by nginx itself.
2. The auth module of his test case is not nginx auth basic module, it's nginx PAM auth module. Why this module cannot work, should be debugged with https://github.com/sto/ngx_http_auth_pam_module. Although, it does not concern proxy_connect module, because it works for GET request.

Note that I have checked proxy_connect module can work under auth basic module (Note that my test case worked for CONNECT request with auth basic header, NOT for GET request).

[chobits]

Note that requests under CONNECT tunnel, such as some GET/POST requests cannot be checked by auth basic module for your proxy_connect nginx.

This module only handles CONNECT request, the raw stream (maybe GET/POST request, or anyother data stream) under CONNECT tunnel cannot be parsed and is passed to backend server directly.

(The similar problem for this case, see #108 (comment). )

[deba12]

I think this module should supports
proxy authentication
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/407

[moralrebuild]

@chobits
Let me explain what I observed.

1. I build this module from its master branch with Nginx 1.16.0.
2. the nginx configuration

server {
	    listen 443 ssl;
	    listen [::]:443 ssl;
	    server_name my.domain;
	    auth_basic "server auth";
	    auth_basic_user_file /etc/nginx/htpasswd;
	    ssl_certificate /etc/letsencrypt/live/my.domain/fullchain.pem;
	    ssl_certificate_key /etc/letsencrypt/live/my.domain/privkey.pem;
	    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
	    ssl_session_timeout 5m;
	    ssl_protocols TLSv1.3 TLSv1.2 TLSv1.1;
	    ssl_prefer_server_ciphers on;
	    ssl_session_cache builtin:1000 shared:SSL:10m;
	    access_log on;
	    root         /var/www/html;
	    index index.html index.htm index.nginx-debian.html;

	    # dns resolver used by forward proxying
	    resolver                       8.8.8.8;

	    # forward proxy for CONNECT request
	    proxy_connect;
	    proxy_connect_allow            443 22;
	    proxy_connect_connect_timeout  10s;
	    proxy_connect_read_timeout     10s;
	    proxy_connect_send_timeout     10s;
	    
	    # forward proxy for non-CONNECT request
	    location / {
	        proxy_pass http://$host;
	        proxy_set_header Host $host;
	    }
	}

3. the test results for two use cases: one is forward proxying, another is tunnelling ssh traffic over https
   3.1 curl

curl https://github.com/ -v -x my.domain:443 --proxy-user user01:password01
*   Trying 111.222.33.44...
* TCP_NODELAY set
* Connected to (nil) (111.222.33.44) port 443 (#0)
* Establish HTTP proxy tunnel to github.com:443
* Proxy auth using Basic with user 'user01'
> CONNECT github.com:443 HTTP/1.1
> Host: github.com:443
> Proxy-Authorization: Basic cHVuZ2lzemI6TEdtMzZCaXc5SlkwdTGFqNA==
> User-Agent: curl/7.52.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 400 Bad Request
< Server: nginx/1.16.0
< Date: Mon, 17 Jun 2019 07:44:24 GMT
< Content-Type: text/html
< Content-Length: 255
< Connection: close
<
* Received HTTP code 400 from proxy after CONNECT
* Curl_http_done: called premature == 0
* Closing connection 0

3.2 proxytunnel

proxytunnel -v -E -p my.domain:443 -P pungiszb:LGm36Biw9JY1Laj4 -d my.domain:22     # for testing only purpose, enable one proxy rather than two
SSL client to proxy enabled
Local proxy my.domain resolves to 111.222.33.44
Connected to my.domain:443 (local proxy)
Set SNI hostname to my.domain

Tunneling to my.domain:22 (destination)
Communication with local proxy:
 -> CONNECT my.domain:22 HTTP/1.1
 -> Host: my.domain:9443
 -> Proxy-Authorization: Basic cHVuZ2lzemI6TEdtMzZCaXc5SlkwdTGFqNA==
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 401 Unauthorized
HTTP return code: 401 Unauthorized
 <- Server: nginx/1.16.0
 <- Date: Mon, 17 Jun 2019 07:53:17 GMT
 <- Content-Type: text/html
 <- Content-Length: 179
 <- Connection: keep-alive
 <- WWW-Authenticate: Basic realm="server auth"

nginx debug log :

2019/06/17 15:58:29 [debug] 30468#30468: *50954 accept: 10.79.76.14:60038 fd:13
2019/06/17 15:58:29 [debug] 30468#30468: *50954 event timer add: 13: 60000:354427063
2019/06/17 15:58:29 [debug] 30468#30468: *50954 reusable connection: 1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 epoll add event: fd:13 op:1 ev:80002001
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http check ssl handshake
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http recv(): 1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 https ssl handshake: 0x16
2019/06/17 15:58:29 [debug] 30468#30468: *50954 tcp_nodelay
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL server name: "my.domain"
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_do_handshake: -1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_get_error: 2
2019/06/17 15:58:29 [debug] 30468#30468: *50954 reusable connection: 0
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL handshake handler: 0
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_do_handshake: 1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL: TLSv1.2, cipher: "ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD"
2019/06/17 15:58:29 [debug] 30468#30468: *50954 reusable connection: 1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http wait request handler
2019/06/17 15:58:29 [debug] 30468#30468: *50954 malloc: 000055DA697591D0:1024
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_read: -1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_get_error: 2
2019/06/17 15:58:29 [debug] 30468#30468: *50954 free: 000055DA697591D0
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http wait request handler
2019/06/17 15:58:29 [debug] 30468#30468: *50954 malloc: 000055DA697591D0:1024
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_read: 162
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_read: -1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 SSL_get_error: 2
2019/06/17 15:58:29 [debug] 30468#30468: *50954 reusable connection: 0
2019/06/17 15:58:29 [debug] 30468#30468: *50954 posix_memalign: 000055DA6973B4B0:4096 @16
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http process request line
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http request line: "CONNECT my.domain:22 HTTP/1.1"
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http uri: ""
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http args: ""
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http exten: ""
2019/06/17 15:58:29 [debug] 30468#30468: *50954 posix_memalign: 000055DA6970C250:4096 @16
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http process request header line
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http header: "Host: my.domain:22"
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http header: "Proxy-Authorization: Basic  cHVuZ2lzemI6TEdtMzZCaXc5SlkwdTGFqNA=="
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http header: "Proxy-Connection: Keep-Alive"
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http header done
2019/06/17 15:58:29 [debug] 30468#30468: *50954 event timer del: 13: 354427063
2019/06/17 15:58:29 [debug] 30468#30468: *50954 generic phase: 0
2019/06/17 15:58:29 [debug] 30468#30468: *50954 generic phase: 1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 rewrite phase: 2
2019/06/17 15:58:29 [debug] 30468#30468: *50954 rewrite phase: 4
2019/06/17 15:58:29 [debug] 30468#30468: *50954 post rewrite phase: 5
2019/06/17 15:58:29 [debug] 30468#30468: *50954 generic phase: 6
2019/06/17 15:58:29 [debug] 30468#30468: *50954 generic phase: 7
2019/06/17 15:58:29 [debug] 30468#30468: *50954 generic phase: 8
2019/06/17 15:58:29 [debug] 30468#30468: *50954 generic phase: 9
2019/06/17 15:58:29 [debug] 30468#30468: *50954 access phase: 10
2019/06/17 15:58:29 [debug] 30468#30468: *50954 access phase: 11
2019/06/17 15:58:29 [info] 30468#30468: *50954 no user/password was provided for basic authentication, client: 10.79.76.14, server: my.domain, request: "CONNECT my.domain:22 HTTP/1.1", host: "my.domain:22"
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http finalize request: 401, "?" a:1, c:1
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http special response: 401, "?"
2019/06/17 15:58:29 [debug] 30468#30468: *50954 http set discard body
2019/06/17 15:58:29 [debug] 30468#30468: *50954 HTTP/1.1 401 Unauthorized
Server: nginx/1.16.0
Date: Mon, 17 Jun 2019 07:58:29 GMT
Content-Type: text/html
Content-Length: 179
Connection: keep-alive
WWW-Authenticate: Basic realm="server auth"

[chobits]

hi @moralrebuild

3.1 curl

For "3.1 curl" case, your nginx server config listens 443 with ssl protocol, while curl command (curl https://... -x ...) creates plain CONNECT request. So nginx server refuses this CONNECT request with 400. Note that curl command cannot create tunnel under SSL protocol. You should disable ssl protocol for listening config.

3.2 proxytunnel

For this case, proxytunnel creates tunnel under SSL protocol, in which case CONNECT request and later proxied stream is under ssl protocol.

But nginx auth_basic module can only handle Authorization header, proxytunnel request header Proxy-Authorization: Basic cHVuZ2lzemI6TEdtMzZCaXc5SlkwdTGFqNA== cannot be checked. Try the following directive to bypass header limit:

        proxy_connect ...;
        ...;

        # set Authorization header via Proxy-Authorization header
        rewrite_by_lua '
            ngx.req.set_header("Authorization", ngx.var.http_proxy_Authorization)
        ';