HTTP3/QUIC not working - please switch to BoringSSL (or LibreSSL or quicTLS)

[the-hotmann]

I have read these issues:

• please add httpv3 protocol  #485
• Image nginx: 1.25.4-alpine3.18 does not have support for HTTPS enabled, and after container startup, an error message is reported using HTTP 3 #867

and think that most people now think, that this dockerized version of Nginx will support HTTP3/QUIC - but it does not, since it is using OpenSSL. The The OpenSSL Compatibility Layer at least does not work for me.

Since Nginx itself supports HTTP3/QUIC, but OpenSSL does not LINK this dockerized version of Nginx (which I love!) does not support HTTP3/QUIC, becasue both things must support it:

• the nginx-version
• the SSL/TLS library used

OpenSSL plans to support HTTP3 for servers from the end of 2024 - but just experimental first (in v3.4.x).
Since this is the current situation I would love to ask to add an additional build (especially the alpine ones) with the addition -boringssl which people (liek me) can use to use and test with HTTP3/QUIC before somewhen OpenSSL supports it.

Note:

• enabeling http3/quic in the config works just fine, BUT your browser (or curl) will not actually use it!
• bascially there are two options: 1.) wait ½-1year and use it with OpenSSL. 2.) use BoringSSL to use it right now!

I used this curl command to verify the actual HTTP Version the server is using:

curl -sIk --http3 https://sub.dom.tld -o/dev/null -w '%{http_version}\n'

or

curl -sIk --http3-only https://sub.dom.tld -o/dev/null -w '%{http_version}\n'

Alternatively you could use the Browsers Dev-Tools to check which protocol actually is getting used - but I prefer the curl version. (curl version should be newer than v8.0.0)

Also please keep in mind, that if you want to use HTTP3/QUIC you need to allow the udp-protocol on Port :443:

services:

  nginx:
    image: nginx:1-alpine-slim
    container_name: nginx
    hostname: nginx
    ports:
      - "443:443"
      - "443:443/udp"
    volumes:
      - "./nginx/templates/:/etc/nginx/templates/:ro"
      - "./nginx/ssl/:/etc/ssl/own/:ro"
    healthcheck:
      test: ["CMD-SHELL", "nc -vz -w1 $(hostname) 443"]
      interval: 1s
      timeout: 1s
      retries: 30
    deploy:
      resources:
        limits:
          memory: 500M
    restart: unless-stopped

if you just open the port :443 this applies to the tcp-protocol only!

I would love to get some feedback from the maintainer of this awesome package and I am ofc open for discussion. :)

[thresheek]

Hi @the-hotmann!

nginx in current docker images works absolutely fine with http3 using openssl via compatibility layer shim nginx team developed.

I suppose something is not configured right in your scenario, so let me illustrate it with my test setup:

$ cat nginx.conf
user nginx;
worker_processes 1;
error_log /dev/stderr info;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    access_log /dev/stdout combined;

    server {
    listen 443 quic reuseport;
    listen 443 ssl;

    add_header Alt-Svc 'h3=":443"; ma=86400';

    ssl_certificate /etc/nginx/ssl/cert.pem;
    ssl_certificate_key /etc/nginx/ssl/key.pem;
    location / { return 200 'http3: $http3\n'; }
    }
}

$ ls -1 ssl/
cert.pem
key.pem

$ docker run -d -v $(pwd)/nginx.conf:/etc/nginx/nginx.conf -v $(pwd)/ssl/:/etc/nginx/ssl/ nginx:1.27.1-alpine
2012dd7d365555ddee7c0977d24327fa77160524cfa6a0fe3ade087f8d2236ef

$ docker inspect 2012dd7d365555ddee7c0977d24327fa77160524cfa6a0fe3ade087f8d2236ef | jq '.[].NetworkSettings.Networks.bridge.IPAddress'
"172.17.0.2"

$ docker run -ti --rm alpine/curl-http3:latest curl -v -k --http3 https://172.17.0.2:443/
*   Trying 172.17.0.2:443...
* Server certificate:
*  subject: C=XX; ST=StateName; L=CityName; O=CompanyName; OU=CompanySectionName; CN=CommonNameOrHostname
*  start date: Sep 30 16:58:10 2024 GMT
*  expire date: Sep 28 16:58:10 2034 GMT
*  issuer: C=XX; ST=StateName; L=CityName; O=CompanyName; OU=CompanySectionName; CN=CommonNameOrHostname
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Connected to 172.17.0.2 (172.17.0.2) port 443
* using HTTP/3
* [HTTP/3] [0] OPENED stream for https://172.17.0.2:443/
* [HTTP/3] [0] [:method: GET]
* [HTTP/3] [0] [:scheme: https]
* [HTTP/3] [0] [:authority: 172.17.0.2]
* [HTTP/3] [0] [:path: /]
* [HTTP/3] [0] [user-agent: curl/8.10.1-DEV]
* [HTTP/3] [0] [accept: */*]
> GET / HTTP/3
> Host: 172.17.0.2
> User-Agent: curl/8.10.1-DEV
> Accept: */*
>
* Request completely sent off
< HTTP/3 200
< server: nginx/1.27.1
< date: Mon, 30 Sep 2024 17:12:34 GMT
< content-type: text/plain
< content-length: 10
< alt-svc: h3=":443"; ma=86400
<
http3: h3
* Connection #0 to host 172.17.0.2 left intact

As you can see, nginx and curl talk http3 here.

[the-hotmann]

@thresheek thank you for your swift response. I ofc will test again and report back.

[the-hotmann]

This is my config:

server {

        # Dont show nginx version
        server_tokens           off                                                     ;

        # Listener for HTTP2 & HTTP3
        listen                  443 quic reuseport                                      ;
        listen                  443 ssl                                                 ;

        # HTTP2 & HTTP3 STUFF
        http2                   on                                                      ;
        http3                   on                                                      ;
        http3_hq                on                                                      ;
        quic_retry              on                                                      ;
        #quic_gso               on                                                      ;
        #ssl_early_data         on                                                      ;

        # Server & SSL Settings
        server_name             ${PUBLIC_HOST}                                          ;
        ssl_certificate         /etc/ssl/own/wildcard_domain.cert                       ;
        ssl_certificate_key     /etc/ssl/own/wildcard_domain.key                        ;
        ssl_protocols           TLSv1.2 TLSv1.3                                         ;
        ssl_ciphers             HIGH:!aNULL:!eNULL:!3DES:!ADH:!CAMELLIA:!DSS:!ECDSA:!EXP:!IDEA:!MD5:!PSK:!RC4:!RSA:!SEED:!SHA1;
        ssl_session_timeout     10m                                                     ;
        ssl_session_cache       shared:MozSSL:10m                                       ;
        ssl_dhparam             /etc/ssl/own/dhparam.pem                                ;
        keepalive_timeout       70                                                      ;

        # HEADERS
        proxy_hide_header       Strict-Transport-Security                               ;
        add_header              Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        # HTTP3 Headers
        add_header              QUIC-Status $http3                                      ;
        add_header              x-quic 'h3'                                             ;
        add_header              alt-svc 'h3=":$server_port"; ma=86400'                  ;

        # PROXY BUFFERS
        proxy_busy_buffers_size         512k    ;
        proxy_buffers                   4 512k  ;
        proxy_buffer_size               256k    ;

        fastcgi_buffer_size             128k    ;
        fastcgi_buffers                 4 256k  ;
        fastcgi_busy_buffers_size       256k    ;

        # HTML BASICS
        root /var/www/html;
        index index.html;

        location / { return 200 'http3: $http3\n'; }


}

But when I run:

docker run -it --rm alpine/curl-http3:latest curl -v -k --http3 https://domain.tld:443/

I get the following:

* Host domain.tld:443 was resolved.
* IPv6: (none)
* IPv4: ###IPv4###
*   Trying ###IPv4###:443...
*   Trying ###IPv4###:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: C=DE; ST=###CENSORED###; L=###CENSORED###; O=###CENSORED###; CN=*.###CENSORED###
*  start date: Jan 19 09:56:52 2024 GMT
*  expire date: Jan 23 23:59:59 2025 GMT
*  issuer: C=DE; O=Deutsche Telekom Security GmbH; CN=Telekom Security ServerID OV Class 2 CA
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* Connected to domain.tld (###IPv4###) port 443
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://domain.tld:443/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: domain.tld]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.10.1-DEV]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: domain.tld
> User-Agent: curl/8.10.1-DEV
> Accept: */*
>
* Request completely sent off
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200
< server: nginx
< date: Tue, 01 Oct 2024 14:01:50 GMT
< content-type: application/octet-stream
< content-length: 8
< strict-transport-security: max-age=31536000; includeSubDomains
< x-quic: h3
< alt-svc: h3=":443"; ma=86400
<
http3:
* Connection #0 to host domain.tld left intact

Notice this:

* ALPN: curl offers h2,http/1.1

why does curl (the very same container you ran not offer h3?

Thanks for looking into this! :)

P.S.:

• I run the alpine-slim Images (does this change anything?)
• I only get HTTP2
• my config is generated from /etc/nginx/templates and envsubst-ed to /etc/nginx/conf.d. I did not edit the /etc/nginx/nginx.conf directly, but used the nginx-container as advertised on docker-hub.

[thresheek]

You probably have some other configuration you didnt mention in this snippet. This works fine for me (I've only changed the SSL certificate locations):

user nginx;
worker_processes 1;
error_log /dev/stderr info;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    access_log /dev/stdout combined;

server {

        # Dont show nginx version
        server_tokens           off                                                     ;

        # Listener for HTTP2 & HTTP3
        listen                  443 quic reuseport                                      ;
        listen                  443 ssl                                                 ;

        # HTTP2 & HTTP3 STUFF
        http2                   on                                                      ;
        http3                   on                                                      ;
        http3_hq                on                                                      ;
        quic_retry              on                                                      ;
        #quic_gso               on                                                      ;
        #ssl_early_data         on                                                      ;

        # Server & SSL Settings
        server_name             test;
        ssl_certificate         /etc/nginx/ssl/cert.pem;
        ssl_certificate_key     /etc/nginx/ssl/key.pem;
        ssl_protocols           TLSv1.2 TLSv1.3                                         ;
        ssl_ciphers             HIGH:!aNULL:!eNULL:!3DES:!ADH:!CAMELLIA:!DSS:!ECDSA:!EXP:!IDEA:!MD5:!PSK:!RC4:!RSA:!SEED:!SHA1;
        ssl_session_timeout     10m                                                     ;
        ssl_session_cache       shared:MozSSL:10m                                       ;
        ssl_dhparam             /etc/nginx/ssl/dhparam.pem                                ;
        keepalive_timeout       70                                                      ;

        # HEADERS
        proxy_hide_header       Strict-Transport-Security                               ;
        add_header              Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        # HTTP3 Headers
        add_header              QUIC-Status $http3                                      ;
        add_header              x-quic 'h3'                                             ;
        add_header              alt-svc 'h3=":$server_port"; ma=86400'                  ;

        # PROXY BUFFERS
        proxy_busy_buffers_size         512k    ;
        proxy_buffers                   4 512k  ;
        proxy_buffer_size               256k    ;

        fastcgi_buffer_size             128k    ;
        fastcgi_buffers                 4 256k  ;
        fastcgi_busy_buffers_size       256k    ;

        # HTML BASICS
        root /var/www/html;
        index index.html;

        location / { return 200 'http3: $http3\n'; }


}
}

$ docker run -ti --rm alpine/curl-http3:latest curl -v -k --http3-only https://172.17.0.3:443/
*   Trying 172.17.0.3:443...
* Server certificate:
*  subject: C=XX; ST=StateName; L=CityName; O=CompanyName; OU=CompanySectionName; CN=CommonNameOrHostname
*  start date: Sep 30 16:58:10 2024 GMT
*  expire date: Sep 28 16:58:10 2034 GMT
*  issuer: C=XX; ST=StateName; L=CityName; O=CompanyName; OU=CompanySectionName; CN=CommonNameOrHostname
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Connected to 172.17.0.3 (172.17.0.3) port 443
* using HTTP/3
* [HTTP/3] [0] OPENED stream for https://172.17.0.3:443/
* [HTTP/3] [0] [:method: GET]
* [HTTP/3] [0] [:scheme: https]
* [HTTP/3] [0] [:authority: 172.17.0.3]
* [HTTP/3] [0] [:path: /]
* [HTTP/3] [0] [user-agent: curl/8.10.1-DEV]
* [HTTP/3] [0] [accept: */*]
> GET / HTTP/3
> Host: 172.17.0.3
> User-Agent: curl/8.10.1-DEV
> Accept: */*
>
* Request completely sent off
< HTTP/3 200
< server: nginx
< date: Tue, 01 Oct 2024 16:43:53 GMT
< content-type: text/plain
< content-length: 10
< strict-transport-security: max-age=31536000; includeSubDomains
< quic-status: h3
< x-quic: h3
< alt-svc: h3=":443"; ma=86400
<
http3: h3
* Connection #0 to host 172.17.0.3 left intact