DietPi-LetsEncrypt | IPv6 support on Lighttpd with HTTPS

[vwillcox]

Creating a bug report/issue:

Required Information:

• DietPi version | 6.9
• Distro version | stretch
• Kernel version | 4.14.34-v7+
• SBC device | RPi 2 Model B (armv7l)
• Power supply used | 5V 2.5A
• SDcard used | SanDisk

Additional Information (if applicable):

• Software title | LetsEncrypt/CertBot
• Was the software title installed freshly or updated/migrated? Freshly installed
• Can this issue be replicated on a fresh installation of DietPi? yes
• dietpi-bugreport ID | 7014414c-a92e-4297-a899-19fac8f9bc0a

Steps to reproduce:

1. Install Dietpi-LetsEncrypt & Wordpress (LightHTTPD)
2. run dietpi-LetsEncrypt
3. Fails to verify

Expected behaviour:

• Cert verified and installed

Actual behaviour:

• Certbot fails to verify the domain

Extra details:

• I am testing IPv6 DSLight for my ISP. This means we do not have our own IPv4 address and our router/modem gives out IPv6 to each device.
  Connecting to my Raspberry pi over SSH with either IPv4 (internal only address) or IPv6 (external address) does not validate the certificate from certbot.

Failed authorization procedure. nas.validdomain.tld (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://nas.talktech.info/.well-known/acme-challenge/qiYxllvcQV5mNOpP5NAMVDX2BdR-SECIgIVweSIuFX4: Error getting validation data

[MichaIng]

@vwillcox
Thank you very much for your report.

Indeed DietPi is known to not handle IPv6 very greatly/reliable. @Fourdee does not have IPv6 at home do run needed test and I would need to set up a non-production device for testing. Hmm, have unused NanoPi K1 here, maybe I just found a job for it, as VirtualBox VMs seem to not work at all with IPv6 here.

But let's see how exactly the network/interfaces are configures on your system:

ip l
ip r
ip a
cat /etc/network/interfaces

The RPi seems to have network access, so this is already great.
You say SSH via IPv4 and IPv6 internally works, or what do you mean by does not validate the certificate from certbot? SSH does not use the LetsEncrypt certificates but it's own host keys, in case user authentication key. If access via IPv4 works, this means your router also distributes IPv4 addresses internally besides the IPv6 suffixes. But externally of course it is only available via IPv6.

Ahh hmm, I think I found the issue. Ref: https://gist.github.com/cecilemuller/a26737699a7e70a7093d4dc115915de8
Our default nginx configuration does not contain a IPv6 related listening line: https://github.com/Fourdee/DietPi/blob/master/dietpi/conf/nginx.site-available-default#L6
The same could be for the default Lighttpd config.
Further reading: https://www.cyberciti.biz/tips/linux-unix-lighttpd-ipv6-support.html
Could you try to add server.use-ipv6 = "enable" to your /etc/lighttpd/lighttpd.conf?

G_CONFIG_INJECT 'server\.use-ipv6' 'server\.use-ipv6 \= \"enable\"' /etc/lighttpd/lighttpd.conf 'server.groupname'
systemctl restart lighttpd

Then rerun certbot.

[vwillcox]

root@DietPi:/var/log# ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether b8:27:eb:71:64:3b brd ff:ff:ff:ff:ff:ff
root@DietPi:/var/log# ip r
default via 192.168.0.1 dev eth0
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.23
root@DietPi:/var/log# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether b8:27:eb:71:64:3b brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.23/24 brd 192.168.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 ::ba27:ebff:fe71:643b/64 scope global mngtmpaddr dynamic
       valid_lft forever preferred_lft forever
    inet6 2a02:88d4:4800:100:ba27:ebff:fe71:643b/64 scope global mngtmpaddr dynamic
       valid_lft 1209599sec preferred_lft 604799sec
    inet6 fe80::ba27:ebff:fe71:643b/64 scope link
       valid_lft forever preferred_lft forever
root@DietPi:/var/log# cat /etc/network/interfaces

injected the settings:

server.modules = (
        "mod_access",
        "mod_alias",
        "mod_compress",
        "mod_redirect",
)

server.document-root        = "/var/www"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.use-ipv6 = "enable"
server.port                 = 80

index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )

# default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

however:

root@DietPi:~# systemctl restart lighttpd
root@DietPi:~# systemctl status lighttpd
● lighttpd.service - Lighttpd Daemon
   Loaded: loaded (/lib/systemd/system/lighttpd.service; disabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2018-06-13 18:30:04 BST; 3s ago
  Process: 27671 ExecStart=/usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf (code=exited, status=255)
  Process: 27663 ExecStartPre=/usr/sbin/lighttpd -tt -f /etc/lighttpd/lighttpd.conf (code=exited, status=0/SUCCESS)
 Main PID: 27671 (code=exited, status=255)

Jun 13 18:30:04 DietPi systemd[1]: lighttpd.service: Failed with result 'exit-code'.
Jun 13 18:30:04 DietPi systemd[1]: lighttpd.service: Service hold-off time over, scheduling restart.
Jun 13 18:30:04 DietPi systemd[1]: Stopped Lighttpd Daemon.
Jun 13 18:30:04 DietPi systemd[1]: lighttpd.service: Start request repeated too quickly.
Jun 13 18:30:04 DietPi systemd[1]: Failed to start Lighttpd Daemon.
Jun 13 18:30:04 DietPi systemd[1]: lighttpd.service: Unit entered failed state.
Jun 13 18:30:04 DietPi systemd[1]: lighttpd.service: Failed with result 'exit-code'.

[vwillcox]

Without the IPV6 line it is up and running and responds on IPv6 (http://nas.talktech.info) should take you to it

[MichaIng]

Without the IPV6 line it is up and running

Hmm, okay this seems to be deprecated then.

http://nas.talktech.info => 403 - Forbidden
But okay this means, the server is answering, AFAIK.

---

But I was also mistaken a bid:

• As with Lighttpd, our LetsEncrypt implementation uses the CertBot internal standalone webserver for authentication, thus this has nothing to do with Lighttpd at all.
• I just found a discussion on their GitHub repo and related PR that added IPv6 support: Enable IPv6 support in standalone plugin certbot/certbot#4773
• This is available from version 0.15.0, but Debian Stretch is on 0.10.2: https://packages.debian.org/stretch/certbot

@vwillcox
You could try to install the newer CerBot from stretch-backports repo: https://packages.debian.org/stretch-backports/certbot
apt install -t stretch-backports certbot
Then retry.

To allow APT showing updates for this version, you would need some pin priority entries:

cat << _EOF_ > /etc/apt/preferences.d/certbot
Package: certbot python3-certbot python3-acme python3-josepy python3-parsedatetime
Pin: release a=stretch-backports
Pin-Priority: 990
_EOF_

[vwillcox]

Hi, that has worked! I have a certificate. However now going to https://nas.talktech.info is not working.
I have turned the firewall to allow incoming connections on port 443 but nothing working.

root@DietPi:~# dietpi-letsencrypt
[  OK  ] Root access verified.
[  OK  ] DietPi-Drive_Manager | RootFS R/W access verified.

 DietPi-LetsEncrypt
─────────────────────────────────────────────────────
 Mode: Running CertBot

[  OK  ] DietPi-LetsEncrypt | Lighttpd webserver detected
[ SUB1 ] DietPi-Services > stop
[  OK  ] DietPi-Services | stop : cron
[  OK  ] DietPi-Services | stop : pijuice
[  OK  ] DietPi-Services | stop : lighttpd
[  OK  ] DietPi-Services | stop : php7.0-fpm
[  OK  ] DietPi-Services | stop : mysql
[  OK  ] DietPi-Services | stop : smbd
[  OK  ] DietPi-Services | stop : nmbd
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for nas.talktech.info
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/nas.talktech.info/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/nas.talktech.info/privkey.pem
   Your cert will expire on 2018-09-12. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

[  OK  ] DietPi-LetsEncrypt | Added setting "mod_setenv", to /etc/lighttpd/lighttpd.conf after line     "mod_access"                        ,
Enabling dietpi-hsts: ok
Run "service lighttpd force-reload" to enable changes

Press any key to continue...

[ SUB1 ] DietPi-Services > restart
[  OK  ] DietPi-Services | restart : nmbd
[  OK  ] DietPi-Services | restart : smbd
[  OK  ] DietPi-Services | restart : mysql
[  OK  ] DietPi-Services | restart : php7.0-fpm
[  OK  ] DietPi-Services | restart : lighttpd
[  OK  ] DietPi-Services | restart : pijuice
[  OK  ] DietPi-Services | restart : cron
[ SUB2 ] DietPi-Process_tool > Apply
[  OK  ] DietPi-Process_tool | Completed

[vwillcox]

server.modules = (
        "mod_access",
        "mod_setenv",
        "mod_alias",
        "mod_compress",
        "mod_redirect",
)

server.document-root        = "/var/www"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80

index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny             = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir          = "/var/cache/lighttpd/compress/"
compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )

# default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

[MichaIng]

@vwillcox
Okay I made a bid research about Lighttpd and IPv6.

I also failed to use the first suggested setting. Always get error that port 80 is already in use. Other guides suggest to add the following to enable IPv6, which worked:
$SERVER["socket"] == "[::]:80" { }

This syntax is used to add vhost settings within the braces. It seems $SERVER["socket"] == ":80" { } is just valid for IPv4, which is initiated by default, while the above just covers IPv6, which needs to be defined, and both cannot be combined via wildcard or something.
See also $SERVER["socket"] == ":443" { within our /etc/lighttpd/conf-enabled/letsencrypt.conf, which is the IPv4-only vhost for HTTPS.

So to enable IPv6 for HTTPS as well, you need to define $SERVER["socket"] == "[::]:443" { ... and add the same SSL settings there. To reduce config code, you can use the include statement to include identical SSL settings within separate .conf file to both vhosts. See: https://serverfault.com/a/670669

As IPv6 is used more and more, I think we need to rework our webserver configs to allow both, or add/remove related settings when IPv6 is enabled/disabled via dietpi-config. Same for dietpi-letsencrypt.

I hope the above allows you to configure your webserver accordingly, otherwise ask and I can provide exact steps to apply.

[vwillcox]

Thank you very much @MichaIng all working now! https://nas.talktech.info.
I just changed the socket section to IPv6 as I have no need to run this over IPv4 at the moment

[MichaIng]

@vwillcox
Great to hear. I keep the issue open, as I want to implement IPv6 support to Lighttpd with and without SSL.

So am I right, that you actually just needed to change the letsencrypt.conf from
$SERVER["socket"] == ":443" { to $SERVER["socket"] == "[::]:443" {, where non-SSL IPv6 access worked from the beginning?

[vwillcox]

@MichaIng that is correct. Working fine now and had lots of reports that others can now get to my testing site with https and the certificate is working!
Thank you

[MichaIng]

Addressed with: #4220

It's added statically, which works still fine when IPv6 is disabled via dietpi.txt/dietpi-config/sysctl. But Lighttpd would fail if one manually blacklists the IPv6 kernel module or re-compiles the kernel with IPv6 disabled. Since that causes other issues and is not recommended, I think it's fine with Lighttpd, but we can switch to a dynamic way when receiving related reports.