Run fails on prompt during certonly if certs are not yet due for renewal

[kwisatz]

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 3.8.7
• Ruby: 2.1.5
• Distribution: Debian 8.7
• Module version: 1.0.0

How to reproduce (e.g Puppet code you use)

We changed the certonly plugin from standalone to webroot and the module seems to be trying to regenerate the certs during the puppet run.

However, since the certificates already exist, letsencrypt-auto is prompting for user input, upon which the puppet run fails.

Running the script manually and selecting option 1 (keep existing certificates) did not improve the situation. How does the puppet module know whether to generate certificates or only run the renew cron job but not the certonly script?

What are you seeing

[…]
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: Creating virtual environment...
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: Updating letsencrypt and virtual environment dependencies.......
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: Running with virtualenv: ~/.local/share/letsencrypt/bin/letsencrypt --agree-tos certonly -a webroot --webroot-path /var/tmp/letsencrypt -d domain.tld --webroot-path /var/tmp/letsencrypt -d my.domain.tld --webroot-path /var/tmp/letsencrypt -d cms.domain.tld --webroot-path /var/tmp/letsencrypt -d www.domain.tld
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: Saving debug log to /var/log/letsencrypt/letsencrypt.log
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: You are running with an old copy of letsencrypt-auto that does not receive updates, and is less reliable than more recent versions. We recommend upgrading to the latest certbot-auto script, or using native OS packages.
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: Cert not yet due for renewal
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: 
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: (ref: /etc/letsencrypt/renewal/my.domain.tld-0002.conf)
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: 
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: What would you like to do?
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: -------------------------------------------------------------------------------
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: 1: Keep the existing certificate for now
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: 2: Renew & replace the cert (limit ~5 per 7 days)
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: -------------------------------------------------------------------------------
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: Select the appropriate number [1-2] then [enter] (press 'c' to cancel): An unexpected error occurred:
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: EOFError: EOF when reading a line
Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: Please see the logfiles in /var/log/letsencrypt for more details.
Error: /opt/letsencrypt/letsencrypt-auto --agree-tos certonly -a webroot --webroot-path /var/tmp/letsencrypt -d domain.tld --webroot-path /var/tmp/letsencrypt -d my.domain.tld --webroot-path /var/tmp/letsencrypt -d cms.domain.tld --webroot-path /var/tmp/letsencrypt -d www.domain.tld returned 1 instead of one of [0]
Error: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[my.domain.tld]/Exec[letsencrypt certonly my.domain.tld]/returns: change from notrun to 0 failed: /opt/letsencrypt/letsencrypt-auto --agree-tos certonly -a webroot --webroot-path /var/tmp/letsencrypt -d domain.tld --webroot-path /var/tmp/letsencrypt -d my.domain.tld --webroot-path /var/tmp/letsencrypt -d cms.domain.tld --webroot-path /var/tmp/letsencrypt -d www.domain.tld returned 1 instead of one of [0]
Notice: /Stage[main]/Profile::Letsencrypt/Module::Letsencrypt::Dhparam[my.domain.tld]/Dhparam[/etc/letsencrypt/live/domain.tld/dhparam.pem]: Dependency Exec[letsencrypt certonly my.domain.tld] has failures: true
Warning: /Stage[main]/Profile::Letsencrypt/Module::Letsencrypt::Dhparam[my.domain.tld]/Dhparam[/etc/letsencrypt/live/domain.tld/dhparam.pem]: Skipping because of failed dependencies

What behaviour did you expect instead

letsencrypt-auto certonly should not be run or should interact with the prompt.

Output log

See above

[kwisatz]

There seems to be a specific problem with the config on the host above. On another host were we applied the same changes (from standalone to webroot), the puppet run works fine and it is not attempting to launch letsencrypt-auto:

Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[icinga.mon.my.domain.tld]/Cron[letsencrypt renew cron icinga.mon.my.domain.tld]/ensure: created

What might be specific about the other host is that at some point – IIRC when we added another domain as a SAN – letsencrypt decided to create new configuration files:

root@loadbalancer:~# ls -l /etc/letsencrypt/renewal/
total 12
-rwxr-xr-x 1 root root 516 Nov 11 12:56 api.my.domain.tld-0001.conf
-rw-r--r-- 1 root root 864 Jan 30 10:49 api.my.domain.tld-0002.conf
-rwxr-xr-x 1 root root 496 Nov  6 22:46 api.my.domain.tld.conf

I don't know what caused it to do so and since this is a production site, I'm not really inclined to too much experimentation.
However, a renewed attempt to do a puppet run resulted in it trying to bootstrap the entire thing again, starting from installing dependencies, etc (although valid certs had been generated in November):

Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[api.my.domain.tld]/Exec[letsencrypt certonly api.my.domain.tld]/returns: Bootstrapping dependencies for Debian-based OSes...

OK, that last thing seems to be a bug in the letsencrypt-auto script that causes it to not interpret ~ as $HOME but literally.

[kwisatz]

And that seems to be at least part of the problem.
Now that I first moved to /root and executed puppet agent -t from there, all is fine:

Notice: /Stage[main]/Profile::Letsencrypt/Letsencrypt::Certonly[api.my.domain.tld]/Exec[letsencrypt certonly api.my.domain.tld]/returns: executed successfully
Notice: /Stage[main]/Profile::Letsencrypt/Module::Letsencrypt::Dhparam[api.my.domain.tld]/Dhparam[/etc/letsencrypt/live/my.domain.tld/dhparam.pem]/ensure: created
Notice: Finished catalog run in 47.39 seconds

Another point is that it suddenly and for no obvious reason to me decided to use another domain from the array of domains passed to certonly as the first domain.