Apache + Nginx requires key path to exist

[ZacharyEspiritu]

Apache and Nginx do some preprocessing of the key path specified in the configuration to ensure that the key path is a real file. The key loader function will most likely need to read from a real file as a result.

As stated in the original design document:

Some web servers, like Apache, may do preprocessing of the private key path to verify that the key exists before passing it to OpenSSL. If this is the case, we’ll move to an alternative approach where we represent private keys as a real file which only contains a Cloud KMS key resource ID. This file would be in protobuf format for extensibility purposes.

It is unclear if Apache requires the key file to be PEM-encoded, though some of the error messages I got when testing strongly suggest that it does do some PEM-processing of the key. If this is the case, this would require much more additional legwork to figure out an easy way for users to generate PEM-encoded keys for Apache.

I was not able to determine if Nginx also requires the key file to be PEM-encoded.

Despite this limitation, Luna HSM appears to still use the “interpret the input file path as the actual key resource ID” approach and seems to have some work around for it in Apache (though it’s not immediately clear how it works). See https://www.youtube.com/watch?v=ur4JyzP1FBc for Luna’s guide to integrating the Luna HSM engine with Apache.

[ZacharyEspiritu]

To set up a Debian machine with Apache (for some reason Debian instructions are completely different than other Linux distributions for Apache):

1. Follow the PR instructions from Add OpenSSL engine installation guide to README.md #120 to set up the OpenSSL engine. Install Apache.

2. Make a backup of /etc/apache2/sites-available/default-ssl.conf.

cp /etc/apache2/sites-available/default-ssl.conf /etc/apache2/sites-available/default-ssl.conf.backup

3. At the top of /etc/apache2/sites-available/default-ssl.conf, add SSLCryptoDevice gcloudkms in between the IfModule tag and the VirtualHost tag:

<IfModule mod_ssl.c>
        SSLCryptoDevice gcloudkms
        <VirtualHost _default_:443>
...

4. Change SSLCertificateFile and SSLCertificateKeyFile to point to your desired server certificate / private key. As mentioned in this issue, SSLCertificateKeyFile most needs to be a real file, and can't just be a key resource ID.

5. Enable the server using sudo a2ensite default-ssl.

6. Check that Apache can load the configuration successfully by using sudo apache2ctl configtest.

[ZacharyEspiritu]

For some reason, the initial error message I was getting about a week ago was that it couldn't find the key file (as was expected, since the file didn't exist). Now, the error I get is:

$ sudo apache2ctl configtest
AH00526: Syntax error on line 3 of /etc/apache2/sites-enabled/default-ssl.conf:
SSLCryptoDevice: Invalid argument; must be one of: 'builtin' (none), 'rdrand' (Intel RDRAND engine), 'dynamic' (Dynamic engine loading support)
Action 'configtest' failed.
The Apache error log may have more information.

The reason why the error might have changed is that perhaps initially I forgot to include the SSLCryptoDevice line (though I unfortunately can't remember if that's the case).

Interestingly, the documentation for SSLCryptoDevice says:

To discover which engine names are supported, run the command "openssl engine".

However, running openssl engine using the setup from PR #120 shows that gcloudkms is there:

$ openssl engine -v
(rdrand) Intel RDRAND engine
(dynamic) Dynamic engine loading support
     SO_PATH, NO_VCHECK, ID, LIST_ADD, DIR_LOAD, DIR_ADD, LOAD
(gcloudkms) Google Cloud KMS Engine

So maybe this is a bug with Apache; not immediately sure.

I found a proposed workaround at https://serverfault.com/a/786372, but I unfortunately did not have time to try it out.