Allow the keystore to be password protected

[joshbressers]

Now that we have a keystore we need to add the ability to encrypt it with a secret. Ideally when a node starts up a secret must be passed in somehow.

Some questions we need to answer

• What does this mean for reloading the keystore
• How do we want to pass in the secret securely
• What other secret storage technologies should we support (like vault)
• What happens to node startup if the secret fails to unlock the keystore

[elasticmachine]

Pinging @elastic/es-security

[elasticmachine]

Pinging @elastic/es-core-infra

[albertzaharovits]

I can answer the easy one:

What does this mean for reloading the keystore

As it stands the reload-secure-store API has a request body param that is used to convey the password, or the vault token, for that matter, to some node which will then broadcast it to the full cluster, during reloading. The password/token is never stored, and it is not obfuscated at any step.

[jasontedor]

A few thoughts from a discussion in the @elastic/es-core-infra team meeting:

What does this mean for reloading the keystore

We should require HTTPS to be able to specify a password in the reload settings API. How exactly we do this is open to discussion, but we think that we should remove the ability to specify a password on the current reload settings API, and add a new API that allows specifying a password, and this API will require HTTPS. The idea here is that we want to avoid plaintext transmission of passwords.

How do we want to pass in the secret securely

With systemd there are built-in facilities for this that we can leverage, but that doesn't solve this for us on Sys V init, and the archive packages (e.g., on Windows). We can read from standard input or on an IPC socket (now that Windows is adding support for AF_UNIX), or via a named pipe. We lean towards avoiding environment variables.

What other secret storage technologies should we support (like vault)

We see this question as completely orthogonal to adding password support to the keystore.

What happens to node startup if the secret fails to unlock the keystore

We should fail the node.