Switch to keys.openpgp.org as default keyserver

[eloquence]

The keyserver ecosystem is extremely flaky and poorly maintained. It routinely causes problems during setup and updates as --recv-key operations time out unpredictably (most recently #4100). The keyserver ecosystem is also vulnerable to cert flooding attacks.

Per the discussion below, we want to:

• Switch to keys.openpgp.org as a well-maintained alternative to other keyserver implementations
• Support WKD as a fallback key distribution method in future (can be tracked in separate issues)

2019-07-01: Updated per discussions below - @eloquence

[emkll]

I think this makes sense, and is already widely used by various project to host the keys used to sign apt packages.

There are two docs-level recommendations we should make:

1 - Since we are specifying a URL to an arbitrary file and not a fingerprint to import, we must guard against unwanted key changes, either due to error or the file being changed on server or in transit. We must validate the fingerprint prior to importing it to the gpg keyring (either programmatically for updates, or manually for first install):

gpg {--with-fingerprint, --with-colons} key.asc

2 - Any user that wants to validate signatures/web of trust would need to gpg --refresh-key

[eloquence]

@emkll Thanks, that makes sense.

gpg {--with-fingerprint, --with-colons} key.asc

What's the purpose of --with-colons here? It seems that the output without that option is easier to read when the verification step is performed by a human.

2 - Any user that wants to validate signatures/web of trust would need to gpg --refresh-key

Do you see this as necessary to document? If so, who's the target audience and how would they act on this information?

[KwadroNaut]

1. I agree with @emkll here. The --with-colons is great for copy-pasting and scripts. Relying on human verification is a pretty bad idea since it's not too hard to create keys where the first and last couple of characters are the same. We, humans, tend to ignore those in the middle.

2. I'd suggest that admins do this, every time they do a manual upgrade or during troubelshooting.

Something else to investigate is if it makes sense to postpone this 'annoyance' until clients support https://tools.ietf.org/html/draft-mccain-keylist-04

[eloquence]

WKD seems worth considering as a non-keyserver method for publishing the release key which is supported within recent versions of gpg as one of the --auto-key-locate methods.

https://wiki.gnupg.org/WKD

See draft spec: https://www.ietf.org/id/draft-koch-openpgp-webkey-service-08.txt

Being considered in gpgsync as well: firstlookmedia/gpgsync#157

[redshiftzero]

note that the SecureDrop release signing key does not currently have an email associated with it (seems like WKD and --auto-key-locate both are aimed at retrieving keys by email) though we could add one

[wiktor-k]

WKD seems worth considering as a non-keyserver method for publishing the release key which is supported within recent versions of gpg as one of the --auto-key-locate methods.

Yes, I did some research on replacing keyservers with WKD and it seems GnuPG supports it:

• gpg --locate-key $EMAIL by default uses WKD in modern GnuPG,
• signatures with Signers UID packet will fetch keys over WKD when --auto-key-retrieve is used (something like curl -s https://metacode.biz/.well-known/security.txt | gpg --auto-key-retrieve --verify --output -).
• expired keys that were fetched over WKD will be re-fetched over WKD automatically. This can be used to rotate keys by providing revoked and new keys in one WKD key file. See more details here: https://lists.gnupg.org/pipermail/gnupg-users/2019-January/061442.html

[eloquence]

Sadly given the time available (and reduced infra team availability, which we'd need for a WKD implementation) this won't make it for 0.14.0, although keyservers still remain a common pain point during installation and maintenance tasks (even commands like securedrop-admin logs use keyserver lookups, which fail frequently), so pushed on to 0.15.0.

[wiktor-k]

@emkll:

1 - Since we are specifying a URL to an arbitrary file and not a fingerprint to import, we must guard against unwanted key changes, either due to error or the file being changed on server or in transit. We must validate the fingerprint prior to importing it to the gpg keyring (either programmatically for updates, or manually for first install):

I'd do it the other way: gpg --auto-key-retrieve --verify file.sig and then ask the user to check for "Good signature" and compare the last line of output (primary key fingerprint):

$ gpg --auto-key-retrieve --verify gnupg-2.2.16.tar.bz2.sig
gpg: assuming signed data in 'gnupg-2.2.16.tar.bz2'
gpg: Signature made wto, 28 maj 2019, 17:26:17 CEST
gpg:                using RSA key D8692123C4065DEA5E0F3AB5249B39D24F25E3B6
gpg: Good signature from "Werner Koch (dist sig)" [marginal]
...
gpg: WARNING: This key is not certified with sufficiently trusted signatures!
gpg:          It is not certain that the signature belongs to the owner.
Primary key fingerprint: 031E C253 6E58 0D8E A286  A9F2 2071 B08A 33BD 3F06

If you check the key during import and not during verification then the file could be correctly signed with other key in user's keyring. With auto-key-retrieve it is only one command so it's simpler.

2 - Any user that wants to validate signatures/web of trust would need to gpg --refresh-key
Do you see this as necessary to document? If so, who's the target audience and how would they act on this information?

This is important to detect keys revoked before they're expired. Or, if keyholder is not using expiry at all, to detect any other changes.

@eloquence:

and reduced infra team availability, which we'd need for a WKD implementation

Do you mean infra team to manage .well-known directory on the server? As far as I understood the initial post the implementation effort wouldn't be big, especially if you don't change the signing key often. (For example it could be directly embedded in the application). I'm asking about details because maybe I could help somehow with implementation (if I understood it correctly).

[KwadroNaut]

Since the milestone moved, it might be worthwhile to check if https://keys.openpgp.org/about/news#2019-06-12-launch is worthwhile to use as a stop-gap measure until some moment when the infra team has a bit more time.

[eloquence]

@KwadroNaut That's a fantastic suggestion, thanks. This does seem like a very well-designed alternative to the previously existing keyservers:

• Emails must be verified, if you don't verify them, only the key material -- none of the identity info -- is associated with the key.

• Doesn't support third party signatures, to mitigate against spam

• (So far) very zippy, includes .onion service and API.

I've tested it, and the whole user experience was quite clean and straightforward. Key management via one-time links also seems to work well.

[eloquence]

The urgency of this has gone up for us due to recent cert flooding of existing keyservers.

WKD still remains an appealing option. Tails includes the Debian Stretch version of gpg, 2.1.18, which supports WKD as a lookup method, but does not yet include the fix for updating expired keys. Until that fix lands in Tails (and a while after that, to give folks time to update), we'd have to force a lookup each time using the --auto-key-locate clear,wkd,nodefault options.

(What those parameters mean: clear -- clear previous preferences; wkd -- use WKD, nodefault -- don't check the local keyring.)

Local results should not be overwritten if the key is unchanged. Example output for a key that exists in the local keyring:

$ gpg --auto-key-locate clear,wkd,nodefault --locate-key some@address.example
gpg: key <fingerprint>: "some@address.example" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
gpg: automatically retrieved 'some@address.example' via WKD
<key info>

[eloquence]

Testing has also shown keys.openpgp.org to work well, and be a good drop-in alternative to the currently configured keyservers. The plan of record is to support both keys.openpgp.org and WKD, so we're not too reliant on centralized infrastructure, but to prioritize transitioning to keys.openpgp.org as the default keyserver, since it's an easy drop-in replacement.

@redshiftzero broke this into the following steps:

• Get PR up with securedrop-keyring diff,
• PR gets reviewed/approved,
• backport to 0.14.0,
• publish key to keys.openpgp.org,
• modify updater logic to use keys.openpgp.org only and backport,
• 0.14.0-rc2 is cut, new keyring package is built along with other rc2 debs and committed to lfs repo

In addition, we'll want to:

• update the docs in relevant places
• update the securedrop-admin logs playbook to use keys.openpgp.org for getting the comms key

[redshiftzero]

In the version of the update logic in 0.14.0, when we switch to using keys.openpgp.org, we need to:

• remove this try/except and hard code the keys.openpgp.org keyserver.
• modify this logic to expect either Good signature from "SecureDrop Release Signing Key" or Good signature from "SecureDrop Release Signing Key <securedrop-release-key@freedom.press>". This is because the key as uploaded to keys.openpgp.org only contains the parts of the key that identifies the owner (i.e. "SecureDrop Release Signing Key <securedrop-release-key@freedom.press>"), but if the old release key is available in an admin workstation, we'll expect to see the first.

[redshiftzero]

Done in #4578, #4586 and #4585, closing in favor of more specific followups, e.g. #4584