proposal: x/crypto: deprecate unused, legacy and problematic packages

[FiloSottile]

x/crypto is meant to follow the same philosophy as the rest of the Go cryptography standard libraries: safe, useful subsets of widely used and robust primitives and protocols. Some packages in x/crypto don't clear that bar anymore, and I propose we freeze and deprecate (but not remove) them.

The intended outcomes are

• reducing the maintenance burden, redirecting resources towards more useful work
• avoiding confusion for users who consider inclusion in x/crypto to be an endorsement
• encouraging community maintained replacements where needed

All open and future issues and CLs that are not security relevant will be closed linking to this issue.

/cc @agl with whom we discussed this at RWC, and @golang/proposal-review for approval

Why

There are different reasons for deprecating different packages, and I give a short explanation for each of them below, but here's a summary.

• Some protocols simply don't have a common enough subset, with each user needing something different from it. The result is that our package is not useful as-is to anyone, and forks proliferate more than if we didn't have an x/crypto package at all.
• Some packages simply can't reach the bar of safe-by-default (anymore) because the underlying protocol or primitive is broken.
• Some packages require a maintainer that has deep knowledge of the many relevant specs and the state of the ecosystem. While it makes sense for the Go team to allocate the resources for, say, TLS and X.509, it doesn't for others.
• A few packages are simply not used enough to justify the maintenance effort, and their inclusion in x/crypto is confusing to users.

What packages

• blowfish/ — '90s cipher fundamentally broken due to its block size, its main use is as a bcrypt/ internal and it has no optimized implementation
• bn256/ — a broken elliptic curve that does not provide 128 bits of security anymore, with a community fork (github.com/cloudflare/bn256)
• cast5/ — another '90s cipher, only used in PGP
• md4/ — ancient broken hash function, only useful as target practice for learning how to generate hash collisions on a laptop (which is very fun!)
• openpgp/
  • the protocol predates modern cryptography engineering, which is still causing severe vulnerabilities, and any new project should rely on alternative modern solutions
  • the ecosystem never solved its major UX and key management issues
  • most importantly, this implementation is aging and incomplete because the specification is sprawling and there is no common subset to target; apparently everyone who's relying on it made their own fork to add features or fix issues, which is a much worse outcome than not having a golang.org/x/crypto package at all
• otr/ — a messaging security protocol from 2000's, mostly obsoleted by the Signal double ratchet and other messaging protocols like OMEMO and MLS, and a very rarely used package
• ripemd160/ — a hash function from the '90s only used by Bitcoin, with no optimized implementation
• tea/, xtea/ — ancient block ciphers, apparently used by one single package in the ecosystem
• twofish/ — AES finalist, not used in any popular protocol, with no advantages over AES
• xts/ — a block mode that should only ever be used for full disk encryption, which unsurprisingly no one is implementing in Go, and I honestly didn't even know it was in here

I was also hoping to kill pkcs12/ because it's a legacy file format that relies on primitives even worse than the ones listed above, and the package only does decoding while a better fork on GitHub also does encoding, but it has way too many users so that's only been frozen instead.

[davecgh]

I would prefer to see ripemd160 maintained as it is used by more cryptocurrencies than just Bitcoin. Honestly, Bitcoin is a pretty big deal on its own and would be enough reason to maintain it in my opinion.

Additionally, I performed a quick survey via godoc.org importers and it shows that the ripemd160 package is imported by at least 511 packages at the current time, including many widely known pieces of software such as Btcsuite, Discord, Decred, Ethereum, and Ripple.

[FiloSottile]

@davecgh I am not aware of any reasons to pick RIPEMD-160 for a new design, so the fact that cryptocurrencies inherited it from the Bitcoin paper is not a good enough reason not to deprecate it.

I don't expect it will require any maintenance anyway—it's a hash function, the package is almost certainly fine as is—but deprecating it will send the message to users not to look there if they just need a hash function, and communicates that we would for example not take an optimized assembly implementation for it.

[aksdb]

Just picking two examples to illustrate my opinion:
md4 is used in the NTLM protocol.
pkcs12 is used to store certificates in a lot of real world applications.

Both are dated, one is a PITA. Yet I think x/crypto should not be reduced to state of the art ciphers/protocols since we are not always developing self contained systems where we can safely choose the best available tech, we also have to connect to legacy systems or simply not-that-well-developed third-party systems.

Also I think that especially the pretty self contained and matured algorithms like MD4 don't suffer much from "being unmaintained". I mean: what would you maintain? The algorithm is stable (although "broken"). We cannot do anything about attacks on the algorithm (apart from refraining from future use).

I understand your point with OpenPGP. Though it might also be an option to try and get one of the forks to be reintegrated and actually maintain that.

If a living protocol (that still receives updates; has a moving spec) rots ... yeah, throw it out. The sophisticated implementations however, that are simply not "maintained" because there is nothing to maintain there ... I would like to keep them.

[nadimkobeissi]

• otr/ — a messaging security protocol from 2000's, mostly obsoleted by the Signal double ratchet and other messaging protocols like OMEMO and MLP, and a very rarely used package

The reasons given for obsoleting the otr package are perfectly sensible, but if it's not too much of a burden I would argue for nevertheless keeping this package: this is because in the XMPP world, the OTR protocol is still used as the standard secure messaging protocol and Signal has virtually no adoption there.

Aside from this argument I admittedly have nothing else with which to claim that keeping otr is relevant.

[tredoe]

• blowfish can not be compared with AES since the former has a fixed block size of 64 bits; the later, just like Serpent and Twofish, have block sizes of 128 bits.

• The fact of that a algorithm been used into a public or known project doesn't mean that it doesn't been used at private projects. At my case, I've used mainly Ripemd160 instead of both MD5 and SHA1 due to greater security, and I use it instead of SHA2 according to the requirements about security and perfomance that my project needs.

• Twofish and also Serpent have greater security than Rijndael, but the later was the AES winner due to greater performance. A lot of people not only chooses an algorithm due to performance else by greater security.

• The block mode XTS is a block mode very safe, only recommended for full-disk encryption. So, do you want to remove it because you didn't know about it, and the coders of open source are not using it, don't you?

  They are coders that build commercial and/or private projects.
  At my case, I'm using it like alternative to cryptsetup.

[jdevelop]

I'd rather keep openpgp (and would consider making it compatible with GnuPg2 ).

Still the problems there are quite surprising

Would be cool to have someone to take a look.

[charles-dyfis-net]

I'd similarly be very disappointed to see openpgp removed. Frankly, the implementation even as it exists is much higher-quality than the upstream GnuPG one, incomplete as it may be; removing it would thus be a loss to the community as a whole.

Admittedly, I'm one of those folks with a private fork, but I have approval from my employer's legal team to submit our changes upstream; if getting off my arse and doing so is the cost of keeping it in-tree (and thus being sure that the existing functionality continues to work with future versions of Go), I'm happy to do so.

[bradfitz]

@tredoe @charles-dyfis-net, this issue is about freezing+deprecating them, not removing them.

[dominikh]

@bradfitz To be fair, deprecating them has almost the same effect as removing them; people will look elsewhere for alternatives that aren't marked as deprecated. Which I imagine is the intended effect. Unfortunately, the alternatives may not always be of higher quality. For example, I'd much rather see people use x/crypto/md4 over some other, random md4 implementation.

I don't think the "these algorithms are broken" holds as an argument. A lot of legacy still relies on them, unfortunately. As such, we should rather look at the cost of not deprecating a package. One can of course argue that another well-organized party that isn't Go should maintain legacy packages.

Have we explored why these packages made it into x/crypto in the first place?

[bradfitz]

To be fair, deprecating them has almost the same effect as removing them

I disagree. The effects are quite different: removing them breaks users, and deprecating them does not. Deprecating them is just adding a comment saying, "Yo, new applications shouldn't use this, and also we're not adding new features so don't send PRs."

people will look elsewhere for alternatives that aren't marked as deprecated. Which I imagine is the intended effect.

Sounds like the intended effect is reducing maintenance cost by not accepting changes to add features or optimize the performance.

[aksdb]

I think the "deprecated" marker would move the same thinking you have here down the line. Your argument for deprecation (or removal) is that the code is old and the algorithms should not be used. While the code is actually most likely still fine and just doesn't need any maintenance (if it does, someone will report it and or create a PR).
If it's marked "deprecated" it will lead to managers/decision makers/etc. thinking "oh, that code is not maintained, we cannot use it. go find something else".
I had discussions like that myself. A library for a microcontroller we were working with was over 7 years old and did not have a single change. It took a considerate amount of time to convince my superior that this did not matter since a) the code was well documented b) had good test coverage c) the microcontroller in question was that old as well - there was simply nothing that could have changed in the meantime.
The same holds for the algorithms in question. MD4, cast5, blowfish, twofish, etc. will not change. They may not be a good choice for cryptographic strength ... but they may still be the right choice to implement a specific interface. And then it's better to have a central source that does not imply "we will get rid of this soon" instead of a bunch of third party sources.

[dominikh]

If it's marked "deprecated" it will lead to managers/decision makers/etc. thinking "oh, that code is not maintained, we cannot use it. go find something else".

Case in point: dominikh/go-tools#413

Everywhere in the standard library, deprecated means "stop using this, move to an alternative".