crypto/x509: unexpected name mismatch error

[sneis]

What version of Go are you using (go version)?

Sorry, I'm not actually using Go itself, but I'm using etcd (https://github.com/etcd-io/etcd; apparently written in go) and got an error about subject and issuer names not matching in my certificate chain, which I believe can be tracked down to go's crypto/x509 implementation.

What did you do?

I tried to verify a certificate chain, where the Issuer DN of sub-CA certificate was specified using the ASN.1 type UTF8String, while the subject DN of the (renewed) CA certificate used PRINTABLESTRING.

What did you expect to see?

RFC 5280 (dated May 2008), section 7.1 says:
RFC 3280 required only binary comparison of attribute values encoded in UTF8String, however, this specification requires a more comprehensive handling of comparison.
And then it goes on to give details on how to compare DN's and then refers to RFC 4518 on how to compare strings of different ASN.1 types. To me, this sounds like at least comparing the exact same ASCII string when encoded as PRINTABLESTRING in one instance and as UTF8STRING in the other still should result in recognizing the two string as equal and thus the certificate chain should be validated successfully (as is the case e.g. for OpenSSL).

What did you see instead?

I got a name mismatch, just as if the ASN.1 encoding of the DN's would be "blindly" compared byte by byte as described by the outdated RFC 3280 (crypto/x509/verify.go:570 seems to indicate that this actually is the case). Of course, for now, there's the obvious workaround of ensuring the ASN.1 encodings of DN's are identical, but in the long run, that shouldn't be necessary (and if you're switching to a different software (version) for generating certificates, ensuring continuing compatibility might not be trivial).

[bcmills]

CC @FiloSottile @agl

[bcmills]

Could you give some concrete examples of certificate chains that exhibit this problem? (Are the certificates in question published publicly?)

[sneis]

I did a quick attempt at creating a set of suitable dummy certfificates (Basically create a dummy Root with openSSL, create a certificate signed by that root, change the string type defined in openssl.cnf from "utf8only" to "default", refresh the root certificate with one with longer validity). If you unzip the attached certs.zip, using OpenSSl, both
openssl verify -verbose -CAfile cacert.pem subcacert.pem
and
openssl verify -verbose -CAfile cacert-new.pem subcacert.pem
successfully verify the certificate in subcacert.pem, while I claim that with the routines from crypto/x509 subcacert.pem can only be validated against cacert.pem while validation against cacert-new.pem is going to fail.
certs.zip

[FiloSottile]

@sleevi Can you help me with some context on how often non-exact comparisons are necessary in chain building? I do believe we are off-spec here, but it's also an extremely sensitive area, and I am uncomfortable relaxing comparison rules. In Go we strive to keep complexity under control by implementing only the important subsets of the specs. Is this something we can get away without?

[sleevi]

@FiloSottile You're totally on-spec. Just the old spec ;)

RFC 2459 allows for byte-for-byte comparison of basically everything but PrintableString, and doesn't require implementations to collapse the string types.

RFC 3280 continues this - implementations MAY assume different strings types are different strings.

While RFC 5280 does change this up, I do want to highlight an important part of the profile - namely:

   When encoding attribute values of type DirectoryString, conforming
   CAs MUST use PrintableString or UTF8String encoding, with the
   following exceptions:

      (a)  When the subject of the certificate is a CA, the subject
           field MUST be encoded in the same way as it is encoded in the
           issuer field (Section 4.1.2.4) in all certificates issued by
           the subject CA.  Thus, if the subject CA encodes attributes
           in the issuer fields of certificates that it issues using the
           TeletexString, BMPString, or UniversalString encodings, then
           the subject field of certificates issued to that CA MUST use
           the same encoding.

Now, in practice, I'm only aware of a single mainstream implementation implementing the rules defined in RFC 5280, Section 7.1 - Microsoft Windows. Other implementations among popular OSes and libraries (Mozilla NSS, Apple macOS and iOS, Google Chrome and Android, most of the *SSL variants like wolf/cyrus/matrix/ya/etc, including OpenSSL/BoringSSL), do not implement that, and at best assert conformity with 3280 rather than 5280, because 5280's rules require a full LDAP stringprep table. It's been a while, but I think the Sun Java implementation tried "similar, but not quite" with respect to LDAP StringPrep.

RFC 5280, Section 4.2.1.10 also anticipates this, when in discussing nameConstraints, it states

   When applying restrictions of the form directoryName, an
   implementation MUST compare DN attributes.  At a minimum,
   implementations MUST perform the DN comparison rules specified in
   Section 7.1.  CAs issuing certificates with a restriction of the form
   directoryName SHOULD NOT rely on implementation of the full ISO DN
   name comparison algorithm.  This implies name restrictions MUST be
   stated identically to the encoding used in the subject field or
   subjectAltName extension.

Namely, while RFC 5280 requires you do X, CAs should not expect clients to do X.

Now, while I stated that OpenSSL/BoringSSL/NSS do not implement the RFC 5280 algorithm, I'm aware #31440 (comment) claims to the contrary. The reason for this seeming inconsistency is the fact that OpenSSL/BoringSSL/NSS do something closer to the RFC 3280 rules that implementations MAY do; namely, it folds UTF8String and PrintableString into an internal canonical encoding before comparison. Some versions of macOS do this as well. However, this canoncalization is not the StringPrep profile mentioned by RFC 5280, and merely taking advantage of the liberalness of the MAY in 3280 to treat values in different types as equivalent to some internal string representation. However, that liberalness is, in practice, a violation of RFC 3280 itself, since they internally convert UTF8String to a canonical (whitespace folded, case normalized) form, which 3280 states that attribute values other than PrintableString are case sensitive and only PrintableString is compared in that way.

There are, unfortunately, some (legacy) public CAs that relied on this franken-hybrid approach, and did not follow the profile of using ensuring issuer and subject were consistently encoded. I'm not aware of any meaningful or significant compatibility issues you'll run into publicly. On internal CAs, this most commonly seems to be from internal CAs developed using older versions of OpenSSL (which did not default to UTF8String) and certificates issued with newer versions (which do default to UTF8String), and configuration being done by config files rather than by copying the Issuer->Subject.

I'm not sure if all of this info was helpful, @FiloSottile , but my $.02 would be that unless Golang is shipping with a full StringPrep implementation (... which has its own sharp edges), that the most consistent thing to do with the ecosystem would be the 3280 behaviour and note it as a "Known Issue". The second most consistent thing, which is more of the "make up the rules but others did it first", is to fold UTF8String and PrintableString into the same internal representation, then case fold and whitespace fold. Which is a violation of all the RFCs, but... matches others.

[sleevi]

Ah, and I somehow missed that #31440 (comment) mentioned that, yes, it's the "utf8only" v "default" switch in OpenSSL.

[FiloSottile]

Thanks @sleevi for the full overview. Given this context, I'm inclined to keep things as is, in particular because this is the first time it comes up in 10 years. Go applies a very strict complexity budget, and it seems we will not take a compatibility hit sufficient to justify the complexity of any of the solutions.

I will leave this open so we get to know if it affects more people.