Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes #11920 raise a clean Python error on DSA signing failure due to nilpotent #11921

Merged
merged 1 commit into from
Nov 11, 2024

Conversation

alex
Copy link
Member

@alex alex commented Nov 8, 2024

No description provided.

@alex
Copy link
Member Author

alex commented Nov 8, 2024

@davidben question for you (since this testcase is from BoringSSL), this isn't a case of key invalidity that should be caught on key load, right? Catching it on signing is the only thing that makes sense? (That's what BoringSSL appears to do, so I assume its correct.)

@alex alex force-pushed the dsa-sign-error branch 2 times, most recently from 1a000fd to b396b02 Compare November 8, 2024 14:46
@alex
Copy link
Member Author

alex commented Nov 8, 2024

@hannob it looks like OpenSSL 3.0 just hangs on these :-( I'm not sure what the right solution is there. Probably OpenSSL ought to backport the fix, unsure if it's CVE worthy...

@alex
Copy link
Member Author

alex commented Nov 8, 2024

Ok, looks like the fix was backported in openssl/openssl@5a53d73, but never made its way into Ubunut. Great.

@hannob
Copy link
Contributor

hannob commented Nov 8, 2024

Actually... I reported this years ago to OpenSSL as a security issue, and initially, they didn't consider this fix-worthy at all. They said that malformed private keys aren't part of their threat-model.

@alex
Copy link
Member Author

alex commented Nov 8, 2024

I'm sure all the CDNs that accept keys from their users love that.

@davidben
Copy link
Contributor

davidben commented Nov 8, 2024

@davidben question for you (since this testcase is from BoringSSL), this isn't a case of key invalidity that should be caught on key load, right? Catching it on signing is the only thing that makes sense? (That's what BoringSSL appears to do, so I assume its correct.)

Well, to say whether something is "correct" presumes a baseline level of well-definedness and importance that DSA does not have. 😄

First, it is not an invalid DSA key per se, but an invalid DSA group. NIST calls them "domain parameters". It's like picking arbitrary coefficients for an ECC key, and then finding that those coefficients do not meet the mathematical preconditions of any of the algorithms and things go into space.

Now, what is the correct handling for invalid domain parameters? Well, the correct way to handle domain parameters is to agree on values ahead of time so that, when it comes down to code, they're hard-coded in the system anyway. Unlike key material, domain parameters are not meant to vary, and they often have preconditions that are expensive to check in key import or use (e.g. primality).

But that's not how DSA ended up being used. People treated the domain parameters as part of the key, even though the expectations on the preconditons are very different. And so now we have to worry about how all our algorithms behave when their preconditions fail. Here, when p is composite, the Z/pZ "field" (it's not a field) has nontrivial nilpotent elements. Pick one of those as the generator and g^k will almost always be zero. DSA signing will then not succeed.

So, what do you do? Well, we pretended it was part of the key, so ideally we'd check it as part of key import, not signing. But preconditions for domain parameters are often hard to check, and correct use of DSA would only use trusted domain parameters, so it's expensive. The other option is to say, well, if you pass invalid domain parameters, we're not promising any kind of cryptographic soundness (because none of the preconditions for the system hold), or much of anything. Just that the function won't infinite loop or hit language-level UB or something. It will cleanly output something garbage, or cleanly fail.

This is a mess, and if DSA mattered, it would perhaps be worth a whole process to revamp the API, migrate from arbitrary parameters to a set of named, pre-defined parameters. But DSA is only decreasing in relevance, so we just went with the second option and added an iteration limit.

(Oh, and then on top of all these, there's the mess around when you check preconditions for keys in the first place because OpenSSL's APIs are not very good about this. I think we've talked about this a bit with RSA_check_key and such. But at least importing from a file does give some place for the library to assemble all this. For RSA, I have plans to rework things in BoringSSL to make that better. DSA... well, DSA is not really relevant.)

@alex
Copy link
Member Author

alex commented Nov 8, 2024 via email

@davidben
Copy link
Contributor

davidben commented Nov 8, 2024

Yeah, I think LibreSSL opted to stick a primality check in there. I take it you are testing against older OpenSSLs that still infinite loop?

(Nitpick: The fix wasn't specifically for nilpotent "generators". It was to cap the number of iterations and eventually give up. It was a while ago, so I forget if we ended up finding other types of invalid inputs that also tickled this.)

@alex
Copy link
Member Author

alex commented Nov 8, 2024 via email

@reaperhulk reaperhulk merged commit da437d1 into pyca:main Nov 11, 2024
60 checks passed
@alex alex deleted the dsa-sign-error branch November 11, 2024 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants