RH2104724: Avoid import/export of DH private keys

[franferrax]

Search this PR in Red Hat Jira

RH2104724 / RH2104725: Avoid import/export of DH private keys

Description

As analyzed in OPENJDK-824, NSS doesn't support wrapping/unwrapping of Diffie-Hellman private keys (CKK_DH), so we can't import/export them from the NSS PKCS#11 software token.

In addition, as part of 78df8b5 work, we started to blindly consider a private key extractable when the plain key support is enabled, preventing DH private keys from being instantiated as the opaque P11PrivateKey. This now causes an error, as an attempt is made to extract these keys when instantiating the P11DHPrivateKey full-data object.

This work:

1. Avoids the import/export of DH private keys, going back to the opaque P11PrivateKey for them (instead of P11DHPrivateKey)
2. Removes the useless code from the FIPS key importer (abcd095), which hasn't triggered any issue, but it will never work
   • NOTE: the FIPS key exporter doesn't have code to handle CKK_DH private keys.

[franferrax]

Failing test cases from OPENJDK-824 are passing in a local build of this PR + #16, executed with:

separator="$(printf "%$(tput cols)s" | tr " " "=")"
highlighter="s/^\(failed\|error\)\(:\|$\)/$(tput bold && tput setaf 1)\0$(tput sgr0)/gi;
             s/^passed\(:\|$\)/$(tput bold && tput setaf 2)\0$(tput sgr0)/gi;
             s/^ignored\(:\|$\)/$(tput bold && tput setaf 3)\0$(tput sgr0)/gi"

function test_env_create() {
    git clone --quiet "https://github.com/rh-openjdk/$1" && pushd "$1" >/dev/null &&
        echo -e "$(tput bold)${separator}\n= $1\n${separator}$(tput sgr0)"
}

function test_env_destroy() {
    popd >/dev/null && rm -rf "$1" && echo -e "\n"
}

function run_tests() {
    export JAVA_HOME="$(find $HOME -wholename */jdk17u/build/*/images/jdk -print -quit)"

    test_env_create ssl-tests || return
    make TEST_PKCS11_FIPS=1 SSLTESTS_SSL_CONFIG_FILTER=SunJSSE,Default,TLSv1.2,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 \
         SSLTESTS_CUSTOM_JAVA_PARAMS=-Djdk.tls.ephemeralDHKeySize=2048 ssl-tests | sed "$highlighter"
    test_env_destroy ssl-tests

    test_env_create CryptoTest || return
    make KeyAgreementTests | sed "$highlighter"
    test_env_destroy CryptoTest

    unset JAVA_HOME
}
run_tests && unset run_tests test_env_destroy test_env_create highlighter separator

Output from rh-openjdk/ssl-tests (excerpt):

Main
PASSED: SunJSSE/Default: TLSv1.2 + TLS_DHE_RSA_WITH_AES_256_GCM_SHA384

Output from rh-openjdk/CryptoTest (excerpt):

running: cryptotest.tests.KeyAgreementTests
0)	SunPKCS11-NSS-FIPS: 	DH~DH	 (KeyAgreement)
Passed
1)	SunPKCS11-NSS-FIPS: 	DH~DiffieHellman	 (KeyAgreement)
Passed
2)	SunPKCS11-NSS-FIPS: 	ECDH~ECDH	 (KeyAgreement)
Passed
Total checks: 3, failed: 0
All KeyAgreement passed
PASSED: cryptotest.tests.KeyAgreementTests

[gnu-andrew]

Looks good to me.

[franferrax]

I forgot to remove the DHKF and DHKFLock static variables from the start of the FIPSKeyImporter class.