Signing with SHA512

[vgottardi]

cryptoauth-openssl-engine/engine_atecc/engine_meth/eccx08_ecdsa_sign.c

Line 88 in 40f0634

if (dgst_len != MEM_BLOCK_SIZE) {

- This line of code makes sure the digest length matches the ECC508 buffer size. That means only 256-bit digests are supported (SHA256).

Longer digests could also be supported (e.g. SHA512). According the the NIST 186-4 section 6.4: "When the length of the output of the hash function is greater than the bit length of n, then the leftmost n bits of the hash function output block shall be used in any calculation using the hash function output during the generation or verification of a digital signature."

The proposed fix is to change this validation to only reject digests that are shorter than 256 bit (dgst_len < MEM_BLOCK_SIZE). This will remove the need to compile OpenSSL with -DOPENSSL_NO_SHA512.

The changed code works fine as tested with TLS client certificate authentication connecting to a plain Linux/Apache or Windows/IIS server.

[bryan-hunt]

Moving forward the engine doesn't require compilation of openssl in order to be built and the sha256 restriction was specified by using sigalgs and cipher specification in the client application:

     if(!SSL_CTX_set_cipher_list(pSSLContext, "ECDHE-ECDSA-AES128-GCM-SHA256"))
     {
          ERROR(" Unable to set cipher suite");
     }

     if(!SSL_CTX_set1_sigalgs_list(pSSLContext, "ECDSA+SHA256"))
     {
          ERROR(" Unable to set sigalgs");
     }
	
     if(!SSL_CTX_set1_client_sigalgs_list(pSSLContext, "ECDSA+SHA256"))
     {
          ERROR(" Unable to set client sigalgs");
     }

However this does appear to be a better solution.

The line reference in the next release would be:

cryptoauth-openssl-engine/cryptoauthlib/lib/openssl/eccx08_ecdsa_sign.c

Line 76 in dfe4a30

if (!dgst || dgst_len != ATCA_BLOCK_SIZE)