TLSv1.3 causes panic on certified FIPS 140-3 linux instances

[michel-laterman]

Using an ubuntu certified AMI (which has a FIPS 140-3 certification), and the latest microsoft/go (v1.24.2 at the time of reporting), operations that require TLSv1.3 cause a panic.

This can be tested with the following program:

package main

import (
        "crypto/tls"
        "log"
        "net/http"
)

func main() {
        client := http.Client{
                Transport: &http.Transport{
                        TLSClientConfig: &tls.Config{
                                MinVersion: tls.VersionTLS13,
                        },
                },
        }

        resp, err := client.Get("https://www.google.com")
        if err != nil {
                panic(err)
        }
        log.Default().Printf("Request status: %d\n", resp.StatusCode)
}

When executed with GOEXPERIMENT=systemcrypto CGO_ENABLED=1 go run main.go, the following panic occurs:

panic: EVP_KDF_derive
    openssl error(s):
    error:1C800069:Provider routines::invalid key length
        ../providers/implementations/kdfs/hkdf.c:163

goroutine 5 [running]:
crypto/tls/internal/tls13.ExpandLabel[...](0xc0000258c8, {0xc0000b4620, 0x20, 0x20}, {0x72c473, 0x2}, {0x0, 0x0, 0x0}, 0xc)
    /usr/local/go/src/crypto/tls/internal/tls13/tls13.go:41 +0x27f
crypto/tls.(*cipherSuiteTLS13).trafficKey(0x9b7540, {0xc0000b4620, 0x20, 0x20})
    /usr/local/go/src/crypto/tls/key_schedule.go:29 +0x10d
crypto/tls.(*halfConn).setTrafficSecret(0xc000190220, 0x9b7540, 0xc0000b45e0?, {0xc0000b4620?, 0x20?, 0x72e255?})
    /usr/local/go/src/crypto/tls/conn.go:234 +0x67
crypto/tls.(*clientHandshakeStateTLS13).establishHandshakeKeys(0xc000025c48)
    /usr/local/go/src/crypto/tls/handshake_client_tls13.go:525 +0x374
crypto/tls.(*clientHandshakeStateTLS13).handshake(0xc000025c48)
    /usr/local/go/src/crypto/tls/handshake_client_tls13.go:134 +0x73e
crypto/tls.(*Conn).clientHandshake(0xc000190008, {0x7a3c40, 0xc000192000})
    /usr/local/go/src/crypto/tls/handshake_client.go:379 +0x810
crypto/tls.(*Conn).handshakeContext(0xc000190008, {0x7a3c40, 0xc0000dc0f0})
    /usr/local/go/src/crypto/tls/conn.go:1568 +0x39a
crypto/tls.(*Conn).HandshakeContext(...)
    /usr/local/go/src/crypto/tls/conn.go:1508
net/http.(*persistConn).addTLS.func2()
    /usr/local/go/src/net/http/transport.go:1703 +0x6e
created by net/http.(*persistConn).addTLS in goroutine 20
    /usr/local/go/src/net/http/transport.go:1699 +0x309
exit status 2

Where the key length check from the OpenSSL provider is:

int ossl_kdf_check_keylen(OSSL_LIB_CTX *ctx, size_t len)
{
#if !defined(OPENSSL_NO_FIPS_SECURITYCHECKS)
    if (ossl_securitycheck_enabled(ctx))
        return len >= 14;
#endif /* OPENSSL_NO_FIPS_SECURITYCHECKS */
    return 1;
}

but the nonce length is hard coded to 12 bytes.

[ebeahan]

Possibly related: #1465

[qmuntal]

Thanks for reporting. I can't reproduce this issue on Azure Linux 3 using FIPS mode, but there is enough info in this issue to investigate. Will report back.

[qmuntal]

Possibly related: #1465

Don't think so, this is a failure while deriving a key with HKDF, not encrypting with AES GCM.

Looking at the stack traces, the secret parameter (aka the pseudo random key) length passed to hkdf.Expand is 32 bytes, which is longer than the minimum of 14 bytes required by FIPS 140-3 (as per NIST.SP.800-131Ar2). The length parameter (aka nonce/iv length in this step of the TLS 1.3 handshake) is 12 bytes, which is shorter than 14 bytes, but AFAIK there is no restriction on the derived key length. Note that the 12-byte nonce is expected, as that's the AEAD nonce length defined in RFC 8446, Section 7.

I've tested with the OpenSSL FIPS provider and the SymCrypt provider, and they both successfully derive a key. Canonical might have patched the HKDF EVP_KDF algorithm to restrict the output key length given that the ossl_kdf_check_keylen function you shared doesn't exist in the OpenSSL repo, but can't confirm because the Canonical Pro patches are behind a paywall.

I'm aware that OpenSSL provides the TLS 1.3 EVP_KDF algorithm, which is an specialization of the HKDF EVP_KDF algorithm (the one we use) just for TLS 1.3. I suppose that Canonical doesn't perform the ossl_kdf_check_keylen check if using that algorithm. AFAIK, it is valid to perform the TLS 1.3 steps outside of the FIPS module boundary (see https://atsec-information-security.blogspot.com/2021/03/the-impact-of-tls-13-and-acvts-on-fips.html), so our implementation is compliant with FIPS 140-3, although not compatible with the Canonical OpenSSL FIPS module due to it being more strict on the HKDF key length output.

@michel-laterman you probably have the Canonical OpenSSL FIPS provider source code in your ubuntu certified AMI. Can you check if my asumptions are correct?

[michel-laterman]

Yes, your asumption is correct:

static int kdf_hkdf_derive(void *vctx, unsigned char *key, size_t keylen,
                           const OSSL_PARAM params[])
{
    KDF_HKDF *ctx = (KDF_HKDF *)vctx;
    OSSL_LIB_CTX *libctx = PROV_LIBCTX_OF(ctx->provctx);
    const EVP_MD *md;

    if (!ossl_prov_is_running() || !kdf_hkdf_set_ctx_params(ctx, params))
        return 0;

    if (!ossl_kdf_check_keylen(libctx, keylen)) {
        ERR_raise(ERR_LIB_PROV, PROV_R_INVALID_KEY_LENGTH);
        return 0;
    }

    md = ossl_prov_digest_md(&ctx->digest);
    if (md == NULL) {
        ERR_raise(ERR_LIB_PROV, PROV_R_MISSING_MESSAGE_DIGEST);
        return 0;
    }
    if (ctx->key == NULL) {
        ERR_raise(ERR_LIB_PROV, PROV_R_MISSING_KEY);
        return 0;
    }
    if (keylen == 0) {
        ERR_raise(ERR_LIB_PROV, PROV_R_INVALID_KEY_LENGTH);
        return 0;
    }

    switch (ctx->mode) {
    case EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND:
    default:
        return HKDF(libctx, md, ctx->salt, ctx->salt_len,
                    ctx->key, ctx->key_len, ctx->info, ctx->info_len, key, keylen);

    case EVP_KDF_HKDF_MODE_EXTRACT_ONLY:
        return HKDF_Extract(libctx, md, ctx->salt, ctx->salt_len,
                            ctx->key, ctx->key_len, key, keylen);

    case EVP_KDF_HKDF_MODE_EXPAND_ONLY:
        return HKDF_Expand(md, ctx->key, ctx->key_len, ctx->info,
                           ctx->info_len, key, keylen);
    }
}

static int kdf_tls1_3_derive(void *vctx, unsigned char *key, size_t keylen,
                             const OSSL_PARAM params[])
{
    KDF_HKDF *ctx = (KDF_HKDF *)vctx;
    const EVP_MD *md;

    if (!ossl_prov_is_running() || !kdf_tls1_3_set_ctx_params(ctx, params))
        return 0;

    md = ossl_prov_digest_md(&ctx->digest);
    if (md == NULL) {
        ERR_raise(ERR_LIB_PROV, PROV_R_MISSING_MESSAGE_DIGEST);
        return 0;
    }

    switch (ctx->mode) {
    default:
        return 0;

    case EVP_KDF_HKDF_MODE_EXTRACT_ONLY:
        return prov_tls13_hkdf_generate_secret(PROV_LIBCTX_OF(ctx->provctx), md,
                                               ctx->salt, ctx->salt_len,
                                               ctx->key, ctx->key_len,
                                               ctx->prefix, ctx->prefix_len,
                                               ctx->label, ctx->label_len,
                                               key, keylen);

    case EVP_KDF_HKDF_MODE_EXPAND_ONLY:
        return prov_tls13_hkdf_expand(md, ctx->key, ctx->key_len,
                                      ctx->prefix, ctx->prefix_len,
                                      ctx->label, ctx->label_len,
                                      ctx->data, ctx->data_len,
                                      key, keylen);
    }
}

The checks canonical does is more strict.

[qmuntal]

You can work around this issue by disabling TLS 1.3. The real fix is adding support for the TLS 1.3 EVP_KDF algorithm in the OpenSSL backend: https://github.com/golang-fips/openssl and using it from the Microsoft build of Go if someone (you? Canonical?) do the work of supporting this new KFD algorithm and validating that it works on Ubuntu Pro FIPS.

[nicholasberlin]

golang-fips/openssl#272

[nicholasberlin]

@qmuntal the canonical patch restricts to 14 bytes by testing keylen, which I believe is the desired final length of the output key.

golang/go seems to restrict based on the length of the pseudorandom input key:

https://github.com/golang/go/blob/71d9505998fe224ebb7380616392f71f7ebf26bf/src/crypto/hkdf/hkdf.go#L77-L79

Think that's a bug?

[qmuntal]

I think that the Go implementation is correct, and that canonical is being too strict by limiting the length of the output key, as NIST.SP.800-131Ar2 only sets limits on the HKDF inputs, not on the output.

[nicholasberlin]

#1654