Currently, the stack silently ignores DTLS frames with an unexpected CID.
However, in a system which performs CID-based demultiplexing before passing
datagrams to the Mbed TLS stack, unexpected CIDs are a sign of something not
working properly, and users might want to know about it.

This commit introduces an SSL error code MBEDTLS_ERR_SSL_UNEXPECTED_CID
which the stack can return in response to an unexpected CID. It will
conditionally be put to use in subsequent commits.

This commit modifies the CID configuration API mbedtls_ssl_conf_cid_len()
to allow the configuration of the stack's behaviour when receiving an
encrypted DTLS record with unexpected CID.

The implementation is complete now.

The previous implementation of mbedtls_ssl_conf_cid() relied on
MBEDTLS_SSL_UNEXPECTED_CID_IGNORE being defined as 1.

Files modified via

sed -i 's/MBEDTLS_SSL_CID\([^_]\|$\)/MBEDTLS_SSL_DTLS_CONNECTION_ID\1/g' **/*.c **/*.h **/*.sh **/*.function

This commit modifies mbedtls_ssl_get_peer_cid() to also allow passing
NULL pointers in the arguments for the peer's CID value and length, in
case this information is needed.

For example, some users might only be interested in whether the use of
the CID was negotiated, in which case both CID value and length pointers
can be set to NULL. Other users might only be interested in confirming
that the use of CID was negotiated and the peer chose the empty CID,
in which case the CID value pointer only would be set to NULL.
It doesn't make sense to pass a NULL pointer for the CID length but a
non-NULL pointer for the CID value, as the caller has no way of telling
the length of the returned CID - and this case is therefore forbidden.

This commit adds the command line option 'bad_cid' to the UDP proxy
`./programs/test/udp_proxy`. It takes a non-negative integral value N,
which if not 0 has the effect of duplicating every 1:N CID records
and modifying the CID in the first copy sent.

This is to exercise the stacks documented behaviour on receipt
of unexpected CIDs.

It is important to send the record with the unexpected CID first,
because otherwise the packet would be dropped already during
replay protection (the same holds for the implementation of the
existing 'bad_ad' option).

This allows to test PSK-based ciphersuites via ssl_client2 in builds
which have MBEDTLS_X509_CRT_PARSE_C enabled but both MBEDTLS_FS_IO and
MBEDTLS_CERTS_C disabled.

A similar change is applied to the `crt_file` and `key_file` arguments.

This allows to test PSK-based ciphersuites via ssl_server2 in builds
which have MBEDTLS_X509_CRT_PARSE_C enabled but both MBEDTLS_FS_IO and
MBEDTLS_CERTS_C disabled.

All of them are copied from (former) CRT and key files in `tests/data_files`.
For files which have been regenerated since they've been copied to `certs.c`,
update the copy.

Add declarations for DER encoded test CRTs to certs.h

Add DER encoded versions of CRTs to certs.c

fix comment in certs.c

Don't use (signed) char for DER encoded certificates

Consistently use `const char *` for test CRTs regardless of encoding

Remove non-sensical and unused PW variable for DER encoded key

Provide test CRTs in PEM and DER fmt, + pick suitable per config

This commit modifies `certs.h` and `certs.c` to start following the
following pattern for the provided test certificates and files:

- Raw test data is named `NAME_ATTR1_ATTR2_..._ATTRn`

  For example, there are
     `TEST_CA_CRT_{RSA|EC}_{PEM|DER}_{SHA1|SHA256}`.

- Derived test data with fewer attributes, iteratively defined as one
  of the raw test data instances which suits the current configuration.

  For example,
     `TEST_CA_CRT_RSA_PEM`
  is one of `TEST_CA_CRT_RSA_PEM_SHA1` or `TEST_CA_CRT_RSA_PEM_SHA256`,
  depending on whether SHA-1 and/or SHA-256 are defined in the current
  config.

Add missing public declaration of test key password

Fix signedness and naming mismatches

Further improve structure of certs.h and certs.c

Fix definition of mbedtls_test_cas test CRTs depending on config

Remove semicolon after macro string constant in certs.c

This should allow to run ssl-opt.sh successfully in the default
configuration minus MBEDTLS_PEM_PARSE_C minus MBEDTLS_FS_IO.

This allows to auto-generate them from scripts.

This commit updates the Crypto submodule to the merge commit of

  ARMmbed#133

which mirrors the change of the DHM selftest, making the latter
succeed in builds with !MBEDTLS_PEM_PARSE_C.

When parsing a substructure of an ASN.1 structure, no field within
the substructure must exceed the bounds of the substructure.
Concretely, the `end` pointer passed to the ASN.1 parsing routines
must be updated to point to the end of the substructure while parsing
the latter.

This was previously not the case for the routines
- x509_get_attr_type_and_value(),
- mbedtls_x509_get_crt_ext(),
- mbedtls_x509_get_crl_ext().
These functions kept using the end of the parent structure as the
`end` pointer and would hence allow substructure fields to cross
the substructure boundary. This could lead to successful parsing
of ill-formed X.509 CRTs.

This commit fixes this.

Care has to be taken when adapting `mbedtls_x509_get_crt_ext()`
and `mbedtls_x509_get_crl_ext()`, as the underlying function
`mbedtls_x509_get_ext()` returns `0` if no extensions are present
but doesn't set the variable which holds the bounds of the Extensions
structure in case the latter is present. This commit addresses
this by returning early from `mbedtls_x509_get_crt_ext()` and
`mbedtls_x509_get_crl_ext()` if parsing has reached the end of
the input buffer.

The following X.509 parsing tests need to be adapted:
- "TBSCertificate, issuer two inner set datas"
  This test exercises the X.509 CRT parser with a Subject name
  which has two empty `AttributeTypeAndValue` structures.
  This is supposed to fail with `MBEDTLS_ERR_ASN1_OUT_OF_DATA`
  because the parser should attempt to parse the first structure
  and fail because of a lack of data. Previously, it failed to
  obey the (0-length) bounds of the first AttributeTypeAndValue
  structure and would try to interpret the beginning of the second
  AttributeTypeAndValue structure as the first field of the first
  AttributeTypeAndValue structure, returning an UNEXPECTED_TAG error.
- "TBSCertificate, issuer, no full following string"
  This test exercises the parser's behaviour on an AttributeTypeAndValue
  structure which contains more data than expected; it should therefore
  fail with MBEDTLS_ERR_ASN1_LENGTH_MISMATCH. Because of the missing bounds
  check, it previously failed with UNEXPECTED_TAG because it interpreted
  the remaining byte in the first AttributeTypeAndValue structure as the
  first byte in the second AttributeTypeAndValue structure.
- "SubjectAltName repeated"
  This test should exercise two SubjectAltNames extensions in succession,
  but a wrong length values makes the second SubjectAltNames extension appear
  outside of the Extensions structure. With the new bounds in place, this
  therefore fails with a LENGTH_MISMATCH error. This commit adapts the test
  data to put the 2nd SubjectAltNames extension inside the Extensions
  structure, too.

Some functions within the X.509 module return an ASN.1 low level
error code where instead this error code should be wrapped by a
high-level X.509 error code as in the bulk of the module.

Specifically, the following functions are affected:
- mbedtls_x509_get_ext()
- x509_get_version()
- x509_get_uid()

This commit modifies these functions to always return an
X.509 high level error code.

Care has to be taken when adapting `mbetls_x509_get_ext()`:
Currently, the callers `mbedtls_x509_crt_ext()` treat the
return code `MBEDTLS_ERR_ASN1_UNEXPECTED_TAG` specially to
gracefully detect and continue if the extension structure is not
present. Wrapping the ASN.1 error with
`MBEDTLS_ERR_X509_INVALID_EXTENSIONS` and adapting the check
accordingly would mean that an unexpected tag somewhere
down the extension parsing would be ignored by the caller.

The way out of this is the following: Luckily, the extension
structure is always the last field in the surrounding structure,
so if there is some data remaining, it must be an Extension
structure, so we don't need to deal with a tag mismatch gracefully
in the first place.

We may therefore wrap the return code from the initial call to
`mbedtls_asn1_get_tag()` in `mbedtls_x509_get_ext()` by
`MBEDTLS_ERR_X509_INVALID_EXTENSIONS` and simply remove
the special treatment of `MBEDTLS_ERR_ASN1_UNEXPECTED_TAG`
in the callers `x509_crl_get_ext()` and `x509_crt_get_ext()`.

This renders `mbedtls_x509_get_ext()` unsuitable if it ever
happened that an Extension structure is optional and does not
occur at the end of its surrounding structure, but for CRTs
and CRLs, it's fine.

The following tests need to be adapted:
- "TBSCertificate v3, issuerID wrong tag"
  The issuerID is optional, so if we look for its presence
  but find a different tag, we silently continue and try
  parsing the subjectID, and then the extensions. The tag '00'
  used in this test doesn't match either of these, and the
  previous code would hence return LENGTH_MISMATCH after
  unsucessfully trying issuerID, subjectID and Extensions.
  With the new code, any data remaining after issuerID and
  subjectID _must_ be Extension data, so we fail with
  UNEXPECTED_TAG when trying to parse the Extension data.
- "TBSCertificate v3, UIDs, invalid length"
  The test hardcodes the expectation of
  MBEDTLS_ERR_ASN1_INVALID_LENGTH, which needs to be
  wrapped in MBEDTLS_ERR_X509_INVALID_FORMAT now.

Fixes #2431.

- Explain the use of explicit ASN.1 tagging for the extensions structuree
- Remove misleading comment which suggests that mbedtls_x509_get_ext()
  also parsed the header of the first extension, which is not the case.

Previously, a test exercising the X.509 CRT parser's behaviour
on unexpected tags would use a '00' byte in place of the tag
for the expected structure. This makes reviewing the examples
harder because the binary data isn't valid DER-encoded ASN.1.

This commit uses the ASN.1 NULL TLV '05 00' to test invalid
tags, and adapts surrounding structures' length values accordingly.
This eases reviewing because now the ASN.1 structures are still
well-formed at the place where the mismatch occurs.

Lengths below 128 Bytes must be encoded as a single 'XX' byte in DER,
but two tests in the X.509 CRT parsing suite used the BER but non-DER
encoding '81 XX' (the first byte 10000001 indicating that the length
is to follow (high bit) and has length 1 byte (low bit)).

Most tests use an sha256WithRSAEncryption OID which isn't recognized
unless RSA and SHA-256 are enabled.

When MBEDTLS_PSA_INJECT_ENTROPY is used, we now require also defining
MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES. When
MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES is defined, we do not add entropy
sources by default. This includes the NV seed entropy source, which the
PSA entropy injection API is built upon.

The PSA entropy injection feature depends on using NV seed as an entropy
source. Add NV seed as an entropy source for PSA entropy injection.

Fixes e3dbdd8 ("Gate entropy injection through a dedicated configuration option")

When MBEDTLS_SSL_ENCRYPT_THEN_MAC is enabled, but not
MBEDTLS_SSL_SOME_MODES_USE_MAC, mbedtls_ssl_derive_keys() and
build_transforms() will attempt to use a non-existent `encrypt_then_mac`
field in the ssl_transform.

    Compile [ 93.7%]: ssl_tls.c
    [Error] ssl_tls.c@865,14: 'mbedtls_ssl_transform {aka struct mbedtls_ssl_transform}' ha
s no member named 'encrypt_then_mac'
    [ERROR] ./mbed-os/features/mbedtls/src/ssl_tls.c: In function 'mbedtls_ssl_derive_keys'
:
    ./mbed-os/features/mbedtls/src/ssl_tls.c:865:14: error: 'mbedtls_ssl_transform {aka str
uct mbedtls_ssl_transform}' has no member named 'encrypt_then_mac'
         transform->encrypt_then_mac = session->encrypt_then_mac;
                  ^~

Change mbedtls_ssl_derive_keys() and build_transforms() to only access
`encrypt_then_mac` if `encrypt_then_mac` is actually present.

Add a regression test to detect when we have regressions with
configurations that do not include any MAC ciphersuites.

Fixes d56ed24 ("Reduce size of `ssl_transform` if no MAC ciphersuite is enabled")

When testing a configuration where no ciphersuites have MAC, via
component_test_when_no_ciphersuites_have_mac(), perform a targeted test
of only encrypt-then-MAC tests within ssl-opt.sh.

psa: Add NV seed as an entropy source when needed

Previously, even in the Chacha20 and Chacha20-Poly1305 tests, we would
test that decryption of an empty buffer would work with
MBEDTLS_CIPHER_AES_128_CBC.

Make the cipher used with the dec_empty_buf() test configurable, so that
Chacha20 and Chacha20-Poly1305 empty buffer tests can use ciphers other
than AES CBC. Then, make the Chacha20 and Chacha20-Poly1305 empty buffer
tests use the MBEDTLS_CIPHER_CHACHA20 and
MBEDTLS_CIPHER_CHACHA20_POLY1305 cipher suites.

Remove the "Decrypt empty buffer" test, as ChaCha20 is a stream cipher
and 0 bytes encrypted is identical to a 0 length buffer. The "ChaCha20
Encrypt and decrypt 0 bytes" test will test decryption of a 0 length
buffer.

* origin/pr/2654:
  Create link to include/mbedtls only when testing is enabled

* origin/pr/2679:
  test: Remove redundant 0-byte decryption test
  test: Check empty buffer decryption for chachapoly

Relative include paths should be avoided. The build system will
determine where to pull in includes from. Specifically, `#include
"../mbedtls/config.h"` shouldn't be used. Use `#include
"mbedtls/config.h` instead, so that the submodule-building makefiles can
change which directory to use to get mbedtls include files from.

Fixes ARMmbed#141

Previously, even in the Chacha20 and Chacha20-Poly1305 tests, we would
test that decryption of an empty buffer would work with
MBEDTLS_CIPHER_AES_128_CBC.

Make the cipher used with the dec_empty_buf() test configurable, so that
Chacha20 and Chacha20-Poly1305 empty buffer tests can use ciphers other
than AES CBC. Then, make the Chacha20 and Chacha20-Poly1305 empty buffer
tests use the MBEDTLS_CIPHER_CHACHA20 and
MBEDTLS_CIPHER_CHACHA20_POLY1305 cipher suites.

Remove the "Decrypt empty buffer" test, as ChaCha20 is a stream cipher
and 0 bytes encrypted is identical to a 0 length buffer. The "ChaCha20
Encrypt and decrypt 0 bytes" test will test decryption of a 0 length
buffer.

psa: Avoid use of relative include paths

test: Check empty buffer decryption for chachapoly

Fix the dependency on libmbedcrypto.a, which is now located under
crypto.

Fix #2682

For unit tests and sample programs, CFLAGS=-m32 is enough to get a
32-bit build, because these programs are all compiled directly
from *.c to the executable in one shot. But with makefile rules that
first build object files and then link them, LDFLAGS=-m32 is also
needed.

Typo rproduce -> reproducible
Call mbedtls_entropy_func

* origin/pr/2678:
  Update crypto submodule to 1.1.0d2
  all.sh: Perform targeted EtM tests for MAC-less configs
  ssl: Don't access non-existent encrypt_then_mac field