Merging Mbed Crypto back into Mbed TLS

[yanesca]

We are going to merge Mbed Crypto back into Mbed TLS. This means that there will be a single repository, https://github.com/ARMmbed/mbedtls, hosting both the cryptography part of the library and the X.509 and TLS parts. Mbed TLS will be the implementation of the mbedtls_xxx cryptography, X.509 and TLS APIs as well as the reference implementation of the PSA Cryptography API.

Rationale

Mbed Crypto and Mbed TLS were separated to facilitate independent development of the two products. However this has had limited benefits, because the two products share most of their infrastructure (build and test scripts, continuous integration, etc.) and are maintained by the same team. Conversely, separating the product has costs: synchronizing changes between the two repositories adds maintenance overhead, building and testing is more complicated. Merging Mbed Crypto back into Mbed TLS avoids these overheads.

PSA Cryptography Implementation will continue to actively happen in Mbed TLS repository. In future, we intend to have a PSA Cryptography reference implementation as a separate product, without the legacy of the mbedtls_xxx cryptography APIs. Mbed Crypto as it currently is does not fulfill this purpose due to the need to maintain the existing mbedtls_xxx APIs.

Timeline

Note: all dates are tentative and subject to change.

• Mid-February 2020: announcement of the repository merge.
• Late February 2020: release of Mbed TLS 2.21, the last release with Mbed Crypto as a submodule.
• Late February 2020: freeze on structural changes to Mbed Crypto and Mbed TLS. Business as usual otherwise.
• Mid- to late March 2020: merge of https://github.com/ARMmbed/mbed-crypto into https://github.com/ARMmbed/mbedtls.
• Late March 2020: Freeze of the Mbed Crypto repository. All work in progress (issues, pull requests) will be migrated to the Mbed TLS repository. PSA Cryptography Implementation continues in Mbed TLS repository.
• April 2020 or later: Release of Mbed TLS 2.22, with the crypto files directly in the source tree instead of being in a submodule and subdirectory.
• Future: Creation of a PSA Crypto reference implementation that is a separate product, independent of mbedtls_xxx APIs. Mbed TLS will retain some mbedtls_xxx cryptography APIs for backward compatibility during
  a transition period.

Impact for users of Mbed TLS

You can keep using Mbed TLS as usual. If you clone the git repository, you won't need submodule support in your infrastructure anymore. The crypto part of the library will no longer be in the crypto/ subdirectory.

Impact for users of Mbed Crypto

You can build libmbedcrypto from the Mbed TLS repository (this has always been possible). If you want to build only the crypto library in its default configuration, run scripts/config.py crypto before you build. For more details, see our guide on building Mbed/PSA Crypto from the Mbed TLS repository below.

Impact for contributors to Mbed Crypto and Mbed TLS

During the transition period, we will keep merging pull requests in both repositories. However, there will be a short freeze on merging pull requests that make structural changes such as adding or removing files, or modify files that are shared between the two repositories. Changes that only affect crypto-only files can still be made in the Mbed Crypto repository, and changes that only affect X.509/TLS-only files can still be made in the Mbed TLS repository.

Once we merge the crypto files back into Mbed TLS, changes to crypto code will need to be made there. After this point, Mbed Crypto pull requests can be re-raised in Mbed TLS with few or nor modifications, since Mbed TLS will have the same file structure. If you have an open pull request in the Mbed Crypto repository at the cutover date, we'll let you know how to proceed.

The project remains committed to the continued development and maintenance of both Mbed TLS and PSA Cryptography API reference implementation. Merging Mbed Crypto back into Mbed TLS repository is to avoid the overheads in the short term as explained in Rationale Section above.

[yanesca]

Building Mbed/PSA Crypto from the mbedtls repo

This comment explains how to build the Mbed Crypto library (the current
incarnation of the PSA Crypto library) from the mbedtls repository. This will
be especially useful after mbed-crypto has been merged back to the mbedtls
repository, which will then be the only source for it.

The intended audience is users of Mbed Crypto or PSA Crypto who are currently
getting it from the mbed-crypto repository, and will need to get it from the
mbedtls repository after the merge. Users who are already getting Mbed Crypto
from the mbedtls repository and building it as part of Mbed TLS are
unaffected.

Summary

• Run scripts/config.py crypto before you build;
• build as usual;
• keep libmbedcrypto.a or libmbedcrypto.so;
• discard the other library files if you don't need them.

When using the default config.h

You can just build as usual (using make or cmake+make) and only keep
libmbedcrypto.a from the build results, discarding the other libraries.

Alternatively, if you want to avoid building the X.509 and TLS libraries
before discarding them, you can run scripts/config.py crypto before running
make (possibly after cmake). This will still create empty files for the
X.509 and TLS libraries, but not waste resources building them.

When starting from the default config.h and adapting it with config.py

If you're using a custom config.h derived from the default one by running a
number of scripts/config.py set/unset XXX commands, then you can just append
scripts/config.py crypto to your list of existing calls to config.py and
build as usual.

Again, this will create empty files for the X.509 and TLS libraries, but not
waste resources building them. Just discard those files and use
libmbedcrypto.a as usual.

Unlike the case of using the default config, running scripts/config.py crypto is required here, to avoid check_config.h complaining if your custom
crypto build is not compatible with the default TLS options.

When using a user config file

If you're starting from the default config.h but customizing it using a user
config file, by setting the MBEDTLS_USER_CONFIG_FILE, then you can just run
scripts/config.py crypto before your build as usual. This will modify the
default config.h but will keep your user config file unchanged.

Again, this will create empty files for the X.509 and TLS libraries, but not
waste resources building them. Just discard those files and use
libmbedcrypto.a as usual.

Unlike the case of using the default config, running scripts/config.py crypto is required here, to avoid check_config.h complaining if your custom
crypto build is not compatible with the default TLS options.

When using a standalone config.h

If you're using a custom stand-alone config.h, that is one that's maintained
independently and replaces the default one (either by overwriting it with your
file, or setting MBEDTLS_CONFIG_FILE in your build environment), then you
can just keep your file as is (assuming you're only setting crypto options in
it) and build as usual.

Again, this will create empty files for the X.509 and TLS libraries, but not
waste resources building them. Just discard those files and use
libmbedcrypto.a as usual.

If you custom standalone config.h was not clean and included X.509 or TLS
settings despite targeting Mbed Crypto, you might get errors from
check_config.h. These would be resolved by running scripts/config.py -f <path_to/your_config.h> crypto just one time. After that you can keep using
and maintaining your standalone config.h as usual.

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/IOTCRYPT-1049

[gilles-peskine-arm]

Mbed Crypto is now merged back into Mbed TLS (via Mbed-TLS/mbedtls#3085).