Make legacy crypto APIs internal

[mpg]

As part of making PSA Crypto the main crypto API in 4.0, we're making (most of) the legacy crypto API internal (that is, headers would be visible to other crypto modules but not to applications).

This issue is an attempt at summarizing the consensus that was reached last time we discussed this.

Categorized list of crypto headers

Cipher & related:

• internalize: aes.h, aria.h, block_cipher.h, camellia.h, ccm.h, chacha20.h, chachapoly.h, cmac.h, des.h, gcm.h, poly1305.h,
• under discussion: cipher.h Remove cipher.h in 4.0 #8451
• keep for now: nist_kw.h (note: includes cipher.h! If we remove cipher.h we'll need to change types in nist_kw.h - only mbedtls_cipher_id_t is used AFAICS) → Move NIST_KW to PSA API #9382.

Hashes & related:

• internalize: md5.h, ripemd160.h, sha1.h, sha256.h, sha3.h, sha512.h,
• under discussion: md.h Evolution of md.h in 4.0 #8450
• remove hkdf.h

Randomness:

• internalize: entropy.h, ctr_drbg.h, hmac_drbg.h - require Study: PSA random generation drivers for 4.0 (replace entropy.h) #8150 to avoid functionality loss MBEDTLS_ENTROPY_HARDWARE_ALT in 4.0 #9618 to avoid critical functionality loss

Asymmetric crypto:

• internalize: bignum.h, dhm.h, ecdh.h, ecdsa.h, ecjpake.h, ecp.h, rsa.h - note: potential feature loss by removing ecp.h and rsa.h (and bignum.h) but came to an agreement accepting it. Note: important feature loss (restartable/interruptible) unless [DI] Use interruptible sign/verify in X.509 & TLS #7292, [DI] Add interruptible APIs to PSA for ECDH #7293, [DI] Use interruptible ECDH in TLS #7294 and resulting EPIC(s) are done first. Note impact on fuzzers.
• under discussion: pk.h Evolution of pk.h in 4.0 #8452
• keep for now: lms.h (note: does not include other headers)

Supporting modules:

• pkcs12.h, pkcs5.h -> internalize (used by PK parse and soon extended PSA key import)
• base64.h -> internalize (used by PEM only)
• pem.h -> semi-internal? (used by X.509, PK parse/write and soon extended PSA key import/export)
• asn1.h, asn1write.h -> semi-internal? (used by X.509, PK parse/write and soon extended PSA key import/export, but also basic RSA import/export)
• oid.h -> split and redesign, partly internalized: Split numeric string conversions out of the OID module #9379, Redesign OID API for smaller code size #9380
• platform.h, platform_time.h, platform_util.h, threading.h -> keep?
• constant_time.h -> ?? (only one function now)
• build_info.h error.h, version.h, -> keep but split crypto vs non-crypto I guess?
• psa_util.h -> keep if it helps the transition, or internalize
• private_access.h -> keep
• memory_buffer_alloc.h -> ??
• timing.h -> ??

Notes

Above, by "crypto headers" I mean all headers that are part of libmbedcrypto - that includes things that are does not actually provide crypto functionality, like platform support.

There seems to be a need to a special "semi-internal" status for header that should be visible to both crypto modules and X.509, but perhaps not to applications (pem.h, asn1.h, asn1write.h).

This issue it mainly to record what we've discussed, and host further discussion if needed. For execution, we probably want to split this in a series of task, for example per area as listed above.

There will probably be interactions with splitting out PSA crypto to its own repo too.

[mpg]

Ping @dave-rodgman - I think you took notes during the meeting, please correct my list if it's not in line with those notes.

[mpg]

Note: some legacy modules, when made internal, can actually be removed as well - when the PSA implementation of that feature doesn't actually use the legacy module.

For example for FFDH, when making dhm.h internal, we can remove dhm.c entirely, as psa_crypto_ffdh.c does not depend on it. We can also move the definition of the constants for the domain parameters to psa_crypto_ffdh.c (currently taken from dhm.h) and then remove dhm.h as well.

Similar considerations might apply to other modules, I've not tried making a complete list yet.

(Note: currently psa_crypto_ecp.c calls functions from ecdh.c but I don't think there's a good reason for that, IMO it should be calling mbedtls_ecp_mul() directly instead.)

[mpg]

(Note: currently psa_crypto_ecp.c calls functions from ecdh.c but I don't think there's a good reason for that, IMO it should be calling mbedtls_ecp_mul() directly instead.)

Ah, actually there's a reason, it's that Everest dispatch is implemented in ecdh.c not ecp.c. IMO that's not a good reason in the long term: Everest should be a PSA driver like everything else instead of having its own special dispatch mechanism in ecdh.c.

[yanesca]

Almost everything here (that is not covered by a separate issue) will be done as part of the repo-split. What remains here to do is to document it clearly. The lack of interface stability must be clearly explained and the affected APIs marked as such.

Open questions:

• Do we want internal/non-stable APIs to show up on readthedocs?
• Do we want to mark individual functions as internal or header placement/comment in the header should be enough?

[gilles-peskine-arm]

Do we want internal/non-stable APIs to show up on readthedocs?

Definitely not.

Do we want to mark individual functions as internal or header placement/comment in the header should be enough?

Separate headers. In the past we've gotten pretty loud complaints when we moved a function declaration from a public header with a comment “internal use only” to no longer being in a private header.

Any legacy function that we don't want to promise keeping for as long as 4.x is around should be in a separate header. I propose to create a subdirectory include/mbedtls/unstable for such headers.

[valeriosetti]

For example for FFDH, when making dhm.h internal, we can remove dhm.c entirely, as psa_crypto_ffdh.c does not depend on it. We can also move the definition of the constants for the domain parameters to psa_crypto_ffdh.c (currently taken from dhm.h) and then remove dhm.h as well.

I've just created the issue for removing DHM module -> #9956

[Faless]

Hi all, I'm a maintainer of Godot Engine which, as some of you may know, uses mbedTLS as the cryptographic library.

Prompted by the recent email about an upcoming mbedTLS 4.0 release I ended up finding this issue and went through the list to see how this would affect our integration.

As far as I can tell, we should be able to migrate almost all API usage to the PSA counterparts (we also have a 'lite embed' option which only bundle a very restricted set of files, but we can probably handle that ad hoc).

One thing I noticed though, is:

base64.h -> internalize (used by PEM only)

As far as I can tell, there's no PSA counterpart to do base64, and we do use the mbedTLS version for doing base64.

While we could pull the internal header when bundling mbedTLS in the engine, we wouldn't be able to use it when mbedTLS is provided by the OS as a shared library (and would likely have to still bundle that source file directly in such scenario).

Is there any chance that base64 functions could be left exposed, or maybe exposed via PSA crypto?

[gilles-peskine-arm]

@Faless Thank you very much for the feedback! I'm glad that you found a way to migrate most of your code.

About the “lite embed” option, please note that if you rely on low-level crypto APIs, many of those will probably be still be around in 4.0, but with declarations only in private headers, and they're very likely to change without notice in subsequent releases.

About Base64, we've reevaluated whether PEM and Base64 should be public and we've decided to keep both public. (I'll update the document in that pull request shortly.)