Deadlock in mbedtls_x509write_crt_pem()

[chmorgan]

mbedtls 2.12, Fedora 28 linux (with mbedtls package from rawhide). Saw the same behavior in the mbedtls 2.9.0 package.

It looks like the issue is that the mutex isn't recursive so the earlier lock results in a deadlock.

If someone wants to confirm I can submit a pull request. This issue is holding up some usage of mbedtls on Linux for me so I'd like it fixed as soon as possible.

Backtrace:

#0 0x00007ffff78af7fd in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007ffff78a8d94 in pthread_mutex_lock () from /lib64/libpthread.so.0
#2 0x00007ffff7f79008 in threading_mutex_lock_pthread () from /lib64/libmbedcrypto.so.3
#3 0x00007ffff7f57258 in mbedtls_ctr_drbg_random () from /lib64/libmbedcrypto.so.3
#4 0x00007ffff7f503e4 in mbedtls_mpi_fill_random () from /lib64/libmbedcrypto.so.3
#5 0x00007ffff7f72548 in mbedtls_rsa_private () from /lib64/libmbedcrypto.so.3
#6 0x00007ffff7f73473 in mbedtls_rsa_rsassa_pkcs1_v15_sign () from /lib64/libmbedcrypto.so.3
#7 0x00007ffff7fa46e1 in mbedtls_x509write_crt_der () from /lib64/libmbedx509.so.0
#8 0x00007ffff7fa48fc in mbedtls_x509write_crt_pem () from /lib64/libmbedx509.so.0
#9 0x0000000000407408 in CertAndKey::WriteCertificateToBuffer (this=0x7ffff5c632c0,

#1:

int mbedtls_ctr_drbg_random( void *p_rng, unsigned char *output, size_t output_len )
{
int ret;
mbedtls_ctr_drbg_context *ctx = (mbedtls_ctr_drbg_context *) p_rng;

#if defined(MBEDTLS_THREADING_C)
if( ( ret = mbedtls_mutex_lock( &ctx->mutex ) ) != 0 ) <-------------------------- second lock
return( ret );
#endif

#5.

mbedtls_rsa_private()
...
if( rsa_check_context( ctx, 1 /* private key checks /,
f_rng != NULL / blinding y/n */ ) != 0 )
{
return( MBEDTLS_ERR_RSA_BAD_INPUT_DATA );
}

#if defined(MBEDTLS_THREADING_C)
if( ( ret = mbedtls_mutex_lock( &ctx->mutex ) ) != 0 ) <--------------------- first lock
return( ret );
#endif

/* MPI Initialization */
mbedtls_mpi_init( &T );

...

[chmorgan]

Alright, looks like a false alarm. Got screwed up by a set of include files that wasn't defining the pthread mutex conditional being included while the F28 mbedtls library is built with pthread mutex enabled.

[hanno-becker]

Thanks @chmorgan for the clarification - indeed I hadn't been able to confirm the issue so far since the RSA and CTR DRBG contexts have independent mutexes.

Closing this issue for now - feel free to reopen if you suspect there is a problem with the library even after resolving your local issues.

[chmorgan]

@hanno-arm I did do a pretty careful audit of a some code so there was some benefit, only a handful of hours on my part until I figured it out. Valgrind wasn't able to catch it either, either with its normal lack checking tool or helgrind.

I did find that the mutex is_valid check was a bit loose. Not having that probably took a few hours to figure out. Let me push a PR so you can see what I mean.

[chmorgan]

@hanno-arm here is the pr, #1934