mbedtls_mpi_sub_abs: fix buffer overflow in error case

Fix a buffer overflow in mbedtls_mpi_sub_abs() when calculating
|A| - |B| where |B| is larger than |A| and has more limbs (so the
function should return MBEDTLS_ERR_MPI_NEGATIVE_VALUE).

Fix Mbed-TLS#4042

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>

ChangeLog.d/mpi_sub_abs.txt

@@ -0,0 +1,7 @@
+Security
+   * Fix a buffer overflow in mbedtls_mpi_sub_abs() when calculating
+     |A| - |B| where |B| is larger than |A| and has more limbs (so the
+     function should return MBEDTLS_ERR_MPI_NEGATIVE_VALUE). Only
+     applications calling mbedtls_mpi_sub_abs() directly are affected:
+     all calls inside the library were safe since this function is
+     only called with |A| >= |B|. Reported by Guido Vranken in #4042.

library/bignum.c

@@ -1354,6 +1354,12 @@ int mbedtls_mpi_sub_abs( mbedtls_mpi *X, const mbedtls_mpi *A, const mbedtls_mpi
     for( n = B->n; n > 0; n-- )
         if( B->p[n - 1] != 0 )
             break;
+    if( n > A->n )
+    {
+        /* B >= (2^ciL)^n > A */
+        ret = MBEDTLS_ERR_MPI_NEGATIVE_VALUE;
+        goto cleanup;
+    }
 
     carry = mpi_sub_hlp( n, X->p, B->p );
     if( carry != 0 )