[.NET Framework] SSL handshake randomly fails when using client certificate

[NitinKMathur]

Hi,

I have a requirement to use HttpClient to call a couple of APIs from a server. The SSL handshake requires a client certificate. I have the client certificate as X509Certificate object and the private key. Both of these objects are in memory and not in disc. I need to call the API concurrently for many users. With low concurrency (< 10) everything is working fine, but as I increase concurrency (say 50) the SSL handshake starts failing with error:

The underlying connection was closed: An unexpected error occurred on a send

This is the function to create handler that is used in HttpClient constructor.

Original certificate was in X509Certificate format, I generated PFX/PKCS12/X509Certificate2 object using X509Certificate and Private key.

        public static WebRequestHandler GetHttpClientHandler(X509Certificate2 clientCertificate = null)
        {
            var handler = new WebRequestHandler();
            handler.AllowAutoRedirect = false;
            handler.UseCookies = false;
            handler.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
            if (clientCertificate != null)
            {
                handler.ClientCertificateOptions = ClientCertificateOption.Manual;
                handler.ClientCertificates.Add(clientCertificate);
            }
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
            return handler;
        }

Any idea what could be issue here?

[davidsh]

Is this using .NET Core? Or .NET Framework?

[NitinKMathur]

@davidsh it's .NET framework.

[davidsh]

cc: @karelz

@NitinKMathur Do you have a small repro you can post here? Otherwise, it is not possible to really figure out this.

[NitinKMathur]

@davidsh let me isolate the code and post. Btw, when this error is expected to occur?

[karelz]

I don't think such error is expected. Without minimal repro, we won't know.
BTW: .NET Framework issues are not tracked on GitHub - see our main page for info how to report them: https://github.com/dotnet/corefx#how-to-engage-contribute-and-provide-feedback
However, it is the same set of people, so let's first narrow down the repro here and then if it is a bug in .NET Framework, we can move it later (GH is easier for back-and-forth communications).

[NitinKMathur]

@karelz @davidsh, thanks for your time to look into this issue. I tried reproducing the issue with same client certificate as i can't simulate different client certificate for each session. With same certificate the issue doesn't happen. Is there any other way to look into this? Any logs/errors/exceptions that may help?

In my case I am getting different client certificate from server. Initial requests are without client certificate, once certificate is delivered client opens another connection on a different port using client cert. Issue is seen when client switches the connection from port1 (without client cert) to port2 (with client cert).

[jhudsoncedaron]

Believe specific error messages. Is it most likely you have too many connections open for the server or its firewall to handle.

[karelz]

I agree with @jhudsoncedaron. Your situation is suspicious - try to look at network traffic to find out if the connections are indeed closed outside of your client. Use network packet capture tools like Wireshark.

[NitinKMathur]

Hi @karelz @jhudsoncedaron @davidsh I figured out the issue. Sorry to bother you all. The issue was not related to HttpClient lib. The client certificate was not being generated properly under high concurrency due to application issue. So, you may close this bug. Thanks again all for your time 👍

[karelz]

Happy to hear you were able to root-cause it.
Yet another reminder how trying to get minimal repro is valuable :)