Many instances of Rejected connection for thumbprint mismatch in Diagnostics

[kendaleiv]

When using AzureWebFarm.OctopusDeploy 1.0.1.16 and Octopus.Client 2.3.1, Octopus Deploy 2.1.3.1223 diagnostics is displaying many instances of:

Rejecting connection: the client at the_ip:the_port provided a certificate with thumbprint [redacted], while the server trusts [redacted] and [redacted] for polling endpoints.

@robdmoore suggested I make an issue to get this sorted. The tentacles wire up themselves to the server as polling tentacles.

[robdmoore]

Are there any interesting exceptions logged in the LogEntity table in table storage for your diagnostics storage?

Also, to confirm - from memory you said the machines did show up in the Environments tab on your Octopus server it's just that the initial deployments failed yeah?

[kendaleiv]

Is this the problem? Sorry for not-fixed-width font uglyness, triple backtick puts the content on a single line.

---

System.Exception: Unable to connect to the Octopus Deploy server. See the inner exception for details. ---> System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure. at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result) at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.ConnectStream.WriteHeaders(Boolean async) --- End of inner exception stack trace --- at System.Net.HttpWebRequest.GetResponse() at Octopus.Client.OctopusClient.DispatchRequest[TResponseResource](OctopusRequest request, Boolean readResponse) at Octopus.Client.OctopusClient.EstablishSession() --- End of inner exception stack trace --- at Octopus.Client.OctopusClient.EstablishSession() at System.Lazy1.CreateValue() at System.Lazy1.LazyInitValue() at Octopus.Client.OctopusClient.get_RootDocument() at Octopus.Client.OctopusRepository.BasicRepository1.Paginate(Func2 getNextPage) at Octopus.Client.OctopusRepository.BasicRepository1.FindOne(Func2 search) at AzureWebFarm.OctopusDeploy.Infrastructure.OctopusDeploy.DeployAllCurrentReleasesToThisMachine()

[kendaleiv]

Instances do show in Environments.

[robdmoore]

So for context - that AzureWebFarm.OctopusDeploy.Infrastructure.OctopusDeploy.DeployAllCurrentReleasesToThisMachine call is here: https://github.com/MRCollective/AzureWebFarm.OctopusDeploy/blob/master/AzureWebFarm.OctopusDeploy/Infrastructure/OctopusDeploy.cs#L56

And that is called after https://github.com/MRCollective/AzureWebFarm.OctopusDeploy/blob/master/AzureWebFarm.OctopusDeploy/Infrastructure/OctopusDeploy.cs#L29 is called.

You can see the originating calls here: https://github.com/MRCollective/AzureWebFarm.OctopusDeploy/blob/master/AzureWebFarm.OctopusDeploy/WebFarmRole.cs#L40 (when the role starts that method gets called).

Hence that means that when the tentacle communicates with the server it works, but when the REST API is used from OctopusClient it's failing with that HTTPS message.

I'm not sure why, but I imagine @nblumhardt or @PaulStovell will be able to jump in with some ideas.

[kendaleiv]

When I access the web UI in the browser Chrome warns about an untrusted certificate, if that factors in here. I didn't see a way to resolve this in the server side management.

[robdmoore]

Ah ding ding ding. I suspect that is the problem.

You should be able to use ServicePointManager to ignore the certificate validation error: http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.servercertificatevalidationcallback.aspx

You can add this call to the constructor of your WebRole.cs file in your web project of your web farm.

You can short circuit to always return true, but that's a bit of a security hole for your infrastructure so I'd say you should check the thumbprint of the presented certificate is one you expect (I.e. the one you installed on your octopus server).

Obviously, ideally you would just get a real certificate so you don't have to introduce a potential security hole.

[robdmoore]

Sorry, could not would for that last paragraph

[kendaleiv]

Seems to be the issue. Seeing how most (arguably all) should be using https, it might be nice if this was clearly called out somewhere in documentation.

Thanks for your help!

[robdmoore]

Awesome.

On the AzureWebFarm side ill change the readme so there is A clear instruction about this.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you think you've found a related issue, please contact our support team so we can triage your issue, and make sure it's handled appropriately.