Interesting issue occurred with the SSO service on one of my BizTalk machines.  I saw below error message in my Application event log on the SSO server while starting up the ENTSSO service. ENTSSO service would not start because of this. I saw below error message even if I did not have SQL server configured for SSL communication.

Log Name:     Application
Source:        ENTSSO
Event ID:      10503
Task Category: Enterprise Single Sign-On
Level:         Error
Keywords:      Classic
Computer:     ServerName
The SSO service failed to start.
 Error Code: 0x800710D9, Unable to read from or write to the database.

I also saw below specified error message in my System event log on the SSO server itself.

Log Name:      System
Source:       Schannel
Event ID:      36882
Task Category: None
Level:         Error
Computer:     ServerName
The certificate received from the remote server was issued by an untrusted certificate authority. Because of this, none of the data contained in the certificate can be validated. The SSL connection request has failed. The attached data contains the server certificate.

The above error shows that SSO service is trying to communicate to the SQL server over SSL protocol but, the certificate got from the SQL server is not valid. This is obviously due to fact that I did not have SQL server  configured for SSL communication.

Then, why does SSO service is trying to communicate with the SQL server over SSL?
Because we have informed SSO service to do so. There is a registry key under the SSO hive which specifies whether to use SSL protocol to communicate with the SQL server.
If this key is set to “Yes”, SSO would try to make SSL communication which was the case on my server. I changed the value to “No” and I was able to start the SSO service without any issues.
Note that this key does not get created by default. You would have to run the below command to create it under SSO hive.
ssoconfig -setSSL yes (Needs to be run from the SSO installation folder)

This reminded me that I had created the above key to test SQL SSL functionality on the BizTalk server but forgot to turn it off when I disabled SSL on the SQL end :(.

I hope this helps.

Written BY
Yakshit Gohel

Reviewed BY
Jainath V R

Microsoft GTSC India