Troubleshooting SSL related issues with IIS

Troubleshooting SSL related issues with IIS

  • Comments 39

In this post I am going to discuss troubleshooting SSL server certificate related issues in IIS. Please remember I am no SSL expert...I am an IIS Asp.Net guy and will discuss issues with regard to SSL with IIS.

We have seen a lot many times that the issues could have been resolved by our customers if they were aware of some targeted troubleshooting techniques.

Here I will be walking through the steps which you can follow yourself before requiring to call Microsoft PSS.

Problem Description

The most common error associated with SSL not working for an IIS website is "The page cannot be displayed" or "Cannot find server or DNS Error".

Troubleshooting steps to be followed:

First thing first: You notice the above error when you browse to your website over SSL (HTTPS). Do you also see the same behavior when you access the website over HTTP? If yes then there is a different problem you need to address first. SSL doesn't come into picture when we access a site over HTTP.
So if you are facing the problem in both the scenarios ensure you first troubleshoot why the request is not reaching the IIS server...maybe a DNS configuration etc.

My topic today is SSL with IIS so I won't be focusing on HTTP.

We have the SSL port listed in IIS mmc as shown below. By default it's 443. You can configure it to listen on a different port.

image

Now, also check whether the issue is happening for all the users, i.e. External and internal users to the network. If the issue is happening only for let's say external users check whether the requests are coming over a firewall like ISA etc. At times Firewalls may block the SSL port. You may want to check the firewall settings to unblock the SSL port. Also firewalls like ISA requires certificate to be published on it as well apart from IIS server. Ensure that the certificate that we have installed on ISA is valid. In case of ISA, if you have renewed the cert on IIS server but not on ISA users will still see the old certificate and not the new certificate. Ensure that you replace the old certificate on ISA with the new certificate. Also you may want to check if the security gateway (if any) such as NFUSE (Citrix) controllers is still referencing the old certificate.

Also at times if you see a wrong certificate being displayed for your Web Site, ensure that you do not have kernel-mode SSL enabled on your web server.Although this improves the performance but I recommend to have it disabled when you have multiple SSL enabled-websites on your server. Kernel-mode SSL is only recommended if you have just one site, mostly static content and no client certificates. In all other cases you should use User-mode SSL.

Please find this link to enable/disable kernel-mode SSL. If it is set to 1 it means Kernel-mode SSL is enabled, if absent or set to 0 it means disabled and user-mode SSL will be used. Ensure that you stop and restart http service as shown below after doing the changes. No reboot is required for the above change.

>net stop http

>net start http

>net start w3svc

[There has been few concerns from people as to why this is suggested. I do not have a proper answer to it at this point, but I have seen such issues in some support incidents where disabling Kernel mode SSL had resolved the above problem]

------------------------------------------------------------------------------------------------------------------------------------------------------------------

Now once you have isolated the issue to be only with HTTPS and all users let's proceed with the following steps:

Run netstat -an (or fport.exe) for IIS 5 (and netstat -ano for IIS 6) and verify whether IIS is listening on the SSL port.

When you start Internet Information Services (IIS) 6.0 on Microsoft Windows Server 2003, IIS binds to all IP addresses on the server, not just the IP addresses that are assigned to Web sites. This is because of socket pooling enabled for IIS by default.

On IIS 6.0, if the website is listening on "All unassigned" IP address or some specific IP address (like 192.134.123.209 etc) and SSL port 443, verify that Local address entry is 0.0.0.0:443 in netstat output.

> netstat -ano

Active Connections

  Proto  Local Address          Foreign Address        State           PID
  TCP    0.0.0.0:21             0.0.0.0:0              LISTENING       2396
  TCP    0.0.0.0:25             0.0.0.0:0              LISTENING       2396
  TCP    0.0.0.0:80           0.0.0.0:0            LISTENING       4
  TCP    0.0.0.0:81             0.0.0.0:0              LISTENING       4
  TCP    0.0.0.0:83             0.0.0.0:0              LISTENING       4
    ...................

  TCP   0.0.0.0:443          0.0.0.0:0             LISTENING       4
  TCP    0.0.0.0:445            0.0.0.0:0              LISTENING       4
  TCP    0.0.0.0:446            0.0.0.0:0              LISTENING       4

Here the PID should show you which process is listening on that port.
Now let's say I have configured my website to run on a specific SSL port and that port is being used by a different process, then website will not start up.

Also you won't see the netstat output showing anything like this depending upon the port being used:

TCP    0.0.0.0:80             0.0.0.0:0              LISTENING       4

TCP    0.0.0.0:443           0.0.0.0:0              LISTENING       4

You can run netstat -ano again and see the PID corresponding to the process which is listening on that port. Stopping or disabling that service/process should allow Websites to be up and listening on the desired port. For IIS 5.x you can run Fport.exe to find out which process is listening on a specific port.

The first thing that I check for (strictly my personal preference) when we have issues with SSL is MACHINEKEYS. Yes, this is a very important (I prefer to call strategic) part of your checklist while dealing with SSL issues. In my personal experience it has resolved a lot of issues for our customers. By default machinekeys folder should have permission for Administrator, System and Everyone according to this KB278381. Ensure that we inherit the above permission for all child objects under the machinekeys folder, that way all the machine keys have the necessary permissions.

This is done by clicking the "Advanced" button on the security settings page, Clicking the check box "Replace permissions entries on all child objects...." and
Apply. This will propagate the settings to the contents in the folder.

Another reason for the above error ('The page cannot be displayed') can be if you do not have a private key corresponding to the SSL server certificate. You can check that by opening the IIS mmc -> <<YourWebSite>> ->Properties ->Directory Security ->Secure Communications ->View certificate ->General -> "You have a private key that corresponds to this certificate" as shown below.

image

If you do not see the above then SSL won't work for your website.
You can try running the following command to recover the private key from the certificate thumbprint.

To associate the certificate with its private key, we run the following command:

> certutil -repairstore my "c9 66 67 68 49 97 88 7f 05 b8 89 b7 b3 f7 37 c8 c4 da 5f 16"

Note: "c9 66 67 68 49 97 88 7f 05 b8 89 b7 b3 f7 37 c8 c4 da 5f 16" is the Thumbprint of the missing certificate. You can see this if you double-click on the .cer file, choose the Details tab and select Thumbprint.

image

Make sure you use the 2003 version of certutil.exe (with the associated certadm.dll and certcli.dll), or you will not have access to the repairstore command.

If the above command does not get back the private key or for some reason it still does not work you may prefer to get a new certificate from the CA.

One of the best tools that Microsoft has come up with is SSL Diagnostics to troubleshoot SSL related issues. Just download it to the IIS server and run it and if there is an error due to configuration of IIS or certificate it will show up. You can also use this tool to issue a test certificate for your website to check whether the problem was occurring because of your server certificate or some other IIS configuration related issues. Things that can be shown quickly through this tool includes on "no private key", "SSL port being used by a different process", "machine keys not having enough permission", "IP address conflicts" etc.

Another scenario can be wherein if you view the certificate, it will state that "You have a private key that corresponds to this certificate," and if you run SSL Diagnostics, you'll get the error as below:

#WARNING: You have a private key that corresponds to this certificate but CryptAcquireCertificatePrivateKey failed

When you initially go through the IIS Certificate Wizard to create a new certificate, a file is added to the MachineKeys folder. This is the private key.

If you add permissions to the new file in the MachineKeys folder before you process the pending request in the IIS certificate wizard, the certificate will work properly. If you've already gone past that point and got a new certificate, then necessary permissions should be added to the MachineKeys folder and a new certificate request must again be submitted.

Now consider a scenario wherein you are able to access the site through SSL Diagnostics tool (through 'Simulate SSL Handshake') for the same certificate but not through IE. What I mean is your SSL handshake gives no error etc in SSL Diagnostics although when you access the website through IE over SSL it still fails. In that case there could be an IE setting that you may want to check for such a behavior. Check this out KB811834.

Now one last scenario. At times we see customers coming up with requests wherein they want to use the same server certificate for multiple websites.

This is not possible. You cannot use the same certificate for multiple websites on the same server. Check this KB187504.

Now there is a workaround in IIS 6.0 Windows 2003 SP1, wherein you can use Wildcard certificates to get a similar functionality. Here is the link.

If you have a Load balanced or clustered environment you can export the same certificate to all the servers and install them on the related websites. It should work fine.

Hope this helps....let me know if there are any questions around the topic.

Cheers!!!

Leave a Comment
  • Please add 2 and 7 and type the answer here:
  • Post
  • Your article is very good, but my problem is little bit different. We use HTTPS for downloading. The problem is the HTTPS works intermittently. When the user is downloading the file from externally after some time the download breaks with error “Connection with the server was reset” and then if the user tries to download again the links doesn’t work “Page Cannot be displayed”. I verified in “httpsys” log file but there were no error logs.

    I also verified with SSL Diagnostic tool, every thing seems fine. I see one warning error with “Basic constraint” value is “Subject Type=End Entity Path Length Constraint=None” in the certificate. Is there some wrong with this parameter or some thing else?

    Your help is appreciated.

    Srividya_manam@yahoo.com

  • Recently a colleague of mine was working on a customer's case which was a Critical level incident. High

  • Just wanted to say thanks!  My SSL private key didn't import correctly, so nothing was working even though it all appeared correct in the IIS UI.  Doing the the "certutil -repairstore my" worked like a charm.

  • Hi, maybe you can make clear my situation. From client's part all work very good with https and trouble is that iis must verify authentication of client from centre of authentication (OpenLdap). It's sharepoint with forme based authentication (FBA). Trought 389 (witout SSL) all work fine, when i change from "useSSL=false" to "useSSL=true" I receive error. When I verify connection with lde.exe from IIS server to authentication ceneter all work fine with LDAPS (secured LDAP). How can I achieve same effect in sharepoint?

    Best regards!!!

    Kalvador

  • Hi Kalvador,

    Unfortunately i don't have expertise on SharePoint, so not sure if i can be of much help. BTW, is this Sharepoint site accessed with IE as a client that is having the problem over SSL? If yes can you try accessing this over SSL with IE and let me know if you get a security alert or not. If you get one that could be one of the probable reasons. What exactly is the error you get...can you share the error message shown or logged in somewhere...

  • Hi, Saurabh Singh, thanks for quick reply. Actually that error is between sharepoint server and OpenLDAP server (on unix like).  That is Microsoft's omission. The membership provider (ASP.NET) "add name="LdapMembership" type="Microsoft.Office.Server.Security.LDAPMembershipProvider, &#xD;&#xA;    Microsoft.Office.Server, Version=12.0.0.0, Culture=neutral, &#xD;&#xA;    PublicKeyToken=71E9BCE111E9429C" server="name.domain.com" port="636" useSSL="true" useDNAttribute="false" userNameAttribute="uid" userContainer="ou=people,dc=domain,dc=com" userObjectClass="person" userFilter="(ObjectClass=person)" scope="Subtree" otherRequiredUserAttributes="uid,cn" />" maybe  don't support SSL. I'm trying find solution for my occasion. Many people talk that only special connector (membership LDAP provider) can be for that solution. Any suggestions will be appreciated.

    Best rerards!!!

    Kalvador

  • HI again.... Singh, now it's work. Thanks for any support :) It was the problem with certificates

    BB

  • Hi BB, Great to know the issue is resolved :-)

  • Hi Saurabh;

    I don't know if this fits in here. We currently hosting multiple websites into two win2k3 IIS6.0 using host header, we have a load balancer on top of  it. Our problem on is that we have to implement SSL cert into 1 website (website B), making it complicated we already have 1 website (website A) running SSL. website A & B has different domain.

    From Microsoft they ask us to use a different IP for website B, but we could not have it as a solution because in time there would be other sites from IIS that would be using SSL. Another solution was to use other port, which is also not appicable on our part.

    Have you encounter such scenario or may know someone who encounter such?

    please help.

    thanks.

    James

    djg_magdangal@yahoo.com

  • Hi James,

    If that is the case wherein you cannot use different IP addresses or ports I feel we are very limited in implementing SSL for multiple sites. Also as you mentioned these sites are having different end domains, wildcard won't be an option as well.

  • Hi Saurabh;

    Thanks for your immediate response. May I know if you have anyone that could help us out on this problem?as for now we have tried asking microsoft for solutions and yet they only provide us with using different IP or using other port.

    Anyone in here would know or encounter same scenario?

  • great post!

    I do not think this is correct: "You cannot use the same certificate for multiple websites on the same server."

    So long as you are using a separate IP or port you can reuse an SSL certificate for a separate IIS site.

  • Hi James, this is a design limitation, so I doubt anyone can help us here.

    Moshe, yes I should have been more specific, thanks I will add a note.

  • Thanks a lot for crunching all the certificate related issues in jst one page. this helps a ton!!

    Thank you

  • Thanks you, excellent article.

Page 2 of 3 (39 items) 123