Crawling and Kerberos the saga continues

"OK, I’ve confirmed this with a bit of independent testing—Search can successfully crawl web applications configured to use Kerberos Authentication *as long as the URL corresponding to the web application being crawled does not contain a port number*.

Listed immediately below is an event viewer log entry from a crawl of a Kerberos authenticated web application whose IIS binding is to standard TCP port 80.

Event Type:        Success Audit

Event Source:    Security

Event Category:                Logon/Logoff

Event ID:              540

Date:                     1/9/2008

Time:                     11:24:53 AM

User:                     DOMAIN\CrawlAccount

Computer:          KERBTEST1

Description:

Successful Network Logon:

                User Name:        CrawlAccount

                Domain:                               DOMAIN

                Logon ID:                             (0x0,0xA917D1)

                Logon Type:       3

                Logon Process:  Kerberos

                Authentication Package:               Kerberos

                Workstation Name:       

                Logon GUID:      {XXXXXXGUIDXXXXX}

                Caller User Name:           -

                Caller Domain:   -

                Caller Logon ID: -

                Caller Process ID: -

                Transited Services: -

                Source Network Address:            IP.AD.DR.ESS

                Source Port:       2213

I created a new web application using Kerberos Authentication and bound it to non-standard port 50000, and confirmed that I could access it via IE using Kerberos Authentication.  Here’s the event viewer log for that:

Event Type:        Success Audit

Event Source:    Security

Event Category:                Logon/Logoff

Event ID:              540

Date:                     1/9/2008

Time:                     3:11:12 PM

User:                     DOMAIN\User

Computer:          KERBTEST1

Description:

Successful Network Logon:

                User Name:        User

                Domain:                               DOMAIN

                Logon ID:                             (0x0,0xEAD974)

                Logon Type:       3

                Logon Process:  Kerberos

                Authentication Package:               Kerberos

                Workstation Name:       

                Logon GUID:      {GUID}

                Caller User Name:           -

                Caller Domain:   -

                Caller Logon ID: -

                Caller Process ID: -

                Transited Services: -

                Source Network Address:            IP.AD.DR.ESS

                Source Port:       3126

Next up, I ran a crawl while taking a netmon sniff between the index machine and the WFE.  Below, you will find the crawl log indicating the access denied and a relevant frame fragment from the sniff proving that the client (the .NET Framework in this case) is not constructing an SPN with the correct port number.

Search crawl log:

Crawled Content Status - 4 Items: 0 Successes, 4 Errors, 0 Warnings

1 - 4

                           URL      Last Content Source      Last Time Crawled

http://kerbtest1.DOMAIN.com...

Access is denied. Check that the Default Content Access Account has access to this content, or add a crawl rule to crawl this content. (The item was deleted because it was either not found or the crawler was denied access to it.)     Local Office SharePoint Server sites 1/9/2008 3:34 PM         

http://kerbtest1.DOMAIN.com...

Access is denied. Check that the Default Content Access Account has access to this content, or add a crawl rule to crawl this content. (The item was deleted because it was either not found or the crawler was denied access to it.)     Local Office SharePoint Server sites 1/9/2008 3:34 PM         

sps3://kerbtest1.DOMAIN.com...

Access is denied. Check that the Default Content Access Account has access to this content, or add a crawl rule to crawl this content. (The item was deleted because it was either not found or the crawler was denied access to it.)     Local Office SharePoint Server sites 1/9/2008 3:34 PM         

http://kerbtest1.DOMAIN.com...

Access is denied. Check that the Default Content Access Account has access to this content, or add a crawl rule to crawl this content. (The item was deleted because it was either not found or the crawler was denied access to it.)     Local Office SharePoint Server sites 1/9/2008 3:34 PM         

Netmon sniff frame fragment:

Kerberos: TGS Request Realm: DOMAIN.COM Sname: HTTP/kerbtest1.DOMAIN.com

This frame fragment came from the indexing machine, and stipulates the Service Principal Name in the Sname: value.  Note that the Sname: value is HTTP/kerbtest1.DOMAIN.comm, with no port number.  Since the web application is hosted in an IIS Virtual Server that is bound to port 50000, and since the SPN that is in the AD contains the port number (perfectly valid, technically accurate and meaningful),  this SPN in this request invalid.  Since it is the .NET Framework that is acting as the client in this context, it is the .NET Framework that is not properly constructing the SPN.

So, Search can crawl web applications that use Kerberos authentication after all, but there is a caveat: Those web applications must not use a non-standard port number.  They must use either TCP port 80 or SSL port 443.

Please let me know if there are any questions.

Thanks,

Mark"