I had a great follow-up question from my last post on Kerberos in SharePoint:

"We are using Kerberos for our MOSS based server and there is one page which renders a list based on the current user's membership groups. The issue is that once the user is removed from the group, he should not see the list contents. However if the Kerberos token is obtained prior to the removal from the group, the user continues to see the list until the ticket expires. In other words, the ticket has to be purged in order to get the latest changes from AD."

The old adage that for every "yin" there is a "yang" holds true here. The reason why Kerberos performs so much better than NTLM over long-running conversations between a client and server is that NTLM requires the server to ask a DC to validate the client's response to a challenge every time authentication is required, whereas with Kerberos the server never has to contact a DC to validate a client's ticket, which once obtained by a client is valid by default for 10 hours. (This number was chosen as it is slightly longer than a typical working day).

The trade-off here is that a ticket can't be invalidated by the KDC. Once a client has one, it's good until it's expiry date or until the logon session is terminated (i.e. the client logs off) at which point the ticket cache is cleared. Since group membership information is embedded in a ticket, this means you can't use group membership to quickly grant or deny client access to a resource. With NTLM, group membership information is returned every time the server authenticates the client (although even with NTLM you can run into group membership caching problems if tokens are cached by the server).

Software solutions sometimes work round this by allowing you to deny access based on a client identity - however this isn't always particularly manageable and isn't an option in SharePoint.

A more extreme option is to configure the lifetime of a ticket. In Windows this is a policy applied to the whole Kerberos realm (Domain) via group policy. I recommend taking great care changing this setting as it may affect adversely performance in your domain - but it is an option. The specifc location and settings in the GPO are described here.

In general, I rarely come across situations where it is actually a business necessity to immediately deny a client access to a resource. Since Kerberos is an intranet protocol if you have a situation where a security concern arises with a user then you can do a number of things - remote log-off or shut-down their machine, or send a heavies from security round to see them for example!

If you absolutely need to do this in SharePoint, then granting and removing access to SharePoint based on SharePoint group membership (rather than Windows group membership) is probably your best bet.

If you want to know more about Kerberos I suggest starting with Keith Brown's guide items 59-62 and if that doesn't satiate you then move on to this monster:

http://technet2.microsoft.com/windowsserver/en/library/4a1daa3e-b45c-44ea-a0b6-fe8910f92f281033.mspx?mfr=true

Good luck!

James.

Hi,

This definitely isn't the first blog post on this topic, and it certainly won't be the last - but hopefully it will bring some peace and understanding to those struggling to get Kerberos working in a live SharePoint deployment.

First off, I must credit Martin Kearn's excellent posts here that were invaluable in getting me started - and will be invaluable to you too.

Why Use Kerberos?

There are a number of good resources that aid understanding Kerberos - the best (imho) being from Keith Brown's .NET security guide, published on the web. If you want to know what Kerberos is, what SPNs are, what delegation is etc. go read items 59-63 of his book here:

http://www.pluralsight.com/wiki/default.aspx/Keith.GuideBook.HomePage

But to summarise the key points from a SharePoint perspective, if you want better authentication performance than NTLM, use Kerberos. If you want to pass your user's credentials through to back end resources (like for example accessing the database exposed through a Business Data Catalog) - then you'll need Kerberos delegation.

Crucial SPN Knowledge

Lets just define the parts of an SPN first - an SPN looks something like this ( a typical SPN for a SQL Server in this case ):

MSSqlSvc/bob.domain.local:1433

Where "MSSqlSvc" is the service class, "bob.domain.local" is the host and 1433 is the port. The port part (including the colon) is optional.

SPNs are basically arbitrary strings that applications "know" how to assemble. They are stored in AD in a multi-valued attribute called servicePrincipalName on user principals. (Multi-valued attributes are what they sound like, attributes that can have none or more values and are typically displayed as a comma-delimited list.)

SetSPN is the tool you should use to create SPNs. When you use SetSPN -A you are just setting this attribute. It's perfectly valid to do this with adsiedit.msc or any other AD editing tool - I don't recommend you do this because SetSPN does some validation to ensure the SPN follows syntax conventions that a direct edit would bypass - I'm just making the point that there is no magic going on and that SPNs are just strings. To set an SPN (with SetSPN or otherwise) you must have permission to write to ServicePrincipalName attribute on the target principal (and that is your least privilege requirement) - although it's a good idea to be able to read them as well.

When a client application wishes to authenticate to a server application with Kerberos, the client must acquire a ticket from the KDC ( = Key Distribution Center which is a role played by Domain Controllers in Windows). To do this, it must tell the KDC which SPN it wants a ticket for. The KDC will take this SPN and look for a principal that has a matching string in its servicePrincipalName attribute.

At the point a couple of things can go wrong: (1) The KDC is unable to find a match in the AD. (2) The KDC finds more than one match in the AD. In either case, you're stuffed and Kerberos authentication will fail. At this point most applications will silently fall back to NTLM authentication and everything will appear to work - unless you need delegation or you have demanded Kerberos authentication. Not many apps demand Kerberos, they usually "Negotiate" an authentication protocol. If you are trying to delegate, then what often happens is that the machine at the end of the second hop gets a NULL identity turning up (see Keith Brown's explanations linked to earlier).

So remember: you can not have two identical SPNs registed to different accounts in your forest. If you do this, neither will work.

So how does the client know which SPN to ask for? Well that's very interesting. It turns out the answer is largely "because it does". The service class and host + port parts of an SPN are completely arbitrary. The client must therefore "know" these things to correctly form an SPN. The one thing it doesn't know (or need to know) is the identity of the principal running the service - the KDC figures this out and passes back a ticket that only the principal against which the SPN is registered will be able to decrypt.

In the case of Internet Explorer, the SPN requested is formed by using a service class of HTTP and a host as specified in the URL, converted to an FQDN (fully qualified domain name) if possible.

Here's the rub: IE never appends the port number to the host name when contacting the KDC for a ticket - even if your server is on a port other than port 80! So for all those SharePoint sites you have on non-80 ports, if you've been specifying a port number in your setspn command, then you haven't been using Kerberos!

Because of the way IE works, its typical to register two SPNs for every web-site. One with the netbios Host name and one with the FQDN host name - this keeps all the bases covered. Just don't put in the port!

So if I ask for http://moss:10000/sites/test for example, then IE will pass an SPN of "HTTP/moss.sharepoint.local" to the KDC (assuming moss is resolved to a box in my sharepoint.local domain) or just "HTTP/moss" if it can't be resolved.

UPDATE: Thanks to Brad Turner for his research on this point - it turns out that there was a fix released for IE that enabled a special registry key setting that changes IE's behaviour so the port number IS included. However, I don't recommend you do this because it is a client side fix, and it's probably hard to guarantee all your clients will have this setting. Even if you use Group Policy to enforce a registry setting, I still don't like it. If you really want to know the details, drop me a line; but be aware I will raise my eyebrow at you!

If you are using host-headers on your website, then use those in your SPN. Remember it's whatever is in the address bar that IE will use to request the SPN. A host-header is set on the dialog you get from the "Advanced..." button on the "Web Site" tab of a web site in IIS 6.0 Manager. It allows IIS to have multiple web-sites on the same port and IIS looks at the address (host header) requested by the client to work out which website to direct the request to. For example, in my MOSS installation I have something like:

portal.sharepoint.local, mysite.sharepoint.local, ssp.sharepoint.local etc. all pointing to the same IP address. The host header can be specified in Central Admin when creating web applications.

You Probably Need Multiple Host Headers

This throws up an interesting point that in most cases this means you will need multiple host-headers in order to be able to use Kerberos in a least privilege environment. Every application pool in a least privilege environment has its own identity. This means that your web applications, which are sitting in different application pools, can't be differentiated by simply using different ports. If you did this, since IE ignores the port when forming its SPN request, the SPNs for each application pool identity would have to be the same - and you can't register the same SPN against multiple accounts! (Oh how I wish this would throw an error!) Thus, a different host header is required to get a Kerberos ticket for each application pool identity in which your SharePoint web applications reside.

How Do I Know Its Working?

Good question. The easiest way is to download Kerbtray.exe (don't worry this says Windows 2000 - it works on all versions). This tool lets you examine the Kerberos tickets for the logon session of the current interactive user only. This means that you can't do something like "Run As" to see the tickets of a service - because you'll be in a different logon session even though its the same principal. However, you can run this tool in a couple of useful places:

• Run it during initial set up of the farm. At this point the setup account connects to SQL server so you should see a ticket for your SQL box - MSSqlSvc/sql.sharepoint.local:1433 for example.
• Run it on any client machine connecting to a SharePoint web site. You should see a ticket for the web site in the list - HTTP/portal.sharepoint.local for example.

Also, on SQL Server 2005 you can run a query to check how clients are authenticating. You should see "KERBEROS" in the auth_scheme column:

SELECT login_name, program_name, host_name, auth_scheme
FROM sys.dm_exec_connections C INNER JOIN sys.dm_exec_sessions S
ON C.session_id  = S.session_id

Delegation

If you want to delegate then you need to do more than just create SPNs. You must allow the services that will delegate right to do so. This is described in Keith Brown's guide again. For SharePoint, the services that are most likely to need rights to delegate are the application pool accounts and the SSP service account. You almost certainly don't need to give the computer accounts delegation rights!

A common scenario is BDC delegation where you want your end-users to authenticate against a database directly when querying a BDC web-part for auditing purposes. In this case you just need the application pool hosting the site hosting the web part to have delegation rights - and that's it!

Hope that helps!

James

 ADDENDUM:

Since writing this article the infrastructure update has added some new functionality that includes custom SPNs for SSPs. It's covered in a new technet article and you can read all about it here: http://technet.microsoft.com/en-us/library/cc263449.aspx#section14

Passing a variable number of parameters to a stored procedure is a problem that’s been around and solved for a while – in fact there’s a good article on several approaches for passing parameters in a comma-delimited string here:

http://www.sommarskog.se/arrays-in-sql.html

Today I’d like to talk about an alternative approach made possible in SQL Server 2005 – passing parameters with the XML data type.

Let’s imagine that we have a table of employees with an EmployeeID. We want to select some arbitrary subset of this table by passing in a list of the EmployeeIDs of the desired employees.

The XML we are going to pass will look something like this:

<employees><employee id=”1” /><employee id=”3” />…..</employees>

This can be built up quite simply in .NET using an XmlTextWriter as the following code snippet shows:

StringBuilder xmlEmployeeListBuilder = new StringBuilder();

StringWriter stringWriter = new StringWriter(xmlEmployeeListBuilder);

XmlTextWriter xmlWriter = new XmlTextWriter(stringWriter);

xmlWriter.WriteStartDocument();

xmlWriter.WriteStartElement("employees");

… other code

// add an employee id to xml parameter string

// repeat this section as necessary

xmlWriter.WriteStartElement("employee");

xmlWriter.WriteAttributeString("id", <variable containing id>);

xmlWriter.WriteEndElement();

… other code

xmlWriter.WriteEndElement();

xmlWriter.WriteEndDocument();

xmlWriter.Close();

string xmlEmployeeList = xmlEmployeeListBuilder.ToString();

Now let’s look at the stored procedure in full and then break it down:

SET QUOTED_IDENTIFIER ON

GO

SET ANSI_NULLS ON

GO

CREATE PROCEDURE SelectSpecificEmployees

      @EmployeeList xml

AS

SET NOCOUNT ON;

CREATE TABLE #EmployeeList

(

      EmployeeId int

);

INSERT #EmployeeList

SELECT employee.value('.', 'int')

FROM @EmployeeList.nodes('/employees/employee/@id') T(employee);

SELECT Employees.EmployeeId, FirstName, LastName

FROM Employees INNER JOIN #EmployeeList ON Employees.EmployeeId = #EmployeeList.EmployeeId;

DROP TABLE #EmployeeList;

GO

The first thing to note is the two SET statements outside the stored procedure declaration. These SET options are different than all other set options in that whenever a stored procedure executes, it uses the settings for QUOTED_IDENTIFIER and ANSI_NULLS that were in place at the time the stored procedure was created!

Changing these settings inside a stored procedure has no effect and produces no errors, and neither does the setting at the database level have any effect on the stored procedure.

If you don’t have these options set as shown, then when you try and run this stored procedure you’ll get an error like:

“INSERT failed because the following SET options have incorrect settings: 'QUOTED_IDENTIFIER'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or query notifications and/or xml data type methods.”

You have been warned!

The next step is to create a temporary table to hold the employee ids parsed from the XML.

Now we insert the employee ids into the temporary table. To do this, we use the nodes() method of an XML data type. This executes an XQuery expression that returns a nodeset against the xml instance and places the results into a table. The table is scoped to the SQL statement in which it is declared.

In the example above, we declare a table called T with a column of employee, and insert the rows from table T into our temporary #EmployeeList table.

That’s all the hard work done – now we just join our #EmployeeList to the actual Employees table and return the matching employees.

Comparing this approach to the amount of code required to deal with comma-delimited strings, I think this is a neater, shorter solution to the problem. How about you?

I've been working with Windows Workflow Foundation for over a year - and was lucky enough to implement a project that ended up being the first world-wide enterprise system in production based on Windows Workflow.

At the time, I struggled to understand the concepts behind workflow before documentation was publicly available. Even now that the RTM is out there, the SDK documentation remains a reference text that necessarily stops short of a full exploration of the concepts.

Essential Windows Workflow Foundation by Dharma Shuka and Bob Schmidt carries on in the great tradition of texts like Don Box's Essential COM (the author of the foreword) and is simply a brilliantly concise discourse on the fundamentals of Windows Workflow. I am in awe of Dharma and Bob's ability to explain this technology which represents a paradigm shift in how we will architect our applications from here on out. The more I look, the more I see applications of the Windows Workflow Framework.

Get this book. Read it. You will not regret it.

Amazon.com:

http://www.amazon.com/Essential-Workflow-Foundation-Microsoft-Development/dp/0321399838/sr=8-1/qid=1165363738/ref=pd_bbs_sr_1/105-2963730-3310866?ie=UTF8&s=books

Amazon.co.uk:

http://www.amazon.co.uk/Essential-Workflow-Foundation-Microsoft-Development/dp/0321399838/sr=8-1/qid=1165363788/ref=pd_ka_1/202-2200016-7087067?ie=UTF8&s=books 

XMLSerializer provides a simple means of serializing and deserializing object graphs to and from XML. However, simplicity brings a price, and there are limitations that I have seen people come up against.

The most common of these is that XMLSerializer will only work on public properties and needs setters to be able to deserialized, and this can create awkward design decisions - such as exposing properties that you really don't want to just to enable serialization. The second common obstacle is that there are no hooks to detect when a class is being constructed during deserialization - this is sometimes necessary when special work needs to be done (or not done) during deserialization.

It is possible to customize serialization by implementing IXmlSerializable - problematic (and undocumented) in .NET 1.1 but much improved in .NET 2.0. However this can create a lot of extra work and maintenance that is a real pain.

With the arrival of Windows Communication Foundation in .NET FX 3.0, an alternative is available that you might consider. The DataContractSerializer (DCS - renamed from XmlFormatter in the betas) provides an Xml Serializer than can be used independantly of WCF.

The key differences are:

• Hooks are providing for refining control of (de)serialization - particularly useful for handling versioning issues. By applying any of four special attributes to methods in the target class you can have them called either before or after (de)serialization.
• The serializer is "opt-in" rather than "opt-out" - which makes (imho) for much cleaner code. In XmlSerializer you could use XmlIgnore to have the serializer ignore certain properties. With the DCS you explicitly mark what you want to serialize.
• Finally, ANY field or property can be serialized - even if they are marked private.

Check out DataContractSerializer in the SDK!

This book is destined to become a classic. Benefiting from the main author's close relationship with the product team, this text provides a distilled insight into the machinations of SQL Server that are hard to garner from elsewhere. Managing to be both a reference text and a step-by-step tutorial, this book will furnish you with all tools you need to effectively author and tune T-SQL to highest standards. I particularly commend the chapter detailing a tuning methodology.

Having said all this, bewarned that this text is not for the novice or beginner. While it is excellently written, it is an advanced text pitched at those with plenty of prior T-SQL experience, and will require careful study. That being said, for those who put in the effort, the rewards will be plentiful.

My only criticism is that a book on T-SQL this good didn't come sooner. A fanatastic work I will be returning to again and again.

I've just picked up the companion sequel (no pun intended!) T-SQL Programming - looks to be just as compelling!

I've been taking a look at a new feature in SQL Server 2005 called "Common Table Expressions" (hereafter CTEs). This is an extra-ordinally powerful extenstion to Transact SQL that removes the need for managing temporary tables in many common scenarios.

In this example code I will set up a typical Employee table that is self-referencing; each Employee has a manager also in the Employee table. I've tested this code on IDW-15 (June CTP) only, but its been in the builds for a while so should work on the April CTP with no problems.

Here is the setup code:

You get back a list of employees and their sales total.

Big deal, we could have just done that in one simple query anyway. So lets look at something more interesting... recursive CTEs! Here is a sample that gets back a list of an employee and all employees that reports to him and all employees that report to them... etc. Basically the subtree of the orgtree rooted at a particular employee.

Awesome! Now let's look at a really cool example... this query uses aggregates to return the total sales of each employee and all the employees below them...

I think you'll agree than CTEs are going to be very popular indeed!

I've been looking at the new encryption functionality in SQL Server 2005. Here's some sql that executes on IDW15 - June CTP (most of it should work on IDW14 - April CTP apart from DecryptByKeyAutoCert I think...

This sample code sets up a database and a table that will contain some data (national insurance numbers) we want to store encrypted. It shows how to set up a master key, certicate and encryption key and how to use these to insert and select data. It also demonstrates how to set up a view that takes away a lot of the pain.

It does expose that in IDW15 at least you need to give CONTROL level permission to a certificate to make this work  - which I hope gets fixed in the next release... with CONTROL you can do anything to a certificate, including dropping and altering it and removing the private key!!! The last of these works even if the cert is used to decrypt keys in the database - which presents an untenable security problem. What we want is just to require REFERENCES level permission (I think).

Other than this little niggle, the functionality is awesome! I have been doing encryption/decryption in the middle tier: it's great to be able to move this to the database - particularly as this makes it much easier to create reports in report server that otherwise had to use a custom assembly to decrypt data.

Copy the below into management studio and walk through it.

create database SecurityDemo
go
use SecurityDemo
go

-- Create a master key - this is used as the root of the encryption
-- hierarchy: all keys and certs are encrypted with this
-- it is scoped to the database
create master key encryption by password = 'pass@word1'

-- Now we will create a simple table with a column that will store
-- national security numbers in encrypted form

create table people (
 id int identity constraint pk_people_id primary key clustered,
 firstname nvarchar(50),
 lastname nvarchar(50),
 encrypted_ninumber varbinary(128) )
go

-- Now lets create a certicate - we will use this to encrypt a key
-- this will give a warning about an invalid start date, but
-- we can ignore this since sql doesn't care about cert dates
create certificate ni_cert with subject = 'NI Certificate'
go

-- Now create a symmetric key to encrypt/decrypt the data
create symmetric key ni_key with algorithm = aes_256
encryption by certificate ni_cert
go

-- Now we will create a procedure to insert and encrypt the data
create procedure dbo.insert_people (
 @firstname nvarchar(50),
 @lastname nvarchar(50),
 @ninumber nchar(9)
) as

-- open the symmetric key
open symmetric key ni_key decryption by certificate ni_cert

-- insert and encrypt the data
insert people(firstname, lastname, encrypted_ninumber)
 values(
  @firstname,
  @lastname,
  EncryptByKey(Key_GUID('ni_key'), @ninumber)
)

-- close the key now
close symmetric key ni_key
go

-- now lets insert some data
insert_people 'James', 'World', 'JZ123488B'
go

-- looking at this data, you can see its nice and encrypted
select * from people
go

-- ok, so lets create a procedure to decrypt this
create procedure dbo.select_person (
 @id int
) as

open symmetric key ni_key decryption by certificate ni_cert

select
 id,
 firstname,
 lastname,
 convert(nchar(9),DecryptByKey(encrypted_ninumber)) as ninumber
from
 people

close symmetric key ni_key
go

-- and try it out
exec select_person 1
go

-- how about if we want another use to call this spoc?
create login bob with password = 'pass@word1'
go
create user bob for login bob
go

-- this won't work as bob has no permissions
execute as user='bob' -- lets us run in the context of bob
exec select_person 1
revert -- puts us back to our dbo context
go

-- let's give him some and then the above should work
grant execute on dbo.select_person to bob
grant references on symmetric key::ni_key to bob
-- eek!!! currently need to grant *control* permission to bob
-- in IDW15... hope this improves!
grant control on certificate::ni_cert to bob

-- now it works
execute as user='bob' -- lets us run in the context of bob
exec select_person 1
revert -- puts us back to our dbo context
go

-- lets take away bob's permissions
revoke execute on dbo.select_person to bob
revoke references on symmetric key::ni_key to bob
revoke control on certificate::ni_cert to bob
go

-- a new feature lets us create a view that automatically decrypts
-- the data
create view view_people as
 select
  id,
  firstname,
  lastname,
  convert(
   nchar(9),
   DecryptByKeyAutoCert(cert_id('ni_cert'),
   null, encrypted_ninumber)
  ) AS ninumber
 from
  people
go

select * from view_people
go

-- can bob look at this view? clearly he needs permission on the view
grant select on view_people to bob
go

-- lets try...
execute as user='bob'
select * from view_people
revert
go

-- hmmm! we got null - this is what anyone gets that doesn't
-- have an open key when they try to read encrypted data
-- lets grant him permissions - control of the cert and references on the key
grant control on certificate::ni_cert to bob
grant references on symmetric key::ni_key to bob
go

-- lets try again
execute as user='bob'
select * from view_people
revert
go

-- this works! and best of all we didn't have to manage
-- opening the key
-- if they fix the level of permissions required on the cert
-- then this will be an awesome way to manage encrypted data

-- current weakness of needing CONTROL permission for bob on the cert:

-- we can do this and remove the ability to decrypt our data!
execute as user='bob'
alter certificate ni_cert remove private key
revert
go

-- certs can be backed up and restored, but this still sucks

-- other than that, this is an awesome feature set!

I discovered a cool new attribute in .NET 2.0 called "InternalsVisibleTo". This takes an assembly name and public key token and grants it access to the assembly internals.

One cool application I can think of is that you no longer have to place your unit tests inside the assembly they are testing; instead you can aggregate them into an external assembly and keep the production binaries as small as possible. Of course you can always wrap the tests with compiler directives, but I think that the former approach would be quite useful in many instances.

Check System.Deployment.dll to see an example of this:

[assembly: InternalsVisibleTo("dfsvc, PublicKeyToken=b03f5f7f11d50a3a")]