Kirk Evans Blog

.NET From a Markup Perspective

Clarifying Guidance on SharePoint Security Groups versus Active Directory Domain Services Groups

Clarifying Guidance on SharePoint Security Groups versus Active Directory Domain Services Groups

Rate This
  • Comments 19

I received the following question on the SPYam group on Yammer, and decided to share the information with the broader audience via my blog.

For SharePoint security Microsoft always recommended to use Active directory group inside SharePoint group. But this guidance now changed Microsoft now recommend:

"We do not recommend SharePoint groups to assign permissions to sites. When a SharePoint group is used to assign permissions, a full crawl of the index occurs. Instead, we recommend Active Directory Domain Services (AD DS) groups."

[from http://technet.microsoft.com/en-us/library/ff758656(v=office.14).aspx]

Can someone from Microsoft confirm this please?!

 

Happy to try Smile

Configuring the Environment

To demonstrate the effect of adding a user to a SharePoint group, adding an Active Directory group, and adding users to an Active Directory group, I want to isolate any changes as much as possible from the rest of my environment.  I configured my environment with a new web application CrawlDemo.Contoso.local, and a root site collection using the Team Site template.  The site collection administrator is a single user, “contoso\administrator”. 

In order to isolate the effect of search crawls to just this web application, I configured a new content source that contains my new web application’s URL.

image

This lets me isolate any crawls to just that one web application.  The rest of the settings for the content source:

  • Crawl Settings: Crawl everything under the hostname for each start address
  • Crawl Schedules: Enable Incremental Crawls
  • Content Source Priority: Normal

I do not have continuous crawls configured for this environment, and none of my content sources are currently scheduled, so I can be sure that crawl information will be isolated to this single web application. 

Before continuing, I make sure that the crawl account has Full Read permission to the web application by checking the User Policy for the web application in Central Administration.

image

Note the text in red:

Adding or updating Web application policy with new users or groups will trigger a SharePoint Search crawl over all content covered by that policy. This can reduce search crawl freshness and increase crawl load. Consider using security groups at the policy level and add/remove users from security groups to avoid this.

We’ll come back to that later in this post as it is directly relevant to what we are discussing.

Once everything is configured, I initiate a full crawl of the content source.  Reviewing the Crawl Health report in Central Administration for Crawl Rate, I see that all 37 items were crawled.

image

If we focus our attention on the Summary table to the right of the screen, we can see the total items crawled was 37 items.

image

This means that the full number of items crawled was 37 items.  When we run an incremental crawl, we should only crawl the changed items and not every item.  The rest of this post focuses on the impact that security changes have on crawling.

A Note on Performance

I am in no way indicating that my demonstrations in this post be used for any type of performance benchmarking.  I am running all of these tests on a single laptop on a single drive using virtualized images that are not optimized for performance.  Further, I have other processes running on the machine, so the numbers that I show do not represent a production environment.  That said, we can make some general assumptions based on the data represented.

Understanding the EventCache

In each content database, there is a table named “EventCache” which contains a change log for everything in that content database.  Once the crawl is complete, I query it using the following SQL statement:

select EventType, ObjectType, EventTime
from EventCache with (nolock)
where EventTime > '2013-05-06 16:37:36.750'

Note that it is unsupported to query SharePoint’s databases directly and should not be done in production without the direct involvement of a Microsoft support engineer.  To minimize any potential effects of the query, I use the SQL NOLOCK hint to read uncommitted records and avoid introducing a table lock. 

Initially this query returns zero results because the time stamp used in the query is the last time stamp from when I created the site collection.  Let’s add a few documents to the Documents library in our new site.

image

We query the table again, and this time we see results in the EventCache table in the content database where the documents were added.

image

That makes complete sense, 4 documents added, 4 rows in the database.  We initiate an incremental crawl (not a full crawl) and wait for it to complete.  Once complete, click the Crawl Health Reports link in the search service application in Central Administration. 

image

The important part to call out is the Summary table on the right of the screen that shows 12 items were crawled during this incremental crawl, none of which were security items.

image

When the search crawler tries to determine what changes were applied, it reads the information in the EventCache table to calculate what needs to be crawled. The thing to learn from this section is that there is a table, EventCache, which the crawler uses to determine changes from each of the content databases. 

I run another incremental crawl, and the number of items decreases to 6 crawled items, and another decreases it to 5 items crawled.  No new rows were in the EventCache table, so SharePoint is simply crawling the top level pages to determine if any content changes appear.

Effect of Adding a User to a SharePoint Group

Let’s add a single domain user to the “Crawl Demo Members” group.

image

We now run our SQL query and see that a row has been added to the EventCache table. 

image

The new value, EventType 2097152, indicates a change where a member was added to a group (actually, 3 new rows were added, but I am only showing the relevant new row).  Recall that the search crawler reads the information in the EventCache table to calculate what needs to be crawled.  We run another incremental crawl and then inspect the health reports to see what happened.  Notice the red bar that now shows, the legend for the chart indicates this is a “security only” crawl.

image

Because we added a user to the group, everything in the site needs to be crawled in order to re-calculate the ACLs for the site.  Focusing on the Summary table at the right, we see that 36 security items were crawled simply because we added a user to a group.

image

We next turn to the crawl log to inspect a summary of our crawls so far.

image

Notice the value in the “Type” column has a letter “F” (indicating full crawl) or “I” (indicating incremental crawl).  The time for the incremental crawl went from 2:30 to 2:10 to 1:40 before we added a user to the group.  When we added the user to the group, it jumped up to 2:50 again because the ACLs for the items had to be recalculated.  The entire index had to be crawled to calculate ACLs.

In a large SharePoint environment with many documents, adding a user to a SharePoint group can have a significant impact on crawl performance.

Effect of Adding an Active Directory Group to a SharePoint Group

We just saw what happens when adding Active Directory Domain Services users to a SharePoint group.  Now let’s see the impact of adding an Active Directory group to a SharePoint group.  The effect will be less educational than before because the results will look nearly identical to what we just saw.  I create a new AD group, “MyCrawlGroup1” and add 12 users to the group in Active Directory.

image

Next, I add “MyCrawlGroup1” to the “Crawl Demo Members” SharePoint group.

image

We then go to the EventCache table to see what the changes were.  I get 3 new rows, the most important one is highlighted.

image

As we saw before, this change indicates that a member was added to a group.  We initiate an incremental crawl again, and just like before we see that we have the same number of security-only updates in the index.  Notice the red bar again.

image

Focusing on the Summary table, we see that we again have 36 security items, the same number as when we added a single user to a SharePoint group. 

image

If we check the crawl times, we can see that the time increased again.  This time, the time increased because SharePoint had to calculate the ACLs for the members of the domain group.  While the difference between the time spent calculating the members of the domain group versus adding a user directly is not statistically relevant (the difference of 10 seconds can be attributed to a number of factors), the fact that adding a domain group caused a security crawl is significant.

image

 

The point to understand in this section is that any time you add users or AD groups to SharePoint groups, your next incremental crawl causes the entire index to be crawled in order to calculate the ACLs. 

Effect of Adding an Active Directory Group to a Site

This should be predictable by now.  We will add an Active Directory group directly to a site instead of adding to a SharePoint group.  This time, I create a group with 12 different users in it called “MyCrawlGroup2”. 

image

Next, we will add the group directly to the site, granting this domain group Edit permissions just like the “Crawl Demo Members” group has.  When I go into “Site settings / Site permissions” and click the “Grant Permissions” button, an interesting thing happens, highlighted below.

image

SharePoint defaults to adding the new user or AD group to an existing SharePoint group.  Since we already proved what happens when you add a domain group to a SharePoint group, let’s change the drop-down to “Edit” instead of “Crawl Demo Members [Edit]”, meaning the group now is added to the site with the same level of permissions as the SharePoint group.

We query the EventCache table and highlight the interesting row.

image

EventType 524288 indicates “change assignment add”, and object type 4 means “Web”, so we are changing the permission on the web itself.  As expected, we run the incremental crawl, and you can guess what happens… a security crawl again.  Notice the red bar.

image

image

Inspecting the crawl log, we see that the time increased again.  Hopefully not statistically relevant, but the time has increased a bit in our crawl logs.

image

This should come as no surprise, adding a security object directly to the site permissions will cause the entire index to be crawled to calculate the ACLs.

Effect of Adding Users to Active Directory Groups

By now, you’re hopefully wondering, where’s the payoff for trudging through this post and all the pictures and data so far?  Well, you’re here!  Let’s see what happens when I add 12 more users to the security group “MyCrawlGroup1”, testing if adding users to a domain group that is contained within a SharePoint group causes security crawls. 

image

First, we query the EventCache table to see if any changes were recorded since the last record was written.  Nope!

image

With no changes in the EventCache table, our new users will be able to access the site because we added them to the Active Directory group, but no changes were recorded to SharePoint to trigger a security crawl.  We check the report, and see that the “Security only” red bar no longer shows in the graph.

image

Further, looking at the crawl log, our crawl time significantly decreased from 3:31 to just 1:50.

image

The lesson to learn is that adding users to an Active Directory group that already has permission in a SharePoint group does not trigger a full crawl of the index to calculate changes in permissions.

Now let’s try adding users to our “MyCrawlGroup2”, testing to see if adding users to an Active Directory group that already has permissions to the site causes a full crawl of the index.  I add 12 users to the AD group.

image

Of course, there are no new records in SQL.

image

And our incremental crawl runs again.  The time stays consistent with zero security updates.

image

The lesson here is that adding users to an Active Directory group that has already been granted permission to a SharePoint site directly does not initiate a full crawl of the index to calculate security changes.

Bonus Material: Adding a User Policy to the Web Application

By now, you’ve seen that adding a user or AD group to a SharePoint group or site causes a full index to calculate the permissions.  At the beginning of this post, I called out the User Policy button in Central Administration, noting the text in red:

Adding or updating Web application policy with new users or groups will trigger a SharePoint Search crawl over all content covered by that policy.

Let’s see what happens when we try to add a user.  In a previous post, I covered PowerShell that configures the object cache accounts.  Configuring the object cache accounts requires that you add Full Read permission to the Portal Super Reader and Full Control to the Portal Super Reader account.  I add the two accounts as part of the configuration steps necessary.

image

Next, I check SQL.

image

EventType 8192 means change or modify, and object type 2048 means Security Policy.  As you can reasonably expect by now, this causes a full index to calculate permissions as described in the red text on that page.  Notice the red bar is back.

image

The lesson to learn is that it is important that you configure the object cache accounts so they have permission to the cache (see my blog posting Setting Object Cache Accounts in SharePoint 2010, the same steps apply for SharePoint 2013) while understanding the impact that adding the permission can have on crawl performance.  Adding a user policy to the web application causes a full index to calculate permissions of all objects covered by that policy.  You still need to configure the object cache accounts, but make sure that you plan for the potential impact on crawl performance when you apply this change.

Summary

Hopefully this post shed some light on how security works with SharePoint and the effect that adding users or Active Directory groups can have on crawl performance. Adding users to Active Directory groups that have already been granted permissions to SharePoint does not affect crawl performance.  Adding users to Active Directory groups is the recommendation for managing security permissions in SharePoint.

  • Hi  Kirk,

    Thank you very much for answering me and the great clarification.

    Regards,

    Mahmoud Hamed

  • Kirk, this is a great post! Keep it up mate.

    Tob.

  • Kirk, very insightful post!

    I am missing the general guidance though as the linked MSDN article does not refer to NOT using active directory groups in SharePoint groups. Also the article "Assign Permission Levels in SP 2013" on MSDN clearly states to use Security groups? technet.microsoft.com/.../cc261972.aspx

    I'm confused now what the best practice is and what the change from SP2010 may be.

  • ... found the last sentence: "Adding users to Active Directory groups is the recommendation for managing security permissions in SharePoint."

    Perfect :-) Though I don't understand what changed with SP2013 - I thought this was also the recommendation for SP2010?

  • @moontear - It's actually how SharePoint has functioned the entire time.  The guidance of using AD groups versus SharePoint groups deserves a bit of context.  Self-service permissions in SharePoint works wonderfully for small SharePoint farms with few individual permissions.  It breaks in large farms where, inevitably, someone conceives a solution with individual item permissions that suddenly becomes business critical and grows to millions of documents in a single library.  It is this latter case that is the context of this article, and is also the context of the TechNet article that was quoted in the original question.  Understanding how the product works is a much better approach than relying on absolute statements.  That's why I wrote the article in the manner I did, I wanted to help readers educate themselves on the potential impact and decide for themselves what risk a full crawl of the index poses for their environment.

  • Kirk - this is a great post! Cheers, Craig.

  • Excellent post !! I like the format in which you arrived at the conclusion. Keep it up..

  • Great post, thanks Kirk. I do have a question though. As a workflow guy (predominately using Nintex Workflow) my clients and I struggle with permissions set this way. To reduce maintenance and management we usually try and use the same groups used for permissions as groups to assign tasks to in workflow. This way, if there is an HR team which need permission to a particular site and that same team needs to approve workflow tasks, we can use a single SP Group, with the HR users in it. If the SP Group contains an AD group, then the tasks do not get properly assigned and members of the AD group can't respond. Similarly if you assign a task to an AD group the task can't be completed. In some circumstances I create SP groups that have no specific permissions on a site, and then use this group to assign tasks too. In this case, if users are added or removed from this group (that has no permissions) what is the load on the DB and servers? Typically, the membership of these kinds of groups can change quite frequently so it would be good to know.

    Thanks again.

  • Kevin - See the following articles for more information on fine-grained permissions.  

    technet.microsoft.com/.../gg128955.aspx

    technet.microsoft.com/.../dn169566.aspx

  • Kirk, thanks for the detailed post. I wasn't aware of this impact on crawling but now that I do I think we may rething our direction on where we use AD groups versus SharePoint groups. We're not using SharePoint 2013 yet but having seen a lot of demos and having had a play myself the 'Share' option seems to really be getting pushed and is much more obvious to users now. Am I missing something or does this not seem to fly in the face of some of the MS best practice, though I do appreciate the balance between business functionality and admin headaches :)

  • Derek - actually, this is the same behavior that SharePoint has always had, it's not new to SharePoint 2013.  There are areas where you will likely want to utilize SharePoint groups for specific functionality, such as Approvers for WCM, workflows, alerts, and other capabilities.  This is not an either/or approach, and understanding how the product works is key to making balanced decisions.  The point of this post was to explain why the recommendation is to use AD groups for securing content and the implication of fine-grained permissions.

  • Great post, Kirk!  I love the way you lay everything out so thoroughly!

  • The article is INVALUABLE. Many thanks for saving many of us these days.

    Regards,

    -T.s

  • Great post Kirk!

  • Hey, the first reference at the top of the blog does not seem to contain the quote attributed to it.  More importantly, the first FGP article referenced in the Q&A above now updates the guidance, at least in one sense:

    (technet.microsoft.com/.../gg128955.aspx)

    "Due to a change in SharePoint Server 2013 search continuous crawl capabilities that now include security information, we no longer recommend that you avoid using SharePoint groups to manage dynamic user and group memberships. Prior to SharePoint Server 2013, using dynamic memberships in SharePoint groups could cause search results to not show up correctly for all members until a full crawl occurred after the membership change. Now with the continuous crawl capability including security information a dynamic membership or other security change will be picked up sooner (default value is 15 minutes) and be used by search query result trimming."

    I also vaguely recall hearing that SP 2013 Search handles security changes in a more optimal way than 2010...something to the effect of Search is able to update the security information on items in the search index, without requiring re-processing of items on which only the security has changed.  It's still work, but I imagine it is a fraction of the work that is required to re-index a document.  Thoughts?

Page 1 of 2 (19 items) 12
Leave a Comment
  • Please add 5 and 6 and type the answer here:
  • Post
Translate This Page
Search
Archive
Archives