Kirk Evans Blog

Create the Best App for Office 2013 in 5 Minutes

Kirk Evans[MSFT] — Tue, 07 May 2013 04:27:07 GMT

While speaking in Kuala Lumpur for the Ignite developer training, Chris O’Connor (aka @grumpiewookie) jokingly said that the whole apps for Office thing needed something catchy, something nostalgic. We need some obscure JavaScript library to do something interesting, something entertaining yet educational.

Something like Clippy.

I searched for all of 30 seconds, and sure enough someone built a JavaScript library that lets you put Clippy in a web page! I love their tagline, “Our research shows that people love two things: failed Microsoft technologies and obscure Javascript libraries. Naturally, we decided to combine the two.”

During the lunch hour, I worked on a funny app (with the help of Keenan Newton) and moments later, demo’d an app that featured Clippy. This is not just an image, Clippy is actually animated, makes sounds, and prints whatever message you feed him to the screen!

There are a number of actions you can provide the agents, such as moveTo, animate, speak, congratulate, and a bunch of other tricks that you can use to amuse your fellow geeks.

Creating a Clippy app is ridiculously easy. There are a few steps, but only a handful of lines of code… the rest is just deployment goo. We’ll even go one step further by deploying our app for Office as part of a SharePoint document library so that when someone creates a new document we can be sure that Clippy will be there.

Create a Document in Excel

This demonstration requires Excel 2013 to support the new app model. Open Excel 2013 and create a document. This document will be the template used for our app (we could use a blank Excel document, but that doesn’t look as nice in screen shots now does it?) In Excel 2013 go to the File tab, click New, and choose the Expense Report template. Save the document somewhere on your hard drive.

Creating the Project

There are four steps that we need to perform to create the Clippy app. First, we create a SharePoint-hosted app. Next, we add an App for Office project item to the app. We then add a document library that uses the document as its template, and finally we add Clippy to the App for Office.

Creating the SharePoint Hosted App

This is the really easy part. In Visual Studio 2012, create a new App for SharePoint 2013 named “MyExpenseReportApp” and click OK. Give the project a name, URL for debugging, and changed the hosting model to “SharePoint-hosted”.

Done with that step.

Add an App for Office

Right-click the project node in Visual Studio 2012 and choose Add / New Item. Choose the “App for Office” project item type and name it “ExpenseReportApp”.

On the next page, uncheck Word and PowerPoint, leaving the option for creating a Task pane app in Excel.

On the next page, choose the option to insert the app into an existing document, and browse to your expense report document that you just created.

Click Finish.

Add a Document Library to the SharePoint App

We now need a document library that will use the App for Office as its template for new documents. Right-click the project node in Visual Studio 2012 and choose “Add / New Item”. Choose the “List” project item and name it “ExpenseReports”. Choose the option to create a customizable list template and a list instance using the Document Library content type.

Choose Next.

On the next screen in the wizard, choose the option to use the following document as the template for this library. Click the browse button, and you should be in the folder that contains your app. There is a child folder, “OfficeDocuments”, that contains the expense report Excel document that was bound to your app for Office.

Once you choose the file, click Finish in Visual Studio. You now have a document template that contains an app as the template for a document library. When someone creates a new document for that library, they will see your app.

The document library will be created in the SharePoint app web, the only way that users will be able to access it is if you provide them the URL or add a UI element like a ribbon button. Another easy way to get to the document library is simply to update the app manifest to point to the document library.

When the app starts, it will be on the all items view for your document library.

Add Clippy

Now for a little developer ingenuity. Visit https://www.smore.com/clippy-js and download Clippy from GitHub. There are several folders in the ZIP file that you download, the only ones you need are the 3 files in the Build folder.

Extract those 3 files and add them to Visual Studio 2012 in the Home folder for your App for Office.

Once you have the files, the rest is easy. Open the Home.html file. Add the following CSS reference in the HEAD section.

<link rel="stylesheet" type="text/css" href="clippy.css" media="all">

You can see where I included it:

Finally, go to the bottom of the Home.html page and add the implementation details.

    <script src="clippy.min.js"></script>
    
    <script type="text/javascript">
    clippy.load('Clippy', function (agent) {
        agent.moveTo(0, 200);
        agent.show();
    });
</script>

Here is a screen shot that again shows where I placed it in my code.

Finally, hit F5 to debug your project. If you followed along, the app should open to your “ExpenseReports” document library in your app web. Go to the Files tab and choose New Document / ExpenseReportsContentType.

That will open Excel using the Expense Report template. The document contains a reference to our App for Office, so when the app opens we will see Clippy.

Can’t you just imagine their surprise when they open the new expense report app that you’ve been laboring over, and they are delighted that it comes with a built-in help system!

For More Information

https://www.smore.com/clippy-js – source and documentation for Clippy.js

http://dev.office.com – Learn to build apps for Office 2013

Clarifying Guidance on SharePoint Security Groups versus Active Directory Domain Services Groups

Kirk Evans[MSFT] — Mon, 06 May 2013 18:59:30 GMT

I received the following question on the SPYam group on Yammer, and decided to share the information with the broader audience via my blog.

For SharePoint security Microsoft always recommended to use Active directory group inside SharePoint group. But this guidance now changed Microsoft now recommend:

"We do not recommend SharePoint groups to assign permissions to sites. When a SharePoint group is used to assign permissions, a full crawl of the index occurs. Instead, we recommend Active Directory Domain Services (AD DS) groups."

[from http://technet.microsoft.com/en-us/library/ff758656(v=office.14).aspx]

Can someone from Microsoft confirm this please?!

Happy to try

Configuring the Environment

To demonstrate the effect of adding a user to a SharePoint group, adding an Active Directory group, and adding users to an Active Directory group, I want to isolate any changes as much as possible from the rest of my environment. I configured my environment with a new web application CrawlDemo.Contoso.local, and a root site collection using the Team Site template. The site collection administrator is a single user, “contoso\administrator”.

In order to isolate the effect of search crawls to just this web application, I configured a new content source that contains my new web application’s URL.

This lets me isolate any crawls to just that one web application. The rest of the settings for the content source:

Crawl Settings: Crawl everything under the hostname for each start address
Crawl Schedules: Enable Incremental Crawls
Content Source Priority: Normal

I do not have continuous crawls configured for this environment, and none of my content sources are currently scheduled, so I can be sure that crawl information will be isolated to this single web application.

Before continuing, I make sure that the crawl account has Full Read permission to the web application by checking the User Policy for the web application in Central Administration.

Note the text in red:

Adding or updating Web application policy with new users or groups will trigger a SharePoint Search crawl over all content covered by that policy. This can reduce search crawl freshness and increase crawl load. Consider using security groups at the policy level and add/remove users from security groups to avoid this.

We’ll come back to that later in this post as it is directly relevant to what we are discussing.

Once everything is configured, I initiate a full crawl of the content source. Reviewing the Crawl Health report in Central Administration for Crawl Rate, I see that all 37 items were crawled.

If we focus our attention on the Summary table to the right of the screen, we can see the total items crawled was 37 items.

This means that the full number of items crawled was 37 items. When we run an incremental crawl, we should only crawl the changed items and not every item. The rest of this post focuses on the impact that security changes have on crawling.

A Note on Performance

I am in no way indicating that my demonstrations in this post be used for any type of performance benchmarking. I am running all of these tests on a single laptop on a single drive using virtualized images that are not optimized for performance. Further, I have other processes running on the machine, so the numbers that I show do not represent a production environment. That said, we can make some general assumptions based on the data represented.

Understanding the EventCache

In each content database, there is a table named “EventCache” which contains a change log for everything in that content database. Once the crawl is complete, I query it using the following SQL statement:

select EventType, ObjectType, EventTime
from EventCache with (nolock)
where EventTime > '2013-05-06 16:37:36.750'

Note that it is unsupported to query SharePoint’s databases directly and should not be done in production without the direct involvement of a Microsoft support engineer. To minimize any potential effects of the query, I use the SQL NOLOCK hint to read uncommitted records and avoid introducing a table lock.

Initially this query returns zero results because the time stamp used in the query is the last time stamp from when I created the site collection. Let’s add a few documents to the Documents library in our new site.

We query the table again, and this time we see results in the EventCache table in the content database where the documents were added.

That makes complete sense, 4 documents added, 4 rows in the database. We initiate an incremental crawl (not a full crawl) and wait for it to complete. Once complete, click the Crawl Health Reports link in the search service application in Central Administration.

The important part to call out is the Summary table on the right of the screen that shows 12 items were crawled during this incremental crawl, none of which were security items.

When the search crawler tries to determine what changes were applied, it reads the information in the EventCache table to calculate what needs to be crawled. The thing to learn from this section is that there is a table, EventCache, which the crawler uses to determine changes from each of the content databases.

I run another incremental crawl, and the number of items decreases to 6 crawled items, and another decreases it to 5 items crawled. No new rows were in the EventCache table, so SharePoint is simply crawling the top level pages to determine if any content changes appear.

Effect of Adding a User to a SharePoint Group

Let’s add a single domain user to the “Crawl Demo Members” group.

We now run our SQL query and see that a row has been added to the EventCache table.

The new value, EventType 2097152, indicates a change where a member was added to a group (actually, 3 new rows were added, but I am only showing the relevant new row). Recall that the search crawler reads the information in the EventCache table to calculate what needs to be crawled. We run another incremental crawl and then inspect the health reports to see what happened. Notice the red bar that now shows, the legend for the chart indicates this is a “security only” crawl.

Because we added a user to the group, everything in the site needs to be crawled in order to re-calculate the ACLs for the site. Focusing on the Summary table at the right, we see that 36 security items were crawled simply because we added a user to a group.

We next turn to the crawl log to inspect a summary of our crawls so far.

Notice the value in the “Type” column has a letter “F” (indicating full crawl) or “I” (indicating incremental crawl). The time for the incremental crawl went from 2:30 to 2:10 to 1:40 before we added a user to the group. When we added the user to the group, it jumped up to 2:50 again because the ACLs for the items had to be recalculated. The entire index had to be crawled to calculate ACLs.

In a large SharePoint environment with many documents, adding a user to a SharePoint group can have a significant impact on crawl performance.

Effect of Adding an Active Directory Group to a SharePoint Group

We just saw what happens when adding Active Directory Domain Services users to a SharePoint group. Now let’s see the impact of adding an Active Directory group to a SharePoint group. The effect will be less educational than before because the results will look nearly identical to what we just saw. I create a new AD group, “MyCrawlGroup1” and add 12 users to the group in Active Directory.

Next, I add “MyCrawlGroup1” to the “Crawl Demo Members” SharePoint group.

We then go to the EventCache table to see what the changes were. I get 3 new rows, the most important one is highlighted.

As we saw before, this change indicates that a member was added to a group. We initiate an incremental crawl again, and just like before we see that we have the same number of security-only updates in the index. Notice the red bar again.

Focusing on the Summary table, we see that we again have 36 security items, the same number as when we added a single user to a SharePoint group.

If we check the crawl times, we can see that the time increased again. This time, the time increased because SharePoint had to calculate the ACLs for the members of the domain group. While the difference between the time spent calculating the members of the domain group versus adding a user directly is not statistically relevant (the difference of 10 seconds can be attributed to a number of factors), the fact that adding a domain group caused a security crawl is significant.

The point to understand in this section is that any time you add users or AD groups to SharePoint groups, your next incremental crawl causes the entire index to be crawled in order to calculate the ACLs.

Effect of Adding an Active Directory Group to a Site

This should be predictable by now. We will add an Active Directory group directly to a site instead of adding to a SharePoint group. This time, I create a group with 12 different users in it called “MyCrawlGroup2”.

Next, we will add the group directly to the site, granting this domain group Edit permissions just like the “Crawl Demo Members” group has. When I go into “Site settings / Site permissions” and click the “Grant Permissions” button, an interesting thing happens, highlighted below.

SharePoint defaults to adding the new user or AD group to an existing SharePoint group. Since we already proved what happens when you add a domain group to a SharePoint group, let’s change the drop-down to “Edit” instead of “Crawl Demo Members [Edit]”, meaning the group now is added to the site with the same level of permissions as the SharePoint group.

We query the EventCache table and highlight the interesting row.

EventType 524288 indicates “change assignment add”, and object type 4 means “Web”, so we are changing the permission on the web itself. As expected, we run the incremental crawl, and you can guess what happens… a security crawl again. Notice the red bar.

Inspecting the crawl log, we see that the time increased again. Hopefully not statistically relevant, but the time has increased a bit in our crawl logs.

This should come as no surprise, adding a security object directly to the site permissions will cause the entire index to be crawled to calculate the ACLs.

Effect of Adding Users to Active Directory Groups

By now, you’re hopefully wondering, where’s the payoff for trudging through this post and all the pictures and data so far? Well, you’re here! Let’s see what happens when I add 12 more users to the security group “MyCrawlGroup1”, testing if adding users to a domain group that is contained within a SharePoint group causes security crawls.

First, we query the EventCache table to see if any changes were recorded since the last record was written. Nope!

With no changes in the EventCache table, our new users will be able to access the site because we added them to the Active Directory group, but no changes were recorded to SharePoint to trigger a security crawl. We check the report, and see that the “Security only” red bar no longer shows in the graph.

Further, looking at the crawl log, our crawl time significantly decreased from 3:31 to just 1:50.

The lesson to learn is that adding users to an Active Directory group that already has permission in a SharePoint group does not trigger a full crawl of the index to calculate changes in permissions.

Now let’s try adding users to our “MyCrawlGroup2”, testing to see if adding users to an Active Directory group that already has permissions to the site causes a full crawl of the index. I add 12 users to the AD group.

Of course, there are no new records in SQL.

And our incremental crawl runs again. The time stays consistent with zero security updates.

The lesson here is that adding users to an Active Directory group that has already been granted permission to a SharePoint site directly does not initiate a full crawl of the index to calculate security changes.

Bonus Material: Adding a User Policy to the Web Application

By now, you’ve seen that adding a user or AD group to a SharePoint group or site causes a full index to calculate the permissions. At the beginning of this post, I called out the User Policy button in Central Administration, noting the text in red:

Adding or updating Web application policy with new users or groups will trigger a SharePoint Search crawl over all content covered by that policy.

Let’s see what happens when we try to add a user. In a previous post, I covered PowerShell that configures the object cache accounts. Configuring the object cache accounts requires that you add Full Read permission to the Portal Super Reader and Full Control to the Portal Super Reader account. I add the two accounts as part of the configuration steps necessary.

Next, I check SQL.

EventType 8192 means change or modify, and object type 2048 means Security Policy. As you can reasonably expect by now, this causes a full index to calculate permissions as described in the red text on that page. Notice the red bar is back.

The lesson to learn is that it is important that you configure the object cache accounts so they have permission to the cache (see my blog posting Setting Object Cache Accounts in SharePoint 2010, the same steps apply for SharePoint 2013) while understanding the impact that adding the permission can have on crawl performance. Adding a user policy to the web application causes a full index to calculate permissions of all objects covered by that policy. You still need to configure the object cache accounts, but make sure that you plan for the potential impact on crawl performance when you apply this change.

Summary

Hopefully this post shed some light on how security works with SharePoint and the effect that adding users or Active Directory groups can have on crawl performance. Adding users to Active Directory groups that have already been granted permissions to SharePoint does not affect crawl performance. Adding users to Active Directory groups is the recommendation for managing security permissions in SharePoint.

How to Allow Only Users Who Have a Community Badge to Your SharePoint 2013 Site

Kirk Evans[MSFT] — Mon, 22 Apr 2013 14:00:00 GMT

This post will show how you can secure a web site based on if a user has been given a badge in a SharePoint 2013 community site.

Badges and Reputation in SharePoint 2013 Community Sites

SharePoint 2013 Community Sites provide a new set of features to implement game theory, or gamification. One feature is called “reputation”, where participation earns you points. As you earn points, you earn achievements that are reflected as your reputation. The settings for reputation allow you to define your own points system, defining the points earned for creating a post, replying to a post, having a post marked with 4 or 5 stars or “liked”, and having a post marked as “Best Reply”. You further define the number of points per level, and whether the achievements are represented as an image or as text.

The second feature to point out is called a Gifted Badge. The site owner will define a set of badges that allow you to define the name of each badge.

This allows a community site owner to define a set of badges and assign them to community members. For instance, you might want to create a badge that calls out if someone has achieved a particular certification level. Just highlight a community site member and the “Give Badge” button will be enabled in the ribbon in the Moderation tab.

Once users have been assigned badges, they appear on the front page of the Community site in the “Top contributors” web part.

When I first saw the new Community Sites in SharePoint 2013, I was excited. Of course, there are a ton of things that I would like to see improved, but this is a decent start. To me, the ability to assign badges seemed really cool and I saw a bunch of opportunities such as assigning badges when employees are on-boarded and they complete training courses. Out of the box, all you can do with them is either earn them or assign them. What would be really cool is if you could somehow use them to protect content such that only people who have a specific badge are allowed to use a site. For instance, I could create a badge called “Microsoft Certified Professional” and only users who have that badge are allowed into the “MCPs Only” web site. That would allow me to do this:

That is exactly what we are going to build.

Wanna Come In? Show Me Your Badge!

Badges are simply items in a list, so how would you use that in order to allow or deny someone access to a site? The solution is a custom claims provider. A claims provider is used to augment a user’s claims and to provide name resolution. Augmenting claims simply means adding more claims into the user’s token. For instance, I can add a claim that says what the user’s favorite basketball team is, what their hair color is, whatever information I want to add to their user token. Here is a screen shot where I added a new claim type “http://Microsoft.PFE.Claims/Badges” with a value “Microsoft Certified Master”.

When the user logs in, our custom claim provider will augment the user’s claim set by adding an additional claim indicating if the user has been assigned a badge. If the user has not been assigned a badge, then no claim is added.

This scenario could be adapted for many other scenarios. Perhaps you have a document library that contains your company’s intellectual property, and only people who have authored intellectual property have access to a special site. Maybe you require people to complete training courses before allowing them access to content. There are many ways that you can adapt this scenario.

The Badge Custom Claims Provider

There are a number of claim providers already registered out of the box, and you can register more than one claim provider to augment claims. Using the Get-SPClaimProvider cmdlet in PowerShell, we can see the claim providers that have been registered.

Start by creating a new SharePoint 2013 farm solution using the “SharePoint 2013 – Empty Project” template. Next, add a class that derives from Microsoft.SharePoint.Administration.Claims.SPClaimProvider. Here is a class diagram of what we will implement.

Before we go into the claims provider, let’s get our data access layer stuff out of the way.

Reading Badges From a Community Site

The first step is to create a few classes that store the data we want to work with. The first class is Badge that just holds the ID and Title for the badge.

namespace Microsoft.PFE.ClaimsProviders
{
    public class Badge
    {
        public int ID { get; set; }
        public string Title { get; set; }
    }
}

The next class is BadgeUser, which holds information about the user and the badges that they have been assigned.

using System.Collections.Generic;

namespace Microsoft.PFE.ClaimsProviders
{
    public class BadgeUser
    {
        public string Name { get; set; }
        public string LoginName { get; set; }
        public List<Badge> Badges { get; set; }
    }
}

Now that we’ve defined some classes to hold the data, let’s create a helper class that can fill them with data. The first method allows us to search for any badges that start with a set of characters.

 1: using Microsoft.SharePoint;

 2: using System.Collections.Generic;

3:

 4: namespace Microsoft.PFE.ClaimsProviders

 5: {

 6:     public class BadgeHelper

 7:     {

 8:         /// <summary>

 9:         /// Get the list of badges that start with the partial name.

 10:         /// </summary>

 11:         /// <param name="partialBadgeName">The characters the badge

 12:         /// name should start with.

 13:         /// </param>

 14:         /// <returns>A list of badges that match the criteria.</returns>

 15:         public static List<Badge> GetBadgesByPartialName(string partialBadgeName)

 16:         {

17:

 18:             List<Badge> badges = new List<Badge>();

19:

 20:             //Use RWEP here because the user may not have

 21:             //access to the site.

 22:             SPSecurity.RunWithElevatedPrivileges(delegate()

 23:             {

 24:                 using (SPSite site = new SPSite("http://intranet.contoso.local"))

 25:                 {

 26:                     SPWeb web = site.OpenWeb("Community");

27:

 28:                     SPQuery query = new SPQuery();

 29:                     query.Query = "<Where>" +

 30:                                        "<BeginsWith>" +

 31:                                            "<FieldRef Name='Title' />" +

 32:                                            "<Value Type='TEXT' >" +

 33:                                                partialBadgeName +

 34:                                            "</Value>" +

 35:                                        "</BeginsWith>" +

 36:                                    "</Where>";

37:

 38:                     query.ViewFields = "<FieldRef Name='ID'/>" +

 39:                                         "<FieldRef Name='Title'/>";

40:

 41:                     SPList badgeList = web.Lists["Badges"];

 42:                     SPListItemCollection items = badgeList.GetItems(query);

43:

44:

 45:                     if (null != items)

 46:                     {

 47:                         foreach (SPListItem item in items)

 48:                         {

 49:                             badges.Add(new Badge

 50:                             {

 51:                                 ID = item.ID,

 52:                                 Title = item.Title

 53:                             });

 54:                         }

 55:                     }

56:

 57:                 }

 58:             });

 59:             return badges;

 60:         }

The call on line 22 is necessary because the current user may not have access to the community site that we are obtaining data from. You can see this is a pretty simple method that just gets data from a SharePoint list. The next method looks nearly identical in that it gets a list of data based on an exact match rather than a partial match.

        /// <summary>
        /// A list of badges that match the exact name.
        /// </summary>
        /// <param name="name">The name of the badge to match.</param>
        /// <returns>A list of badges that have the exact name.</returns>
        public static List<Badge> GetBadgesByExactName(string name)
        {            
            List<Badge> badges = new List<Badge>();

            //Use RWEP here because the user may not have 
            //access to the site.
            SPSecurity.RunWithElevatedPrivileges(delegate()
            {
                using (SPSite site = new SPSite("http://intranet.contoso.local"))
                {
                    SPWeb web = site.OpenWeb("Community");

                    SPQuery query = new SPQuery();
                    query.Query = "<Where>" +
                                       "<Eq>" + 
                                            "<FieldRef Name='Title' />" +
                                            "<Value Type='TEXT' >" + 
                                                name + 
                                            "</Value>" +
                                       "</Eq>" +
                                   "</Where>";
                    query.ViewFields = "<FieldRef Name='ID'/>" +
                                        "<FieldRef Name='Title'/>";

                    SPList badgeList = web.Lists["Badges"];
                    SPListItemCollection items = badgeList.GetItems(query);


                    if (null != items)
                    {
                        foreach (SPListItem item in items)
                        {
                            badges.Add(new Badge
                            {
                                ID = item.ID,
                                Title = item.Title
                            });
                        }
                    }

                }
            });
            return badges;
        }

The last method bears a little more explanation. Now we want to get the badges for a particular user. The user may not have access to the site, so we have to use RunWithElevatedPrivileges. In fact, the user may not even be in the site users list, so we have to use EnsureUser in order to obtain a valid reference to the SPUser object for that user.

 1:         /// <summary>

 2:         /// Gets the badges for a particular user

 3:         /// </summary>

 4:         /// <param name="loginName">The login name in encoded claims

 5:         /// format. For instance, "i:0#.w|contoso\administrator".</param>

 6:         /// <returns>The user information and badges for that user.</returns>

 7:         public static BadgeUser GetBadgesForUser(string loginName)

 8:         {

 9:             BadgeUser badgeUser = null;

10:

 11:             //Use RWEP here because the user may not have

 12:             //access to the site.

 13:             SPSecurity.RunWithElevatedPrivileges(delegate()

 14:             {

 15:                 using (SPSite site = new SPSite("http://intranet.contoso.local"))

 16:                 {

 17:                     SPWeb web = site.OpenWeb("Community");

18:

 19:                     //The encoded claim will be in the format

 20:                     //0#.f|ldapmember|kirkevans. Add the i: prefix

 21:                     //to indicate this is an identity claim.

 22:                     if (!loginName.StartsWith("i:"))

 23:                     {

 24:                         loginName = "i:" + loginName;

 25:                     }

26:

 27:                     //EnsureUser will only work if the same login provider

 28:                     //is available in this site. For example, if you are

 29:                     //calling this from a user authenticated in another web

 30:                     //application, and this site does not have FBA configured,

 31:                     //the call to EnsureUser will fail.

 32:                     SPUser user = user = web.EnsureUser(loginName);

33:

 34:                     SPQuery query = new SPQuery();

 35:                     query.Query = "<Where>" +

 36:                                     "<And>" +

 37:                                         "<Eq>" +

 38:                                            "<FieldRef Name='Member' LookupId='TRUE' />" +

 39:                                            "<Value Type='USER' >" + user.ID + "</Value>" +

 40:                                         "</Eq>" +

 41:                                         "<IsNotNull>" +

 42:                                            "<FieldRef Name='GiftedBadgeLookup'/>" +

 43:                                         "</IsNotNull>" +

 44:                                     "</And>" +

 45:                                   "</Where>";

46:

 47:                     query.ViewFields = "<FieldRef Name='Title'/>" +

 48:                                         "<FieldRef Name='Member'/>" +

 49:                                         "<FieldRef Name='GiftedBadgeLookup'/>";

 50:                     SPList memberList = web.Lists["Community Members"];

 51:                     SPListItemCollection items = memberList.GetItems(query);

52:

 53:                     if (items.Count > 0)

 54:                     {

 55:                         badgeUser = new BadgeUser

 56:                         {

 57:                             LoginName = user.LoginName,

 58:                             Name = items[0].Title

 59:                         };

 60:                         List<Badge> badges = new List<Badge>();

61:

 62:                         foreach (SPListItem item in items)

 63:                         {

 64:                             SPFieldLookupValue badgeName =

 65:                                 new SPFieldLookupValue(item["GiftedBadgeLookup"].ToString());

 66:                             //The current data structure only allows

 67:                             //one badge, but we may want to add badges

 68:                             //from another system in the future.

 69:                             badges.Add(new Badge

 70:                             {

 71:                                 ID = badgeName.LookupId,

 72:                                 Title = badgeName.LookupValue

 73:                             });

 74:                         }

 75:                         badgeUser.Badges = badges;

 76:                     }

 77:                 }

 78:             });

 79:             return badgeUser;

 80:         }

 81:     }

 82: }

The call to EnsureUser requires that the provider used to authenticate the user is available to this web application. If we are in one web application that uses FBA claims and then we make a call to another web site (in this case http://intranet.contoso.local), then both web applications must provide the same authentication capabilities. If one web application uses FBA, the site you are calling EnsureUser for must also be configured to use the same FBA provider.

Implementing the Badge Claim Provider

Now that we have the data access stuff out of the way, let’s focus on the really cool part of the solution. We’ll start by implementing the properties as shown in the previous class diagram.

 1: using System;

 2: using System.Collections.Generic;

 3: using Microsoft.SharePoint.Administration;

 4: using Microsoft.SharePoint.Administration.Claims;

 5: using Microsoft.SharePoint.WebControls;

6:

 7: namespace Microsoft.PFE.ClaimsProviders

 8: {

 9:     public class BadgeClaimProvider : SPClaimProvider

 10:     {

 11:         #region Constructor

 12:         public BadgeClaimProvider(string displayName)

 13:             : base(displayName)

 14:         {

 15:         }

 16:         #endregion

17:

 18:         #region Properties

 19:         internal static string ProviderInternalName

 20:         {

 21:             get { return "BadgeClaimsProvider"; }

 22:         }

23:

 24:         public override string Name

 25:         {

 26:             get { return ProviderInternalName; }

 27:         }

28:

 29:         internal static string ProviderDisplayName

 30:         {

 31:             get { return "Badges for User"; }

 32:         }

33:

 34:         private static string BadgeClaimType

 35:         {

 36:             get { return "http://Microsoft.PFE.Claims/Badges"; }

 37:         }

38:

 39:         private static string BadgeClaimValueType

 40:         {

 41:             get { return Microsoft.IdentityModel.Claims.ClaimValueTypes.String; }

 42:         }

43:

 44:         public override bool SupportsEntityInformation

 45:         {

 46:             get { return true; }

 47:         }

48:

 49:         public override bool SupportsHierarchy

 50:         {

 51:             get { return false; }

 52:         }

53:

 54:         public override bool SupportsResolve

 55:         {

 56:             get { return true; }

 57:         }

58:

 59:         public override bool SupportsSearch

 60:         {

 61:             get { return true; }

 62:         }

63:

 64:         #endregion

The ProviderDisplayName property on line 29 can be seen in the DisplayName property when we use the Get-SPClaimProvider cmdlet. The BadgeClaimType (line 34) defines the URI for the claim, and the BadgeClaimValueType (line 39) defines the type of value the claim will contain.

The four properties starting with “Support” indicate to SharePoint what capabilities this provider has.

EntityInformation – provides claims augmentation for a specific user.
Hierarchy – used in the People Picker to create a hierarchical representation of claims.
Resolve – Find an exact match for the claim value.
Search – Search for a value based on a partial word or search pattern.

Our provider will support entity information, search, and resolve.

Claims Augmentation - Providing Entity Information

We need to add a claim to the user’s token that shows a badge that they might have earned. To do that, we first tell SharePoint that we are going to fill a claim with a specific type (“http://Microsoft.PFE.Claims/Badges”) and what type its value will be (Microsoft.IdentityModel.Claims.ClaimValueTypes.String). We then use the FillClaimsForEntity method to augment the claims for the current user. This method accepts a parameter, “entity”, the value of which is the user’s identity claim (for instance, “0#.f|ldapmember|kirkevans”). We then pass that value to our helper method GetBadgesForUser.

 1:         protected override void FillClaimTypes(List<string> claimTypes)

 2:         {

 3:             if (claimTypes == null)

 4:                 throw new ArgumentNullException("claimTypes");

5:

 6:             // Add our claim type.

 7:             claimTypes.Add(BadgeClaimType);

 8:         }

9:

 10:         protected override void FillClaimValueTypes(List<string> claimValueTypes)

 11:         {

 12:             if (claimValueTypes == null)

 13:                 throw new ArgumentNullException("claimValueTypes");

14:

 15:             // Add our claim value type.

 16:             claimValueTypes.Add(BadgeClaimValueType);

 17:         }

18:

19:

 20:         protected override void FillClaimsForEntity(Uri context, SPClaim entity, List<SPClaim> claims)

 21:         {

 22:             if (entity == null)

 23:                 throw new ArgumentNullException("entity");

24:

 25:             if (claims == null)

 26:                 throw new ArgumentNullException("claims");

27:

28:

 29:             var badgeUser = BadgeHelper.GetBadgesForUser(entity.Value);

30:

31:

 32:             if (null != badgeUser)

 33:             {

 34:                 if (null != badgeUser.Badges)

 35:                 {

 36:                     foreach (var badge in badgeUser.Badges)

 37:                     {

 38:                         claims.Add(CreateClaim(BadgeClaimType, badge.Title, BadgeClaimValueType));

 39:                     }

 40:                 }

 41:             }

 42:         }

43:

44:

45:

 46:         protected override void FillEntityTypes(List<string> entityTypes)

 47:         {

 48:             if (null == entityTypes)

 49:             {

 50:                 throw new ArgumentNullException("entityTypes");

 51:             }

 52:             entityTypes.Add(SPClaimEntityTypes.FormsRole);

 53:         }

Enabling the People Picker

So far, all we have done is add a claim to the user’s token. We want to enable the scenario where someone can create a site, select Users and Groups, and use the People Picker to select our badge, allowing any users with that badge access to the site. To do this, we need to implement a few more methods: FillSchema, FillSearch and FillResolve. FillSchema tells the PeoplePicker what information we will display and in what format. FillSearch accepts a string value that we use to find any badges that start with those characters.

        protected override void FillSchema(SPProviderSchema schema)
        {
            if (null == schema)
            {
                throw new ArgumentNullException("schema");
            }

            schema.AddSchemaElement(new SPSchemaElement(PeopleEditorEntityDataKeys.DisplayName,
                "Display Name", SPSchemaElementType.TableViewOnly));
        }

        protected override void FillSearch(Uri context, 
            string[] entityTypes, 
            string searchPattern, 
            string hierarchyNodeID, 
            int maxCount, 
            SPProviderHierarchyTree searchTree)
        {
            if (!EntityTypesContain(entityTypes, SPClaimEntityTypes.FormsRole))
            {
                return;
            }

            List<Badge> badges = BadgeHelper.GetBadgesByPartialName(searchPattern);
            if(null != badges)
            {
                foreach (var badge in badges)
                {
                    PickerEntity entity = CreatePickerEntity();
                    entity.Claim = CreateClaim(BadgeClaimType, badge.Title, BadgeClaimValueType);
                    entity.Description = badge.Title;
                    entity.DisplayText = badge.Title;
                    entity.EntityData[PeopleEditorEntityDataKeys.DisplayName] = badge.Title;
                    entity.EntityType = SPClaimEntityTypes.FormsRole;
                    entity.IsResolved = true;
                    searchTree.AddEntity(entity);
                }
            }
        }

Another way you can use the People Picker is to type a value in. When you type the value instead of using search, this calls the FillResolve method of the registered claims providers.

        protected override void FillResolve(Uri context, 
            string[] entityTypes, 
            SPClaim resolveInput, 
            List<PickerEntity> resolved)
        {
            FillResolve(context, entityTypes, resolveInput.Value, resolved);
        }

        protected override void FillResolve(Uri context, 
            string[] entityTypes, 
            string resolveInput, 
            List<PickerEntity> resolved)
        {
            List<Badge> badges = BadgeHelper.GetBadgesByExactName(resolveInput);
            if (null != badges)
            {
                foreach (var badge in badges)
                {
                    PickerEntity entity = CreatePickerEntity();
                    entity.Claim = CreateClaim(BadgeClaimType, badge.Title, BadgeClaimValueType);
                    entity.Description = badge.Title;
                    entity.DisplayText = badge.Title;
                    entity.EntityData[PeopleEditorEntityDataKeys.DisplayName] = badge.Title;
                    entity.EntityType = SPClaimEntityTypes.FormsRole;
                    entity.IsResolved = true;
                    resolved.Add(entity);
                }
            }
        }

        protected override void FillHierarchy(Uri context, 
            string[] entityTypes, 
            string hierarchyNodeID, 
            int numberOfLevels, 
            SPProviderHierarchyTree hierarchy)
        {
            throw new NotImplementedException();
        }
    }
}

Registering Your Custom Claim Provider

In Visual Studio 2012, add a new farm-scoped feature and a feature receiver. Change the type of the feature receiver to SPClaimProviderFeatureReceiver. You only need to override a few properties here, no need to override FeatureActivating or FeatureDeactivating.

using System;
using System.Runtime.InteropServices;
using Microsoft.SharePoint.Administration.Claims;

namespace Microsoft.PFE.ClaimsProviders.Features.BadgeClaimFeature
{
    /// <summary>
    /// This class handles events raised during feature activation, deactivation, installation, uninstallation, and upgrade.
    /// </summary>
    /// <remarks>
    /// The GUID attached to this class may be used during packaging and should not be modified.
    /// </remarks>

    [Guid("99871a2c-f81a-443c-9625-d2465d2ec29c")]
    public class BadgeClaimFeatureEventReceiver : SPClaimProviderFeatureReceiver
    {


        public override string ClaimProviderAssembly
        {
            get { return typeof(BadgeClaimProvider).Assembly.FullName; }
        }

        public override string ClaimProviderDescription
        {
            get { return "A sample provider written by Kirk Evans"; }
        }

        public override string ClaimProviderDisplayName
        {
            get { return BadgeClaimProvider.ProviderDisplayName; }
        }

        public override string ClaimProviderType
        {
            get { return typeof(BadgeClaimProvider).FullName; }
        }
    }
}

Right-click the project and choose Publish. This will package the solution into a WSP. Once packaged into a WSP, you can add the solution and deploy it using PowerShell.

Add-SPSolution -LiteralPath C:\temp\claims\Microsoft.PFE.ClaimsProviders.wsp
Install-SPSolution microsoft.pfe.claimsproviders.wsp  -GACDeployment

Securing Content Based on a Badge

Now let’s put our new claim provider through its paces. I create a new site titled “MCM Only” with unique permissions. I then determine who to add to the group “MCM Only Members”. When I search for “Microsoft” I see a list of the badges that start with the search term (this called our FillSearch method). When we hover over a result, a popup shows which claim provider the information came from.

Now let’s try our FillResolve method by typing the value in.

When we hover over the underlined value, we choose the value that matches.

Once we choose a value, only users that have that claim will be granted access. For those users that have access to the site because they match a claim, they will be granted access. Notice that it shows in the top nav bar for a user who has the badge.

When I try to access the site, it succeeds.

Further, SharePoint is smart enough to security trim away the site that a user doesn’t have access to based on our claim provider! Now I am logged in as Dan Jump, who does not have a badge.

And when I try to access the site by typing it in the URL, it fails just as we would expect.

Summary

Custom claim providers address a large number of scenarios that people commonly face with SharePoint, enabling the ability to add additional information into the user’s token and then using that information to secure content or assign work to users via the people picker. There are quite a few interesting ideas on using custom claim providers.

Securing SharePoint Content using Profile Attributes

Leveraging Facebook for SharePoint Security

Creating a Claims Provider Based on SharePoint Audiences

Creating a Hierarchical Claim Provider Based on Favorite Basketball Teams

As I survey many of the implementations that have been done for various customers using a mix of HttpModules and code in master pages to provide less than elegant solutions to similar problems, I can see more and more scenarios that could be solved using custom claim providers.

How I Became a SharePoint 2010 MCM

Kirk Evans[MSFT] — Wed, 17 Apr 2013 15:54:57 GMT

I recently attained the Microsoft Certified Master for SharePoint 2010 certification. I have long wanted to write about the experience, but honestly didn’t know how to begin. After quite a few conversations with friends and colleagues, I thought I would share some insight about the process. I am not writing this to dissuade you from pursuing MCSM… in fact, quite the opposite. I think many more SharePoint professionals need to attend the program to receive the most amazing training that they will ever receive anywhere. That said, you should also understand what it takes to achieve certification and what the rest of the Masters went through. They didn’t just attend a training course and then get a certificate of completion, this is a true measure of your mastery of the technology.

IT WAS A LOT OF WORK

Mrs. Robertson, my high school English teacher, would scream at me for putting that heading in all caps, but I mean every single word of it: IT WAS A LOT OF WORK. The MCM certification is just that, a certification. You might be familiar with the Microsoft Most Valuable Professional (MVP) award which is given to recognize technical leaders who contribute to the community in a meaningful way. I was lucky enough to be an MVP at one time and cannot speak highly enough about the hard work and dedication for these people. Where MVP is an award that recognizes technical leadership within the community, MCM is a certification process that requires multiple tests. The program requires that you pass 4 certification exams (two for IT Pro, and 2 for Developer) before you even qualify for an interview to determine if you are accepted into the MCM training program. Once you complete the training program, you have to pass the mother of all written tests, by far the hardest written test I’ve ever taken (worse than my Calculus 3 final exam in college by far). Finally, you sit for a qualification lab test that took every minute of 8.5 hours to complete, a test that I joke most consultants would bill at least a month of time to complete the same amount of work. It was grueling, and worth every minute. It was one of the most difficult and yet satisfying achievements of my career.

The Interview

My interview was with two people, both MCMs, and they grilled me on technical aspects of the product that I had never considered, let alone had any real world experience with. I honestly don’t know of too many people who would have enough technical experience to approach that interview with confidence, let alone have much confidence left after the interview. They asked me about some of the roughest edges in the product, how to solve problems on a global scale as well as how to solve very specific smaller complex problems. I thought I knew a lot about the product, but I was seriously questioning my abilities once I got off the phone. Somehow, I passed, and they gave me feedback on areas that I needed to focus on in order to be successful with the program going forward. For me, unfortunately, the things that I needed to focus on were precisely the things that were emphasized and central components of the training program.

The Training

There are two formats for the technical training portion of the program: a three week immersive training course where you are on site the entire time, and a one week on site plus 10 week remote training course. After talking with colleagues who had gone through the experience, they thoroughly recommended the three week course as the better option. For me, I found that the three weeks on site provided me the best opportunity to succeed as I was able to work side by side with friends until the wee hours of the morning in the lab away from the distractions of work and family. Yes, I said the distractions of family, as much as it pains me to say that this program requires that you set aside all personal responsibilities to focus. I didn’t feel that I would be disciplined enough to attend the training virtually.

The material in the program covers a broad set of topics. For instance, simply saying we covered “ECM” means that we covered document sets, holds, metadata navigation, document ID, retention policies, IRM, workflows, content organizer, and a bunch of other topics. For each topic, we examined the timer jobs required, the interaction of processes (holds leverage search), the limitations of various capabilities, common issues that people have with various functions, and the recommended approach for applying it. We covered things like the content migration API, discussing what information is retained and what is lost, and the implications of using this in various scenarios such as content deployment.

Monday through Friday for three weeks we arrived in the lab around 7:00 am, took a few very short breaks and an hour for lunch, and were bombarded with incredibly deep technical training until around 6:00 pm, sometimes later. The material is all 400 level, you are not introduced to things like search or user profiles, but rather it is assumed you already know how this stuff already works. They teach you the deep inner workings of how things work, like a fantastic deep dive on user profiles by Spence Harbar or another incredibly deep dive on search by Neil Hodgkinson. This training is an invaluable part of the program where you hear some of the most common issues and things that cause the most support cases for the product team. You learn from people who know this stuff more than anyone else.

For each of the modules below, we went into detail about what service applications are used, the required permissions, the implication of topology choices, services required and where they should run, how to manage them and how to troubleshoot them. Below is a summary of various topics covered, I am asked quite often what kind of information is covered in the course. This is not an exhaustive list, I am probably missing a few and some of these are way deeper than I suggest in this list.

ECM – everything mentioned above plus WCM.
Governance – creating a governance plan.
Planning and Information Architecture – planning topologies and how you might segment information based on requirements. Did you know that TechNet has a bunch of information on planning?
Multi Tenancy – how to configure multi-tenancy, why you shouldn’t do this for an on-premise farm, and design considerations.
Service Applications – how they work, core components, how to create your own service application.
User Profile – My Sites, social, architecture, provisioning, synchronization, troubleshooting, dependencies, interactions with managed metadata and search, enterprise considerations.
Identity – Kerberos, FBA, constrained delegation, claims.
Search – Everything from how to provision the search topology, what services are running where, what service accounts are used and what can be federated or delegated, how the crawler works, how search stores information and where, and what happens when you query.
Search Customization – custom search pages, result refiners, extending search web parts, best bets, scopes, XSLT for core results, BDC results, people results, crawl extensibility.
Business Continuity Management – recycle bin, list export and import, site backup and restore, farm backup and restore, mirroring, log shipping, SQL Server Always On. How SQL backups work and what mirroring and log shipping are really doing, what databases can use log shipping and why.
Upgrades – how to upgrade, best practices, things to watch out for, how to handle customizations and site definitions, how to troubleshoot. How to patch, best practices for patching.
Capacity Planning – how to determine the requirements for your farm and how to test your farm to determine if it can handle the load. How to monitor the farm once deployed, and how to address constraints.
Developer – workflows, data access technologies, schema-based deployment (declarative vs. imperative deployment), web templates, LINQ, REST, application lifecycle management, feature upgrades, management and monitoring custom solutions.
Workflow – Developing, configuration, troubleshooting, performance tuning
Business Connectivity Services – capabilities, APIs, PowerShell, security, search, augmenting user profiles, custom connectors, throttles and limits, performance tuning.

Pretty daunting list? Admittedly, there are going to be topics you are strong at and topics you are not as strong at. Prior to attending the training, you should spend quite a bit of time with the pre-reading materials. We joked that it is basically, “read all of TechNet and all of Steve Peschka’s and Spence Harbar’s blog.” From the list above, that’s not far from the truth. The one thing that I didn’t write but is common to all of the modules: PowerShell. You need to know PowerShell, and if you aren’t comfortable with it before training, you sure the heck will be afterwards.

Keep in mind that this was what we covered for SharePoint 2010 MCM, the list of materials has changed since then.

I was a good student in college, I achieved Dean’s List for the last 2 years of college and generally was able to get by with taking notes, memorizing for tests, and then forgetting most of what I learned after finals. This simply won’t work for this program: you must bring a broad set of experiences with you or the training will be over your head. This is a test of experience and stamina. Being attentive for 9-10 hours per day to deep technical content is exhausting and you get to the point where you start trying to figure out “do I really need to listen here or can I zone out for a minute, can this possibly be on the test?” The answer is yes: this is probably on the test. The information presented in class covers topics that you are expected to already know, just with more detail than you might already know. For instance, you might already know a thing or two about ECM, but do you know all the services required to make everything work as expected? The more familiar you are with search, user profiles, identity and authentication and claims, and service application federation before you attend the class, the more you will get out of the class. If you are hearing a topic for the first time (admittedly, much of it was brand new to me), then you are going to struggle. The MCM certification is just as much a test of what you learned in class as it is your own experience, and it’s really difficult to master a concept you just learned about.

The Hands on Labs

You are given an environment to practice hands on labs to bolster the training that you receive during the day. The training lectures runs Monday through Friday from around 8:00 am to 6:00 pm, so you spend as much time nights and weekends working on the labs as you can. There are some labs that are relatively short, only taking a few hours to complete, while others may never be completed. We were often in the lab until midnight or 1am working on problems, surviving on little sleep while knowing we needed to be alert during the training lectures. Weekends were consumed with these labs as well as studying for the written test. You have to balance your time practicing the things you think you know with the things you are not as strong with.

These labs are not detailed walkthroughs, they are often just a description of a task that you need to accomplish that may seem deceptively easy yet take days to complete. At first, I was frustrated at the lack of direction in the labs and how open-ended they were, but I came to love this about the training course. Rather than show you one way to accomplish a task, you are forced to learn how to do the task on your own and understand various aspects of it. The second-to-last sentence in the lab was usually “verify that a document shows up” or “verify that the search result shows this”. The last sentence was usually a question like “what else could you do with this” or “how else could you accomplish this?”

Here are the HOL topics that were covered.

Logical architecture and governance
Topology
ECM
WCM
Content migration
Basic upgrades
Complex upgrades
Patching
Business Continuity Management
Capacity Planning and Performance
Custom service applications
Visual Studio workflows
Authentication
Platform security
Schema based development
Search topology
Search customization
Business Connectivity Services

My favorite lab was the Topology lab. You are handed this lab on the first day, and wow is it trial by fire. Given a set of servers, provision various services with a desired topology. This lab took me almost two weeks to complete because I was stuck on one problem and couldn’t solve it. Time management is a huge part of this course, spending 2 weeks on that one lab was, in hindsight, stupid because I should have spent more time on the other labs making sure I was prepared for the qualification lab. However, I now have a very firm grasp on user profiles and will never forget that the user profile service must be started before provisioning tenants! You need to work through the hands on labs in order to prepare yourself to pass the written test or the qual lab, it is vital that you work through as much of the labs as possible. Time management is key.

I rarely ate dinner out, opting to have food delivered just so that I could get more time in the lab. On weekends, I would try to start a load of laundry while studying notes and then I’d go back to the lab so that I could work with peers, asking for help on some of the more difficult assignments. It is a delicate balance of sleep deprivation and will power. I attended during fall, during college football season, and I just wanted to go to a bar and watch an NFL game or the Georgia Bulldogs play but instead I buried my face in the glow of the computer monitor to keep practicing, keep studying. Those who know how much I love football know this took tremendous willpower.

The Commitment

To put in the work required means you really have to put aside all other commitments. You aren’t going to be effective if you are having to excuse yourself for a conference call when the instructor is going over material that you really need to hear. You really need to make sure that you can excuse yourself from work during the course duration. The hard part means that you need to stay off email, Facebook, and everything else. I was lucky to have a very supportive group of friends at work who covered for me while I was gone, sent reassuring emails telling me that everything was going great and to keep working hard, and chastised me to get the heck off Facebook and Twitter. I can’t thank you guys enough Chad and Eric.

I have a loving wife and two young children. Every day with them is precious to me, and I fight to create work and life balance, although that often means that I am working from home and they know to find me in my home office. I travel frequently, but usually only for a few days at a time. For the MCM program, I got an apartment near campus and lived there for 3 weeks by myself. There was a 2-hour time difference that meant when I woke up at 6:00 am (recall the sleep deprivation that I emphasized before), my kids were already at school. When I was eating lunch, they weren’t yet home from school, and when training was done at night they were already in bed. I missed my kids so much it hurt. By the end of the second week, I damned near had a breakdown when I checked my voicemail and a tearful daughter said “I miss you daddy.” It gets really hard to focus on listening to technical training all day and then working all night in a lab when all you want to do is to hear your family say I love you. My wife was fantastic, she wouldn’t tell me about any of the times the kids got in trouble, when the oven broke, about a tax audit letter we received, she just wanted me to focus. We usually spend so much time talking to each other every night that this was something we both very much missed while I was away at the program, she didn’t have me to share her highs and lows every day and that really made an impact on her.

On weekends, we tried to use Skype, and it was wonderful to see everyone’s faces and hear their voices… but it was only fleeting. I have to admit that this was something I fully didn’t realize how much it would impact me. I didn’t realize at all how much it would impact my family.

The commitment isn’t just you, it’s your family as well.

The Written Test

Once the three weeks of training have concluded, there is a written test. This is not like any other Microsoft certification test that I have ever taken.

I have long questioned the value of Microsoft certification tests, there are so many stories of young kids passing tests or, worse, people passing tests who have never used the technology before. A friend of mine decided to see how many certifications he could achieve within 2 months, most he had little or no experience with, and he achieved 6 of the 8 he attempted. Simply put: this test is nothing like that. During the lectures, I would often listen to the lecture and try to think of what questions could be asked on the test. I made up my own set of questions that I thought could be fair game on the test. Not even close.

The questions were way harder, and often required you to put multiple complex concepts together to arrive at an answer. There should be a camera outside the door as people leave the written test just to see their reactions, many F bombs are dropped by even the most experienced and attentive students. You are under NDA, so you cannot talk about what’s on the test to each other or compare notes, you can only acknowledge each others’ frustrations. You can’t even blow off steam by going to a bar and drinking too many beers because you are now looking forward to the next day where you take the qualification lab.

Many people do not pass the written test on the first try. To me, this is the hardest one to retake because it requires that you study all of your notes and re-read TechNet and Peschka’s blog and Spence’s blog making sure you understand the finer points, realizing you aren’t getting the same test a second time. One strategy is to do a brain dump of as many questions and topics that you saw and what you aren’t sure if you got right and make sure you study those concepts. You really do not want to put off the retake for too long because it becomes increasingly hard to find the time to study like this.

My strategy during the program was to focus my attention towards studying for the written test as much as I could because I knew that would be harder for me to attempt a second time. I passed the written test on the first try, and I was admittedly shocked to see that I passed it.

The Qualification Lab

You are given a lab environment and a set of tasks to complete in 8.5 hours. The test was open-internet, so you could research whatever you needed to, but honestly if you were looking it up in the first place your chances of passing are very slim. It’s nothing that an average SharePoint professional couldn’t reasonably achieve in a month’s time, but you are given just 8.5 hours and are graded on your ability to complete the tasks as described.

Let me emphasize here… a month’s worth of work in 8.5 hours, done accurately. Either you know it or you don’t. If you know it, you can do it quickly and accomplish it within the time limit, if you are unsure then you are going to waste a bunch of time trying to figure it out. The clock is ticking.

A boxed lunch was served, although I didn’t eat much of it because I didn’t want to lose any focus on completing the tasks. It is stressful, because we all know how easy it is to forget a seemingly small step only to spend hours troubleshooting something. Forget one timer job, and you can find yourself beating your head on the keyboard. This is where it is vitally important to have paid attention in class as well as having experience.

I failed the qualification lab. Twice. It was that hard, and from speaking with many others who have attempted the course, they agree that the combination of the written test and qualification lab are more than an accurate measure of the level of competency expected of an MCM. It is stressful, realizing how many times you are trying to accomplish the task as written and fumbling on things you know how to do, things you do every day for work. I remember thinking during my first attempt at the test that I had a reasonable shot at passing… yeah, not even close. The three weeks of lecture and demos, the late nights in the lab, the 16+ hour days on weekends, all that bolsters what you are expected to already know, it doesn’t teach to the test, and there’s only so much partial credit given for incomplete tasks. Simply put, it was by far the hardest test I have ever taken because you had to be both thorough and fast.

The first time I took the test I honestly wasn’t prepared. I did horrible, but that was my strategy the whole time was to focus on the written test and then plan to retake the qual lab. The second time I took the qual lab, I failed by only a few points and it was completely time management that did me in. I knew all the material, I just took too long to do some of the tasks and the clock ran away from me.

I finally passed the qualification lab on my third and final attempt. The first time you take the qualification lab, you are in a room with the proctor. Your retakes can be done remotely while a proctor monitors you.

The Aftermath

Maybe it took more for me to accomplish this certification than it would for you, maybe you have more experience than I did going in and you wouldn’t struggle as much. Plenty of people pass the written and qual labs during rotation. You should be prepared for what happens if you don’t and understand that if you didn’t pass the first time, you are really going to have to apply yourself to be able to pass the second time.

Many people do not pass one or both of the tests on the first attempt. Some of the MCMs that I look up to most (including some of the instructors) failed one or both parts on the first attempt. Some of the SharePoint hardcore geeks that I look up to did not pass and decided not to even attempt a retake. It’s hard to think about making such a huge time investment (pre-reading study, independent lab practice, three weeks of very long days of lectures and labs, two grueling tests) and then having to make yet another huge time investment afterward. For some, they recognize that they don’t have it to give. For me, I wasn’t about to give up.

After attending the program in person, you have 2 retake attempts before you have to repeat the entire program. I know quite a few people who attempted it and decided not to pursue it after failing the first time, recognizing the amount of work that it would take to prepare for a retake.

Prior to attending, I spent quite a few long hours at my desk at work trying to understand search and user profiles. I spent about a week (which is way too short) studying the pre-reading list. After attending the training program, I studied nights and weekend for over a year. I gained new experiences through working with the product day in and out with customers, bolstering my experience with the product. I presented on some of the material at conferences, and made sure that I was able to cover the areas that I did poorly on. I studied nights, weekends, holidays. I was lucky that many of the customer issues I was handling for work were directly related to the material that I was studying and that gave me additional experience. I felt like I lived, breathed, ate, slept, and dreamed of SharePoint. I drove my family crazy by sitting in my office making up scenarios with the product and trying to solve them to make sure I knew how things worked instead of enjoying time with them outside. On vacations I was distracted, I spent an absurd amount of time focusing on making sure that I was ready for my third and final attempt. I rarely blogged, and when I did I tried to blog about the topics that I was studying. Yes, I worked nights and weekends and busted my tail preparing for my 2nd qualification lab attempt. For my 2nd attempt at the qualification lab, I thought I was ready… I barely failed, but it was still a failure. That only strengthened my resolve that I was going to finish what I started.

For the third qualification lab attempt, I worked nights and weekends and holidays and on vacation. The problem was that it was getting tougher to focus. I still had a day job, and I was now focusing less on SharePoint 2010 and more on SharePoint 2013. I was traveling all over the world teaching developers about the new features in SharePoint 2013 for the Ignite program. I was recording videos for MSDN, editing them, reviewing them, and re-recording them and doing much of that during nights and weekends. That stuff helped me study through experience, but it threatened to reduce the amount of studying that I could do to prepare for the third attempt. I finally had to put everything else on hold and focus for a few weeks solid to prepare. Yes, I took vacation just to study for MCM.

This is the part that many don’t plan on, the aftermath of the program. If you don’t pass the first time, it takes a dedicated effort to make sure that you are ready for the next attempt. It takes quite a bit of sacrifice. It’s sometimes hard to explain to my non-technical friends that I sacrificed a significant amount of family time in order to pass a certification exam. This program consumed my life for most of 18 months to the point that I questioned what possible benefit could I receive to justify all this hard work other than a title to add to my resume. I thought several times about quitting, but I just couldn’t quit, that’s not who I am. Instead I sacrificed quite a bit to accomplish a goal that I had set for myself.

I failed the first and second attempt at the qualification lab, and finally passed on my third and final attempt. Funny enough, I am glad I had the opportunity to take the lab three times as I was able to see a wide number of real-world scenarios that made me go back to my lab at home and work on them to completely understand them for the next test attempt. Even if I didn’t understand them enough to pass the test the first time, I sure the heck understand them now! The level of experience that I took away from just the qual labs and studying is invaluable.

The Reward

This was a personal goal, a way to prove to myself that I could take my skills to the next level. Looking back on where I was before I started this program, I can say it was definitely worth my time to better myself as a SharePoint professional. I learned so much throughout the program and made so many friends through the experience that professionally it was worth the investment. I can now speak much more confidently about how the product works. I now have a network of peers who went through the same certification process and passed, and I have this incredibly rich network of professionals to draw from. I can sit in a room with the lead architect of a Fortune 50 company and provide factual guidance on how something should or should not be implemented. I gained the respect of my peers because I showed so much dedication to finishing what I started.

For me, this was much more than trying to just get a promotion at work. Actually, I haven’t yet realized any monetary rewards and I’m not sure that I will, but I have gained a few new opportunities that otherwise I would not have had. For me, the real payoff was the sense of accomplishment, something I can show my kids that I didn’t give up and that I worked extremely hard to achieve a goal. Not everyone will appreciate just what it took for me to accomplish this goal, but my family knows.

I did it. I set a goal, I worked hard but not hard enough, then I buckled down and really pushed through. I am proud of myself, and not at all ashamed to tell anyone how hard I worked towards this goal. I don’t brag about being an MCM and don’t flaunt my credentials, in fact it was a tremendously humbling experience. That said, I am not at all bashful about telling you how hard I worked to get there. And I certainly am telling my kids.

There is nothing more valuable to me than being able to show my kids that it was tough, but if you work hard you can achieve your goals. Once you accomplish your goals, sit back, reflect, and appreciate the hard work that you put into it and share that with others.

Inside SharePoint 2013 OAuth Context Tokens

Kirk Evans[MSFT] — Fri, 05 Apr 2013 17:43:16 GMT

This post will show you how to inspect the SharePoint 2013 context token to better understand how OAuth is used in SharePoint 2013 apps.

First, Some Context

In order to use a context token with SharePoint 2013 apps, you will need to create a provider-hosted app that uses a client ID and a client secret. This requires that the target SharePoint farm has a trust configured to Azure Access Control Services, or ACS. Office365 automatically configures this trust for you, so if you create your app using Office365 then this works just fine. If you create your app using your own SharePoint farm, then you will need to configure low-trust apps for on-premises deployments in order to establish a trust with Azure ACS for your SharePoint farm.

When you use a client ID and a client secret to build your app, you are using Azure ACS as the authentication server. The OAuth flow looks like this:

This is what I jokingly refer to as “the scary slide”. It’s not really that scary once you understand it.

User makes a request to SharePoint
SharePoint requests the context token from Azure ACS
The signed context token is returned
The HTML markup for the SharePoint page is sent to the client
If the SharePoint page contents contain a client app part, then a request is sent to the remote web server for the IFrame’s contents. If the user simply clicked on the app to launch it, then a page in SharePoint called appredirect.aspx is used to redirect to the app’s start URL. In either case, a context token is sent along with that page request in a form parameter called SPAppToken.
The context token is a JWT token. The JWT token is a set of base64 encoded claims. One claim is named “refreshtoken” that has a base64 encoded value. One of the claims is “appctx” which contains a child object, one of the properties is SecurityTokenServiceUri with a value https://accounts.accesscontrol.windows.net/tokens/OAuth/2. Your app uses the TokenHelper.cs class to extract the refreshtoken and then send it to the SecurityTokenServiceUri to request an access token.
The access token is retrieved from ACS.
The access token is passed to the SharePoint site in the HTTP header Authorization that has a value beginning with “Bearer” and has your access token. This token is passed in the HTTP header to the CSOM endpoint _vti_bin/client.svc/ProcessQuery when you use the CSOM.
If the app is authorized, SharePoint returns the data requested.
Finally, the app server returns the requested HTML markup as a response.

The really important thing to understand is what’s passed to your app, the context token. The context token contains a few key pieces of information necessary for the rest of the plumbing to function.

What’s In Your Wallet?

Now that we’ve walked through the steps of the OAuth dance, let’s take a look at what’s in that context token that is critical to the whole process. When you use Visual Studio to create a provider-hosted app, the boiler plate code looks like this:

protected void Page_Load(object sender, EventArgs e)
{
    // The following code gets the client context and Title property by using TokenHelper.
    // To access other properties, you may need to request permissions on the host web.

    var contextToken = TokenHelper.GetContextTokenFromRequest(Page.Request);
    Response.Write(contextToken + "<br/>");
    var hostWeb = Page.Request["SPHostUrl"];

    using (var clientContext = TokenHelper.GetClientContextWithContextToken(hostWeb, contextToken, Request.Url.Authority))
    {
        clientContext.Load(clientContext.Web, web => web.Title);
        clientContext.ExecuteQuery();
        Response.Write(clientContext.Web.Title);
    }
}

Reading that code, you can see that the context token is passed to your app, but it is a base64 encoded string that is unintelligible. I added a Response.Write statement to write the context token to the page, but you can also see it in Fiddler. In the following screen shot, I highlighted the SPAppToken HTTP form variable where the context token is passed.

Once you have that context token, you have the base64 encoded value of the context token, which is a JWT token. The JWT token format is pretty straightforward, and the TokenHelper.cs class in Visual Studio simply parses that JWT token for you. However, for the curious who want to see what it looks like, you can use a tool like http://openidtest.uninett.no/jwt to inspect the JWT token. Copy the context token string into the “Encoded JWT” text box and the client secret into the “Secret” text box, and then press decode.

What is returned is the following JSON object, which is a JWT token that contains a set of claims.

{
    "aud": "4c2df2aa-3d14-4d84-8a79-5a75135e98d0/localhost:44346@d341a536-1d82-4267-87e6-e2dfff4fa325",
    "iss": "00000001-0000-0000-c000-000000000000@d341a536-1d82-4267-87e6-e2dfff4fa325",
    "nbf": 1365177964,
    "exp": 1365221164,
    "appctxsender": "00000003-0000-0ff1-ce00-000000000000@d341a536-1d82-4267-87e6-e2dfff4fa325",
    "appctx": "{\"CacheKey\":\"em1/saZohTOS4nOUZHXMb8QJgyNbkEO86TSe5j9WYmo=\",
\"SecurityTokenServiceUri\":\"https://accounts.accesscontrol.windows.net/tokens/OAuth/2\"}",
    "refreshtoken": "IAAAANc8bAVMWZceOsdfgsdfggbfm7oU_aM7D2qofUpQstMsdfgsdfgfYS0OtbZ-
eY9UQGvlYSl5kpPi913G1AwIVBMxoCux8-bhcCCiaGVo-vuFzrXetdhRGPftQdHh-
1rS5cvDuuQ_bw_mjySIyuHNGSavEs8HUgHY9BOVc3pTGZtZ_nS-
1NbDLYObjnznasdfasdfasdfQreLAeeOpVRY1PGsdfgsdfgOITA3BKhjJFz_40YJMubdHmY2OTS
nqwNnUe-rBBCtfvKt4xFWvdRzTzwfW",

    "isbrowserhostedapp": "true"
}

You can now see that the context token contains the refresh token as a base64 encoded value.

What are the claims in the context token?

The following shows the properties for the context token.

aud

Short for “audience”, means the principal the token is intended for. The format is <client ID>/<target URL authority>@<target realm>. Based on this information, you can determine the client ID for your app and the realm. In an on-premise environment, there is typically just one realm, and its identifier matches your farm ID. For Office 365, this is your tenant ID.

iss

Short for “issuer”, this is the principal that issued the token, in the form of <principal ID>@<realm>. The principal ID value 00000001-0000-0000-c000-000000000000 is ACS.

nbf

Short for “not before”, this is the number of seconds after January 1, 1970 (part of the JWT specification) that the token starts being valid.

exp

Short for “expires”, represents the number of seconds after January 1, 1970 that the token stops being valid.

appctxsender

The sender of the token in the form <sender ID>@<realm>. The value 00000003-0000-0ff1-ce00-000000000000 is the identifier for SharePoint. For trivia:

ACS	00000001-0000-0000-c000-000000000000
Exchange	00000002-0000-0ff1-ce00-000000000000
SharePoint	00000003-0000-0ff1-ce00-000000000000
Lync	00000004-0000-0ff1-ce00-000000000000
Workflow	00000005-0000-0000-c000-000000000000

The realm will be the tenant ID for Office 365, or the farm ID for your on-premise deployment.

appctx

Contains two properties, CacheKey and SecurityTokenServiceUri.
CacheKey:
UserNameId + "," + UserNameIdIssuer + "," + ApplicationId + "," + Realm
This is provided so that you can cache the value in a cookie or in session to identify that the user has already authenticated.

SecurityTokenServiceUri:
The URL for Azure ACS where the token is to be validated. The URL is https://accounts.accesscontrol.windows.net/tokens/OAuth/2.

refreshtoken

The contents of the refresh token that are sent to Azure ACS.

isbrowserhostedapp

Indicates if the request initiated from a user interacting with the browser and not an app event receiver

During Ignite training, I tell the attendees that the most important thing you can understand when building apps for SharePoint is OAuth. You need to understand the OAuth dance, the reason for the context token, and it is invaluable to understand what the TokenHelper.cs is doing for you when it does things like GetClientContextFromContextToken. This post goes into some of the details on why this is such an important piece to understand when troubleshooting apps. You can also see why it is mandatory for apps to SSL due to the nature of information being sent in HTTP.

You can find more information about the context token, calculating nbf and exp claim values, and links to additional articles at http://msdn.microsoft.com/en-us/library/office/apps/fp179932.aspx.

For More Information

configure low-trust apps for on-premises deployments

JWT token format

Tips and FAQs: OAuth and remote apps for SharePoint 2013

Access 2013–Not Your Father’s Access

Kirk Evans[MSFT] — Tue, 02 Apr 2013 15:20:21 GMT

I worked this weekend to prepare for a presentation this week at the Dallas SharePoint TechFest, “Access Services 2013 – Not Your Father’s Access!” I am covering the new features of Access 2013 as an app designer for SharePoint 2013. While presenting on Access, I frequently say that while Access has changed drastically, Access remains familiar because key parts of the UI have barely changed since 1993. As proof, I installed Windows XP and Access 2.0 and took some screen shots for comparison.

Creating a table:

Creating a query:

Even the wizards to create tables have pretty much the same elements as they did back in Access 2.0 with the ability to choose from out of box templates.

The thing that is drastically different is the ability for end users to use Access 2013 with Access Services 2013 to create apps for SharePoint 2013. That’s the part that I am very excited about because it provides the ability for end users to create composite applications without requiring developers. Looking forward to speaking at the event!

New Features in Microsoft Office Developer Tools for Visual Studio 2012

Kirk Evans[MSFT] — Mon, 01 Apr 2013 14:12:00 GMT

The Microsoft Office Developer Tools for Visual Studio 2012 have been released and have some fantastic new additions in them for building apps. This blog post covers some of the new features that you’ll want to be familiar with.

New Project Item Templates

There are a few project item templates that have been added since the Preview 2 tools.

Search Configuration Project Item

An interesting addition is the new Search Configuration project item template. To use this, create a search center and then go to Site Settings / Search / Configuration Export.

This will export the search settings as an XML file. When you use the Search Configuration project item in Visual Studio, it prompts you for this XML file. This enables you to deploy search settings such as ranking models and managed properties as part of your app! For more information on deploying search customizations, see the MSDN topic How to: Deploy custom search configurations by using Visual Studio.

App for Office Project Item

Another project item to notice is the App for Office project item that can be added to a SharePoint hosted app or a provider hosted app. This project item enables you to define your app and bind it to a document, which can then be used in your project to set as the default template of a document library. Add the new App for Office to a SharePoint hosted app:

The next screen asks if you want to create a new document or use an existing document:

Add a new list to the project, creating a customizable list based on the Document Library type.

Here’s the really cool part. Prior to the RTM version of the tools, this is where things stopped, and you had to do some work with the element manifest files to deploy the document and set it as the document template. Visual Studio now saves you this extra step, allowing you to choose the document created in the previous step that has a reference to your app.

To test, I changed the start URL of my app to ~appWebUrl/Lists/MyDocs?{StandardTokens} and press F5. The app opens to my new document library, and the content type is available in the New menu.

Create a new “MyDocsContentType” document, and when the document is opened, it will have your app already in it!

Personally, I love this feature. There used to be many more steps involved in this, and this makes the entire process so incredibly simple. Even better, the content for your app is being served from SharePoint, so there’s no need to go search for somewhere to host your app for Office.

For more information on deploying an App for Office as part of your SharePoint app, see the MSDN topic How to: Create an app for SharePoint that contains a document template and a task pane app.

Debugging Remote Event Receivers

One of my favorite additions in the tools is the ability to use Windows Azure Service Bus for debugging remote event receivers. This was a challenge with previous releases of the tools because remote event receivers are basically HTTP calls made from the SharePoint server. If you develop an autohosted app, add a remote event receiver, add a breakpoint and press F5 in Visual Studio, your event receiver will not fire because your web application is running as the address localhost, the remote server cannot resolve “localhost” to your machine. The new developer tools make this simple by leveraging Windows Azure Service Bus. Start by logging into the Azure Portal and add a new namespace.

Once the namespace is created, you can use the “Access Key” button to discover the connection string for your service bus endpoint.

Now go to the properties / SharePoint tab for your app manifest project and paste the Windows Azure Service Bus connection string.

Add a breakpoint, hit F5, and watch the goodness as the web site containing your remote event receiver service is running under localhost and the breakpoint is still hit!

To learn more about remote event receivers in SharePoint 2013, see the MSDN topic Handling events in apps for SharePoint.

App for Office Validation

I was very happy to see that a new feature has been added when publishing your app for Office… validation! This validation is representative of some of the checks that will be made when you submit your app to the Office marketplace. Simply check the checkbox to “Validate my app for the Office Store” on the publish window and choose Finish.

Your app will then be validated and a report shown of various issues that could prevent your app from successfully being submitted to the Office Store.

To learn more about the validation policies for apps submitted to the Office Store, see the MSDN topic Validation policies for the apps submitted to the Office Store (version 1.3).

For More Information

Download Microsoft Office Developer Tools for Visual Studio 2012

How to: Deploy custom search configurations by using Visual Studio

How to: Create an app for SharePoint that contains a document template and a task pane app

Handling events in apps for SharePoint

Validation policies for the apps submitted to the Office Store (version 1.3)

SharePoint 2013 App Only Policy Made Easy

Kirk Evans[MSFT] — Sat, 23 Feb 2013 16:12:59 GMT

This post demonstrates the app only policy for SharePoint 2013 app development and show how to create SharePoint 2013 apps using OAuth tokens with either a trust to ACS as well as S2S. I show how to request the AllowAppOnlyPolicy permission and how to execute actions that the current user is not authorized to perform directly.

Background

As developers start working with SharePoint 2013 apps, you quickly realize how permissions with apps work when performing work on behalf of a user. I can log in as the site collection administrator, and execute an app that writes to a list and see the action fail because permission was denied. The reason it was denied is because the app did not request write permissions to the list it was trying to write to. Similarly, I can log in as a user who only has read permission to a list and invoke an app that has full control permission and see an attempt to write to a list fail because permission was denied. The reason this time is that even though the app had permissions, the user did not.

This is explained in the following diagram. The app is making a call into SharePoint, and it does so by providing the OAuth access token. SharePoint sees that no user credentials were provided in the request and an OAuth access token is present. When acting on behalf of a user, the token will include user information, so the context will be the App+User context.

This is exactly the scenario I just described. The App+User policy is applied, and both the user and the app must have sufficient permission to write to the list.

Can I Elevate Privileges?

When presenting this material to developers, this is usually right about the time someone asks me about the RunWithElevatedPrivileges API. It should be renamed to “RunWithScissors”, because so many developers use it but don’t quite understand what the API is doing, and it is this confusion that usually generates support calls on how to properly use the API in a predictable manner. The new app model does away with this impersonation confusion and makes things quite a bit more straightforward because there is no impersonation capability in the API.

And when I say this to developers looking at apps for the first time, they usually want to throw things at me. How the heck am I supposed to get things done?!? Give users permissions to everything?!?

Let’s analyze the requirement instead of focusing on how we arrive at the solution. We need to perform an action that the user is not entitled to. OK, now that we agree on semantics, how, then, do I use the new app model to perform actions when the user doesn’t have permission? In the scenario presented earlier, how can I create an app that has privileges to do something such as writing to a list even though the current user does not have permissions to write to the list? The answer is the App Only Policy.

Don’t think of this as impersonation or elevation of the current user’s privileges, it’s not. In fact, we are not going to use the permissions of the current user at all.

The App Only Policy

In the previous diagram, we highlighted a flow where an app is calling into SharePoint using the CSOM. Our new scenario of an app performing work that the user does not have permission to means we will use the app only policy. Following the flow diagram from the Start node, there is no user credential provided, an OAuth access token is present, and this time the token does not contain user information. In this case, we are solely evaluating permissions based on the permission of the app and not on the user.

Doing this is actually pretty straightforward. The first step is to request permission to use the app only policy in your app manifest by adding the AllowAppOnlyPolicy attribute to the AppPermissionRequests node with a value of true.

When your app requests this permission, a subtle change is reflected in the trust dialog. You now see the text “Let it share its permissions with other users.”.

This text is letting you know that the app is requesting permission to perform actions that the user may not have permission to. In this dialog, the app will be granted Write permission (which means read, create, edit, or delete items) in the Announcements list and will be able to do so whether the user has permission or not.

This capability is only available to provider-hosted and Azure auto-hosted apps. It is not available to SharePoint-hosted apps. In a SharePoint-hosted app, there is necessarily always an app+user context.

App Only Context with S2S

Creating a provider-hosted app that leverages only policy using S2S is actually very easy. The first step is to make sure that you add the AllowAppOnlyPolicy attribute in the app manifest. The next step is to use the TokenHelper::GetS2SAccessTokenWithWindowsIdentity method, passing a null for the WindowsIdentity parameter.

string appOnlyAccessToken = 
   TokenHelper.GetS2SAccessTokenWithWindowsIdentity(_hostWeb, null);

If the app manifest has the AllowAppOnlyPolicy attribute set to true, this call will succeed and return a valid OAuth access token that does not contain user information. Once you have the access token, then you can use the TokenHelper::GetClientContextWithAccessToken method to obtain a CSOM client context.

using (ClientContext clientContext = 
    TokenHelper.GetClientContextWithAccessToken(_hostWeb.ToString(), appOnlyAccessToken))
{
    List list = clientContext.Web.Lists.GetByTitle("Announcements");
    ListItemCreationInformation info = new ListItemCreationInformation();
    Microsoft.SharePoint.Client.ListItem item = list.AddItem(info);
    item["Title"] = "Created from CSOM";
    item["Body"] = "Created from CSOM " + DateTime.Now.ToLongTimeString();

    item.Update();
    clientContext.Load(item);
    clientContext.ExecuteQuery();
}

To test this, deploy the app and then log on as a user that only has read permission to the list. Execute the code, and a new item is created even though the user does not have permission to create items in the list. The Created By and Modified By fields in the list will reflect that it was only the app’s permissions that were used to create the item.

If we had included a user identity (as the code generated by Visual Studio does by default), then the created by and modified by fields would look a little different, showing that code was executed on behalf of an individual.

App Only Context with ACS

Creating an app-only context with a provider-hosted app using a trust to ACS is just slightly more involved. A trust to ACS is automatically established when you create your O365 tenant. If you are self-hosting your SharePoint farm, then a trust to ACS can be established between your SharePoint farm and ACS.

We use a similar approach as before, but this time we need to extract the context token string and validate the token in order to retrieve information such as the authentication realm. Provided this information, we can now request an app-only token using TokenHelper::GetAppOnlyAccessToken. Once we have the access token, we can now obtain the client context using TokenHelper::GetClientContextWithAccessToken.

string _hostWeb = new Uri(Request.QueryString["SPHostUrl"]);
string _contextTokenString = TokenHelper.GetContextTokenFromRequest(Page.Request);

//Get context token.
SharePointContextToken contextToken =
    TokenHelper.ReadAndValidateContextToken_contextTokenString, Request.Url.Authority);

//Get app only access token.
string appOnlyAccessToken = 
    TokenHelper.GetAppOnlyAccessToken(contextToken.TargetPrincipalName, 
            _hostWeb.Authority, contextToken.Realm).AccessToken;

Response.Write("<h2>Valid app-only access token retrieved</h2>");
Response.Write("<p>" + appOnlyAccessToken + "</p>");
Response.Flush();

using (ClientContext clientContext = 
    TokenHelper.GetClientContextWithAccessToken(_hostWeb.ToString(), appOnlyAccessToken))
{
    List list = clientContext.Web.Lists.GetByTitle("Announcements");
    ListItemCreationInformation info = new ListItemCreationInformation();
    Microsoft.SharePoint.Client.ListItem item = list.AddItem(info);
    item["Title"] = "Created from CSOM";
    item["Body"] = "Created from CSOM " + DateTime.Now.ToLongTimeString();

    item.Update();
    clientContext.Load(item);
    clientContext.ExecuteQuery();
}

The result is the same as before… a user that does not have write permission to a list is able to execute the app, which has been granted the AllowAppOnlyPolicy in the permission requests in the app manifest, and the write to list operation now succeeds.

For More Information

App authorization policy types in SharePoint 2013

SharePoint Low-Trust Apps for On-Premises Deployments

Configuring LDAP for FBA in SharePoint 2010 or SharePoint 2013 with PowerShell

Kirk Evans[MSFT] — Thu, 31 Jan 2013 18:14:00 GMT

This post provides PowerShell script to easily configure forms based authentication using LDAP in SharePoint 2010 or 2013.

A long time ago, I wrote a post that shows how to configure the SQL Server Provider for FBA in SharePoint 2010. I also wrote an accompanying post on how to automate configuring FBA with SqlMembershipProvider in SharePoint 2010 using PowerShell. Those posts also apply to SharePoint 2013 as well, you just need to change the path to the STS web.config to point to the path in SharePoint 2013.

Today I was troubleshooting a configuration that uses LDAP to authenticate users using the LdapMembershipProvider, enabling LDAP authentication in SharePoint using FBA. Admittedly, I stumbled a few times due to the slight differences between the settings in Central Administration, the STS, and the application’s web.config settings. Once I got it working, I decided to create a PowerShell script that would let me easily configure an LDAP provider for SharePoint 2010 or SharePoint 2013 and take the guesswork out of it while making the process less error-prone.

What Does It Do?

The code isn’t terribly difficult, and follows the directions from the TechNet article Configure forms-based authentication for a claims-based Web application (SharePoint Server 2010) closely. It uses PowerShell to update the web.config for the web application, Central Administration, and the STS for every server in your farm.

The “main” part of the script is at the bottom of the page. This is where we ask for all servers in the farm that are application servers (not SQL, Active Directory, or Exchange servers) and iterate through all of them. Note that the currently logged in account must have read-write permissions to all of the servers in the farm and have access to the C drive on each. Also notice that a backup file is created in the directory using the form “yyyy MM dd HH mm.web.config.bak” so that, should something go wrong, you can easily revert back to a previous version of the configuration file.

I did not edit the script to overwrite previous changes, I leave that as an exercise to the reader. The good news is that you can just go back to a previous backup file (one of the .bak files that this script creates) and simply rename it to web.config and you are back to where you started.

What Do I Need to Change?

The only part of the script that you should need to change are the variables at the bottom of the page.

#Update with path to Central Administration web.config 
$pathToCentralAdminConfig = "c:\inetpub\wwwroot\wss\VirtualDirectories\37552\web.config" 
#Update with path to web application's web.config 
$pathToWebApplicationConfig = "c:\inetpub\wwwroot\wss\VirtualDirectories\DevTeam.Contoso.lab80\web.config" 
#Update with the correct LDAP server 
$ldapServer = "dc.contoso.lab"
#Update with the correct container name
$userContainer = "OU=EMPLOYEES,DC=CONTOSO,DC=LAB"

The article Configure forms-based authentication for a claims-based Web application (SharePoint Server 2010) has a good walkthrough of the settings and changes. The ldapServer variable is the LDAP server that you will authenticate against, and the userContainer variable is the container that holds your users.

If you want to find out what the correct container name is, Mirjam van Olst provides a great trick in her blog post “Configuring claims and forms based authentication for use with an LDAP provider in SharePoint 2010”. Just go to Active Directory Users and Computers, right-click the container name and choose All Tasks / Resultant Set of Policy (Planning).

The resulting screen will provide the container name.

The configuration for SharePoint 2010 and SharePoint 2013 are identical, except the path to the STS web.config. If you are using SharePoint 2013, you will also want to update the path to the STS web.config in the Main function within the script.

$stsConfigPath = "\\$name\c$\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\WebServices\SecurityToken\web.config"

Note also that the script assumes a membership provider name of “LdapMember” and role provider name of “LdapRole”. These are the values you will enter into Central Administration when configuring FBA.

Show Me The Code!

The code is available as an attachment to this post as well. As usual, this code is provided as-is, no warranties, use at your own risk.

function CreateBackupFile($xmlDoc, $path)
{
    $date = Get-Date    
    $dateString = $date.ToString("yyyy MM dd H mm")
    $backupPath = $path.Replace("web.config", "$dateString.web.config.bak")
    $xmlDoc.Save($backupPath)
}

function AddPeoplePickerWildcard($xmlDoc)
{    
    
    
    $newPeoplePickerNode = $xmlDoc.selectSingleNode("/configuration/SharePoint/PeoplePickerWildcards/add[@key='LdapMember']");
    if(!$newPeoplePickerNode)
    {

        $peoplePickerNode = $xmlDoc.selectSingleNode("/configuration/SharePoint/PeoplePickerWildcards")
        $newPeoplePickerMemberNode = $xmlDoc.CreateNode("element", "add", "")
        
        $peoplePickerKeyAttr = $xmlDoc.CreateAttribute("key");
        $peoplePickerKeyAttr.Value = "LdapMember";
        $newPeoplePickerMemberNode.Attributes.Append($peoplePickerKeyAttr)
        
        $peoplePickerValueAttr = $xmlDoc.CreateAttribute("value");
        $peoplePickerValueAttr.Value = "*"        
        $newPeoplePickerMemberNode.Attributes.Append($peoplePickerValueAttr)
        
        $peoplePickerNode.AppendChild($newPeoplePickerMemberNode)
        
        $newPeoplePickerRoleNode = $xmlDoc.CreateNode("element","add","")
        
        $peoplePickerKeyAttr = $xmlDoc.CreateAttribute("key");
        $peoplePickerKeyAttr.Value = "LdapRole";
        $newPeoplePickerRoleNode.Attributes.Append($peoplePickerKeyAttr)
        
        $peoplePickerValueAttr = $xmlDoc.CreateAttribute("value");
        $peoplePickerValueAttr.Value = "*"        
        $newPeoplePickerRoleNode.Attributes.Append($peoplePickerValueAttr)        
        
        $peoplePickerNode.AppendChild($newPeoplePickerRoleNode)
        
    }
    
}


function AddMembership($xmlDoc, $ldapServer, $userContainer, $userFilter)
{
    $membershipAddNode = $xmlDoc.selectSingleNode("/configuration/system.web/membership/providers/add[@name='LdapMember']")
    
    if(!$membershipAddNode)
    {
        #The membership node doesn't exist
        $membershipNode = $xmlDoc.selectSingleNode("/configuration/system.web/membership")
        $providerNode = $null
        
        if(!$membershipNode)
        {
            $membershipNode = $xmlDoc.CreateNode("element", "membership", "")        
            $providerNode = $xmlDoc.CreateNode("element","providers","")
            $membershipNode.AppendChild($providerNode)
            $xmlDoc.selectSingleNode("/configuration/system.web").AppendChild($membershipNode)                
        }
        
        $providerNode = $xmlDoc.selectSingleNode("/configuration/system.web/membership/providers")
        $membershipAddNode = $xmlDoc.CreateNode("element","add","")
        
        $membershipNameAttr = $xmlDoc.CreateAttribute("name")
        $membershipNameAttr.Value = "LdapMember"
        $membershipAddNode.Attributes.Append($membershipNameAttr)
        
        $membershipTypeAttr = $xmlDoc.CreateAttribute("type")        
        $membershipTypeAttr.Value = "Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
        $membershipAddNode.Attributes.Append($membershipTypeAttr)
        
        
        $membershipServerAttr = $xmlDoc.CreateAttribute("server")
        $membershipServerAttr.Value = $ldapServer
        $membershipAddNode.Attributes.Append($membershipServerAttr)

        $membershipPortAttr = $xmlDoc.CreateAttribute("port")
        $membershipPortAttr.Value = "389"
        $membershipAddNode.Attributes.Append($membershipPortAttr)
        
        $membershipuseSSLAttr = $xmlDoc.CreateAttribute("useSSL")
        $membershipuseSSLAttr.Value = "false"
        $membershipAddNode.Attributes.Append($membershipuseSSLAttr)

        $membershipuserDNAttribute = $xmlDoc.CreateAttribute("userDNAttribute")
        $membershipuserDNAttribute.Value = "distinguishedName"
        $membershipAddNode.Attributes.Append($membershipuserDNAttribute)
        
        $membershipuserNameAttribute = $xmlDoc.CreateAttribute("userNameAttribute")
        $membershipuserNameAttribute.Value = "sAMAccountName"
        $membershipAddNode.Attributes.Append($membershipuserNameAttribute)
        
        
        $membershipuserContainer = $xmlDoc.CreateAttribute("userContainer")
        $membershipuserContainer.Value = $userContainer
        $membershipAddNode.Attributes.Append($membershipuserContainer)
              
        $membershipuserObjectClass = $xmlDoc.CreateAttribute("userObjectClass")
        $membershipuserObjectClass.Value = "person"
        $membershipAddNode.Attributes.Append($membershipuserObjectClass)
        

        $membershipuserFilter = $xmlDoc.CreateAttribute("userFilter")
        $membershipuserFilter.Value = $userFilter
        $membershipAddNode.Attributes.Append($membershipuserFilter)

        $membershipScope = $xmlDoc.CreateAttribute("scope")
        $membershipScope.Value = "Subtree"
        $membershipAddNode.Attributes.Append($membershipScope)
        
        $membershipotherRequiredUserAttributes = $xmlDoc.CreateAttribute("otherRequiredUserAttributes")
        $membershipotherRequiredUserAttributes.Value = "sn,givenname,cn"
        $membershipAddNode.Attributes.Append($membershipotherRequiredUserAttributes)
        
        $providerNode.AppendChild($membershipAddNode)

    }
}

function AddRoles($xmlDoc, $ldapServer, $userContainer, $userFilter, $groupFilter)
{
    #Check to see if it was already created, and if not, create it
    $rolesAddNode = $xmlDoc.selectSingleNode("/configuration/system.web/roleManager/providers/add[@name='LdapRole']")
    if(!$rolesAddNode)
    {
        $rolesNode = $xmlDoc.selectSingleNode("/configuration/system.web/roleManager")
        $providerNode = $null
        
        if(!$rolesNode)
        {
            $rolesNode = $xmlDoc.CreateNode("element", "roleManager", "")        
            $providerNode = $xmlDoc.CreateNode("element","providers","")
            $rolesNode.AppendChild($providerNode)
            $xmlDoc.selectSingleNode("/configuration/system.web").AppendChild($rolesNode)                
        }
        $rolesNode = $xmlDoc.selectSingleNode("/configuration/system.web/roleManager")
        $rolesEnabledAttr = $xmlDoc.CreateAttribute("enabled");
        $rolesEnabledAttr.Value = "true";
        $rolesNode.Attributes.Append($rolesEnabledAttr)
        
        $providerNode = $xmlDoc.selectSingleNode("/configuration/system.web/roleManager/providers")
        $rolesAddNode = $xmlDoc.CreateNode("element","add","")
        
        $roleNameAttr = $xmlDoc.CreateAttribute("name")
        $roleNameAttr.Value = "LdapRole"
        $rolesAddNode.Attributes.Append($roleNameAttr)
        
        $roleTypeAttr = $xmlDoc.CreateAttribute("type")
        $roleTypeAttr.Value = "Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
        $rolesAddNode.Attributes.Append($roleTypeAttr)

        
        $rolesserverAttr = $xmlDoc.CreateAttribute("server")
        $rolesserverAttr.Value = $ldapServer
        $rolesAddNode.Attributes.Append($rolesserverAttr)

        $rolesPortAttr = $xmlDoc.CreateAttribute("port")
        $rolesPortAttr.Value = "389"
        $rolesAddNode.Attributes.Append($rolesPortAttr)
        
        $rolesuseSSL = $xmlDoc.CreateAttribute("useSSL")
        $rolesuseSSL.Value = "false"
        $rolesAddNode.Attributes.Append($rolesuseSSL)
        
        
        $rolesgroupContainer = $xmlDoc.CreateAttribute("groupContainer")
        $rolesgroupContainer.Value = $userContainer
        $rolesAddNode.Attributes.Append($rolesgroupContainer)        

        $rolesgroupNameAttribute = $xmlDoc.CreateAttribute("groupNameAttribute")
        $rolesgroupNameAttribute.Value = "cn"
        $rolesAddNode.Attributes.Append($rolesgroupNameAttribute)           

        $rolesgroupNameAlternateSearchAttribute = $xmlDoc.CreateAttribute("groupNameAlternateSearchAttribute")
        $rolesgroupNameAlternateSearchAttribute.Value = "samAccountName"
        $rolesAddNode.Attributes.Append($rolesgroupNameAlternateSearchAttribute)    
        
        $rolesgroupMemberAttribute = $xmlDoc.CreateAttribute("groupMemberAttribute")
        $rolesgroupMemberAttribute.Value = "member"
        $rolesAddNode.Attributes.Append($rolesgroupMemberAttribute)    
                
                
        $rolesuserNameAttribute = $xmlDoc.CreateAttribute("userNameAttribute")
        $rolesuserNameAttribute.Value = "sAMAccountName"
        $rolesAddNode.Attributes.Append($rolesuserNameAttribute)    
        
        $rolesdnAttribute = $xmlDoc.CreateAttribute("dnAttribute")
        $rolesdnAttribute.Value = "distinguishedName"
        $rolesAddNode.Attributes.Append($rolesdnAttribute)            


        $rolesgroupFilter = $xmlDoc.CreateAttribute("groupFilter")
        $rolesgroupFilter.Value = $groupFilter
        $rolesAddNode.Attributes.Append($rolesgroupFilter)     
                                
        $rolesuserFilter = $xmlDoc.CreateAttribute("userFilter")
        $rolesuserFilter.Value = $userFilter
        $rolesAddNode.Attributes.Append($rolesuserFilter)    

        $rolesscope = $xmlDoc.CreateAttribute("scope")
        $rolesscope.Value = "Subtree"
        $rolesAddNode.Attributes.Append($rolesscope)    
                                        
                                        
        $providerNode.AppendChild($rolesAddNode)
    }
}

function ProcessCentralAdmin($path, $ldapServer, $userContainer)
{

    $content = Get-Content -Path $path
    [System.Xml.XmlDocument] $xd = new-object System.Xml.XmlDocument
    $xd.LoadXml($content)

    CreateBackupFile $xd $path
    

    
    #Add People Picker Wildcard
    AddPeoplePickerWildcard $xd
    
    #Add Roles
    AddRoles $xd $ldapServer $userContainer "((ObjectClass=person)" "((ObjectClass=group)"
    $roleNode = $xd.selectSingleNode("/configuration/system.web/roleManager")
    $defaultRoleProviderAttr = $xd.CreateAttribute("defaultProvider")
    $defaultRoleProviderAttr.Value = "AspNetWindowsTokenRoleProvider"
    $roleNode.Attributes.Append($defaultRoleProviderAttr)
            
    #Add Membership
    AddMembership $xd $ldapServer $userContainer "(ObjectClass=person)"
    
    $membershipNode = $xd.selectSingleNode("/configuration/system.web/membership")
    $defaultMembershipProviderAttr = $xd.CreateAttribute("defaultProvider")
    $defaultMembershipProviderAttr.Value = "LdapMember"
    $membershipNode.Attributes.Append($defaultMembershipProviderAttr)        
       
    $xd.Save($path)
}

function ProcessWebApplication($path, $ldapServer, $userContainer)
{
    $content = Get-Content -Path $path
    [System.Xml.XmlDocument] $xd = new-object System.Xml.XmlDocument
    $xd.LoadXml($content)

    CreateBackupFile $xd $path  

    
    #Add People Picker Wildcard
    AddPeoplePickerWildcard $xd
            
    #Add Membership
    AddMembership $xd $ldapServer $userContainer "(&(ObjectClass=person))"
    
    #Add Roles
    AddRoles $xd $ldapServer $userContainer "(&(ObjectClass=person))" "(&(ObjectClass=group))"
    
    $xd.Save($path)
}

function ProcessSTS($path, $ldapServer, $userContainer)
{
    $content = Get-Content -Path $path
    [System.Xml.XmlDocument] $xd = new-object System.Xml.XmlDocument
    $xd.LoadXml($content)
    
    CreateBackupFile $xd $path
       
    
    #People picker wildcard is not necessary in STS config
            
    #Check to see if the system.web element exists, and if not, create it 
    $sysWebNode = $xd.SelectSingleNode("/configuration[system.web]")
    if(!$sysWebNode)
    {
        $config = $xd.SelectSingleNode("/configuration");
        $sysWebNode = $xd.CreateNode("element","system.web","")
        $config.AppendChild($sysWebNode)
    }
    
    #Add Membership
    AddMembership $xd $ldapServer $userContainer "(&(ObjectClass=person))"
    
    #Set LdapMember as default in STS
    $membershipNode = $xd.selectSingleNode("/configuration/system.web/membership")
    $defaultMembershipProviderAttr = $xd.CreateAttribute("defaultProvider")
    $defaultMembershipProviderAttr.Value = "LdapMember"
    $membershipNode.Attributes.Append($defaultMembershipProviderAttr)        
    
    #Add Roles
    AddRoles $xd $ldapServer $userContainer "(&(ObjectClass=person))" "(&(ObjectClass=group))"
    
    #Set LdapRole as default in STS
    $roleNode = $xd.selectSingleNode("/configuration/system.web/roleManager")
    $defaultRoleProviderAttr = $xd.CreateAttribute("defaultProvider")
    $defaultRoleProviderAttr.Value = "LdapRole"
    $roleNode.Attributes.Append($defaultRoleProviderAttr)

    $xd.Save($path)
}

function Main($pathToWebApplicationConfig, $pathToCentralAdminConfig, $ldapServer, $userContainer)
{
    $servers = Get-SPServer | ?{$_.Role -eq "Application"}
    foreach($server in $servers)
    {
        $name = $server.Name
                        
        $webAppConfigPath = $pathToWebApplicationConfig.ToLower().Replace("c:\", "\\$name\c$\")

        $centralAdminConfigPath = $pathToCentralAdminConfig.ToLower().Replace("c:\", "\\$name\c$\")

        $stsConfigPath = "\\$name\c$\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebServices\SecurityToken\web.config"

        ProcessWebApplication $webAppConfigPath $ldapServer $userContainer

        if(Test-Path $centralAdminConfigPath)

                ProcessCentralAdmin $centralAdminConfigPath $ldapServer $userContainer

        ProcessSTS $stsConfigPath $ldapServer $userContainer

#Update with path to Central Administration web.config

$pathToCentralAdminConfig = "c:\inetpub\wwwroot\wss\VirtualDirectories\37552\web.config"

#Update with path to web application's web.config

$pathToWebApplicationConfig = "c:\inetpub\wwwroot\wss\VirtualDirectories\DevTeam.Contoso.lab80\web.config"

#Update with the correct LDAP server

$ldapServer = "dc.contoso.lab"

#Update with the correct container name

$userContainer = "OU=EMPLOYEES,DC=CONTOSO,DC=LAB"



Main $pathToWebApplicationConfig $pathToCentralAdminConfig $ldapServer $userContainer

Summary

That’s it! I have used this several times in my test environment and it seems to work for either an existing or new environment, but you’ll want to test to make sure any assumptions I make in the script about availability of elements are accurate.

For More Information

Configure forms-based authentication for a claims-based Web application (SharePoint Server 2010)

Configuring claims and forms based authentication for use with an LDAP provider in SharePoint 2010

Enable Exchange Mailboxes for All Users in an OU

Kirk Evans[MSFT] — Wed, 23 Jan 2013 23:06:19 GMT

This is more of a note to myself than anything, but what the heck, why not share it with everyone else?

I frequently build new SharePoint farms for development and testing, and I usually create a new domain controller and populate it with users. One of my favorite scripts is by Mark Rhodes, “Adding 285 Contoso Users with Pictures to your Development Environment Active Directory”. The result is just as the title indicates:

Much less frequently I set up Exchange in the environment as well. Usually I add the users with Mark’s script first, and then at some point later I install Exchange. The problem is that now I have 285 users in the Employees OU that I want to have mailboxes, but I certainly don’t want to create them by hand. Further, I don’t want the users in the Users container to have mailboxes. The result? A little PowerShell.

Open the Exchange Management Shell, which opens an instance of PowerShell. Next, use the following PowerShell (substituting, of course, your searchBase parameter.

Import-module activedirectory

$users = Get-ADUser -LDAPfilter '(name=*)' -searchBase {OU=Employees,DC=contoso,DC=lab} 
foreach($user in $users)
{
   Enable-Mailbox -Identity $user.SamAccountName
}

That script will iterate through the users, filtered by the OU that you want using the searchBase parameter. I am sure there is room for improvement here, but it works in my lab!