Kirk Evans Blog

SharePoint Error 6398 - You do not have permission to run 'SP_TRACE_CREATE'

Kirk Evans[MSFT] — Thu, 09 Feb 2012 17:45:18 GMT

Working with a customer, we noticed a large number of errors in the Windows Application event log for error 6398. The errors look like:

The Execute method of job definition Microsoft.SharePoint.Diagnostics.SPSqlDeadlockDiagnosticProvider (ID c020fbe2-7771-495c-95dd-ad7fb680e2e5) threw an exception. More information is included below.

You do not have permission to run 'SP_TRACE_CREATE'.

and

The Execute method of job definition Microsoft.SharePoint.Diagnostics.SPIOIntensiveQueryDiagnosticProvider (ID 85c74911-f645-4c61-977b-81914d9eb69b) threw an exception. More information is included below.

You do not have permission to run 'SP_TRACE_CREATE'.

SharePoint 2010 introduced the Health and Usage Data Collection component. One of the (if not the most) definitive sources for understanding this capability is by reading the blog series that Todd Carter did on Extending the SharePoint 2010 Health & Usage. The errors above mention a diagnostic provider that is attempting to perform an unauthorized action.

Start by looking at the diagnostic providers in your farm. When you install the SharePoint 2010 Administration Toolkit, it includes a fantastic component called SharePoint Diagnostic Studio (SPDIAG). This component is able to query the usage data store and provide reports that help you manage the health and performance of your SharePoint 2010 environment. When you create a new project, it will install the diagnostic providers and start collecting health data.

SPDIAG will install several diagnostic providers that collect additional data. You can view these in the Central Administration web UI, but PowerShell makes this task simple using Get-SPDiagnosticsProvider.

PS C:\Windows\system32> Get-SPDiagnosticsProvider

Name                                                    Retention  MaxTotalSi Enabled   
                                                                   zeInBytes            
----                                                    ---------  ---------- -------   
job-diagnostics-blocking-query-provider                 14         6200000000 False     
job-diagnostics-sql-dmv-provider                        14         1000000... True      
job-diagnostics-uls-provider                            14         1000000... True      
job-diagnostics-performance-counter-sql-provider        14         6200000000 True      
job-diagnostics-performance-counter-wfe-provider        14         1000000... True      
job-diagnostics-event-log-provider                      14         6200000000 True      
job-diagnostics-changed-objects-provider                14         6200000000 True      
job-diagnostics-sql-blocking-report-provider            14         6200000000 True      
Search Health Monitoring - Trace Events                 14         1290000... True      
job-diagnostics-io-intensive-query-provider             14         1000000... True      
job-diagnostics-change-types-provider                   14         6200000000 True      
job-diagnostics-sql-memory-provider                     14         1000000... False     
job-diagnostics-sprequestusage-provider                 14         6200000000 True      
job-diagnostics-sql-deadlock-provider                   14         6200000000 True

Compare this to the results of an out-of-box installation without SPDIAG, and you can see that SPDIAG adds several diagnostic providers. The ones that are throwing the 6398 are the three jobs that are trying to collect information about SQL.

job-diagnostics-sql-deadlock-provider
job-diagnostics-sql-blocking-report-provider
job-diagnostics-io-intensive-query-provider

These diagnostics providers use SP_TRACE_CREATE and RECONFIGURE to gather data needed to support several reports in SPDIAG.

From http://technet.microsoft.com/en-us/library/hh144782.aspx:

Some SPDiag diagnostic timer jobs require sysadmin or sqladmin privileges

Some SPDiag 3.0 diagnostics jobs require that the farm account has the sysadmin or sqladmin role assigned on the SQL Server instance where the SharePoint 2010 Products databases are located. If the farm account does not have these roles assigned, it will have insufficient privileges to run diagnostic jobs that are required for certain reports to gather data.

SP_TRACE_CREATE and RECONFIGURE are available to the sysadmin and serveradmin fixed server roles implicitly. These permissions are not available to the db_owner database role. The account that is trying to create the trace is the SharePoint farm account. By default, this account has db_owner database permissions, but it does not have server permissions and is not included in a fixed server role. That’s why you see the error above.

At this point, you have a few choices:

If you find value in the blocking, IO, and deadlock reports for proactively managing your environment, then add the farm account to the fixed sysadmin or sqladmin role.
If you want to use SPDIAG but do not want to grant that level of permission to the SharePoint farm account, you can still use it without those diagnostic providers, you just won’t get the database specific reports (blocking, IO, deadlocks).

To disable these providers, you can run the following PowerShell script. Remove the line breaks in the IF statement below.

$providers = Get-SPDiagnosticsProvider
foreach($provider in $providers)
{
   if(($provider.Name -eq "job-diagnostics-sql-blocking-report-provider") 
-or ($provider.Name -eq "job-diagnostics-io-intensive-query-provider") 
-or ($provider.Name -eq "job-diagnostics-sql-deadlock-provider"))
    {
        $provider.IsDisabled = $true;
        $provider.Update();
    }
}

We run the Get-SPDiagnosticsProvider PowerShell script again to confirm our changes.

PS C:\Windows\system32> Get-SPDiagnosticsProvider

Name                                                    Retention  MaxTotalSi Enabled   
                                                                   zeInBytes            
----                                                    ---------  ---------- -------   
job-diagnostics-blocking-query-provider                 14         6200000000 False     
job-diagnostics-sql-dmv-provider                        14         1000000... True      
job-diagnostics-uls-provider                            14         1000000... True      
job-diagnostics-performance-counter-sql-provider        14         6200000000 True      
job-diagnostics-performance-counter-wfe-provider        14         1000000... True      
job-diagnostics-event-log-provider                      14         6200000000 True      
job-diagnostics-changed-objects-provider                14         6200000000 True      
job-diagnostics-sql-blocking-report-provider            14         6200000000 False     
Search Health Monitoring - Trace Events                 14         1290000... True      
job-diagnostics-io-intensive-query-provider             14         1000000... False     
job-diagnostics-change-types-provider                   14         6200000000 True      
job-diagnostics-sql-memory-provider                     14         1000000... False     
job-diagnostics-sprequestusage-provider                 14         6200000000 True      
job-diagnostics-sql-deadlock-provider                   14         6200000000 False

Note that there have been other cases where error 6398 is thrown because of SQL aliases (see the For More Information section below).

For More Information

SharePoint 2010 Administration Toolkit

SharePoint 2010 Diagnostics Studio

Extending the SharePoint 2010 Health & Usage

SPDiagnosticsProvider Class

Timer Event 6398 – SPDatabaseServerDiagnosticsPerformanceCounter

SQL Aliases and SP_TRACE_CREATE errors on SharePoint 2010

Moving From Heroes to Mentors

Kirk Evans[MSFT] — Sat, 04 Feb 2012 14:59:46 GMT

In July 2010, I switched roles from Developer Platform Evangelism to become a Premier Field Engineer for SharePoint. My role in DPE was amazing as it afforded me time for activities that I love, like participating in user groups, speaking at conferences, writing blogs, and sometimes posting videos to Channel9. After joining PFE, I largely stopped those activities because I wanted to focus on the technology, immersing myself to succeed in my new role. Occasionally I will still squeak a blog post out, maybe speak at a conference or two, but I am nowhere near as engaged with the community as I had been previously.

Going Through The Change

Our mission is to show our customers how to improve the health of their environment proactively, teaching them how to better manage and operate their environment. We joke that the most successful PFE will work themselves out of a job, but that’s the whole point… if our customers don’t need our services any longer, we’ve succeeded. Unfortunately, the technologies we work with are incredibly complex and provide many variations in the way our customers implement them. Heck, SharePoint is a booming business for just that very reason.

PFE has some of the brightest minds in the industry within its group. These are people with deep troubleshooting skills, people who routinely work with customers to improve their experience with Microsoft products. When your SharePoint farm goes down, SQL crashes, Exchange becomes unavailable, whatever the problem, these are the people who you work with to fix the issue and help prevent the issue from ever occurring again. These are people who are very deep with the technology but don’t have experience with writing or public speaking. Wouldn’t it be great to learn from someone with that depth of experience?

This past fall, Premier Field Engineering held a series of internal conferences called GeekReady. At this event, the General Manager for PFE, Shannan Collins, unveiled a new direction for PFE called “Heroes to Mentors”. There wasn’t much tactical information announced, it was very much a vision for where PFE is headed, in fact it was intentionally vague to tell the field to pick up the ball and run with it. Heroes to Mentors means changing the culture of PFE, something that is much more than an exec issuing an edict. Heroes to Mentors implies change at every level within PFE, from the execs to the Directors to Managers to the engineers themselves. It implies that engineers should work harder to share knowledge internally and continue to raise the technical capabilities of our team. It implies that we work to share that knowledge externally with our customers and partners. It means that we are given leeway to start doing many of the things that I loved so much from my days in DPE.

What’s In It For You

I hear regularly from user group leaders across the country asking for me to participate in events like SharePoint Saturday, speaking at a user group event, even speaking at conferences. Often I have to decline since I don’t have a business reason to travel and afford the T&E. The unfortunate part is that I don’t often pass that opportunity or introduction on to another PFE in that area to see if they might be interested. I am changing that, and will be encouraging my peers to reach out to their local user group leaders to see how they can contribute. I am also personally going to make an effort to participate in user group and SharePoint Saturday events more frequently. While I still am not able to travel to as many events as I would like (I’m not flying to Antarctica, for instance), I am going to say no a whole lot less often. And I am going to make sure to introduce other PFE folks in the area that will want to participate as well.

Another initiative that I am resurrecting is my Channel9 video blog. No, there’s no new content there yet… I am working on the recordings to make sure that they are much more than marketing introductions. I want to introduce you to some of the brightest folks in the industry, but also have them show you something valuable. Just watch http://twitter.com/kaevans or this blog for announcements. And if there are PFE that are reading this, look for announcements about opportunities to fulfill your Heroes to Mentors commitments

It’s not just me. There are some PFE that are already heavily engaged in the community. This includes folks like Bob Fox, Ryan Campbell, Jason Gallicchio, Josh Gavant, Eric Harlan. These guys are also working to draw some of our colleagues out of their hiding spaces. We will be working to find better ways to make ourselves more accessible. We are working internally to help folks get started, trying to find ways to better engage our colleagues, and working on ways that we can be more effective in creating content and sharing it with the rest of the community. I have a few more things that I am not yet ready to share, but watch this space as we continue to make announcements.

There are many other projects that PFE works on that you are familiar with that have made amazing contributions to the community. The PAL tool written by Clint Huffman is a great example of a tool that PFE regularly uses for troubleshooting but some of our customers are not yet familiar with how to use the tool. Folks like Greg Varveris, who worked on the SPDisposeCheck tool, are working on some fantastic new initiatives that our customers need to see. I can’t wait to show you what Greg is working on with fellow PFE Tom Fuller. I am making the commitment to break out of my SharePoint shell and to start finding ways to show you what the rest of PFE is doing.

Looking Forward

This is just the beginning. I am incredibly excited about this change. Admittedly, culture change is very difficult, and it will be slow as we move our colleagues from the mindset of being a hero to becoming a mentor both internally at Microsoft and externally with our customers, partners, and community.

Creating SharePoint Folders and Items with PowerShell

Kirk Evans[MSFT] — Tue, 24 Jan 2012 15:04:00 GMT

I was working on a customer problem and needed a very large list to try to reproduce a problem. I typically fire up Visual Studio and create Yet Another Console App. You know, one of the hundred or so “ConsoleApplication1” projects that litter your My Documents/Visual Studio/Projects folder? I decided to force myself to do this with PowerShell instead, and the result was not only quick, but also something I could do on a machine that doesn’t have Visual Studio installed.

Not only did I need many items in the list, I also needed a weird structure… 10 folders, each having 10 folders, each having 10 folders, each having 50 items.

Creating a folder in SharePoint is easy, but not straightforward. You have to use the SPFileSystemObjectType.Folder enumeration value when creating a list item. You can see the syntax for that below. Since we want to create nested folders, you can see how we use the relative URL to accomplish that as well.

Here is the script I came up with. Note! Before you run this in your environment, understand that it creates 50,000 items! You might want to change the number of loops to reduce the number of items being created, or better yet, just read the code and adapt it for your situation.

$spAssignment = Start-SPAssignment
$mylist = (Get-SPWeb -identity http://portal.sharepoint.com -AssignmentCollection $spAssignment).Lists["LargeList"]

for($i=1; $i -le 10; $i++)
{
    $folder = $mylist.AddItem("", [Microsoft.SharePoint.SPFileSystemObjectType]::Folder)
    $folder["Title"] = "folder$i"
    $folder.Update();
    for($j=1; $j -le 10; $j++)
    {
        $s1folder = $mylist.AddItem($folder.Folder.ServerRelativeUrl, 
[Microsoft.SharePoint.SPFileSystemObjectType]::Folder)
        $s1folder["Title"] = "subfolder$j"
        $s1folder.Update();
         for($k=1; $k -le 10; $k++)
        {
            $s2folder = $mylist.AddItem($s1folder.Folder.ServerRelativeUrl, 
[Microsoft.SharePoint.SPFileSystemObjectType]::Folder)
            $s2folder["Title"] = "subsubfolder$k"
            $s2folder.Update();
                        
            for($l=1; $l -le 50; $l++)
            {
                
                #Create item
                
                $newItem = $mylist.AddItem($s2folder.Folder.ServerRelativeUrl,
[Microsoft.SharePoint.SPFileSystemObjectType]::File, $null)
                $newItem["Title"] = "Item $i $j $k $l"
                $newItem["FirstName"] = "FirstName $i $j $k $l"
                $newItem["LastName"] = "LastName $i $j $k $l"
                $newItem["Company"] = "Company $i $j $k $l"
                $newItem.Update()
            }
        }   
    }
}
Stop-SPAssignment $spAssignment

That’s it, once you read the code you can see there’s not much to it. And now you’ll be able to get rid of all those throw-away ConsoleApplication1 projects in your My Documents folder!

For More Information

Creating SharePoint Folders and Items with PowerShell

SharePoint 2010 CAML List Joins

Kirk Evans[MSFT] — Sat, 21 Jan 2012 00:26:06 GMT

This post demonstrates how to use the JOIN syntax in CAML.

A customer pinged me asking about how to write a JOIN query in CAML. Honestly, I haven’t done this since in a long time, so it took awhile to shake off the cobwebs. I tried to make things explicit and easy to reproduce in your environment, please let me know in the comments of this post if something is unclear.

SharePoint 2010 adds the ability to create relational lists. To use the JOIN operator in CAML, your lists must have a defined relation. Let’s start by creating the lists and their relationships.

Creating the Lists

I first add a standard Contacts list named “DSE” and populate it with some data.

Go to the list settings for your new contact list and hover over the column names. Many SharePoint developers forget this, the name you see in the web UI is the .Title property, which may be different than the internal name that SharePoint uses (the .InternalName property of the SPField object). For a quick way to see what the actual field name is, hover over the column name and look in the address bar in your browser, and we see that the “Last Name” column is actually the “Title” field.

Next, I create a custom list named “Projects” and add a column named “Manager”. The type is a lookup column. Note the name “Manager”, we’ll refer to this name in our code.

In the additional column settings section for the “Manager” column, I set the lookup to the ID column for the DSE list, and additionally show the First Name and Last Name.

When I add a new item to the Projects list, I get a drop-down that lets me select a value from the lookup column.

The result looks like this:

Now that we have the lists, a lookup field, and some data, let’s query it.

Querying Using JOINS and SPQuery

Nothing speaks louder than a code sample. At the top of my code, I add a few using directives.

using System;
using System.Linq;
using Microsoft.SharePoint;

Next, the code to query the list.

The first thing to notice is the SPQuery.Joins property which lets me provide 1 or more joins for the list. The list must already have a lookup column and a relation defined to the list being joined to. Notice in the Joins property that we refer to the name that we provided when we created the lookup column, “Manager”.

The second thing to notice is the ProjectedFields property. This is where we tell SharePoint how to project the lookup columns into the result. In a contacts list, the “First Name” column has an internal name of “FirstName”. The “Last Name” column is actually the Title column (see above for how to determine the .InternalName of a field).

Finally, the ViewFields property lets us define which fields are included in the result. We use the same name that we used in the ProjectedFields property. This name could be anything, so long as the name in ProjectedFields and the name in ViewFields match.

class Program
{
static void Main(string[] args)
{
using (SPSite site = new SPSite("http://spstc.sharepoint.com"))
{
    SPWeb web = site.RootWeb;

    SPQuery query = new SPQuery();

    query.Joins = "<Join Type='INNER' ListAlias='DSE'>" +
                    "<Eq>" +
                        "<FieldRef Name='Manager' RefType='Id'/>" +
                        "<FieldRef List='DSE' Name='ID'/>" +
                    "</Eq>" +
                    "</Join>";
    query.ProjectedFields =
        "<Field Name='DSEFirstName' Type='Lookup' " +
                "List='DSE' ShowField='FirstName'/>" +
        "<Field Name='DSELastName' Type='Lookup' " +
                "List='DSE' ShowField='Title'/>";

    query.ViewFields = "<FieldRef Name='Title'/>" +
                        "<FieldRef Name='DSEFirstName'/>" +
                        "<FieldRef Name='DSELastName'/>";
    SPList customerList = web.Lists["Projects"];
    SPListItemCollection items = customerList.GetItems(query);
    foreach (SPListItem item in items)
    {
        SPFieldLookupValue dseLastName =
            new SPFieldLookupValue(item["DSELastName"].ToString());
        SPFieldLookupValue dseFirstName =
            new SPFieldLookupValue(item["DSEFirstName"].ToString());

        Console.WriteLine("{0}  {1}   {2}",
                item.Title,
                dseLastName.LookupValue,
                dseFirstName.LookupValue);
    }
}
}
}
}

The results are pretty unimpressive, but it proves that our CAML query works.

And there you have it, your first CAML JOIN query.

Querying Using LINQ

Just for kicks, I will throw in a LINQ sample, too. LINQ doesn’t support the JOIN operator, nor does it support a projection to a referenced entity. What you can do, though, is just reference the entity in the results.

I opened the Visual Studio 2010 Tools command window and ran the following command:

spmetal /web:http://spstc.sharepoint.com /namespace:Microsoft.PFE.DSE.Samples /code:PFE.cs

This generates the strongly-typed entities for me behind the scenes with metadata that tells LINQ how to query the results. I include the result, PFE.cs, in my Visual Studio project and run the following code. Notice I add the ctx.Log setting to log output to the Console window. That lets me include a picture of a WHOLE BUNCH of CAML in the screen shot below.

using (PFEDataContext ctx = new PFEDataContext("http://spstc.sharepoint.com"))
{
    ctx.Log = Console.Out;
    var results = from c in ctx.Customers
                  select c;
    foreach (var item in results)
    {
        Console.WriteLine("{0}  {1}   {2}",
        item.Title,
        item.DSE.LastName,
        item.DSE.FirstName);
    }
}

Just to show the results…

Using your own JOIN operation can be more efficient, but test the results to be sure. Often times LINQ will generate more efficient queries, but not always.

Consolidate SharePoint 2010 Application Pools

Kirk Evans[MSFT] — Mon, 19 Dec 2011 16:00:00 GMT

Working with a customer, we saw that they had 20 or so web application pools, each with 1 web application each. The software boundary for SharePoint 2010 is to have no more than 10 web application pools. Because each application pool can have multiple web applications assigned to it, and easy fix is to just move the web applications to new application pools.

I have seen several blogs and even forum posts that indicate you can just change the application pool in IIS. This is not sufficient, because SharePoint maintains a reference to the application pool and the managed account assigned to the application pool. If you change it in IIS without telling SharePoint, you can cause orphaned configuration data which may prevent you from being able to apply updates or upgrades.

There’s no UI to change the application pool in Central Administration, the only way to change this is through the object model. Luckily this is pretty simple to do using PowerShell.

$webService = [Microsoft.SharePoint.Administration.SPWebService]::ContentService 
$pool = $webService.ApplicationPools["SharePoint - AuthTest80"] 
$app = Get-SPWebApplication  http://teamsapp
$app.ApplicationPool = $pool
$app.Update()
$app.ProvisionGlobally()

The trick here is to use the ProvisionGlobally method. If you leave this out, it changes the setting in SharePoint’s configuration data, but does not make a change in IIS. ProvisionGlobally will make the change in IIS as well, and will do this for every server in your farm… another huge benefit for doing this through the object model instead of manually updating the setting in IIS for every server in your farm!

For More Information

SharePoint Server 2010 capacity management: Software boundaries and limits

SPWebApplication.ProvisionGlobally Method

Extract a WSP Solution from SharePoint 2010

Kirk Evans[MSFT] — Mon, 05 Dec 2011 16:00:00 GMT

This post shows how to download a WSP from SharePoint 2010 using PowerShell.

Background

A customer of mine had a scenario where they needed to download a WSP from SharePoint 2010. In the Central Administration screen, a link is not available for you to download the WSP solution.

Likewise, with stsadm, there’s not a command to download a solution.

Implementation

I could have written this with a C# console application, but I’ve been playing with PowerShell and found that the 3 lines of PowerShell code that it takes is much easier than trying to deploy a binary to a customer’s environment.

$farm = Get-SPFarm
$file = $farm.Solutions.Item("extendeddiagnosticproviders.wsp").SolutionFile
$file.SaveAs("c:\temp\extendeddiagnosticproviders.wsp")

Note that in order to run this command, you need to be a member of the Farm Administrators group and have permission to the configuration database. To configure the permissions for SQL, I recommend using Add-SPShellAdmin.

For More Information

SPSolutionCollection

SPSolution class

Add-SPShellAdmin

Playing with SharePoint 2010 MUI

Kirk Evans[MSFT] — Mon, 28 Nov 2011 16:00:00 GMT

This post shows how to set up a SharePoint environment to explore the multilingual user interface (MUI).

Background

I admit to being a typical American. When I write applications, I hardly ever use resource files to support any language other than English. My friend Derek Noonan sent me this map that makes me laugh (“Here be dragons!”) because it is painfully true… most Americans don’t speak a second language aside from knowing how to ask for another beer or directions to the bathroom.

I have seen demos of the SharePoint 2010 Multilingual User Interface, but I haven’t ever tried to make it work in my environment. I decided to configure it, document the installation process, and show some of the capabilities. If you are interested in learning more, see the For More Information section at the bottom of this post.

Even if you aren’t currently supporting multiple languages, I encourage you to understand the MUI and its capabilities. Some day you may find yourself with a task of supporting multiple languages, understanding the out of box capabilities can save you many hours of work.

Setup

I was a complete MUI novice, I wasn’t even sure what to install to make it work. It turns out to be pretty easy (albeit a bit time consuming) to set up MUI in SharePoint 2010. You do not need to install language packs for the operating system, you only need to install the SharePoint language pack for each language you want to support and SP1 for each language pack (and the latest CU).

To start with, my laptop probably looks just like yours. Its display language is English with no other display languages installed, and I installed SharePoint 2010 using the downloaded media from MSDN. I applied the latest updates (at the time of this writing, the latest service pack is SP1, and the latest Cumulative Update is August 2011 CU) and ran PSConfig. This is a typical developer machine that supports exactly 1 language: English. The steps are:

Install the Language Packs
Install Service Pack 1 for Server Language Pack 2010 for each language pack
Apply the latest cumulative update

For 4 languages, that’s a total of 9 installations. Here is what my download folder looks like:

Install the Language Packs

The first step is to download the language pack(s) for SharePoint 2010. The first time I tried this, I ended up installing the English language pack. Showing how typical of an American I am, a friend (thanks, Mads!) pointed out there is a drop-down that doesn’t just change the language of the web page you are looking at, but also changes which language pack you are going to download.

Once you select a language, download and install each language pack that you want to use. I chose French, German, Japanese, and Spanish language packs. When a language pack is completed, there is a checkbox asking you if you want to run the SharePoint Product Configuration Wizard (PSConfig). Uncheck this… we will run PSConfig later.

Install Service Pack 1 for Server Language Pack 2010 for Each Language Pack

Once you install a language pack, you need to also install Service Pack 1 for Server Language Pack 2010 for each language pack that you installed. Just like with the previous download screen, you need to change the drop-down to the language that you want to download.

Apply the Latest Cumulative Update

I already have August CU applied to my environment, but you should reapply the latest CU again after installing the language packs. When I tried to run the August CU again on my machine, the installer reported the hotfix was already installed. It can’t hurt to be thorough.

Run PSConfig

Notice how we didn’t say to run PSConfig yet? Thankfully, you can do all 9 installations first, and after all the installations are complete run PSConfig on each server in your farm.

Playing with MUI

To get started, create a site and access its site settings page. Choose Language Settings, and select which languages you want to allow the users to switch to.

Once you enable alternate languages, users will now have the option to change the display language.

I selected German, and the site changed to the following:

Let’s poke around and see what else is localized.

Reports:

Site columns:

Content types:

Very cool!

What About Content and Code?

Now, you might be asking yourself, “what about content?” If a user types something in a wiki page, is that automatically translated? The answer here is no. You can experiment by creating a new site page and then editing it with some text. Change the site language, and the text remains the same. OK, understandable.

What about my code that displays strings, can it be localized? The answer here is a resounding yes. The .NET Framework provides extensive support for localization, as does SharePoint, but only when you leverage it. The code is very simple:

using System;
using System.ComponentModel;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
using System.Globalization;
using Microsoft.SharePoint.Utilities;

namespace MUIDemo.HelloMUIWebPart
{
    [ToolboxItemAttribute(false)]
    public class HelloMUIWebPart : WebPart
    {
        protected override void CreateChildControls()
        {
            Label hello = new Label();
            hello.Text = "Hello";
            Controls.Add(hello);

            Controls.Add(new System.Web.UI.HtmlControls.HtmlGenericControl("br"));

            Label localGreeting = new Label();
            localGreeting.Text = Properties.Resources.Greeting;
            Controls.Add(localGreeting);

            Controls.Add(new System.Web.UI.HtmlControls.HtmlGenericControl("br"));

            Label currentUIculture = new Label();
            currentUIculture.Text = "Current UI Culture: " + CultureInfo.CurrentUICulture;
            Controls.Add(currentUIculture);

            Controls.Add(new System.Web.UI.HtmlControls.HtmlGenericControl("br"));

            Label date = new Label();
            date.Text = "Date: " + System.DateTime.Now.ToString("d", CultureInfo.CurrentUICulture);
            Controls.Add(date);

            Controls.Add(new System.Web.UI.HtmlControls.HtmlGenericControl("br"));

            Label time = new Label();
            time.Text = "Time: " + System.DateTime.Now.ToString("t", CultureInfo.CurrentUICulture);
            Controls.Add(time);                    
        }
    }
}

The trick here is deploying the satellite assemblies. Build your project with your new resource files and add them to your Visual Studio 2010 project, following the steps in How to: Deploy Localized Sandboxed Solutions by Using Satellite Assemblies.

The English version of our page:

Change the site language to French, and compare the changes. The static string “Hello” remains un-localized, and the text that the user entered into the wiki page remains unaffected. However, the Greeting resource string is localized as are the date and time, according to the current UI culture.

Just for good measure, let’s change the site language to German to compare the changes.

For More Information

How to: Deploy Localized Sandboxed Solutions by Using Satellite Assemblies

Walkthrough: Localizing a Web Part

Understanding the Multilingual User Interface (MUI)

Add Microsoft.SharePoint.PowerShell Snap-In to All PowerShell Windows

Kirk Evans[MSFT] — Mon, 14 Nov 2011 14:46:25 GMT

This post will show you how to ensure Add-PSSnapin "Microsoft.SharePoint.PowerShell" has been executed in all PowerShell windows.

Background

I have been doing a lot of work in PowerShell lately, and found myself frequently in the PowerShell ISE window writing scripts. I was constantly adding the line Add-PSSnapin "Microsoft.SharePoint.PowerShell" at the top of every script I wrote, and I wanted it to just already be there like it is when I run the SharePoint 2010 Management Shell. You can add the SharePoint snap-in to all PowerShell windows, so it will be available when you run PowerShell ISE and you don't have to remember to add the snap-in to every script you run.

Implementation

Credit goes to SharePoint and PowerShell guru Gary Lapointe for this tip. Open up PowerShell ISE and run the following to create a profile script if one doesn’t exist and edit it in the ISE:

if (!(test-path $profile.AllUsersAllHosts)) {new-item -type file -path $profile.AllUsersAllHosts –force}
powershell_ise $profile.AllUsersAllHosts

That will open a new tab in PowerShell ISE allowing you to edit profile.ps1. If you simply execute $profile.AllUsersAllHosts, you will see the path where this file is stored (be default it is C:\Windows\System32\WindowsPowerShell\v1.0).

In PowerShell ISE, you will now have a new tab where you can edit this file. In that new tab, add the following code and then save the file.

$ver = $host | select version
if ($ver.Version.Major -gt 1) {$host.Runspace.ThreadOptions = "ReuseThread"} 
if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null) 
{
    Add-PSSnapin "Microsoft.SharePoint.PowerShell"
}

The edit window looks like this:

Now, close PowerShell ISE and then open it again. It will take a little longer than usual to open the window because it is executing the code from Profile.ps1 and adding the Microsoft.SharePoint.PowerShell snap-in. To test it, run a command such as Get-SPFarm.

That simple little trick has saved me hours of frustration while editing scripts, and has enabled me to really start leveraging PowerShell ISE. Thanks to Gary for his great insight at: http://social.technet.microsoft.com/Forums/en/sharepoint2010setup/thread/fcb77654-0f13-42e0-b181-6e52242fe9d6.

If you are interested in PowerShell and SharePoint and haven’t visited Gary’s blog, do so immediately… it is a great resource. http://blog.falchionconsulting.com.

Update: Spence Harbar let me know that he has a post doing something very similar, executing the same script that the SharePoint 2010 Management Shell executes. http://www.harbar.net/archive/2010/05/03/adding-sharepoint-2010-poweshell-cmdlets-to-your-powershell-ise.aspx

For More Information

http://social.technet.microsoft.com/Forums/en/sharepoint2010setup/thread/fcb77654-0f13-42e0-b181-6e52242fe9d6

Adding SharePoint 2010 PoweShell cmdlets to your PowerShell ISE

Add-PSSnapIn

SharePoint 2010 Products administration by using Windows PowerShell

Add-PSSnapIn Microsoft.SharePoint.PowerShell Failing after SharePoint Install

Create DNS Entries From a File

Kirk Evans[MSFT] — Wed, 02 Nov 2011 21:55:05 GMT

This post will show how to create DNS entries from a file using the DNSCmd.exe tool.

Background

I have a kind of edge case that I am trying to troubleshoot for my customer. They have a lot of sub-companies, and they created a web application per company. Fast forward a few years, they now have 143 web applications. For those keeping score, that’s 44 more than the maximum supported limit for SharePoint. We are working to collapse those web applications to host-named site collections. In order to test the process, I needed to implement a tool where I could create that many web applications in my environment. The first challenge was creating that many DNS A records.

Implementation

After searching the web and coming up fairly empty on doing this with WMI or other technologies, I decided to take the hack way out and use DNSCmd.exe. This utility is present on the server when DNS is installed on the server. It becomes a pretty trivial exercise, then, to write a program that runs DNSCmd.exe, provides the necessary command line arguments, and generates the DNS entries.

I put all of my new DNS entries into a file, with each new host name on a single line:

CompanyA
CompanyB
CompanyC
CompanyD

Then, it’s just a matter of reading the file and using the System.Diagnostics.Process.Start() method, providing the right command line arguments.

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Diagnostics;

namespace CreateDNSEntries
{
    class Program
    {
        static void Main(string[] args)
        {
            CreateDNSENtries();
        }

        static void CreateDNSENtries()
        {
            string[] sites = File.ReadAllLines("C:\\temp\\dnsnames.txt");
            foreach (string site in sites)
            {         
                string passRaw = "YourPasswordGoesHere";

                System.Security.SecureString pass = new System.Security.SecureString();
                for (int i = 0; i < passRaw.Length; i++)
                {
                    pass.AppendChar(passRaw[i]);
                }

                Process.Start("dnscmd.exe", "/recordadd sharepoint.com " 
                    + site + " A 192.168.2.5", "administrator", pass, "sharepoint");                
            }
        }
    }
}

The command that we are running uses the /recordadd command line argument, this is the stuff you will probably want to change.

Domain: sharepoint.com
Host: site
Record type: A
IP address: 192.168.2.5

We also use the domain administrator’s credentials to run the utility, using the System.Security.SecureString class to provide the password.

Run your new command line utility on your DNS server to add the entries. Simple, huh? It took me longer to fail searching for an answer than it did to whip together a hack solution. Sometimes it’s faster to just roll up your sleeves.

For More Information

DNSCmd Syntax

SecureString Class

Process.Start Method

Set IIS Log Fields with C#

Kirk Evans[MSFT] — Wed, 02 Nov 2011 17:29:00 GMT

This post shows how to set the log fields for IIS web sites programmatically. Code is attached at the end of the post.

Background

I have a customer that has many web applications, and we need to set the IIS log fields to do some investigation. James Blackwell posted a great sample of how to do this using VBScript (http://blogs.msdn.com/b/jb/archive/2008/02/03/script-for-enabling-full-iis-logging.aspx). I had a need to do this via C#, and most of the work was simply converting James’ sample to use the System.DirectoryServices.DirectoryEntry class to manipulate the sites. The reason I chose to use this instead of IIS7’s new Microsoft.Web.Administration namespace is because I am working with an IIS6 web server, however the code below works with both IIS6 and IIS7.

The Value of IIS Logs

I see a lot of customers who have IIS logging enabled but don’t do anything with the data. The IIS logs can be invaluable to tell you what’s going on with your server at a particular time. For instance, the operations team gets a call that their web site is slow. We look, and don’t see anything wrong with the site. We look at the IIS logs and see that the particular page in question is rendering very quickly. We compare this against Fiddler and see that the customer introduced a ton of JavaScript that is slowing down the rendering of the page in the browser. SharePoint is doing its job just fine. Using IIS logs can considerably reduce the amount of time spent finding the needle that is poking the haystack.

A fantastic resource for measuring your site’s performance is the whitepaper, “Analyzing Microsoft SharePoint Products and Technologies Usage”. It covers how to use tools like LogParser to get a better understanding of how your servers are performing. Of course, to process the data means that your server needs to log the data in the first place.

The Implementation

We will create a .NET 2.0 console application and reference the System.DirectoryServices assembly. You can query more than just Active Directory using the DirectoryEntry class, as we will see in this post. We need to set the LogExtFiles property for a web site, which expects a DWORD, which is a 32-bit integer. To set the flags, we first need to determine what the values are by looking them up in the LogExtFileFlags documentation. That is where I got the values for the LogExt* constants below.

The rest is simple, just choose which fields you want to set and add them together to form the value. For instance, if you only want to set LogExtFileDate and LogExtFileTime, the value is 1 + 2 = 3. In my example below, I set all values except LogExtFileCookie because this field can cause your log files to grow considerably.

using System;
using System.Collections.Generic;
using System.Text;
using System.DirectoryServices;

namespace SetWebSiteLoggingFields
{
    class Program
    {
        const int LogExtFileDate            = 1;        
        const int LogExtFileTime            = 2;
        const int LogExtFileClientIp        = 4;     
        const int LogExtFileUserName        = 8;    
        const int LogExtFileSiteName        = 16;
        const int LogExtFileComputerName    = 32;
        const int LogExtFileServerIp        = 64;
        const int LogExtFileMethod          = 128;
        const int LogExtFileUriStem         = 256;
        const int LogExtFileUriQuery        = 512;
        const int LogExtFileHttpStatus      = 1024;
        const int LogExtFileWin32Status     = 2048;
        const int LogExtFileBytesSent       = 4096;
        const int LogExtFileBytesRecv       = 8192;
        const int LogExtFileTimeTaken       = 16384;
        const int LogExtFileServerPort      = 32768;        
        const int LogExtFileUserAgent       = 65536;
        const int LogExtFileCookie          = 131072;
        const int LogExtFileReferer         = 262144;        
        const int LogExtFileProtocolVersion = 524288;
        const int LogExtFileHost            = 1048576;
        const int LogExtFileHttpSubStatus   = 2097152;  


        static void Main(string[] args)
        {
            try
            {
                string serverName = args[0];
                SetLogFields(serverName);
            }
            catch(Exception oops)
            {
                Console.WriteLine(oops.Message);
                Console.WriteLine();
                DisplayUsage();
            }
        }

        static void DisplayUsage()
        {
            Console.WriteLine("Enables all logging fields on all web sites.");
            Console.WriteLine("Usage:");
            Console.WriteLine("\tSetWebSiteLoggingFields <serverName>");
            Console.WriteLine();
            Console.WriteLine("Example:");
            Console.WriteLine("\tSetWebSiteLoggingFields WFE1");
            Console.WriteLine();

        }

        
        static void SetLogFields(string serverName)
        {            
            DirectoryEntry w3svc = new DirectoryEntry(
                                string.Format("IIS://{0}/w3svc", serverName));

            foreach (DirectoryEntry site in w3svc.Children)
            {
                Console.WriteLine("Site ID: " + site.Name);

                if (null != site.Properties["LogExtFileFlags"].Value)
                {
                    Console.WriteLine("Setting log fields for Site ID: " + site.Name);

                    site.Properties["LogExtFileFlags"][0] =
                        LogExtFileDate +
                        LogExtFileTime +
                        LogExtFileClientIp +
                        LogExtFileUserName +
                        LogExtFileSiteName +
                        LogExtFileComputerName +
                        LogExtFileServerIp +
                        LogExtFileMethod +
                        LogExtFileUriStem +
                        LogExtFileUriQuery +
                        LogExtFileHttpStatus +
                        LogExtFileWin32Status +
                        LogExtFileBytesSent +
                        LogExtFileBytesRecv +
                        LogExtFileTimeTaken +
                        LogExtFileServerPort +
                        LogExtFileUserAgent +
                        LogExtFileReferer +
                        LogExtFileProtocolVersion +
                        LogExtFileHost +
                        LogExtFileHttpSubStatus;
                    site.CommitChanges();

                }
            }  
        } 
    }
}

For More Information

Script for enabling full IIS logging

How to Use Microsoft.Web.Administration

LogExtFileFlags

Analyzing Microsoft SharePoint Products and Technologies Usage

Making The Case for Async Web Parts

Kirk Evans[MSFT] — Mon, 24 Oct 2011 15:22:00 GMT

This post will show how to make a call to HttpWebRequest asynchronously from a web part and wait on the result or time out.

Years ago, I was fortunate to meet Jeff Richter and attend a session he gave on .NET performance. He discussed I/O completion ports and showed why any calls from an ASP.NET page to a web service or another page absolutely must be done asynchronously. In short, when you make a call such as HttpWebRequest.GetResponse from an ASP.NET page, you are blocking the thread. Instead, replacing it with an asynchronous pattern such as HttpWebRequest.BeginGetResponse will use an I/O completion port to wait for the response, freeing the processing thread to do more work.

I recently worked with a customer that wrote a web part that uses HttpWebRequest to make calls to a backend service to obtain some data. Remember that an ASP.NET page invokes its child controls synchronously, calling methods on your controls like CreateChildControls and Render, and this is true for web parts as well. If your web part performs work that can take a few seconds (such as calling another web page), then consider what happens if you place 3 of those web parts on a page. If you use HttpWebRequest.GetResponse (the synchronous call), then each web part will be invoked serially. As a more concrete example, if your web part takes 5 seconds to make an HttpWebRequest call, placing 3 of those controls on your page makes the total page render time 15 seconds.

Confirming That Web Parts Are Synchronous

While I was working on a solution, a colleague indicated he thought that web parts would be invoked asynchronously. That’s not true, but let’s test that theory by writing some code and measuring. I created a class called Helper with a method GetWebPageSource.

        public string GetWebPageSource(string url)
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
            request.Method = "GET";
            WebResponse response = request.GetResponse();

            using (Stream receiveStream = response.GetResponseStream())
            {
                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
                StreamReader readStream = new StreamReader(receiveStream, encode);
                Result = readStream.ReadToEnd();
            }
            response.Close();
            return Result;
        }

My web part will call this method in its OnPreRender method.

using System;
using System.ComponentModel;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
using System.Threading;
using Microsoft.SharePoint.Utilities;

namespace Microsoft.PFE.DSE.Samples.Async.WebPageSourceWebPart
{
    
    [ToolboxItemAttribute(false)]
    public class WebPageSourceWebPart : WebPart
    {
        [WebBrowsable]
        [WebDisplayName("Url")]
        [WebDescription("The URL to obtain the source from")]
        [Category("Web Page Source Settings")]
        [Personalizable(PersonalizationScope.Shared)]
        public string Url { get; set; }


        Helper _helper;
        string _pageSource = string.Empty;


        protected override void OnPreRender(EventArgs e)
        {
            using (SPMonitoredScope monitoredScope = new SPMonitoredScope("OnPreRender to " + Url)) 
            {

                if ((!string.IsNullOrEmpty(Url)) && 
                  (WebPartManager.DisplayMode == WebPartManager.BrowseDisplayMode))
                {
                    
                    _helper = new Helper();
                    _pageSource = _helper.GetWebPageSource(Url);
                    
                }
            }
        }


        protected override void Render(HtmlTextWriter writer)
        {
            
            writer.AddAttribute(HtmlTextWriterAttribute.Class, "WePageSourcePart");
            writer.RenderBeginTag(HtmlTextWriterTag.Div);
            writer.WriteEncodedText(_pageSource);
            writer.RenderEndTag();
        }
    }
}

What would be a good way to measure the performance? The easiest way is to use the Developer Dashboard in SharePoint 2010.

Just to exaggerate the problem, I added 8 web parts to the page that obtain data from various web sites (Microsoft.com, MSN.com, MSDN.Microsoft.com, etc). The part to pay special attention to here is that 3 of the calls are to a page that I created that takes 5 seconds to render. Total page rendering time is 17917.48 ms, or nearly 18 seconds to render my page. Clearly the web parts are processed serially and calls to CreateChildControls, Render, etc are done synchronously.

We know that users are impatient and won’t put up with a page that takes 18 seconds to render. Think about the performance impact… while you tie up the thread in ASP.NET, it is not available to service additional calls. If 100 users access this page, you have 100 ASP.NET threads tied up for 18 seconds. What do you think will eventually happen as these users get impatient and hit F5? More requests, more threads tied up, and soon your server will become unresponsive because there are no threads available to service requests. Instead of slow pages, your users will see error pages, and now you have a huge perception problem on your hands because the users are unhappy that you spent all this time and money building a solution and the site can’t handle even a few hundred users.

Making It Async

While it sounds like a dire situation (it is in its current state), the good news is that it can easily be fixed. For instance, ASP.NET provides an incredibly handy mechanism for asynchronous pages, which is about as simple as adding an Async=True page directive. The problem in my case was that I am using a SharePoint publishing page, and the SPPageParserFilter class does not allow the Async=True page directive. We can still achieve the same effect, though, by directly using the same APIs in the same manner. We already know that many of the APIs that make remote calls, such as HttpWebRequest, provide an asynchronous call mechanism. It turns out to be very straightforward to change our web part code to call the asynchronous API and wait on the result. First, let’s change the Helper class to expose the asynchronous methods.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;

namespace Microsoft.PFE.DSE.Samples.Async
{
    public class Helper
    {
        public string Result { get; set; }

        public IAsyncResult BeginGetWebPageSource(string url)
        {
                        
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
            request.Method = "GET";
            return request.BeginGetResponse(null, request);            
        }

        public void EndGetWebPageSource(IAsyncResult iar)
        {
            HttpWebRequest request = iar.AsyncState as HttpWebRequest;
            WebResponse response = request.EndGetResponse(iar);
            
            using (Stream receiveStream = response.GetResponseStream())
            {
                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");                
                StreamReader readStream = new StreamReader(receiveStream, encode);
                Result = readStream.ReadToEnd();                
            }
            response.Close();
        }

        public string GetWebPageSource(string url)
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
            request.Method = "GET";
            WebResponse response = request.GetResponse();

            using (Stream receiveStream = response.GetResponseStream())
            {
                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
                StreamReader readStream = new StreamReader(receiveStream, encode);
                Result = readStream.ReadToEnd();
            }
            response.Close();
            return Result;
        }
    }
}

This should be familiar, we are just using the asynchronous API. The interesting part to point out is that we return an IAsyncResult from BeginGetWebPageSource. Additionally, we provide a null as the first argument to BeginGetResponse, indicating we do not have a callback method. We’ll point this out again in just a moment.

The next part that we need to change is that we need to invoke our new asynchronous calls. We will do this on a background thread, so we don’t want ASP.NET to continue processing… we need to wait until the response is received, otherwise our control would complete rendering before the HttpWebRequest ever returns, so we would not see any data. There are a few ways to achieve this, but the easiest for our purposes is to use a WaitHandle.

using System;
using System.ComponentModel;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
using System.Threading;
using Microsoft.SharePoint.Utilities;

namespace Microsoft.PFE.DSE.Samples.Async.WebPageSourceWebPart
{
    

    [ToolboxItemAttribute(false)]
    public class WebPageSourceWebPart : WebPart
    {
        [WebBrowsable]
        [WebDisplayName("Url")]
        [WebDescription("The URL to obtain the source from")]
        [Category("Web Page Source Settings")]
        [Personalizable(PersonalizationScope.Shared)]
        public string Url { get; set; }


        [WebBrowsable]
        [WebDisplayName("IsAsync")]
        [WebDescription("Specifies if the control should be process asynchronously")]
        [Category("Web Page Source Settings")]
        [Personalizable(PersonalizationScope.Shared)]
        public bool IsAsync { get; set; }

        IAsyncResult _result;
        Helper _helper;
        string _pageSource = string.Empty;

        protected override void OnLoad(EventArgs e)
        {            
            if ((!string.IsNullOrEmpty(Url)) && 
                (WebPartManager.DisplayMode == WebPartManager.BrowseDisplayMode))
            {
                _helper = new Helper();
                if (IsAsync)
                {
                    _result = _helper.BeginGetWebPageSource(Url);
                }
                
            }
        }

        protected override void OnPreRender(EventArgs e)
        {
            using (SPMonitoredScope monitoredScope = new SPMonitoredScope("OnPreRender to " + Url)) 
            {

                if ((!string.IsNullOrEmpty(Url)) && 
                    (WebPartManager.DisplayMode == WebPartManager.BrowseDisplayMode))
                {
                    if (IsAsync)
                    {
                        if (null != _result)
                        {
                            WaitHandle[] handles = new WaitHandle[1];
                            handles[0] = _result.AsyncWaitHandle;

                            //Wait for 5 seconds or until the result is returned
                            int index = WaitHandle.WaitAny(handles, 10000);
                            if (index == WaitHandle.WaitTimeout)
                            {
                                //The page timed out before we got a result
                                _pageSource = "Timeout before the result was returned";
                            }
                            else
                            {
                                _helper.EndGetWebPageSource(_result);
                                _pageSource = _helper.Result;
                            }
                        }
                    }
                    else
                    {
                        _helper = new Helper();
                        _pageSource = _helper.GetWebPageSource(Url);
                    }
                }
            }
        }


        protected override void Render(HtmlTextWriter writer)
        {
            
            writer.AddAttribute(HtmlTextWriterAttribute.Class, "WePageSourcePart");
            writer.RenderBeginTag(HtmlTextWriterTag.Div);
            writer.WriteEncodedText(_pageSource);
            writer.RenderEndTag();
        }
    }
}

In the OnLoad method, we check to see if the user indicated the web part should be processed asynchronously or not (so that we can easily compare the performance of synchronous versus asynchronous just by checking a checkbox). If it is asynchronous, then the page starts accessing the backend data asynchronously. In the OnPreRender method, we check again to see if this should be processed asynchronously or not. If it is async, then we use the IAsyncResult returned from the HttpWebRequest.BeginGetResponse method to access its AsyncWaitHandle property. This is how we will know when the external call is completed. Further, we can provide a 10 second timeout to stop processing in case the page we are getting data from takes too long to respond.

To test this, I added 8 web parts to the page and checked the IsAsync property in the web part settings.

The nice thing is that the web requests are not issued unless the page is in BrowseDisplayMode, allowing you to make changes to multiple web parts while in EditDisplayMode. If you just press the apply button to make changes, the page stays in edit mode, allowing you to change other web parts. Once you hit OK, the page goes back to BrowseDisplayMode, and you have to wait for all of your HttpWebRequests to complete.

Now that we set each web part’s IsAsync property to true, let’s see if that changed things. We again turn to the developer dashboard.

Wow! What a difference, our page makes the same number of calls using HttpWebRequest, but they are done in parallel instead of serially. Our page that used to take nearly 18 seconds to render now takes just over 5 seconds (5447.34 ms) to render! Our users are happy, our boss gives us a much needed day off, we can eat a home cooked meal for once instead of cheap pizza during yet another late night at the office. This happens because the requests are processed in parallel.

The Alternatives

Now that I’ve got you interested to learn more, read the material in the For More Information section, particularly the two posts in Tess’ blog. Make sure that you catch exceptions and tune your environment’s maxConnections property if you intend on making a lot of calls from the server.

However, you very much need to think about why you are making calls from the server in the first place. Even though we significantly increased our page performance, it still takes a minimum of 5 seconds to render. Frankly, that’s horrible and should be avoided at all costs. Remember that your primary job as a developer is to make web pages as fast as possible, and taking a dependency on a system out of your control is just asking for problems. A good alternative here would be to add some script to the page that makes the call from the client side, completely relieving the server of that job. Think carefully if that work can be pushed to the client and off the server, because you would much rather 100 client machines take that load than your web server.

For More Information

Creating ASP.NET Applications Using Wait Handles

ASP.NET Performance Case Study: Web Service Calls Taking Forever

ASP.NET Case Study: Hang on WaitOne, WaitAny or WaitMultiple

Capturing a Memory Dump Using DebugDiag 1.2

Kirk Evans[MSFT] — Tue, 06 Sep 2011 20:54:53 GMT

In previous posts, I talked about using WinDBG to debug. Another incredibly handy tool to have at your disposal is the Debug Diagnostic Tool, or DebugDiag 1.2. This tool can be used to automatically capture a memory dump when an exception occurs. To understand some of the tools at your disposal to troubleshoot issues, see my previous posts:

Those posts show how to read a dump file, but don’t go into how to capture a dump file. I want to write a dump file when an exception occurs so I can see what the stacktrace for it is, the state of local variables, and what modules were loaded. For instance, I am seeing a System.ArithmeticException in my logs, but I can’t seem to figure out where it is coming from. It would be nice to capture a full memory dump when the problem occurs. DebugDiag does just that.

Open the tool and click the Add Rule button and create a new Crash rule.

Next, select the target type. Since I am debugging a SharePoint application, I know it is occurring in a single web application, so I choose “A specific IIS web application pool”.

I know that my SharePoint application uses the “SharePoint – 80” web application pool, so I choose that one.

In the next window, choose “Exceptions” from the Advanced Settings panel at the bottom of the window.

After you click the Exceptions button, you can configure a new Exception. We want to capture a dump whenever a System.ArithmeticException occurs, so we choose Exception Code E0434F4D – CLR (.NET) 1.0 – 3.5 Exception from the list, and type System.ArithmeticException in the .NET Exception Type text box. Finally, change the Action Type to Full Userdump so that a memory dump is created when the condition is met.

The next window allows you to name the rule and to configure the location where the userdump file is created.

The last window asks you if you want to activate the rule.

Once we configure the rule, you can see the rule and the number of userdumps that have been generated by the rule.

To test this, I created a very simple application page that throws a new System.ArithmeticException.

using System;
using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;

namespace ArithmeticExceptionPage.Layouts.ArithmeticExceptionPage
{
    public partial class ApplicationPage1 : LayoutsPageBase
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            throw new System.ArithmeticException("oops");
        }
    }
}

When I browse to the page, the exception is thrown, and DebugDiag sees the exception. The Userdump Count column is incremented to 2 and the .DMP file is written to the directory that I configured. Be mindful that the length of time required to create the dump depends on a few factors, including how much memory your process is consuming and how fast your disk is.

Looking at the directory, I can right-click on the file and see a new shell extension to Analyze Crash/Hang Issue.

DebugDiag runs for awhile, and generates a nice report letting me know what happened.

Of course, we already knew that the problem was a System.ArithmeticException, but the resulting stack information can often give us a better idea of what is going on with the process. And if we don’t see our problem in the generated report, we know how to open WinDBG and take a look ourselves.

For More Information

Intro to WinDBG for .NET Developers

WinDBG and PssCor2 for SharePoint Developers

Get Social Data from SharePoint via Web Service

Kirk Evans[MSFT] — Fri, 29 Jul 2011 15:16:24 GMT

I am working on a project to create newsletters to send to my customers, and experimenting with a few ways to collect the information so that I can easily author the newsletter later.

One way that I am experimenting with is to use the Tagging feature of SharePoint 2010. As I visit different web sites that are external to my farm, I can tag that site with my own URL, which saves the tag and the URL in my profile. Later, I can view the tags and notes that I have applied.

My idea was to add the Tags and Noteboard shortcut to my favorites bar in IE, making it easy to just click the link to save it to my profile.

When I click the link, I get the following window popup:

Later, I can visit my MySite and see the items that I have tagged. But how can I use this to possibly automate the generation of a newsletter? I really don’t want to go back to my MySite and copy and paste everything I tagged.

One way is to use the SocialData web service in SharePoint 2010. This service is located at:

http://<host>/<site>/_vti_bin/SocialDataService.asmx

For my MySite, this means the URL would be:

http://my/sites/kirkevans/_vti_bin/SocialDataService.asmx

There are methods such as GetTagUrlsOfUser, GetTagsOfUser, and GetTagUrlsOfUserByKeyword that we can use to retrieve data. This makes obtaining the data a simple WCF call. In Visual Studio 2010, add a service reference to the SocialDataService.asmx endpoint, and then edit the app.config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <behaviors>
      <endpointBehaviors>
        <behavior name="allowImpersonationBehavior">
          <clientCredentials>
            <windows allowedImpersonationLevel="Impersonation" />
          </clientCredentials>
        </behavior>
      </endpointBehaviors>
    </behaviors>
    <bindings>
      <basicHttpBinding>
        <binding name="SocialDataServiceSoap">
          <security mode="TransportCredentialOnly">
            <transport clientCredentialType="Ntlm"
                       proxyCredentialType="Ntlm"
                       realm="" />            
          </security>
        </binding>
      </basicHttpBinding>
    </bindings>
    <client>
      <endpoint address="http://my/sites/kirkevans/_vti_bin/socialdataservice.asmx"
                binding="basicHttpBinding"
                bindingConfiguration="SocialDataServiceSoap"
                contract="MySiteSocialData.SocialDataServiceSoap"
                behaviorConfiguration="allowImpersonationBehavior"
                name="SocialDataServiceSoap" />
    </client>
  </system.serviceModel>
</configuration>

WCF is talking to an ASMX service, so there are certain aspects of communication that it guesses at. It correctly guesses that we are using basic HTTP communication, but it cannot infer the details of authentication, we have to provide those details.

WCF has to be told to use impersonation to use the current user’s credentials to access the backend service. We do this by creating the endpointBehavior.
Our SharePoint site uses NTLM for authentication. We tell WCF to use NTLM by configuring the HTTP binding. I also cleaned up the binding configuration a bit to make it easier to read.

Now that we have configured how to talk to the service endpoint and provided security details, the last thing is the easy part… writing the code to call the service.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplication2
{
    class Program
    {
        static void Main(string[] args)
        {
            MySiteSocialData.SocialDataServiceSoapClient proxy = 
                new MySiteSocialData.SocialDataServiceSoapClient();            
            proxy.ChannelFactory.Credentials.Windows.ClientCredential = 
                System.Net.CredentialCache.DefaultNetworkCredentials;

            string[] taggedItems =  
                proxy.GetTagUrlsOfUserByKeyword("Newsletter","sharepoint\\kirkevans");
            foreach (var item in taggedItems)
            {
                Console.WriteLine(item);                
            }
        }
    }
}

The result is a list of URLs that I have tagged with the term “Newsletter”:

Summary

I am still looking for an easy way to collect URLs and some text about them, and to be able to collect that data to generate a newsletter later. I am not sure if I will end up using tags and notes or if there is some other way to achieve this, but you’ve got to admit that it’s pretty cool to be able to tag various URLs and retrieve them later!

STSADM Crashes After In-Place Upgrade to SharePoint 2010

Kirk Evans[MSFT] — Wed, 20 Jul 2011 16:28:46 GMT

I recently performed an in-place upgrade of SharePoint 2007 to SharePoint 2010, and the upgrade appeared successful. I tried to run a few STSADM commands and got the following:

System.MissingMethodException: Method not found: 'Void Microsoft.SharePoint.SPRequestManager.Dispose()'.

at Microsoft.SharePoint.StsAdmin.SPStsAdmin.Main(String[] args)

After a few moments of panic and a quick search, I found a post that gave me a hint at the cause. To make it easier to run STSADM commands on my server, I had added the path to the BIN directory to the %PATH% environment variable:

When I opened a command window and ran STSADM, it was trying to run the SharePoint 2007 version of STSADM. The fix was simple, change the path variable to point to the SharePoint 2010 version:

If you want to confirm this on your SharePoint 2010 server, you can either use the System Properties that I showed above, or use PowerShell to see your environment variables:

Once I was running the correct version of STSADM, everything worked as expected.

Control Access to SharePoint Designer

Kirk Evans[MSFT] — Mon, 11 Jul 2011 12:41:09 GMT

You will often want to control who has access to use SharePoint Designer on your site. There are a number of ways to control who has permission, you will need to determine the combination that applies to your intended use of the site and your governance policy.

Determine Your Information Architecture

Inside Microsoft’s intranet, there are a huge number of SharePoint sites available for just about everything I could ever need. There are sites to access my HR information, fill out expense reports, schedule vacation, log my billable hours, read news about the company, participate in discussion boards, learn about upcoming product releases, and a slew of other sites. Some, such as the HR and corporate intranet news site, are completely read-only to me. I have no ability to contribute information, I can only read what’s available. On others, I might be part of a select group that is able to write information, such as participate in discussion lists or enter timecard and vacation information. Still others, I might own a web site and control who has access to the web site, but I do not control other sites within the same site collection. And the last scenario is similar to a MySite where I own the entire site collection and everything in it.

In this example, there is a defined information architecture. I do not universally have permissions to edit any site that I visit within the farm. Many sites are read-only to me. On sites where I am allowed to write information, I am typically restricted to write information only in certain areas of the site, for instance a particular list, I am not allowed to change the look and feel or grant others permissions to the site. As a concrete example, the Premier organization of which I am a part of has a number of different sites within a site collection. I can fill out a form on one site, but otherwise the site is read-only to me. My team has another site where we can add documents and links and sample code, but otherwise the site is read-only to me. The point is that your information architecture should help structure appropriate use and govern who has access to various capabilities and features depending on business need. This is not a blanket decision across the board, users are granted access and various permissions based on what they need to do with the site.

How Microsoft’s Internal Information Architecture Implies Governance

The policy within Microsoft’s intranet is pretty simple: some sites are managed, and some are not.

There is an HR site, for example, that tens of thousands of people access regularly. This site is on dedicated hardware and isolated from the shared resources farm. Only a handful of people have write permissions, while tens of thousands of people have read access and visit the site regularly. This site is business critical to the entire organization and was purposefully architected due to its importance. This wasn’t support by accident, but rather its storage and isolation was carefully planned when the site was being created. The site is architected to handle the number of users that access the site, including peak periods such as open enrollment, and is architected with high availability in mind. To get a site like this means I have to pay for it because it requires dedicated resources (SQL storage, disks, backups, etc). Since I am paying for a dedicated environment, I get to do pretty much whatever I want with it, such as deploy server-side custom code. The main point to note is that it is critical to the business and therefore budget is applied to ensure it is available.

On the other side are the unmanaged resources. I can request a team site in a shared resources farm, and I will be the site collection owner. There are a number of things that aren’t available to me in the shared resources farm. I cannot add custom code, and a number of services are unavailable to me such as analytics and PerformancePoint. While there are a few limitations, generally it’s pretty open, and I am able to use SharePoint Designer and grant others permission to use SPD. I can create workflows and lookup lists and a whole bunch of stuff that makes SharePoint admins typically shudder. The reason that I can do this is because it was planned and architected to allow self-service creation. The environment has disk space allocated to accommodate the growth, and not everything is lumped into one giant database. Site quotas are strictly enforced: if you need more than what is provided in the shared resources farm, then you need to purchase a managed solution from the IT group. Large list throttling is strictly enforced, and a number of other checks and balances are enforced that limit what I can do. If my site is unused for a period of time I have to renew it before it is deleted. If my site in the shared resources farm uses too many resources, the IT group will make me purchase a managed solution.

You can use this to your benefit in choosing whether to allow SharePoint Designer or not. In one part of your farm, you probably don’t want to enable it, while in another part (where everyone keeps their team sites), you want to let the site collection owner decide if they want to use it or not. Further, it’s not an all-or-nothing scenario, you can selectively choose what users can do with SPD if they have access to a particular site.

Control Access to SharePoint Designer

A nice feature in SharePoint 2010 is the ability to limit where SharePoint Designer can be used. Open Central Administration and go to General Application Settings / SharePoint Designer Settings. Here you can configure SharePoint Designer settings for an entire web application.

You may have a site that is highly branded, and all branding and configuration is created through Visual Studio and managed through code releases. In this case, you might want to completely disable SharePoint Designer for the web application. You might use this in a publishing scenario where nobody has access to use SPD in the production farm. Content is created in an authoring farm, possibly using SharePoint Designer, and then pushed to the production farm using content publishing. What may not be immediately evident is that you can use this same approach within the same farm and publish content from one web application to another.

The settings above apply to the entire web application. If we disable SharePoint Designer, we will see the following when we try to open the site in SPD:

If you have multiple site collections within a web application, you may want to choose whether to allow SPD for each site collection. Go to the Site Settings page at the root site of the site collection and you will find the link for SharePoint Designer settings for each site collection.

If SPD were enabled at the web application level, then we would have a screen that looks very similar to the page above. However, since we disabled SPD at the web application level, here is what it looks like if we try to override the settings at the site collection level. You’ll see that the web application settings overrule any settings at the site collection level, and use of SPD is disabled for all site collections in the web application.

Notice there are additional options besides turning SPD on or off. A common issue that customers have with SPD is that a user can detach a page from its definition and completely customize the page. Clearing the checkbox removes that permission, so users will not be able to detach from the site definition. Another common request is to restrict users from being able to make changes to master pages. By clearing the option, you can restrict if users can change master pages using SPD.

What if you want something more granular other than turning features of SPD on or off for everyone in a web application, or everyone in a site collection? SharePoint 2010 provides a more granular permissions model that lets you restrict what users can or cannot do.

Control Permissions to SharePoint Designer Features

By default, SharePoint 2010 provides 33 permission levels and six default groups. They are provided in the following table.

	Limited Access	View Only	Read	Contribute	Design	Full Control
List Permissions
Manage Lists
Override Check Out
Add Items
Edit Items
Delete Items
View Items
Approve Items
Open Items
View Versions
Delete Versions
Create Alerts
View Application Pages

Site Permissions
Manage Permissions
View Web Analytics Data
Create Subsites
Manage Web Site
Add and Customize Pages
Apply Themes and Borders
Apply Style Sheets
Create Groups
Browse Directories
Use Self-Service Site Creation
View Pages
Enumerate Permissions
Browse User Information
Manage Alerts
Use Remote Interfaces
Use Client Integration Features
Open
Edit Personal User Information

Personal Permissions
Manage Personal Views
Add/Remove Personal Web Parts
Update Personal Web Parts

One of the interesting things about SharePoint is how it security trims the user interface based on what permissions you have. For instance, as a site collection administrator, I have full permissions to the site. The site actions menu shows all of the commands, including Edit in SharePoint Designer:

To contrast, as a member of the Visitors group, I am given Read permissions. The site actions menu looks very different because the UI is security trimmed based on my permissions.

A user with Contribute permissions has the ability to create and edit site pages in a team site. Notice in the permissions grid above, we show that users with Contribute do not have the permission to add and customize pages, yet the UI shows that they have permission to add and edit wiki pages in the menu. This permission name is a little misleading, because you can still add new content in the form of new wiki pages, and can edit those pages, even add web parts to wiki pages as you see in the following screen shot:

What the Add and Customize Pages permission means is that without this permission, you cannot edit files at the root of the site (such as default.aspx in a team site) or files that reside in folders outside of lists and libraries.

While I can add and edit wiki pages, notice there’s no link to Edit in SharePoint Designer as there is with a user who has Full Control. I cannot add arbitrary pages (such as my own .ASPX or a new web part page). Additionally, if we try to open the site in SharePoint Designer 2010, we receive the following error:

Let’s create a new permission group called Page Creators that only grants the Add and Customize Pages permission.

I grant this permission to a user that already has Contribute permissions. The permissions are cumulative, meaning if something is checked in one permission group but unchecked in others, the user will have that permission. You can see that the user retains Contribute permissions and is additionally granted Add and Customize Pages, which now lets them open in SharePoint Designer.

Where a user with Contributor access only has the Edit Page, New Page, and View All Site Content options, a user that is granted the Add and Customize Pages now has a link to Edit in SharePoint Designer as well as access to Site Settings. If we go to Site Settings, we can see that it, too, is security trimmed with only a few options available.

If we open the site in SharePoint Designer 2010, our options are security trimmed. We cannot access the site master pages, cannot create new lists, and cannot manipulate workflows. About the only thing we can do is to add a new web part page to the site.

By manipulating permissions, we can control what users are able to do with our site using SharePoint Designer and limit their capabilities to only those activities that they need to have for a particular site.

Do NOT Remove the Use Remote Interfaces Permission

Looking at the list of permissions, you may be tempted to disable the Use Remote Interfaces permission because it mentions using “SharePoint Designer interfaces to access the Web site” — a reasonable conclusion, but just don’t do it! This permission has a dependent permission, Use Client Integration Features, and removing this permission disables all SharePoint integration with Office programs like Word, Excel, and PowerPoint. It also has the unintended effect of disabling your ability to access SharePoint’s web services.

[via http://blogs.msdn.com/b/sharepointdesigner/archive/2008/11/25/locking-down-sharepoint-designer.aspx]

For More Information

Governance features (SharePoint Server 2010)

Locking Down SharePoint Designer

Modifying the Page Layout for Enterprise Wiki Pages in SharePoint 2010

Kirk Evans[MSFT] — Fri, 08 Jul 2011 00:28:50 GMT

In this post, I will show how to modify the page layout for Wiki pages in SharePoint 2010.

There are many reasons you might want to edit the page layout for a Wiki page. For instance, you might want to move the Modified By and Last Modified Date fields to appear below the content. You might create a new content type based on the Wiki Page content type that adds a new field (such as a managed metadata field), or you might want to add additional functionality to the page via code behind.

It’s helpful to understand that pages in an Enterprise Wiki site use the same concepts as publishing pages. That is, there is a content type that defines what columns are available, and a page layout that positions those fields on the page. Here is the page at render time, we see the Last Modified Date and Modified By fields at the top of each page.

The page layout defines the fields that are used when editing the page and what the page looks like in edit mode versus render mode. Here, we can see the field controls change based on them being in edit mode. Notice the Last Modified Date and Modified By fields are read-only. We want to move those to the bottom of the page.

We could customize the out of box Wiki page layout (EnterpriseWiki.aspx), but it’s a better idea to create a new page layout and copy the contents. To create a custom layout, open the site in SharePoint Designer, go to the Page Layouts menu, and choose New Page Layout based on the Enterprise Wiki Page content type.

I highlighted the EnterpriseWiki.aspx out of box file. You can open that file, copy its contents, paste into your new CustomEnterpriseWiki.aspx page, then make your edits. Here, I moved the Modified By and Last Modified Date information to appear below the Wiki text.

Save, check in, publish, and approve changes. Go to the site settings, Page Layout and Template Settings, and add the new page layout.

Go back to the site, create a new Wiki page. Notice that existing pages use the old page layout (you can edit the page and choose the page layout), but all new pages by default use the new page layout.

Notice that the Last modified date and modified by fields now appear below the Wiki content.

For More Information

How to Create Custom SharePoint 2010 Page Layouts using SharePoint Designer 2010

How can I add a contact-button to a wiki homepage and page layout?

Programmatically creating Wiki Pages

Code-Behind Page Layouts with Visual Studio 2010

Adding a custom control to the SharePoint 2010 Team Site Wiki Page Template ( wkpstd.aspx ) programmatically

Querying Active Directory

Kirk Evans[MSFT] — Mon, 04 Jul 2011 15:26:42 GMT

In this post, we show how to query Active Directory using out of box tools as well as custom code.

Since joining Premier Field Engineering, I realized that an essential part of understanding SharePoint requires an understanding of Active Directory and LDAP. You need to have a decent grasp on the basics of LDAP to effectively configure and troubleshoot problems with user profile imports. I’ll admit, this is an area that I used to be very weak in, but it is not at all difficult to ramp up and become proficient.

Querying Using Active Directory Tools

I set up a domain called SharePoint.com and added an Organizational Unit (OU) called “Contractors”, then added a few users using the “Active Directory Users and Computers” MMC snap-in on the domain controller.

Besides the MMC tools, other tools are installed when you create a domain controller. Open the tool LDP.exe on the domain controller. In the Connection menu, choose “Connect” and leave the Server string empty, indicating you want to connect to the local directory.

Next, in the Connection menu, choose Bind and bind as the currently logged in user.

On the View menu, choose Tree, and navigate to an individual user. This is an incredibly useful tool to see the properties for a user and their values:

Note: You can install LDP to another server, such as a SharePoint web front end, by adding the Active Directory Lightweight Directory Services role to a Windows Server 2008 instance. This can be useful in troubleshooting connectivity from a SharePoint web front end to the domain controller, allowing you to query the DC from the web front end as the same user account that is running user profile imports.

In the screen shot above, notice the property “cn” with value “Andrew Fuller”, and the “userPrincipalName” property with value “andrewfuller@SharePoint.Com". We are going to show how to query these same properties using C#.

Querying Using C#

To get started, add a reference in your project to System.DirectoryServices.

The code to connect to Active Directory is very simple. We start by creating a new instance of the DirectoryEntry class. Think of the Active Directory domain as a tree, the DirectoryEntry class lets us point to a specific node in the tree. Referring to the screen shots above, we want to point to the Contractors node.

DirectoryEntry entry = new DirectoryEntry("LDAP://OU=Contractors,DC=SharePoint,DC=Com");

Just like with SQL queries, we need to tell the query engine what properties we want to include in the results. We do this by using the PropertiesToLoad.Add method. Also, just like SQL queries provide a WHERE clause to filter results, we can filter results in Active Directory to only include users with the Filter property.

ds.PropertiesToLoad.Add("name");                
ds.PropertiesToLoad.Add("userPrincipalName");

ds.Filter = "(&(objectClass=user))";

Now that we have told the query engine where we want to connect, what we want in the results, and how we want to filter results, we execute the query using the FindAll method.

SearchResultCollection results = ds.FindAll();

The results are returned in a SearchResultCollection, each SearchResult object contains a hashtable of properties and values. Think of each SearchResult as a single user. The properties are the same properties that we included in the query. Each property can have multiple values, but our results contain just a single value that we access through the 0 index:

foreach (SearchResult result in results)
{
    Console.WriteLine("{0} - {1}", 
        result.Properties["name"][0].ToString(),
        result.Properties["userPrincipalName"][0].ToString());
}

Part of the reason this is so cool is because it becomes incredibly handy to troubleshoot problems connecting to Active Directory and querying. Knowing what is happening under the hood and how to use tools like LDP become invaluable tools in your toolbelt, especially for SharePoint developers doing things with user profiles.

The Full Code Listing

using System;
using System.DirectoryServices;

namespace SharePointConsoleApplication2
{
    class Program
    {
        static void Main(string[] args)
        {
            DirectoryEntry entry = new DirectoryEntry("LDAP://OU=Contractors,DC=SharePoint,DC=Com");

            using (DirectorySearcher ds = new DirectorySearcher(entry))
            {
                ds.PropertiesToLoad.Add("name");
                ds.PropertiesToLoad.Add("userPrincipalName");

                ds.Filter = "(&(objectClass=user))";

                SearchResultCollection results = ds.FindAll();

                foreach (SearchResult result in results)
                {
                    Console.WriteLine("{0} - {1}",
                        result.Properties["name"][0].ToString(),
                        result.Properties["userPrincipalName"][0].ToString());
                }
            }
        }                
    }
}

The results:

Andrew Fuller - andrewfuller@SharePoint.Com
Nancy Davolio - NancyDavolio@SharePoint.Com
Janet Leverling - JanetLeverling@SharePoint.Com
Margaret Peacock - MargaretPeacock@SharePoint.Com

For More Information

Howto: (Almost) Everything In Active Directory via C#

Introduction to System.DirectoryServices, Part 2

Getting Search Results

Using ADSI, LDAP, and Network Management Functions With Active Directory

Programmatically Creating a SharePoint Content Type

Kirk Evans[MSFT] — Fri, 29 Apr 2011 14:32:51 GMT

I have been working on a project that requires creating a content type programmatically. Admittedly, it took me awhile to figure it out. You can create a content type for SharePoint declaratively in a feature using XML similar to the following:

<?xml version="1.0" encoding="utf-8"?>
<Elements Id="2fdb55a0-75c1-4ad4-b709-82b2e1393f34"
          xmlns="http://schemas.microsoft.com/sharepoint/">
  <ContentType ID="0x010100C568DB52D9D0A14D9B2FDCC96666E9F2
007948130EC3DB064584E219954237AF39
0075425CE93BDC404F8B042629FC235785"                   
               Name="TermsAndConditionsType"
               Group="Custom Content Types"
               Description="Custom Content Type for Terms and Conditions">
    <FieldRefs>
      <FieldRef ID="{f55c4d88-1f2e-4ad9-aaa8-819af4ee7ee8}"
                Name="PublishingPageContent" />
    </FieldRefs>
  </ContentType>
</Elements>

That really long ID that is broken into three lines needs to be on a single line, it was formatted for readability. The challenge is that when we try to delete the content type, we get an error that the content type is part of an application and cannot be deleted. So, I needed to instead create the content type programmatically.

The above XML works out to the following code:

using (SPSite site = new SPSite("http://sp2010dev:44365"))
{
    SPWeb web = site.RootWeb;
                
    HtmlField field = (HtmlField)web.Fields["Page Content"];

    SPContentTypeId id = new SPContentTypeId("0x010100C568DB52D9D0A14D9B2FDCC96666E9F2
007948130EC3DB064584E219954237AF39
0075425CE93BDC404F8B042629FC235785");
    SPContentType termsAndConditionsType = new
        SPContentType(
            id,            
            web.ContentTypes,
            "TermsAndConditionsType");
                
    web.ContentTypes.Add(termsAndConditionsType);
    termsAndConditionsType = web.ContentTypes[id];
    termsAndConditionsType.Group = "Custom Content Types";
    termsAndConditionsType.Description = "Custom Content Type for Terms and Conditions";
    termsAndConditionsType.Update();
    SPFieldLink l = new SPFieldLink(field) { DisplayName = "Page Content" };
    termsAndConditionsType.FieldLinks.Add(l);
    termsAndConditionsType.Update();                
}

Again, the really long string needs to be in a single line.

User Controls and Server Controls in SharePoint

Kirk Evans[MSFT] — Thu, 28 Apr 2011 20:37:00 GMT

ASP.NET developers have enjoyed using user controls and server controls in their development since ASP.NET was created. This post talks about user controls and server controls and shows how to implement them in SharePoint.

User Controls

A user control is a control that is associated with a .ASCX extension and was originally intended for a developer to re-use within a single project. ASP.NET developers know it is really easy to create a new user control, drag and drop some controls onto a visual designer, put some logic in the control’s code-behind, and re-use the control within a single project. This is because the .ASCX file associated with the control enables you to create markup for your control:

<%@ Assembly Name="$SharePoint.Project.AssemblyFullName$" %>
<%@ Control Language="C#" 
        AutoEventWireup="true" 
        CodeBehind="EchoControl.ascx.cs" 
        Inherits="ControlsDemo.ControlTemplates.ControlsDemo.EchoControl" %>


Enter some text:
<asp:TextBox ID="TextBox1" runat="server"></asp:TextBox>
<asp:Button ID="Button1" runat="server" Text="Button" />
<asp:Label ID="Label1" runat="server" Visible="false"></asp:Label>

Because you can add markup for your control, you get the nice WYSIWYG designers for your control. This allows you to drag items from the toolbox, such as a button, a label, and a text box.

Another nice feature of user controls is that you can quickly add event handlers for your control. For instance, I double-click the button to add some code-behind, and Visual Studio pops up the code editor to allow me to add some code.

using System;
using System.Web.UI;
using System.Web.UI.WebControls;

namespace ControlsDemo.ControlTemplates.ControlsDemo
{
    public partial class EchoControl : UserControl
    {     
        protected void Button1_Click(object sender, EventArgs e)
        {
            Label1.Text = "You entered: " + 
                TextBox1.Text + " at " + 
                System.DateTime.Now.ToLongTimeString();
            Label1.Visible = true;
        }
    }
}

Notice that our code-behind derives from System.Web.UI.UserControl. For existing ASP.NET developers, you will be happy to see that it’s the same old UserControl class that’s been there since ASP.NET was introduced. By using the .ASCX, Visual Studio 2010 provides the same WYSIWYG, drag-and-drop features that you have with ASP.NET development. In your SharePoint 2010 project, add a new User Control to your project, add the markup to the .ASCX file, and put some code in the code-behind. The project structure will look like this:

When the solution is compiled, the code (the .cs classes) are compiled into an assembly that is deployed to the GAC. The ASCX file points to our assembly, which SharePoint deploys to the global assembly cache. SharePoint deploys your .ASCX file to a special location:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\CONTROLTEMPLATES

If you open that directory, you will see lots of out-of-box controls, such as Welcome.ascx, the control that is registered in the v4.master master page in SharePoint 2010. Once the control is deployed to the ControlTemplates directory structure, an end user can then use the control in their master page or page. For instance, we’ll use SharePoint Designer 2010 to edit a master page to use our control. At the top of the file, register the control:

<%@ Register TagPrefix="demo" TagName="EchoControl"  
src="~/_controltemplates/ControlsDemo/EchoControl.ascx" %>

You can see where to do that in this screen shot:

Once our control is registered, we use it in our page, and the visual design is used in SharePoint Designer 2010 to give a WYSIWYG rendering to the end user.

We save the master page and then look at the final product:

User controls are a great way to add functionality to a SharePoint site for end users without requiring a web part. You might have noticed that Visual Studio 2010 includes a new project item template called “Visual Web Part”. This project simply creates a .ASCX control that is loaded by your web part.

Server Controls

A server control is a compiled control that renders on the server. Where user controls derive from System.Web.UI.UserControl, a server control (also known as a custom control) typically derives from System.Web.UI.WebControl. The main difference is that server controls do not have a corresponding .ASCX control. For some scenarios, this can be highly beneficial because the page parser does not need to parse the .ASCX control.

Creating a server control is very easy. In Visual Studio 2010, add a new ASP.NET Server Control item to your project from the Web group.

The generated template is just a suggestion, you can edit the class to suit your needs. Here is a useful control for SharePoint developers that simply shows the name of the server that is currently processing the request.

using System;
using System.Web.UI.WebControls;
using System.Web.UI;

namespace ControlsDemo
{
    [ToolboxData("<{0}:ServerNameControl runat=server></{0}:ServerNameControl>")]
    public class ServerNameControl : WebControl
    {
        protected override void CreateChildControls()
        {
            Label l = new Label();
            l.Text = System.Environment.MachineName;
            Controls.Add(l);   
        }
    }
}

This can be useful for troubleshooting inconsistent behavior among web front end servers in a load-balanced farm. When Visual Studio builds the project, the class is compiled into our assembly and deployed to the GAC. Once the control is registered in the GAC, we can use it in our master page similar to our previous example. The syntax is different, because before we could point the Src attribute to the .ASCX file, this time we need to register the control and import the namespace.

<%@ Register 
    TagPrefix="demo2" 
    Namespace="ControlsDemo" 
    Assembly="ControlsDemo, Version=1.0.0.0, Culture=neutral, PublicKeyToken=01374ea14aa626aa" %>
<%@ Import Namespace="ControlsDemo" %>

Once we register the control, we can now use it in our page.

<demo:ServerNameControl runat="server" id="serverNameControl"/>

Only this time, instead of seeing the nice WYSIWYG UI in SharePoint Designer 2010, we see an error message, “The type is not registered as safe.”

WTF?!?!

To understand what’s going on, you need to understand how the page is processed:

When a page with a user control is requested, the following occurs:

The page parser parses the .ascx file specified in the Src attribute in the @ Register directive and generates a class that derives from the System.Web.UI.UserControl class.

The parser then dynamically compiles the class into an assembly.

If you are using Visual Studio, then at design time only, Visual Studio creates a code behind file for the user control, and the file is precompiled by the designer itself.

Finally, the class for the user control, which is generated through the process of dynamic code generation and compilation, includes the code for the code behind file (.ascx.cs) as well as the code written inside the .ascx file.

[via http://support.microsoft.com/kb/893667]

The page parser will parse the page and determine if the page uses code-behind or includes in-line script. I have blogged about the impact of marking your controls as safe for pages here and here, which gives some insight to the problem. The reason that our user control example worked is because it was deployed to the ControlTemplates directory, an action that only a farm administrator would have privileges to do. Thus, any control in the ControlTemplates directory structure is marked as safe out of the box:

      <SafeControl Src="~/_controltemplates/*"
                   IncludeSubFolders="True"
                   Safe="True"
                   AllowRemoteDesigner="True"
                   SafeAgainstScript="True" />

The server control does not have a corresponding .ASCX file, so we need to explicitly mark it as safe. Visual Studio 2010 makes this very easy. Add a new Module to your project and give it a name.

Click the module in the Solution Explorer pane, and edit the Safe Controls property. Add a new safe control, taking care to add the right namespace. You can use an asterisk for the typename if any type in the namespace is safe, or you can use the class name of a specific class that is safe as I did here:

You can delete the sample text file that is generated. Now, when we look at the web.config file, we see a SafeControl entry for our control:

      <SafeControl Assembly="ControlsDemo, Version=1.0.0.0, Culture=neutral, PublicKeyToken=01374ea14aa626aa"
                   Namespace="ControlsDemo"
                   TypeName="ServerNameControl"
                   Safe="True"
                   SafeAgainstScript="False" />

Now, when we use the control in SharePoint Designer 2010, we have the markup correct but see nothing in the WYSIWYG view:

If we view the page in the browser, we can see the effect of the control being rendered (highlighted in yellow to show you where the server name is rendered).

You can do some amazing things with ASP.NET controls, and you can bring many of your existing controls forward so that end users can take advantage of them in their pages and master pages.

Debugging JavaScript with the IE Developer Tools

Kirk Evans[MSFT] — Mon, 25 Apr 2011 10:47:02 GMT

An interesting question was posted on one of the distribution lists today.

“The client wanted to look into this MSDN drag and drop typed left navigation bar to work on Sharepoint 2010, does anyone make this work or have idea about this?”

Click the control that is circled, and the menu is expanded or contracted. It’s basic JavaScript, but the more interesting part is how you can use the IE Developer Tools to inspect, format, and debug any page’s JavaScript. To use the Developer Tools in IE (introduced in IE8, I am showing IE9 in this post) press F12. Click the “Select element by click” icon, or press Ctrl + B. Then, select the item in the page that you want to inspect.

The lower pane changes to show the page source and the CSS rules that are applied, even showing which rules were overridden (very useful for figuring out CSS issues).

When we clicked the element in the page, it highlighted the following HTML for us:

<img id="ResizeImageIncrease" class="cl_nav_resize_open" 
title="Expand" onmousedown="onIncreaseToc()" alt="Expand" src="http://i3.msdn.microsoft.com/Hash/a19e30a4020fe81d2b1209058013a360.png">

Pretty easy to see the onmousedown handler calls “onIncreaseToc()”. Let’s find that function. In the top-right of the developer tools pane, there is a search window. Switch to the Script tab and enter onIncreaseToc into the search window, and navigate through the results until you find the function declaration for onIncreaseToc.

You can see that the script is minified and somewhat unreadable. Click the Configuration toolbar item and choose “Format JavaScript”

Now that the page is formatted nicely, you can easily set breakpoints in the code.

Click the Start Debugging button in the developer tools pane and watch your breakpoint get hit! You can set watches, inspect locals, and inspect the call stack.

Step through the code just like you would in Visual Studio, using F11 (step into), F10 (step over), and shift + F11 (step out).

In the Console tab, you can enter script commands to inspect various elements.

And this is just the start of what you can do with this tool. How about a great script profiling tool, built right into every instance of Internet Explorer, that profiles the performance of your script? Just go to the Profiler tab and start profiling, interact with your page, and then stop profiling to see the call results.

And the best thing is that it’s already available with every copy of Internet Explorer 8 and above, so you don’t have to install any add-ons to debug. This is extremely helpful when trying to reproduce behavior at someone’s desktop where you can’t install your favorite add-ons or debugging tools.

TinyGet for SharePoint

Kirk Evans[MSFT] — Thu, 14 Apr 2011 14:50:52 GMT

The IIS 6 Resource Kit Tools includes a tool called TinyGet to quickly issue an HTTP request to a page from a command line, and it works with IIS6 and higher (yes, you can install the resource kit tools on an IIS7 machine without problems).

Why would you want to issue HTTP GET requests from the command line? I can’t tell you how many times I have written HttpWebRequest code in a .NET command-line executable to issue a request to a page just so that I could verify the output, only to throw the program away once I am done with it. Using TinyGet and LogParser together, you can verify content in pages!

Get a Page as the Current User

I needed this yesterday to simulate some load on my site so that I could see what happens when I call a single page many times, without having to try to refresh a bunch of browser windows quickly.

tinyget -a:2 -u:CurrentUser -srv:sp2010dev.sharepoint.com -uri:/SitePages/SlowWebPart.aspx –data

This will access the URL “http://sp2010dev.sharepoint.com/SitePages/SlowWebPart.aspx” as the current user using NTLM authentication, and display the data in the command window. Want to make sure that a particular string is in the output? Pipe the results into the FINDSTR command. For instance, we want to make sure that the string “You entered: 5” is in the output.

tinyget -a:2 -u:CurrentUser -srv:sp2010dev.sharepoint.com -uri:/SitePages/SlowWebPart.aspx –data | FINDSTR  “You entered: 5”

Using TinyGet to Warm Up / Verify Your Sites

Got a demo? Create a script that issues a request to various pages to make sure they are JIT’d and cached in memory. Besides demos, how can you use this in real life? How about when you have a users reporting outages for specific sites, and you want to verify which ones are responding? You could create a script that verifies what sites are running.

http://blogs.technet.com/b/operationsguy/archive/2010/11/16/hey-dude-one-of-your-servers-is-broken-part-2.aspx

Using TinyGet to Troubleshoot Memory Issues

QA or operations has reported memory issues and you want to try to reproduce it and troubleshoot it in your environment?

http://blogs.msdn.com/b/tess/archive/2008/02/15/net-debugging-demos-lab-3-memory.aspx

WinDBG and PssCor2 for SharePoint Developers

Kirk Evans[MSFT] — Wed, 13 Apr 2011 18:17:40 GMT

In this post, we’ll see some steps about how to troubleshoot a production SharePoint site using WinDBG and PssCor2.dll. For an introduction to setting up your environment and getting started, see my blog post “Intro to WinDBG for .NET Developers”. In this post, we’ll apply some of our WinDBG skills to a SharePoint environment and show you how you can figure out what’s going on in your site.

In this example, a customer has raised the issue that their SharePoint site is very slow. Let’s assume that you have looked at IIS logs and notice server render times are abnormally long and you want to find out what the server is doing.

Capture a Memory Dump in IIS

The first thing we need to do is figure out which process we need to capture memory for. In Windows Server 2008, open a command window to “c:\windows\system32\inetsrv” and type the following command:

appcmd list wp

This will list the worker processes for IIS, showing you the process ID (aka PID) and its associated application pool. Here’s the output on my machine.

Once you know the PID (in this screenshot, I want to troubleshoot PID 5292), open up task manager on the machine. Select the checkbox “Show processes from all users” and find the W3WP.exe process with the ID you just found. Right-click and choose “Create Dump File”.

Note that this will pause all threads for the process while the memory dump file is created, the length of time depends on the amount of memory the process is using.

Inspect the Crash Dump in WinDBG with PssCor2

Open WinDBG (making sure that you’ve already configured your environment for PssCor2.dll and set your symbol path) and choose “Open Crash Dump”. Open the .DMP file that task manager created for you. I copied PssCor2.dll into the same directory as WinDBG.exe, so I type:

.load psscor2

Next, type “!help” to make sure that the psscor2.dll extension was loaded. Once you see the help instructions, we can start evaluating the memory dump file.

!aspxpages

So far, this is my favorite command in the psscor2.dll extension. This handy little guy asks for all the .ASPX pages in memory. It shows the URL to the page, how long the thread has been running, what the thread ID is, and what the HTTP verb is for the response.

Looking at our screen shot, I can see a bunch of requests to “SlowWebPart.aspx” that have been running for 20 or more seconds. Most have a thread ID of “XXX”, but a few of them have a thread ID that we can inspect (like 50 and 55). One of the first things I do when trying to analyze a slow SharePoint site is to run this command to see how long things are taking. If you see one page frequently at the top, it is running the longest and is the culprit for consuming resources. If you see lots of different pages, you can look at the callstacks to see what everyone is doing and what they are waiting on.

~[thread]s

When you hit a breakpoint in Visual Studio, you can see the call stack, showing the chain of methods that were called up until the breakpoint was hit. If you want to do something similar in WinDBG, you need to tell it thread contains the CLR stack you are interested in. You can get a list of managed threads with the command:

!threads

In WinDBG, you need to switch to a specific thread context to see its stack. In the previous !aspxpages example, we already know that there are threads 50 and 55 that we can take a look at, so we would issue the following statement:

~50s

The output is shown below.

Notice the yellow highlights in the screen shot. The first:

0:000> ~50s

The command prompt is “0:000>”, and we issued the command to switch the thread context to thread ID 50. The second:

0:050> !clrstack

Notice the command prompt has changed to “0:050>”. This tells us that the current context is thread id 50. Looking at the stack, we see that our ASPX page was loaded, the CreateChildControls method was called on an object called MyWebPart.VisualWebPart1.VisualWebPart1, and then we see “MyWebPart.ServiceReference1.ServiceClient.GetData(Int32)” called before a bunch of WCF stuff. We can infer from looking at the stack that a web part or control is in the middle of calling a WCF service. We also saw earlier by using !aspxpages that this thread has been running for 24 seconds.

~* e !clrstack

We will follow the steps in Tess Ferrandez’ demo video (watch the entire video, but scroll to 24:48 in the video) to list the CLR stack for every thread in memory.

~* e !clrstack

This statement says “for every thread, dump the CLR stack”. The output is very long, but we see a bunch of CLR stacks that look just like the screen shot above. This is our most likely suspect, a web part that is calling a web service synchronously and is waiting on a response.

!DumpStackObjects (dso)

Since we switched our context thread to thread ID 50, we want to inspect objects in the stack, kind of like looking at the Locals window in Visual Studio.

!dso

The output looks like:

I highlighted some of the results. Notice that the objects are shown as they are pushed onto the stack (oldest on stack at the bottom of the page). We can see our web part user control, and we can see some WCF stuff. Just like you would poke around in Visual Studio locals window to inspect various objects, we can inspect objects in WinDBG.

!DumpObject (do) and !DumpField

Notice the underlined item at the top of the WCF box. When you use !dso to dump the stack objects, the second column has a header column that says “Object”. This is actually the address for the object. We can use that piece of information to inspect the objects. For instance, we can inspect the ASCX user control by using its object address.

!do 00000001479219e0

The output is:

Look at the very bottom of that output, we see that the user control contains a Label control. Want to see what’s in it? We can use the !dumpfield command. To see how to use it, use !help dumpfield to see the syntax. When we used !do, our object address was 00000001479219e0. We use that to inspect specific fields on that object, like drilling down into an object in the locals window in Visual Studio.

!dumpfield -field Label1 -string 00000001479219e0

The output shows the fields for the Label control. Pretty cool! We can use this trick to inspect the various objects in the stack. In our screenshot above (under the !DumpStackObjects heading), I underlined a System.String object with address 0000000141389b58. We can use the !dumpobject command to inspect it:

!do 0000000141389b58

I highlighted in the result that the URI for the web service we are calling is “http://localhost:5555/service.svc”. Now we not only know that we have a bunch of web parts that are waiting on a response (and have been for a long time) from a service, but we also know the URI of the service. We can then do some tests to confirm that the web service is responding quite slowly!

We can then use this to determine a course of action. We have a web part that is blocking threads, waiting on a response from a web service that is not under our control. We shouldn’t block the thread waiting on a synchronous response from a web service, instead we should use an asynchronous request with a timeout. Our plan is then to change the web service call to be asynchronous, load test it using a tool like the Web Performance Test in Visual Studio 2010, and confirm that it now meets our performance expectations even when the web service we are calling is performing poorly.

Lest you think this example is contrived, let me assure you that it is not. This same behavior happens if you use HttpWebRequest synchronously, consume an RSS feed synchronously, call a WCF service synchronously, etc. I have seen a lot of people make web service calls from ASP.NET controls synchronously, this can kill your SharePoint site’s performance.

For More Information

Video introduction to WinDBG for .NET Developers by Tess Ferrandez

.NET Debugging Demos Lab 4: High CPU Hang – Review also by Tess

Intro to WinDBG for .NET Developers

Drill Into .NET Framework Internals to See How the CLR Creates Runtime Objects

Intro to WinDBG for .NET Developers

Kirk Evans[MSFT] — Mon, 11 Apr 2011 17:53:24 GMT

When your code goes into production, you usually no longer have access to its binaries when they reach their final destination. Whether that is someone’s desktop or a set of servers, you no longer have access to directly observe your code and its environment. Operating system patches are applied, network policies are changed, firewall rules are restricted, disks are configured… as your code lives its life in its new home, there’s a wide range of things that may change in its environment and affect how it behaves (or rather misbehaves). You liberally littered your code with lines of logging logic to learn in these lulls (long alliteration!), and that gives you an idea of where the code is not performing as expected, but you still are unaware of the exact reason (and thus, the fix) that your code is not working as expected.

Your challenge now is to try to figure out what is going wrong without wasting the customer’s time doing troubleshooting, because there’s nothing that a business user loves more than being asked by a technical guy which button is he really clicking to get that error. You don’t have the luxury (should have thrown that in the alliteration sentence previously) of spending days or weeks doing troubleshooting, you need to know what is happening right now.

In a perfect world, you would have the stack trace, you’d be able to inspect locals, you could debug the code. Well, it turns out, you can do just that… and never attach to the customer’s environment.

Download WinDbg and Get Started

Download the Debugging Tools for Windows to your local developer machine. You can get them as part of the Windows SDK. Choose the Debugging Tools for Windows in the Common Tools section if you only want the debugging tools for your current machine’s platform. If it is an x86 machine, then only the x86 tools are installed. If your machine has an Intel 64-bit processor, then only the x64 tools are installed. If you choose the redistributable version, then you get all three (x86, x64, and Itanium). After you download, install to your local developer machine (not the customer’s machine).

One tip is to change the installation path. By default, windbg will be copied to the Program Files directory. Instead, change the path to something like “d:\debug”. This will make it easier to add extensions.

Now that you’ve installed, in the start menu you will see a new program group, “Debugging Tools for Windows (x64)”, and a new program in it called “WinDbg”.

Install PssCor2

The next step is to install the extensions for managed code. By default, WinDbg is a tool designed for unmanaged code debugging, but an extension ships with the .NET Framework called SOS.dll that enables managed code debugging. An additional WinDbg extension called PssCor2 has been created that is a superset of SOS and provides additional functionality for managed code developers. This extension will allow you to inspect managed threads, the managed object heap, and inspect the CLR stack, among other things. Download PssCor2 and unzip it.

I unzipped PssCor2 to my “d:\debug” directory, this will make it much easier to use as you will see later in this post.

Another tip: copy the psscor2.dll file and copy it to the d:\debug directory where windbg.exe resides. This will make loading the extension much easier for you.

Set Your Symbol Path

Have you ever noticed the .PDB file that is generated in your program’s bin/debug folder when you compile in Visual Studio? This is the file that contains debugging symbols for your assembly. It does not contain instructions or executable code, but rather provides the ability for a debugger to translate your compiled code’s instructions into something you can read. Microsoft provides a set of symbols on a public server for your debugger program to use. These symbols are located at http://msdl.microsoft.com/download/symbols.

The symbols will be downloaded from Microsoft and cached locally on your machine. Choose a path to store the symbols. For instance, I store mine at “d:\debug\symbols”. After you use WinDBG, you will see the symbols downloaded to your machine.

Now that you understand the symbols are downloaded and cached locally, here’s the first tricky part: setting your symbol path in WinDbg. Start WinDbg, and on the File menu choose “Symbol File Path…” Specify your symbol path using the following syntax:

srv*d:\debug\symbols*http://msdl.microsoft.com/download/symbols

Lest you think this is a misprint or you are having problems getting this to work, here’s exactly what’s in my symbol path.

I prefer to keep everything (symbols, extensions, dump files) under a single folder at the root of a drive (for instance, “d:\debug”) because it is easier to locate all the pieces when you need them.

Create a Problem

To get started, we’ll need some code to work with. I’ll use a simple Console application.

using System;

namespace Microsoft.PFE.Samples
{
    public class Program
    {
        static void Main()
        {
            Console.WriteLine("Enter a message:");
        string input = Console.ReadLine();
            Data d = new Data
            {
                ID = 5,
                Message = input,
                CurrentDateTime = System.DateTime.Now
            };
            Console.WriteLine("You entered: " + d);
        }
    }

    public class Data
    {
        public int ID {get; set;}
        public string Message {get; set;}
        public DateTime CurrentDateTime {get; set;}


        public override string ToString()
        {
            Console.ReadLine();
            return string.Format("ID:{0} {1} at {2}", ID, Message, 

CurrentDateTime.ToLongTimeString());
        }
    }
}

You can see that this is an over-glorified “echo” program. I compile the program using CSC.exe, making sure we are using the version of CSC from Windows SDK 7.0, which is the .NET Framework 3.5 version.

csc.exe Program.cs

PssCor2 works with the .NET 3.5 framework, so if you are compiling with Visual Studio, then change the framework version to 3.5 before compiling. Run the program and enter a string to see the program work correctly. ~~There is also a PssCor4.dll, which works with .NET 4.0~~. [Update 4/13/2011 – thanks to an astute reader for pointing out PssCor4 is not available]

The customer’s complaint is that they have to hit enter twice in the Console window and cannot figure out why. Our program is doing something we didn’t intend, so let’s see what’s going on. Of course, for our contrived example, we know that it’s because of the second Console.ReadLine that was mistakenly placed in the ToString method of the Data class, so let’s use WinDbg to find it.

Run the program and enter a string, but do not hit Enter at the second ReadLine prompt. We are now at the point that we want to capture what’s happening, and we can do this in a dump file.

Take a Dump

Rather than try to remote desktop into a customer’s machine or worse, asking them to install Visual Studio so you can do some debugging, you can have them capture a dump of the process so that you can debug it later. Windows 7 and Windows 2008 have this great feature for capturing dumps from a process using Task Manager. To see what’s going on in the customer’s environment without asking them to install utilities, just have them start Task Manager (either from Control Panel or using Ctrl + Alt + Del). Find the process, right-click, and choose “Create Dump File”.

After we create the dump file, we see the following message:

The dump file is just a snapshot of memory. However large the dump file is on disk is the size of the memory used for the process. You can zip the file to significantly reduce its size, and download it to your machine for offline analysis.

There are other ways to create a dump file. The one we created is a full dump, and its size is 74.9 MB (just for our small Console application). You can use other tools to create a dump. For instance, I like to use Process Explorer from SysInternals to capture a dump (using the Full Dump option generates the same thing as right-clicking a process in Task Manager).

Other tools used to create dumps include using ADPlus (included in the Debugging Tools for Windows), or even DebugDiag. Using DebugDiag lets you create rules for when dumps are captured, such as when a particular exception is raised. This can be very helpful for grabbing a snapshot of memory as the error occurs so that you can troubleshoot errors in ULS logs in SharePoint.

Start Exploring with WinDbg

Now that we have a dump file to debug, we go back to WinDbg and start exploring. From the file menu, choose “File / Open Crash Dump” to open the dump file in WinDbg. As soon as you open it, you should see text:

Loading Dump File [D:\debug\program6.dmp]
User Mini Dump File: Only registers, stack and portions of memory are available

Symbol search path is: srv*d:\debug\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Version 7600 MP (8 procs) Free x64
Product: WinNt, suite: SingleUserTS
Machine Name:
Debug session time: Sun Feb  6 10:43:57.000 2011 (GMT-6)
System Uptime: not available
Process Uptime: 0 days 1:05:48.000
.........................
ntdll!NtRequestWaitReplyPort+0xa:
00000000`76d2ff7a c3              ret

Inside that text, you can see the path for the dump file and the symbol search path. At the bottom of the screen is a text box where you will enter commands.

List Modules

Let’s start by listing the modules that were loaded in the process when the dump file was created. In the text box at the bottom of the dump window, type “lm” to list the modules. The output looks like this:

0:000> lm
start             end                 module name
00000000`00120000 00000000`00128000   program    (deferred)             
00000000`742b0000 00000000`74379000   msvcr80    (deferred)             
00000000`76ac0000 00000000`76bba000   user32     (deferred)             
00000000`76bc0000 00000000`76cdf000   kernel32   (pdb symbols)          d:\debug\symbols\kernel32.pdb\D5E268B5DD1048A1BFB011C744DD3DFA2\kernel32.pdb
00000000`76ce0000 00000000`76e8b000   ntdll      (pdb symbols)          d:\debug\symbols\ntdll.pdb\0F7FCF88442F4B0E9FB51DC4A754D9DE2\ntdll.pdb
000007fe`f3fb0000 000007fe`f4134000   mscorjit   (deferred)             
000007fe`f5030000 000007fe`f5f0b000   mscorlib_ni   (deferred)             
000007fe`f7650000 000007fe`f7ffe000   mscorwks   (deferred)             
000007fe`f8010000 000007fe`f80a0000   mscoreei   (deferred)             
000007fe`f80a0000 000007fe`f810f000   mscoree    (deferred)             
000007fe`fcb70000 000007fe`fcb7f000   CRYPTBASE   (deferred)             
000007fe`fcc40000 000007fe`fcc4f000   profapi    (deferred)             
000007fe`fcf20000 000007fe`fcf8b000   KERNELBASE   (deferred)             
000007fe`fd0e0000 000007fe`fd2e2000   ole32      (deferred)             
000007fe`fd4d0000 000007fe`fd59a000   usp10      (deferred)             
000007fe`fd6f0000 000007fe`fe476000   shell32    (deferred)             
000007fe`fe480000 000007fe`fe4ae000   imm32      (deferred)             
000007fe`fe840000 000007fe`fe84e000   lpk        (deferred)             
000007fe`fe9d0000 000007fe`feaab000   advapi32   (deferred)             
000007fe`feb50000 000007fe`fec7e000   rpcrt4     (deferred)             
000007fe`fec80000 000007fe`fecf1000   shlwapi    (deferred)             
000007fe`fed00000 000007fe`fed67000   gdi32      (deferred)             
000007fe`fee10000 000007fe`fef19000   msctf      (deferred)             
000007fe`fef20000 000007fe`fefbf000   msvcrt     (deferred)             
000007fe`fefd0000 000007fe`fefef000   sechost    (deferred)

The thing you are looking for here is the existence of mscorwks for a .NET 3.5 application. PssCor2 works with .NET 3.5. If you have a .NET 4.0 process that you want to debug, you can do this as well, but you need to use the SOS.dll extension that ships with the .NET Framework. For an interesting look at changes introduced in .NET 4.0, see http://debuggingblog.com/wp/2009/07/07/windbg-extension-sos-in-clr-40net-framework-40-ctp-net-runtime-dll-renamed-and-sos-commands-just-got-richer/.

For SharePoint developers, this can be an invaluable tool for debugging things like feature receivers and event handlers that never seem to fire. You can list the loaded modules to see if your assembly is ever loaded into memory. If it’s not, then you most likely have a configuration issue, significantly limiting the number of places to go searching for things to fix. For ASP.NET developers, this can be extremely helpful in figuring out why an HttpModule is not firing, indicating the most likely problem is in the web.config file.

Load PssCor2

OK, we saw the modules that are loaded, let’s dive into our code a little and see what we can discover. Remember that we have an extension for WinDbg that we need to load. To load PssCor2.dll as an extension into WinDbg, use the following command:

.load d:\debug\psscor2\amd64\psscor2.dll

My machine is a 64-bit Intel machine, so I will load the AMD64 version of PssCor2.dll. The version you are debugging with and the architecture of the dump that you are debugging must match. If you are debugging an x86 process, you need to load the x86 version of PssCor2.dll.

To verify it was loaded, type the following command:

!help

The output of this is:

0:000> .load d:\debug\psscor2\amd64\psscor2.dll
0:000> !help
-------------------------------------------------------------------------------
PSSCOR is a debugger extension DLL designed to aid in the debugging of managed
programs. Functions are listed by category, then roughly in order of
importance. Shortcut names for popular functions are listed in parenthesis.
Type "!help <functionname>" for detailed info on that function. 

Object Inspection                  Examining code and stacks
-----------------------------      -----------------------------
DumpObj (do)                       Threads
DumpArray (da)                     CLRStack
DumpStackObjects (dso)             IP2MD
DumpAllExceptions (dae)            BPMD
DumpHeap                           U
DumpVC                             DumpStack
GCRoot                             EEStack
ObjSize                            GCInfo
FinalizeQueue                      EHInfo
PrintException (pe)                COMState
TraverseHeap
DumpField (df)
DumpDynamicAssemblies (dda)
GCRef
DumpColumnNames (dcn)
DumpRequestQueues
DumpUMService

Examining CLR data structures      Diagnostic Utilities
-----------------------------      -----------------------------
DumpDomain                         VerifyHeap
EEHeap                             DumpLog
Name2EE                            FindAppDomain
SyncBlk                            SaveModule
DumpThreadConfig (dtc)             SaveAllModules (sam)
DumpMT                             GCHandles
DumpClass                          GCHandleLeaks
DumpMD                             VMMap
Token2EE                           VMStat
EEVersion                          ProcInfo 
DumpModule                         StopOnException (soe)
ThreadPool                         MinidumpMode 
DumpHttpRuntime                    FindDebugTrue
DumpIL                             FindDebugModules
PrintDateTime                      Analysis
DumpDataTables                     CLRUsage
DumpAssembly                       CheckCurrentException (cce)
RCWCleanupList                     CurrentExceptionName (cen)
PrintIPAddress                     VerifyObj
DumpHttpContext                    HeapStat
ASPXPages                          GCWhere
DumpASPNETCache (dac)              ListNearObj (lno)
DumpSig
DumpMethodSig                      Other
DumpRuntimeTypes                   -----------------------------
ConvertVTDateToDate (cvtdd)        FAQ
ConvertTicksToDate (ctd)
DumpRequestTable
DumpHistoryTable
DumpBuckets
GetWorkItems
DumpXmlDocument (dxd)
DumpCollection (dc)

Examining the GC history
-----------------------------
HistInit
HistStats
HistRoot
HistObj
HistObjFind
HistClear

mscordacwks.dll

I prefer not to try to run windbg on the server with the problem. Instead, we capture a dump, which creates a snapshot of memory at a moment in time, and download that file to my local Windows 7 machine. If I am troubleshooting a problem on a Windows Server 2008 R2 machine, I want to capture the dump and download it to my Windows 7 machine. When I try to inspect it using psscor2, I get the following error:

CLRDLL: CLR DLL load disabled
Failed to load data access DLL, 0x80004005
Verify that 1) you have a recent build of the debugger (6.2.14 or newer)
            2) the file mscordacwks.dll that matches your version of mscorwks.dll is
                in the version directory
            3) or, if you are debugging a dump file, verify that the file
                mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.
            4) you are debugging on the same architecture as the dump file.
                For example, an IA64 dump file must be debugged on an IA64
                machine.

You can also run the debugger command .cordll to control the debugger's
load of mscordacwks.dll. .cordll -ve -u -l will do a verbose reload.
If that succeeds, the PSSCOR command should work on retry.

If you are debugging a minidump, you need to make sure that your executable
path is pointing to mscorwks.dll as well.

A quick Bing search yields a blog post that shows how to work around the mscordacwks issue by copying the version of mscordacwks from the server running SharePoint and copying to the same directory where windbg is running. The version on my Windows Server 2008 R2 machine is 4952, so I copy to my d:\debug directory and rename it to “mscordacwks_AMD64_AMD64_2.0.50727.4952.dll”.

Once it is copied and renamed, run the command from the error output above to do a verbose reload.

.cordll -ve -u –l

Once you get mscordacwks loaded, everything should work out fine. You will know you are successful when you see the following line in windbg:

CLR DLL status: Loaded DLL mscordacwks_AMD64_AMD64_2.0.50727.4952.dll

Inspect the CLR Stack

Let’s look at the managed stack with the following command:

!clrstack

That outputs the following:

0:000> !clrstack
OS Thread Id: 0xa48 (0)
*** WARNING: Unable to verify checksum for mscorlib.ni.dll
Child-SP         RetAddr          Call Site
000000000012e910 000007fef5a910e9 DomainNeutralILStubClass.IL_STUB(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)
000000000012ea30 000007fef5a91202 System.IO.__ConsoleStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Int32, Int32 ByRef)
000000000012ea90 000007fef538065a System.IO.__ConsoleStream.Read(Byte[], Int32, Int32)
000000000012eaf0 000007fef53a28ca System.IO.StreamReader.ReadBuffer()
000000000012eb40 000007fef5a9435f System.IO.StreamReader.ReadLine()
000000000012eb90 000007ff0017015b System.IO.TextReader+SyncTextReader.ReadLine()
000000000012ebf0 000007fef791d502 Program.Main()

Very cool! You now see how to inspect the stack to see what calls were pushed down onto it. Immediately, we can see that inside our Program.Main function, a Console.ReadLine call was made and we are currently waiting on user input.

Summary

This post was to help you get started with windbg. For a fantastic overview of WinDbg and how to troubleshoot various types of issues, watch the video “Debugging .NET Applications with WinDbg” by Tess Ferrandez.

Acknowledgements

A huge thanks to fellow PFE Chad Ray. He has helped me through many issues while I ramp up to learn how to use WinDbg more effectively.

For More Information

Process Explorer from SysInternals

PssCor2 Debugging Extension

Download the Debugging Tools for Windows

SOS.dll (SOS Debugging Extension)

“Failed to load data access DLL, 0x80004005” – OR – What is mscordacwks.dll?

DebugDiag

Adding jQuery to Every Page in SharePoint with Delegate Controls

Kirk Evans[MSFT] — Wed, 06 Apr 2011 22:50:00 GMT

A customer asked how they could add jQuery to every page in SharePoint. They wanted to modify core.js on the disk, but of course we all know that Microsoft does not support modifying files that reside in the _layouts folder. Of course there are supported ways to do this without such hacks.

All of the out-of-box SharePoint master pages (v4.master, default.master, minimal.master, and nightandday.master) include the following in the HEAD section of the HTML markup:

<SharePoint:DelegateControl runat="server" 
          ControlId="AdditionalPageHead" 
          AllowMultipleControls="true"/>

Chris O’Brien has a great write-up of Using the Delegate Control. The DelegateControl provides a plugin framework for SharePoint, telling ASP.NET to find any registered delegate controls, and for each call the LoadControl method to load your control. It’s a very slick way to add controls to pages without modifying the master page.

The solution for us is to deploy the jQuery script to SharePoint, and then deploy a delegate control that references that script location.

Create the Control

This is one of those things that I think every SharePoint developer should become instantly familiar with. You can add your own user controls to SharePoint and reference them from your pages just like you can with ASP.NET. In Visual Studio 2010, create a new empty SharePoint project called “Microsoft.PFE.DSE.Samples” and make it a farm solution.

Note: We could deploy a delegate control using the sandbox as well. I chose to use a farm solution because I am deploying the jQuery library to the SharePoint root (aka 14 hive), which you cannot do with sandboxed solutions.

In Visual Studio, click “Add New Item” and add a new User Control called jQueryControl.ascx.

In the source view for the control, add the following line of code:

<script type="text/javascript" 
    src="/_layouts/Microsoft.PFE.DSE.Samples/jquery-1.5.2.min.js" />

The result looks like this:

That’s how easy it is to create a control and deploy it to SharePoint. You can now use your control in your pages in SharePoint. Now, let’s see how to register it as a delegate control.

Adding the Feature

The next step is to add a feature to the project that tells SharePoint about our control. In Visual Studio, add a new “Empty Element” to your project named “elements”.

Change the resulting Elements.xml to the following:

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">  
  <Control Id="AdditionalPageHead" 
           Sequence="90"             
           ControlSrc="~/_CONTROLTEMPLATES/Microsoft.PFE.DSE.Samples/jQueryControl.ascx" />
</Elements>

Now that the feature element is created, we need to edit the feature that deploys it. Open the feature in the SharePoint project and change the scope to “Web”. Give it a title and description as well.

By deploying your feature with web scope, every page in the web where the feature is activated will include the jQuery script, including pages in the layouts directory (remember that they use master pages with AdditionalPageHead, too). If you change the scope to “Site”, every page in your site collection will include the script, changing to WebApplication means every page in your application gets it, and Farm means every single page in the farm gets it.

Deploying the jQuery Script File to Layouts

The last part of the puzzle is to deploy the jQuery script file to the layouts directory. Right-click your project and add a SharePoint “Layouts” Mapped Folder.

In the resulting folder, add the jQuery javascript file. The results look like this:

The Results

Hit F5 in Visual Studio 2010 to deploy the solution and start debugging. The debugger will automatically attach to the page, and you will see the script loaded in the debugger, without having to modify the master page, the page you are viewing, and certainly you won’t be modifying core.js to make it happen!

The source for the project is attached.

For More Information

Using the Delegate Control

DelegateControl Class

SharePoint Developers–What’s On Your Machine?

Kirk Evans[MSFT] — Tue, 05 Apr 2011 19:19:04 GMT

I recently got a question from a customer about utilities to include on their development machines. My view is that you should trust your developers and let them install the tools they need. However, some devs don’t have managers who trust their developers to do the right thing and they need to request tools in advance. Which got me thinking… what’s on your machine? What do you consider essential for developing software? I challenged myself to go through my machine and recall all of the various utilities, add-ons, programs, scripts, etc that I use regularly and listed them below. There are a lot, and I am sure I am missing a few! I am always on the lookout for new tools and utilities that will make development easier, so… what’s on your machine?

SharePoint Server 2010 + pre-requisites (Standalone)

ADO.NET Data Services Update for .NET Framework 3.5 SP1 for Windows 7 and Windows Server 2008 R2 (enables WCF Data Services for SharePoint) - http://www.microsoft.com/downloads/en/details.aspx?familyid=79d7f6f8-d6e9-4b8c-8640-17f89452148e&displaylang=en

Office 2010 Professional Plus (32-bit)

Microsoft SQL Server 2008 Management Studio Express (or I sometimes have full-blown SQL Server 2008 and tools on the same box) - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=08e52ac2-1d62-45f6-9a4a-4b76a8564a2b&displaylang=en

Microsoft SQL Server Community Projects & Samples - http://sqlserversamples.codeplex.com/

SharePoint Designer 2010 (32-bit) - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d88a1505-849b-4587-b854-a7054ee28d66&displaylang=en

Visual Studio 2010 Ultimate Edition - http://www.microsoft.com/visualstudio/en-us/

Silverlight 4 Tools for Visual Studio - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=b3deb194-ca86-4fb6-a716-b67c2604a139&displayLang=en

Expression Studio 4 Ultimate - http://www.microsoft.com/expression/

Open XML SDK - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c6e744e5-36e9-45f5-8d8c-331df206e0d0&DisplayLang=en

Microsoft Windows SDK for Windows 7 and .NET Framework 4 - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=6b6c21d2-2006-4afa-9702-529fa782d63b&displaylang=en

Psscor2 Managed-Code Debugging Extension for WinDbg - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=5c068e9f-ebfe-48a5-8b2f-0ad6ab454ad4&displayLang=en