Random Musings of Jeremy Jameson : MOSS 2007

Database Default Locations in SQL Server

Jeremy Jameson — Thu, 03 Dec 2009 13:30:00 GMT

I've mentioned before the importance of using multiple "spindles" when working with large SQL Server databases.

Generally speaking, the recommendation is to use different RAID 1+0 arrays for data and log files -- and depending on the size and load of your database, you may also need to isolate data files on individual RAID 1+0 arrays.

For really large OLTP databases that require lots of IOPS, additional arrays might also be required to split the log files (although, honestly, I haven't worked on any projects where this has been necessary).

One of the configuration steps that I always recommend for any Production SQL Server environment is to configure the default database locations to ensure your "data I/O" is isolated from your "log I/O" (both of which should be isolated from other I/O -- specifically, the paging file).

For example, suppose you have the following disk configuration on your SQL Server cluster:

C: (Local Disk) - RAID 1 array for operating system files, program files, and paging file
D: (DATA01) - RAID 1+0 array for data files
L: (LOG01) - RAID 1+0 array for log files
Q: (Quorum)

In this simplistic example, the D:, L:, and Q: drives are LUNs on the backend SAN, whereas the C: drive is DAS (Direct Attached Storage). Hopefully, the Storage team responsible for managing your SAN provides dedicated physical drives for D: and L: -- although, honestly, I've seen several enterprise organizations that don't dedicate these drives and subsequently experience performance issues later on.

Given the above disk configuration, SQL Server Management Studio should be used to configure the default database locations similar to the following:

Server Properties
- Database Settings
  - Default database locations
    - Data: D:\NotBackedUp\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA
    - Log: L:\NotBackedUp\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\DATA

You certainly don't have to use the NotBackedUp folder if you don't want to (that's just the standard that I've been using for years).

Once you've configured the default database locations, anytime you create a new database -- for example, when you create a new Web application or content database in Microsoft Office SharePoint Server (MOSS) 2007 -- the data and log files will be placed on the desired drives (i.e. D: and L:, respectively).

Of course, when creating MOSS 2007 databases for a Production (or Test) environment, you still need to resize them appropriately in order to avoid having them simply auto-grow from a very small initial size (in order to avoid fragmentation and optimize performance).

Suppose you need to determine the default database locations through SQL (perhaps because you are scripting the process to create your databases). If you fire up SQL Server Profiler and start a new trace (the default trace options are fine), you will find that SQL Server Management Studio executes the following SQL statements when viewing the Database Settings page in the Server Properties window:

declare @RegPathParams sysname
declare @Arg sysname
declare @Param sysname
declare @MasterPath nvarchar(512)
declare @LogPath nvarchar(512)
declare @ErrorLogPath nvarchar(512)
declare @n int

select @n=0
select @RegPathParams=N'Software\Microsoft\MSSQLServer\MSSQLServer'+'\Parameters'
select @Param='dummy'
while(not @Param is null)
begin
    select @Param=null
    select @Arg='SqlArg'+convert(nvarchar,@n)

    exec master.dbo.xp_instance_regread
        N'HKEY_LOCAL_MACHINE',
        @RegPathParams,
        @Arg,
        @Param OUTPUT
        
    if(@Param like '-d%')
    begin
        select @Param=substring(@Param, 3, 255)
        select @MasterPath=substring(
            @Param,
            1,
            len(@Param) - charindex('\', reverse(@Param)))            
    end
    else if(@Param like '-l%')
    begin
        select @Param=substring(@Param, 3, 255)
        select @LogPath=substring(
            @Param,
            1,
            len(@Param) - charindex('\', reverse(@Param)))
    end
    else if(@Param like '-e%')
    begin
        select @Param=substring(@Param, 3, 255)
        select @ErrorLogPath=substring(
            @Param,
            1,
            len(@Param) - charindex('\', reverse(@Param)))
    end
    
    select @n=@n+1
end
            
print 'LogPath = ' + @LogPath
print 'MasterPath = ' + @MasterPath

Note that I truncated the actual SQL batch and added the print statements.

Bug with Duplicate Field Names in MOSS 2007

Jeremy Jameson — Tue, 01 Dec 2009 16:12:00 GMT

I encountered a rather nasty bug in Microsoft Office SharePoint Server (MOSS) 2007 yesterday that occurs when a custom field (i.e. site column) has the same name as an existing field. Note that this issue will also occur in Windows SharePoint Services (WSS) v3.

I initially created the following Fields.xml file:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Field ID="{11F7E97D-6282-44b1-8222-2E6E377BCDFC}"
    Name="AnnouncementEndDate"
    Type="DateTime"
    Format="DateOnly"
    Group="Fabrikam Custom Columns"
    DisplayName="Announcement End Date">
  </Field>
  <Field ID="{9D1F7E52-EFBF-431e-9331-7DCA31D9CB7A}"
    Name="AnnouncementStartDate"
    Type="DateTime"
    Format="DateOnly"
    Group="Fabrikam Custom Columns"
    DisplayName="Announcement Start Date">
  </Field>
</Elements>

Next, I created the ContentTypes.xml file:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <ContentType
    ID="0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D400EF37EB6F40C54a21A3872B1E6CA5BC0A"
    Name="Announcement Page"
    Description="Announcement Page is a custom content type ..."
    Group="Fabrikam Content Types">
    <FieldRefs>
      <FieldRef ID="{9D1F7E52-EFBF-431e-9331-7DCA31D9CB7A}" Name="AnnouncementStartDate" />
      <FieldRef ID="{11F7E97D-6282-44b1-8222-2E6E377BCDFC}" Name="AnnouncementEndDate" />
    </FieldRefs>
    <DocumentTemplate TargetName="/_layouts/CreatePage.aspx" />
  </ContentType>
</Elements>

Note

The really long ContentType ID above specifies that Announcement Page inherits from the out-of-the-box Welcome Page. I vaguely recall reading somebody's blog where he or she stated that you shouldn't inherit from the OOTB page types. However, I don't recall the justification -- if there even was one. I haven't encountered any problems with this approach and thus haven't seen any compelling reason to stop doing so.

Then I created a page layout for the new Announcement Page content type.

After building and deploying the corresponding WSP -- and activating the feature -- I verified that I could create a new Announcement Page and specify field values, including Announcement Start Date and Announcement End Date. Consequently I checked in my changes to TFS.

At that point, I started thinking that perhaps AnnouncementStartDate should simply be StartDate and AnnouncementEndDate should simply be EndDate (thinking that we might want to use these fields for more than just announcements at some point in the future). Consequently, I changed the Fields.xml file as follows:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Field ID="{11F7E97D-6282-44b1-8222-2E6E377BCDFC}"
    Name="EndDate"
    Type="DateTime"
    Format="DateOnly"
    Group="Fabrikam Custom Columns"
    DisplayName="End Date">
  </Field>
  <Field ID="{9D1F7E52-EFBF-431e-9331-7DCA31D9CB7A}"
    Name="StartDate"
    Type="DateTime"
    Format="DateOnly"
    Group="Fabrikam Custom Columns"
    DisplayName="Start Date">
  </Field>
</Elements>

I propagated the changes to the ContentTypes.xml file:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <ContentType
    ID="0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D400EF37EB6F40C54a21A3872B1E6CA5BC0A"
    Name="Announcement Page"
    Description="Announcement Page is a custom content type ..."
    Group="Fabrikam Content Types">
    <FieldRefs>
      <FieldRef ID="{9D1F7E52-EFBF-431e-9331-7DCA31D9CB7A}" Name="StartDate" />
      <FieldRef ID="{11F7E97D-6282-44b1-8222-2E6E377BCDFC}" Name="EndDate" />
    </FieldRefs>
    <DocumentTemplate TargetName="/_layouts/CreatePage.aspx" />
  </ContentType>
</Elements>

After rebuilding my Web application, adding and deploying the updated WSP, and activating the feature, I discovered that the Announcement Page content type did not have the Start Date and End Date fields.

Since there were no errors deploying the WSP or activating the feature, I was rather baffled by the issue. Then I discovered that by renaming AnnouncementStartDate to StartDate, I introduced a duplicate field name. Note the following field definition from the OOTB fieldswss.xml file:

    <Field ID="{64cd368d-2f95-4bfc-a1f9-8d4324ecb007}"
        Name="StartDate"
        SourceID="http://schemas.microsoft.com/sharepoint/v3"
        StaticName="StartDate"
        Group="$Resources:Base_Columns"
        Type="DateTime"
        Format="DateOnly"
        DisplayName="$Resources:core,Start_Date;"><!-- _locID@DisplayName="camlidT6" _locComment=" " -->
        <Default>[today]</Default>
    </Field>

Similarly, my custom EndDate field conflicted with the OOTB EndDate field.

My memory might be incorrect, but I vaguely recall getting an error a few years ago when trying to define a second field with the same name as an existing field. Obviously that was long before MOSS 2007 Service Pack 2 (which I am currently running) so it's possible the behavior has changed. [Honestly, while I could try to repro this using the MOSS 2007 RTM version, I'm not going to bother. It would take me about an hour to build out a vanilla MOSS 2007 RTM environment, and that is far longer than I am willing to invest in this issue. It's also possible that I am simply remembering a slightly different issue -- perhaps a duplicate content type name.]

Regardless of whether this is a new problem, or has been around since the original MOSS 2007 release isn't the point. What is important is that you get no warnings or errors when your custom feature mistakenly attempts to define new StartDate and EndDate fields.

Once I identified the crux of the issue, I decided to simply rip out my custom StartDate and EndDate fields and instead use the OOTB fields. After all, as my old mantra goes, always try to use something out-of-the-box instead of customization whenever possible.

In other words, I updated my ContentTypes.xml file to the following:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <ContentType
    ID="0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D400EF37EB6F40C54a21A3872B1E6CA5BC0A"
    Name="Announcement Page"
    Description="Announcement Page is a custom content type ..."
    Group="Fabrikam Content Types">
    <FieldRefs>
      <FieldRef ID="{64cd368d-2f95-4bfc-a1f9-8d4324ecb007}" Name="StartDate" />
      <FieldRef ID="{2684F9F2-54BE-429f-BA06-76754FC056BF}" Name="EndDate" />
    </FieldRefs>
    <DocumentTemplate TargetName="/_layouts/CreatePage.aspx" />
  </ContentType>
</Elements>

Unfortunately, I then discovered that the OOTB EndDate field does not parallel the definition of the OOTB StartDate field:

    <Field ID="{2684F9F2-54BE-429f-BA06-76754FC056BF}"
        Name="EndDate"
        Type="DateTime"
        DisplayName="$Resources:core,End_Time;"
        Format="DateTime"
        FromBaseType="TRUE"
        Group="_Hidden"
        SourceID="http://schemas.microsoft.com/sharepoint/v3/fields"
        StaticName="EndDate" ><!--DisplayName=$Resources:camlid3;-->
        <FieldRefs>
            <FieldRef ID="{7D95D1F4-F5FD-4a70-90CD-B35ABC9B5BC8}" Name="fAllDayEvent" RefType="AllDayEvent" />
        </FieldRefs>
        <Default>[today]</Default>
    </Field>

Notice that with EndDate, Format="DateTime" whereas with StartDate, Format="DateOnly". Also note that the display name (in English - U.S.) is "End Time" -- instead of the expected "End Date". So much for consistency ;-)

Lastly, note that the OOTB fieldswss.xml also contains the following:

    <Field ID="{8A121252-85A9-443d-8217-A1B57020FADF}"
        Name="_EndDate"
        Group="$Resources:Base_Columns"
        Type="DateTime"
        DisplayName="$Resources:End_Date"
        Format="DateTime"
        SourceID="http://schemas.microsoft.com/sharepoint/v3/fields"
        StaticName="_EndDate" >
        <Default>[today]</Default>
    </Field>

Unfortunately, this field definition doesn't specify Format="DateOnly" either -- otherwise I would have just used it instead.

Thus I reverted back to using my original AnnouncementStartDate and AnnouncementEndDate custom fields. It's not ideal, but it works.

Building SharePoint WSPs with Team Foundation Build

Jeremy Jameson — Wed, 18 Nov 2009 13:45:00 GMT

As I noted in my previous post, I recently discovered that my approach for building Web Solution Packages (WSPs) in Microsoft Office SharePoint Server (MOSS) 2007 isn't compatible with Team Foundation Build.

I'm actually a little embarrassed to say this, but when I created the original "DR.DADA" approach for MOSS 2007 development back on the Agilent Technologies project, we were using Visual SourceSafe -- not Team Foundation Server (TFS) -- and a "manual" build process.

I'd used VSS and automated builds on other projects before (using NAnt), but never got around to automating our MOSS 2007 builds on the Agilent project because, honestly, there were just too many other higher priority items. Besides, each build only required a couple of minutes of actual human effort because most of the build was scripted.

Still, an automated daily build (and deployment to DEV) is a really, really good thing to have.

I've been fortunate to be on a few projects since then that have leveraged TFS.

However, up until about a month ago, I hadn't used Team Foundation Build (outside of the Jameson Datacenter, of course) due to the fact that we are leveraging the extranet TFS instance hosted by Microsoft.

Note that Microsoft IT makes it very easy for us to provision new TFS projects on either the extranet or one of several internal TFS instances. Configuring builds using Team Foundation Build on one of the intranet TFS instances is very easy (from what I hear), but I strongly prefer working off the extranet TFS instance because then I don't have to VPN into CorpNet in order to have access to source control.

However, choosing the extranet TFS instance also means we can't configure builds using the out-of-the-box functionality in Team Foundation Server (at least not without setting up a build server on the extranet). Fortunately, I've found a way to schedule "manual" builds that look a lot like automated builds performed using Team Foundation Build.

So, if you are building SharePoint WSPs -- regardless of whether you use the real Team Foundation Build or my "imitation Team Foundation Build" -- you need a way to build the WSPs without referring to referenced assemblies using relative paths.

As I first mentioned in my previous post, relative paths work just fine when compiling from within Visual Studio or using MSBuild from the command line. However, they don't work at all when queuing the builds through Team Foundation Build.

The problem with relative paths is that Team Foundation Build uses a different folder structure when compiling your projects. Specifically, it changes the output folder for all compiled items to be under a new Binaries folder -- not the location specified in the project settings within Visual Studio.

In other words, if you refer to a referenced assembly using something like:

..\..\..\CoreServices\bin\%BUILD_CONFIGURATION%\Fabrikam.Demo.CoreServices.dll

then you will find that this works just fine when building through Visual Studio -- or even when compiling using TFSBuild.proj from the command line (a.k.a. a "Desktop Build"). However, if you then queue the build through Team Foundation Server, you'll find your build fails because the referenced assembly was actually output to a different folder.

If you dive into the log file for the build, you will find that Team Foundation Build modifies the OutDir variable and sets it to something like:

C:\Users\svc-build\AppData\Local\Temp\Demo\Daily Build - Main\Binaries\Debug\

So the trick to building WSPs with Team Foundation Build is to leverage the OutDir variable instead of relying on relative paths to referenced assemblies.

Here is the updated DDF file based on my earlier sample:

;
; This ddf specifies the structure of the .wsp solution cab file.
;
; HACK: OPTION EXPLICIT cannot be used when specifying a variable with the /D option,
; otherwise MakeCAB aborts with an error similar to the following:
;
;        ERROR: Option Explicit and variable not defined: OUT_DIR
;
;.OPTION EXPLICIT    ; Generate errors for undefined variables

.Set CabinetNameTemplate=Fabrikam.Demo.Publishing.wsp

; The following variable must be set when calling MakeCAB (using the /D option)
;.Define OUT_DIR=

.Set DiskDirectoryTemplate=CDROM    ; All cabinets go in a single directory
.Set CompressionType=MSZIP            ; All files are compressed in cabinet files
.Set UniqueFiles=ON
.Set Cabinet=ON
.Set DiskDirectory1=%OUT_DIR%\Package

DeploymentFiles\PackageFiles\manifest.xml

.Set SourceDir=%OUT_DIR%            ; Copy assemblies from %OUT_DIR% folder

Fabrikam.Demo.Publishing.dll
Fabrikam.Demo.CoreServices.dll

.Set SourceDir=                    ; Copy files relative to project folder

.Set DestinationDir=Fabrikam.Demo.Publishing.DefaultSiteConfiguration
DefaultSiteConfiguration\FeatureFiles\Feature.xml

.Set DestinationDir=Fabrikam.Demo.Publishing.Layouts
Layouts\FeatureFiles\Feature.xml
Layouts\FeatureFiles\ProvisionedFiles.xml

.Set DestinationDir=Fabrikam.Demo.Publishing.Layouts\MasterPages
Layouts\MasterPages\FabrikamMinimal.master

;.Set DestinationDir=Fabrikam.Demo.Publishing.Layouts\PageLayouts
;Layouts\PageLayouts\MinimalPage.aspx

.Set DestinationDir=Fabrikam.Demo.Publishing.Layouts\Images
Layouts\Images\FabrikamLogo_32x32.png

.Set DestinationDir=Fabrikam.Demo.Publishing.Layouts\Themes\Theme1
Layouts\Themes\Theme1\BreadcrumbBullet.gif
Layouts\Themes\Theme1\FauxColumn-Fixed-2Col.png
Layouts\Themes\Theme1\FauxColumn-Fixed-3Col.png
Layouts\Themes\Theme1\Fabrikam-Basic.css
Layouts\Themes\Theme1\Fabrikam-Core.css
Layouts\Themes\Theme1\Fabrikam-FixedLayout.css
Layouts\Themes\Theme1\Fabrikam-IE.css
Layouts\Themes\Theme1\Fabrikam-IE6.css
Layouts\Themes\Theme1\Fabrikam-QuirksMode.css
Layouts\Themes\Theme1\Tab-LeftSide.jpg
Layouts\Themes\Theme1\Tab-RightSide.jpg

.Set DestinationDir=ControlTemplates\Fabrikam\Demo\Publishing\Layouts
Layouts\Web\UI\WebControls\GlobalNavigation.ascx
Layouts\Web\UI\WebControls\StyleDeclarations.ascx

.Set DestinationDir=Layouts\Fabrikam
Layouts\MasterPages\FabrikamMinimal.master

Note how I've replaced the BUILD_CONFIGURATION variable with the OUT_DIR variable. Not surprisingly, the OUT_DIR variable in the DDF is specified similar to how BUILD_CONFIGURATION was previously specified when calling makecab.exe. However, unlike the build configuration the OutDir variable will likely contain spaces as well as a trailing slash (which makecab.exe apparently doesn't like). Therefore we must quote the OutDir variable and append with "." if a trailing slash is found.

Here is the corresponding update to the project file:

  <PropertyGroup>
    <BuildDependsOn>
      $(BuildDependsOn);
      CreateSharePointSolutionPackage
    </BuildDependsOn>
    <QuotedOutDir>"$(OutDir)"</QuotedOutDir>
    <QuotedOutDir Condition="HasTrailingSlash($(OutDir))">"$(OutDir)."</QuotedOutDir>
  </PropertyGroup>
  <Target Name="CreateSharePointSolutionPackage" Inputs="@(None);@(Content);$(OutDir)$(TargetFileName);" Outputs="$(ProjectDir)$(OutDir)Package\Fabrikam.Demo.Publishing.wsp">
    <Message Text="Creating SharePoint solution package..." />
    <Exec Command="makecab /D OUT_DIR=$(QuotedOutDir) /F &quot;$(ProjectDir)DeploymentFiles\PackageFiles\wsp_structure.ddf&quot;" />
  </Target>

With these changes, the SharePoint WSP is successfully built regardless of whether it is compiled through Visual Studio, from the command line using MSBuild and the TFSBuild.proj file, or as an automated build using a Team Foundation Build server.

The "Copy Local" Bug in Visual Studio

Jeremy Jameson — Wed, 18 Nov 2009 12:21:00 GMT

If you've ever worked with me on a Microsoft Office SharePoint Server (MOSS) 2007 project -- or if you've read my Sample Walkthrough of the DR.DADA Approach to SharePoint -- then you've probably seen the following comment:

Note: Referenced assemblies must be specified with a path corresponding to the build configuration. If the path is not specified to the referenced assembly, then the build works fine as long as the referenced assembly is not in the GAC.

However, when the referenced assembly is in the GAC (i.e. after a deployment) then MakeCAB will not be able to find the referenced assembly (since it is no longer copied to the current project's bin\Debug or bin\Release folder).

It turns out that I was using a rather elaborate workaround for a problem that is actually much easier to solve.

To workaround the "Copy Local" bug and force a referenced assembly to always be copied to the output folder (regardless of whether the referenced assembly is in the GAC):

In the Solution Explorer window in Visual Studio, expand the References folder for the project and then select the referenced assembly.
In the Properties window, change the value of Copy Local to False, and then change it back to True.

Following these two simple steps explicitly adds <Private>True</Private> to the project file, as shown in the following example:

    <ProjectReference Include="..\CoreServices\CoreServices.csproj">
      <Project>{01C58D27-9818-45D6-A0B6-8EF765CA9397}</Project>
      <Name>CoreServices %28CoreServices\CoreServices%29</Name>
      <Private>True</Private>
    </ProjectReference>

When you add a referenced assembly in Visual Studio using a project reference, Copy Local defaults to True, but Visual Studio doesn't explicitly state this in the MSBuild project file. Toggling the value of Copy Local forces this element to be added to the project file and consequently you no longer need any hacks to reference the assembly in its original output folder.

At this point, you might be wondering why do I bring this up after all this time? After all, hasn't the hack I came up with for building SharePoint Web Solution Packages (WSPs) been working for several years? Well, yes, in most cases it works just fine.

However, there's one fundamental problem that I only recently discovered back in early October: you can't use Team Foundation Build to build the WSP when specifying relative paths to assemblies in the DDF file.

I'll cover this in my next post.

SharePoint Web Part to Redirect from HTTP to HTTPS

Jeremy Jameson — Tue, 10 Nov 2009 13:09:00 GMT

Yesterday, I detailed the steps I recommend for configuring SSL on sites built on Microsoft Office SharePoint Server (MOSS) 2007. I also mentioned that users won't automatically be redirected from HTTP to HTTPS, and how I've previously used a little bit of code to automatically perform this redirection.

Here is a base class that contains the core logic for detecting when a redirect from HTTP to HTTPS is required and automatically redirecting as necessary:

using System;
using System.Diagnostics;
using System.Globalization;
using System.Text;
using System.Web;
using System.Web.UI.WebControls.WebParts;

using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;

using Fabrikam.Demo.CoreServices.Logging;
using Fabrikam.Demo.Web.Properties;

namespace Fabrikam.Demo.Web.UI.WebControls
{
    /// <summary>
    /// Base class for Web Parts that require secure communication (i.e. HTTPS).
    /// </summary>
    public abstract class SslRequiredWebPart : WebPart
    {
        /// <summary>
        /// Redirect from HTTP to HTTPS (if required).
        /// </summary>
        /// <param name="e">An <see cref="System.EventArgs" /> object that
        /// contains the event data.</param>
        protected override void OnInit(
            EventArgs e)
        {
            base.OnInit(e);

            HttpContext context = HttpContext.Current;

            if (context == null)
            {
                throw new InvalidOperationException(
                    "HttpContext.Current is null.");
            }

            HttpContextWrapper contextWrapper = new HttpContextWrapper(context);

            bool forceRedirect = IsSslRedirectRequired(contextWrapper);
            
            if (forceRedirect == false)
            {
                return;
            }

            HttpRequest request = context.Request;

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "SslRequiredWebPart - Redirecting from HTTP to HTTPS"
                    + " for URL ({0})...",
                request.RawUrl);

            Debug.Assert(request.RawUrl.StartsWith(
                "/",
                StringComparison.OrdinalIgnoreCase) == true);

            StringBuilder urlBuilder = new StringBuilder();

            urlBuilder.Append(Uri.UriSchemeHttps);
            urlBuilder.Append(Uri.SchemeDelimiter);
            urlBuilder.Append(request.Url.Host);
            urlBuilder.Append(request.RawUrl);

            string redirectUrl = urlBuilder.ToString();

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "SslRequiredWebPart - Redirecting to URL ({0})...",
                redirectUrl);

            context.Response.Redirect(redirectUrl, false);
            context.ApplicationInstance.CompleteRequest();
        }

        /// <summary>
        /// Determines if an SSL redirect is required.
        /// </summary>
        /// <param name="context">Context information representing an individual
        /// HTTP request.</param>
        /// <returns><c>true</c> if a redirect from HTTP to HTTPS is required,
        /// otherwise <c>false</c>.</returns>
        protected virtual bool IsSslRedirectRequired(
            HttpContextBase context)
        {
            if (context == null)
            {
                throw new ArgumentNullException("context");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "SslRequiredWebPart - RedirectToHttpsWhenSslRequired = {0}",
                Settings.Default.RedirectToHttpsWhenSslRequired);

            if (Settings.Default.RedirectToHttpsWhenSslRequired == false)
            {
                return false;
            }

            HttpRequestBase request = context.Request;

            if (request.IsSecureConnection == true)
            {
                Logger.LogDebug(
                   CultureInfo.InvariantCulture,
                   "SslRequiredWebPart - The connection is secure.",
                   request.RawUrl);

                return false;
            }

            if (request.Url.Host.Contains(".") == false)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "SslRequiredWebPart - The hostname is not a fully-qualified"
                        + " domain name.");

                return false;
            }
            else if (request.Url.Host.Contains("-local.") == true
                || request.Url.Host.Contains("-dev.") == true)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "SslRequiredWebPart - LOCAL or DEV environment detected.");

                return false;
            }

            if (SPContext.Current != null)
            {
                SPControlMode formMode = SPContext.Current.FormContext.FormMode;

                if (formMode == SPControlMode.Edit
                    || formMode == SPControlMode.New)
                {
                    // Never redirect when editing a page                
                    Logger.LogDebug(
                        CultureInfo.InvariantCulture,
                        "SslRequiredWebPart - SSL redirect is not required"
                        + " because the page is being edited.");

                    return false;
                }
            }

            return true;
        }
    }
}

Using this approach, users can browse the anonymous areas of your site using HTTP. However, as soon as they browse to a page that contains a Web Part that requires secure communication, they are automatically redirected from HTTP to HTTPS.

Note how I use a custom application setting (RedirectToHttpsWhenSslRequired) to allow this feature to be turned off. The default value is True (meaning the feature is turned on). However, by changing the property setting to False in the Web.config file, the feature can be disabled as necessary for a particular environment or server (for troubleshooting purposes):

<configuration>
  <configSections>
    <sectionGroup
      name="applicationSettings"
      type="System.Configuration.ApplicationSettingsGroup, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" >
      <section name="Fabrikam.Demo.Web.Properties.Settings"
        type="System.Configuration.ClientSettingsSection, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
        requirePermission="false" />
    </sectionGroup>
  </configSections>
  <applicationSettings>
    <Fabrikam.Demo.Web.Properties.Settings>
      <setting name="RedirectToHttpsWhenSslRequired" serializeAs="String">
        <value>False</value>
      </setting>
    </Fabrikam.Demo.Web.Properties.Settings>
  </applicationSettings>
</configuration>

Also note that there are several scenarios in which a redirect is avoided:

When the connection is already secure (well, duh...we obviously don't want to cause an endless redirect loop)
When the request URL does not specify a fully qualified domain name, but rather an intranet URL (e.g. http://fabrikam). In this scenario, user are expected to be authenticated using Windows Authentication, which does not send credentials in clear text and therefore does not require SSL.
In LOCAL developer environments (e.g. http://www-local.fabrikam.com) and the Development Integration environment (DEV) -- e.g. http://www-dev.fabrikam.com -- since these environments don't typically have SSL certificates installed. This is an example of why a standard environment naming convention is important.
When the page where the Web Part resides is being edited (because we don't want to force a redirect immediately after someone adds the Web Part to a page). This scenario is not expected to occur, since content managers will typically use the intranet URL (e.g. http://fabrikam) for creating and editing pages. However, it is covered just in case the scenario is ever encountered.

Assuming the IsSslRedirectRequired method returns true, the Web Part redirects to an HTTPS connection while preserving all of the query string parameters specified in the original request. Note that this simple approach only supports an HTTP GET on the original request. In other words, HTTP POST requests are "not supported" (however they are not expected to occur either). This is because any form parameters specified in the body of an HTTP POST request would be dropped in the redirect.

Similar code can also be used in a page (e.g. /_layouts/Fabrikam/Login.aspx) if you prefer that approach instead of creating some kind of "Login" Web Part.

Lastly, note that I allow the IsSslRedirectRequired method to be overridden in Web Parts that inherit from SslRequiredWebPart (e.g. LoginFormWebPart). I haven't yet found this to be necessary, but it's there just in case.

Configuring SSL on SharePoint Sites

Jeremy Jameson — Mon, 09 Nov 2009 15:24:00 GMT

If you are using Basic Authentication or Forms-Based Authentication (FBA) with Microsoft Office SharePoint Server (MOSS) 2007 -- or any Web site, for that matter -- then you must configure secure communication (HTTPS) using SSL certificates.

However, you probably didn't select to enable SSL when you originally created your Web application (after all, you most likely don't want to use Basic Auth or FBA exclusively, but rather only for users accessing the site externally). If you followed the process I recommend for customers deploying MOSS solutions, you created the Web application using an intranet URL (e.g. http://fabrikam) and subsequently extended the Web application to the Internet zone (e.g. http://www.fabrikam.com) or Extranet zone (e.g. http://extranet.fabrikam.com) -- utilizing the SharePoint alternate access mappings feature.

Now that you need to enable secure communication, you might once again consider extending the Web application -- say to the Custom zone -- and select the option to use SSL (e.g. https://www.fabrikam.com). After all, this is the recommendation in the following TechNet article:

Update a Web application URL and IIS bindings (Windows SharePoint Services). (Updated 2009-04-23).

http://technet.microsoft.com/en-us/library/cc298636.aspx

Specifically, here is the text that I am referring to:

We do not recommend reusing the same IIS Web site for your HTTP and SSL hosting. Instead, extend a dedicated HTTP and a dedicated SSL Web site, each assigned to its own alternate access mapping zone and URLs.

Wait a minute...let me see if I've got this right...you want me to extend my Web application -- thereby creating yet another copy of my Web.config files -- just to enable SSL? That doesn't seem like a good idea.

After all, managing the multitude of Web.config files across multiple Web application and Web servers in a farm seems to be one of the most challenging aspects for the Release Management and Operations teams (at least in my experience during the past several years). [I can't tell you how many times I've compared Web.config files during the process of troubleshooting various issues and found discrepancies due to manual changes not getting applied consistently. Sure, the risk of this is drastically reduced if you leverage the SPWebConfigModification infrastructure, but there are still rare occasions when a little manual "tweaking" is necessary -- and these often lead to inconsistent Web.config files.]

Therefore, one of my goals with any SharePoint solution is to minimize the number of Web.config files that are used. It seems reasonable to expect two different Web.config files to be used for a site that uses Windows Authentication internally and Forms Authentication externally, since this is configured using the <authentication> element.

For example, in the Default zone you probably want to specify:

    <authentication mode="Windows" />

Whereas in the Internet zone you instead need something like this:

    <authentication mode="Forms">
      <forms loginUrl="/Public/Pages/default.aspx" defaultUrl="/" />
    </authentication>

Okay, so if we want to avoid creating a new Web application -- and the corresponding Web.config file(s) -- how can we support both HTTP and HTTPS on the same Web application? It turns out to be rather simple.

For a starting point, let's assume you've just created your Web application using the default (i.e. intranet) URL -- say, http://fabrikam -- which uses Windows Authentication. This is the site that your content authors will use to manage the site.

The first step is to create an alternate access mapping (AAM).

To configure an alternate access mapping:

On the SharePoint Central Administration home page, click the Application Management tab on the top link bar.
On the Application Management page, in the SharePoint Web Application Management section, click Create or extend Web application.
On the Create or Extend Web Application page, in the Adding a SharePoint Web Application section, click Extend an existing Web application.
On the Extend Web Application to Another IIS Web Site page:
1. In the Web Application section, select the Web application to extend (e.g. http://fabrikam).
2. In the IIS Web Site section, in the Port and Host Header boxes, enter the corresponding values such as 80 and www.fabrikam.com, respectively.
3. In the Security Configuration section, keep the default options (you can configure forms authentication, anonymous access, and SSL later).
4. In the Load Balanced URL section, ensure the default value specified in the URL box is correct (e.g. http://www.fabrikam.com:80) and in Zone dropdown list, select Internet.
5. Click OK.

The next step is to install your SSL certificate on the site. Once you've procured your certificate and installed it through Internet Information Services (IIS) Manager, you then need to add a public URL in SharePoint for HTTPS and add an HTTPS binding in IIS.

To add a public URL to HTTPS:

On the SharePoint Central Administration home page, click the Operations tab on the top link bar.
On the Operations page, in the Global Configuration section, click Alternate access mappings.
On the Alternate Access Mappings page, click Edit Public URLs.
On the Edit Public Zone URLs page:
1. In the Alternate Access Mapping Collection section, select the Web application (e.g. http://fabrikam).
2. In the Public URLs section, copy the URL from the Internet box to the Custom box, and change http:// to https://.
3. Click Save.

To add an HTTPS binding to the site in IIS:

Click Start, point to Administrative Tools, and then click Internet Information Services (IIS) Manager.
In Internet Information Services (IIS) Manager, click the plus sign (+) next to the server name that contains the Web application, and then click the plus sign next to Sites to view the Web applications that have been created.
Click the name of the Web application corresponding to the Internet zone (e.g. SharePoint – www.fabrikam.com80). In the Actions section, under the Edit Site heading, click Bindings….
In the Site Bindings window, click Add.
In the Add Site Binding window:
1. In the Type: dropdown, select https.
2. In the SSL Certificate: dropdown, select the certificate corresponding to the site (e.g. www.fabrikam.com).
3. Click OK.
4. In the Site Bindings window, click Close.

At this point, your SharePoint site supports Windows Authentication both internally (via http://fabrikam) and externally (via http://www.fabrikam.com and https://www.fabrikam.com). The final steps are to enable anonymous access and configure Forms Authentication.

Note that after configuring Forms Authentication, users will not automatically be redirected from HTTP to HTTPS for the login page. To achieve this user experience, I've used a couple of techniques in the past that are based on the same fundmental concept. The first technique utilized code on the login page to automatically detect HTTP connections and redirect to HTTPS. The second technique was essentially the same logic, but rather than utilizing code embedded in the login page, it is encapsulated in a Login Form Web Part. I'll discuss this redirect code in a follow-up post.

AutoEventWireup Issue in MOSS 2007

Jeremy Jameson — Mon, 09 Nov 2009 02:15:00 GMT

I recently promised to finish this blog post that has been sitting in "unpublished" status since June 2008, so here it is...

Have you ever encountered the following error in Microsoft Office SharePoint Server (MOSS) 2007?

An error occurred during the processing of . The attribute 'autoeventwireup' is not allowed in this page.

I just searched for this using Bing and it seems like I'm not the only one who has ever experienced this issue. However, glancing through a few of the top search results, I didn't see any solutions to the error.

The problem occurs when you have a custom master page which includes code and that master page subsequently becomes unghosted. I believe this happens with custom page layouts that are customized as well.

I have to admit that I was completely stumped when I first encountered this error a few years ago while working on the Agilent Technologies project. I eventually tracked down the root cause to be unghosted pages, but we were not using SharePoint Designer to create or customize our master pages, so I couldn't understand why we would occasionally encounter this error.

My speculation is that when the feature/solution containing the custom master page is deactivated, retracted, and deleted (as part of the "DR.DADA" process), SharePoint has some "smarts" within it that essentially equates to:

Hey, this master page (or page layout) is currently in use so removing it could really break the site.
Therefore, I'd better make a copy of it and store it in the database (i.e. unghost it).

Unfortunately, when we subsequently added, deployed, and activated the solution/feature, SharePoint would still attempt to use the unghosted master page and summarily generate the error stating that "the attribute 'autoeventwireup' is not allowed in this page."

Note that this is pure speculation on my part as to what was causing the master page to become unghosted.

However, what I do know for sure is that once I reghosted the master page, the AutoEventWireup error would magically disappear.

Here are the steps to reghost a master page or page layout:

Browse to Site Settings page for your site. Note that if your master page is causing the AutoEventWireup error, you can explicitly specify the URL (e.g. http://fabrikam/_layouts/settings.aspx).
On the Site Settings page, under the Look and Feel section, click Reset to site definition.
On the Reset Page to Site Definition Version page:
1. In the Reset to Site Definition section, ensure the option to Reset specific page to site definition version is selected, and then in the Local URL of the page box, typtype the URL of the master page or page layout that you want reset (e.g. http://fabrikam/_catalogs/masterpage/FabrikamMinimal.master).
2. Click Reset.
3. In the confirmation dialog that appears stating that you will lose all customizations, including web part zones, custom controls, and in-line text, click OK.

Note that in ASP.NET, the default value for the AutoEventWireup attribute is true. Therefore you might assume that you could simply remove the attribute from your custom master page in order to avoid the error when the master page is unghosted. After all, the error clearly states that the AutoEventWireup attribute is not allowed in this page, right?

In other words, the solution to the problem would seem to be simply be a matter of changing something like this...

<%@ Master Language="C#" AutoEventWireup="true" Codebehind="FabrikamMinimal.master.cs"
    Inherits="Fabrikam.Demo.Publishing.Layouts.MasterPages.FabrikamMinimal" %>

...to this:

<%@ Master Language="C#" Codebehind="FabrikamMinimal.master.cs"
    Inherits="Fabrikam.Demo.Publishing.Layouts.MasterPages.FabrikamMinimal" %>

Unfortunately -- at least in my experience -- this doesn't work. It only leads to other errors, such as:

The event handler 'OnPreRender' is not allowed in this page.

The above error occurs when the master page contains something like the following:

            <asp:SiteMapPath ID="BreadcrumbSiteMapPath" Runat="server"
                SiteMapProvider="CurrentNavSiteMapProviderNoEncode"
                RenderCurrentNodeAsLink="true"
                SkipLinkText=""
                OnPreRender="BreadcrumbSiteMapPath_OnPreRender">

I attempted to resolve this by converting the BreadcrumbSiteMapPath_OnPreRender event handler to a method and invoking the method from the Page_PreRender event handler instead. However, that only led to yet another error:

Code blocks are not allowed in this file.

Sensing a very deep "rat hole" at this point, I decided it wasn't worth pursuing this issue any further.

Fortunately, as I've stated before, I don't believe master pages and page layouts deployed through solutions and features should subsequently be customized through SharePoint Designer. In my opinion, these items should be tightly managed through your SCM (software configuration management) process -- in other words, versioned in your source control system and subsequently deployed through a formal change process.

Of course, if your custom master pages and page layouts are very simple (i.e. no code-behind) then you probably will never encounter this problem.

Supportability Concerns with Custom HTTP Modules in SharePoint

Jeremy Jameson — Sat, 07 Nov 2009 13:34:00 GMT

I am sure that I'm not the first one to tell you this, but you can't believe everything you read on the Internet these days ;-)

Case in point...I've seen a number of sources claim that custom HTTP modules are not supported in Microsoft Office SharePoint Server (MOSS) 2007 and Windows SharePoint Services (WSS) v3.

These statements make it sound like if you add your own custom processing to the pipeline of each page request, you will instantly teleport your Production SharePoint farm into that vast wasteland we generally refer to as "Unsupported" -- where your support calls to Microsoft will be summarily terminated shortly after getting your Service Request (SRX) number.

Not so -- at least in this particular case.

Thank goodness, at least for my sake, because I've worked on a number of MCS (Microsoft Consulting Services) projects where we've delivered custom HTTP modules. It's always a little scary that something we deliver as part of an MCS engagement will later be deemed "unsupported."

In general, we are very good about knowing what is -- and is not -- supported, but there's always a chance that something "creative" we do in order to satisfy the business requirements for our customers will later be considered a problem by Microsoft Customer Service and Support (CSS).

There are, in fact, lots of things you can do with SharePoint that will thrust you into the dreaded "unsupported" category, but a custom HTTP module isn't one of them.

How can I say this with such conviction? Well, yesterday I came across the following on MSDN:

Support Details

A HTTP module assembly can be installed in the Web application’s \bin directory or in the global assembly cache.

Because an HTTP module is always called as part of the page processing pipeline, a poorly designed or faulty module can have a detrimental effect on performance or perceived stability of the environment. Thoroughly test each module for performance before deploying it.

If you want to see it for yourself, you can go straight to the source:

HTTP Module

http://msdn.microsoft.com/en-us/library/bb862635.aspx

Obviously you want to be very cautious about using custom HTTP modules, but there are definitely scenarios where this is a good approach.

Introducing the SharePointSmtpHelper Class

Jeremy Jameson — Sun, 18 Oct 2009 14:29:00 GMT

Continuing in the spirit of my previous posts for the SharePointPublishingHelper class and SharePointWebPartHelper class, I'd like to introduce another helper class that you may find useful when building solutions for Windows SharePoint Services (WSS) v3 and Microsoft Office SharePoint Server (MOSS) 2007.

Consider the scenario where you need to send an e-mail message from a SharePoint application. Perhaps an unexpected error occurred and you want to notify the operations team, or perhaps you just want to send a confirmation message to a user who has just submitted some important information.

Whatever the case, you'd like to send an e-mail message using the SMTP server configured in SharePoint Central Administration for the farm. SharePointSmtpHelper makes it really easy to do this.

Note that there's really not much code to the SharePointSmtpHelper class -- thanks to the classes provided by the .NET Framework in the System.Net.Mail namespace. It may look like more code than you might expect, but that's just because of all the overloads of the SendMessage method that I provide.

using System;
using System.Net.Mail;

using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;

namespace Fabrikam.Demo.CoreServices.SharePoint
{
    /// <summary>
    /// Exposes static methods for sending e-mail messages from a SharePoint
    /// Web application using the Simple Mail Transfer Protocol (SMTP). The
    /// SMTP server must be configured in SharePoint Central Administration.
    /// This class cannot be inherited.
    /// </summary>
    /// <remarks>
    /// All methods of the <b>SharePointSmtpHelper</b> class are static and
    /// can therefore be called without creating an instance of the class.
    /// </remarks>
    public sealed class SharePointSmtpHelper
    {
        private static object lockObject = new object();

        private static string smtpServer;
        private static string defaultFromAddress;
        
        private SharePointSmtpHelper( ) { } // all members are static

        /// <summary>
        /// Sends an e-mail message using the outbound SMTP server and
        /// "From address" configured in SharePoint Central Administration.
        /// </summary>
        /// <param name="toRecipients">The recipients of the e-mail message.
        /// In other words, the addresses on the To line of the e-mail.</param>
        /// <param name="subject">The subject line for the e-mail message.
        /// </param>
        /// <param name="body">The e-mail message body.</param>
        public static void SendMessage(
            string toRecipients,
            string subject,
            string body)
        {
            SendMessage(
                toRecipients,
                subject,
                body,
                false,
                null,
                null);
        }

        /// <summary>
        /// Sends an e-mail message using the outbound SMTP server and
        /// "From address" configured in SharePoint Central Administration,
        /// specifying whether the body of the message is formatted using HTML.
        /// </summary>
        /// <param name="toRecipients">The recipients of the e-mail message.
        /// In other words, the addresses on the To line of the e-mail.</param>
        /// <param name="subject">The subject line for the e-mail message.
        /// </param>
        /// <param name="body">The e-mail message body.</param>
        /// <param name="isBodyHtml"><c>true</c> if the mail message body is
        /// formatted using HTML markup. <c>false</c> if the message is simply
        /// plain text.</param>
        public static void SendMessage(
            string toRecipients,
            string subject,
            string body,
            bool isBodyHtml)
        {
            SendMessage(
                toRecipients,
                subject,
                body,
                isBodyHtml,
                null,
                null);
        }

        /// <summary>
        /// Sends an e-mail message using the outbound SMTP server and
        /// "From address" configured in SharePoint Central Administration,
        /// specifying whether the body of the message is formatted using HTML
        /// and also including carbon copy (CC) recipients.
        /// </summary>
        /// <param name="toRecipients">The recipients of the e-mail message.
        /// In other words, the addresses on the To line of the e-mail.</param>
        /// <param name="subject">The subject line for the e-mail message.
        /// </param>
        /// <param name="body">The e-mail message body.</param>        
        /// <param name="isBodyHtml"><c>true</c> if the mail message body is
        /// formatted using HTML markup. <c>false</c> if the message is simply
        /// plain text.</param>
        /// <param name="ccRecipients">The carbon copy (CC) recipients of the
        /// e-mail message.</param>
        public static void SendMessage(
            string toRecipients,
            string subject,
            string body,
            bool isBodyHtml,
            string ccRecipients)
        {
            SendMessage(
                toRecipients,
                subject,
                body,
                isBodyHtml,
                ccRecipients,
                null);
        }

        /// <summary>
        /// Sends an e-mail message using the outbound SMTP server configured
        /// in SharePoint Central Administration for the farm.
        /// </summary>
        /// <param name="toRecipients">The recipients of the e-mail message.
        /// In other words, the addresses on the To line of the e-mail.</param>
        /// <param name="subject">The subject line for the e-mail message.
        /// </param>
        /// <param name="body">The e-mail message body.</param>        
        /// <param name="isBodyHtml"><c>true</c> if the mail message body is
        /// formatted using HTML markup. <c>false</c> if the message is simply
        /// plain text.</param>
        /// <param name="ccRecipients">The carbon copy (CC) recipients of the
        /// e-mail message.</param>
        /// <param name="from">The e-mail address of the sender.</param>
        public static void SendMessage(
            string toRecipients,
            string subject,
            string body,
            bool isBodyHtml,
            string ccRecipients,
            string from)
        {
            if (string.IsNullOrEmpty(toRecipients) == true)
            {
                throw new ArgumentNullException("toRecipients");
            }

            toRecipients = toRecipients.Trim();
            if (string.IsNullOrEmpty(toRecipients) == true)
            {
                throw new ArgumentException(
                    "In order to send an e-mail message, "
                        + " at least one recipient must be specified.",
                    "toRecipients");
            }

            if (string.IsNullOrEmpty(subject) == true)
            {
                throw new ArgumentNullException("subject");
            }

            subject = subject.Trim();
            if (string.IsNullOrEmpty(subject) == true)
            {
                throw new ArgumentException(
                    "In order to send an e-mail message, "
                        + " the subject must be specified.",
                    "subject");
            }

            if (string.IsNullOrEmpty(body) == true)
            {
                throw new ArgumentNullException("body");
            }

            body = body.Trim();
            if (string.IsNullOrEmpty(body) == true)
            {
                throw new ArgumentException(
                    "In order to send an e-mail message, "
                        + " the message body must be specified.",
                    "body");
            }

            EnsureSmtpSettings();

            // If the "from" address is not specified, default to the address
            // configured in SharePoint Central Administration.
            if (string.IsNullOrEmpty(from) == true)
            {
                from = defaultFromAddress;
            }

            SmtpClient smtpClient = new SmtpClient(smtpServer);

            MailMessage message = new MailMessage(
                from,
                toRecipients,
                subject,
                body);

            message.IsBodyHtml = isBodyHtml;

            if (string.IsNullOrEmpty(ccRecipients) == false)
            {
                message.CC.Add(ccRecipients);
            }

            smtpClient.Send(message);
        }

        private static void EnsureSmtpSettings()
        {
            if (string.IsNullOrEmpty(smtpServer) == true)
            {
                if (SPContext.Current == null)
                {
                    throw new InvalidOperationException(
                        "SharePointSmtpClient can only be called from within"
                            + " a SharePoint site (SPContext.Current is null).");
                }

                lock (lockObject)
                {
                    if (string.IsNullOrEmpty(smtpServer) == true)
                    {
                        SPWebApplication webApp =
                            SPContext.Current.Site.WebApplication;

                        if (webApp.OutboundMailServiceInstance != null
                            && webApp.OutboundMailServiceInstance.Server != null)
                        {
                            smtpServer =
                                webApp.OutboundMailServiceInstance.Server.Address;
                        }

                        if (string.IsNullOrEmpty(smtpServer) == true)                            
                        {
                            throw new InvalidOperationException(
                                "The outbound SMTP server is not specified in"
                                + " SharePoint Central Administration.");
                        }

                        defaultFromAddress = webApp.OutboundMailSenderAddress;
                    }
                }
            }
        }
    }
}

Note that I use static members in order to only read the SMTP server configuration from SharePoint Central Administration one time.

Assuming you just want to send a simple message using the default "From address" specified in SharePoint Central Administration, you only need a single line of code that passes three parameters:

            SharePointSmtpHelper.SendMessage(
                "jeremy_jameson@fabrikam.com",
                "Test message",
                "Testing the SharePointSmtpHelper class");

Introducing the SharePointWebPartHelper Class

Jeremy Jameson — Sat, 17 Oct 2009 12:54:00 GMT

In a previous post, I introduced the DR.DADA approach to SharePoint development and how I typically use the concept of a FeatureConfigurator to automatically configure one or more aspects of a SharePoint site when activating my feature.

For example, on my latest project, we needed a login page as well as a legal disclaimer page for a customer service portal based on Microsoft Office SharePoint Server (MOSS) 2007. Thus I created a "PublicSiteConfiguration" feature that, upon activation:

Automatically creates the /Public site
Populates the page content on the /Public/Pages/default.aspx page and adds an instance of a custom Login Form Web Part to the page
Creates the /Public/Pages/Disclaimer.aspx page and populates its default content

I've already introduced the SharePointPublishingHelper class that performs the bulk of the work in creating and configuring publishing pages. In this post, I want to cover the SharePointWebPartHelper class. Like the SharePointPublishingHelper class, the SharePointWebPartHelper class is simply a collection of helper methods for the SharePoint API.

SharePointWebPartHelper makes it really easy to:

Attempt to retrieve a reference to a Web Part by specifying an SPLimitedWebPartManager for a specific page and the title of the Web Part you are trying to find
Ensure a Web Part exists on a page (creating a new Web Part as necessary)
Ensuring a ContentEditorWebPart exists on a page and that it is configured with the specified HTML content and title (as well as removing the "chrome", since -- at least in my experience -- you don't want to show this to users for this type of Web Part)

Note that one could argue that the stuff specific to the Content Editor Web Part doesn't really belong in this class, but nevertheless that's where it ended up after refactoring the code a few years ago on the Agilent Technologies project. [Actually, I believe it originally ended up in an even more generic SharePointHelper class, but I've since refactored it even further into the SharePointWebPartHelper class.]

Anyway, here's the code for the SharePointWebPartHelper class:

using System;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Security.Permissions;
using System.Web.UI.WebControls.WebParts;
using System.Xml;

using Microsoft.SharePoint;
using Microsoft.SharePoint.Security;
using SPWebPartPages = Microsoft.SharePoint.WebPartPages;

using Fabrikam.Demo.CoreServices.Logging;

namespace Fabrikam.Demo.CoreServices.SharePoint
{
    /// <summary>
    /// Exposes static methods for commonly used helper functions for
    /// SharePoint Web Parts. This class cannot be inherited.
    /// </summary>
    /// <remarks>
    /// All methods of the <b>SharePointWebPartHelper</b> class are static and
    /// can therefore be called without creating an instance of the class.
    /// </remarks>
    [CLSCompliant(false)]
    [SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true)]
    public sealed class SharePointWebPartHelper
    {
        private SharePointWebPartHelper() { } // all members are static

        private static WebPart CreateWebPart(
            SPWebPartPages.SPLimitedWebPartManager wpm,
            string webPartId,
            string webPartFilename,
            string zoneId,
            ref int zoneIndex)
        {
            Debug.Assert(wpm != null);
            Debug.Assert(string.IsNullOrEmpty(webPartId) == false);
            Debug.Assert(string.IsNullOrEmpty(webPartFilename) == false);
            Debug.Assert(string.IsNullOrEmpty(zoneId) == false);

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Creating Web Part ({0}) on page ({1})...",
                webPartId,
                wpm.ServerRelativeUrl);

            WebPart webPart = null;

            SPFile wpFile = GetWebPartFileFromGallery(
                wpm.Web.Site,
                webPartFilename);

            Debug.Assert(wpFile != null);

            string errorMessage = string.Empty;

            using (Stream stream = wpFile.OpenBinaryStream())
            {
                XmlReader reader = XmlReader.Create(stream);
                webPart = wpm.ImportWebPart(reader, out errorMessage);
            }

            if (string.IsNullOrEmpty(errorMessage) == false)
            {
                string message =
                    "Error importing Web Part ("
                        + webPartFilename + "): "
                        + errorMessage;

                throw new SPWebPartPages.WebPartPageUserException(message);
            }

            webPart.ID = webPartId;
            wpm.AddWebPart(webPart, zoneId, zoneIndex);

            // HACK: When adding the first, second, third, and fourth Web Parts
            // to a zone:
            //
            //    SPLimitedWebPartManager.AddWebPart(webPart, zoneId, zoneIndex)
            //
            // behaves as expected, meaning that
            // (webPart.ZoneIndex == zoneIndex).
            //
            // However, when adding the fifth and sixth Web Parts to a zone,
            // then (webPart.ZoneIndex == zoneIndex + 1).
            if (webPart.ZoneIndex > zoneIndex)
            {
                Debug.Assert(zoneIndex >= 4);

                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "HACK: webPart.ZoneIndex ({0})"
                        + " is greater than zoneIndex ({1})."
                        + " Setting zoneIndex to {0}...",
                    webPart.ZoneIndex,
                    zoneIndex);

                zoneIndex = webPart.ZoneIndex;
            }

            Debug.Assert(webPart.ZoneIndex == zoneIndex);
            zoneIndex++;
            
            Logger.LogInfo(
                CultureInfo.InvariantCulture,
                "Successfully created Web Part ({0}) on page ({1}).",
                webPartId,
                wpm.ServerRelativeUrl);

            return webPart;
        }

        /// <summary>
        /// Ensures that a specific Web Part exists on a page.
        /// </summary>
        /// <param name="wpm">The Web Part manager for the page.</param>
        /// <param name="webPartId">The unique identifier for the Web Part.</param>
        /// <param name="webPartFileName">The .dwp or .webpart file in the
        /// Web Part gallery (used to import the Web Part if it does not exist).
        /// </param>
        /// <param name="zoneId">The zone the Web Part should reside in.</param>
        /// <param name="zoneIndex">The index of the Web Part relative to other
        /// Web Parts in the zone. If the Web Part is created, the index is
        /// incremented.</param>
        /// <returns>The new or existing Web Part on the page.</returns>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1045:DoNotPassTypesByReference",
            Justification = "The zoneIndex is incremented when the Web Part is"
                + " added.")]
        public static WebPart EnsureWebPart(
            SPWebPartPages.SPLimitedWebPartManager wpm,
            string webPartId,
            string webPartFileName,
            string zoneId,
            ref int zoneIndex)
        {
            if (wpm == null)
            {
                throw new ArgumentNullException("wpm");
            }

            if (string.IsNullOrEmpty(webPartId) == true)
            {
                throw new ArgumentException(
                    "The Web Part Id must be specified.",
                    "webPartId");
            }

            if (string.IsNullOrEmpty(webPartFileName) == true)
            {
                throw new ArgumentException(
                    "The Web Part file name must be specified.",
                    "webPartFileName");
            }

            if (string.IsNullOrEmpty(zoneId) == true)
            {
                throw new ArgumentException(
                    "The Web Part zone must be specified.",
                    "zoneId");
            }

            WebPart webPart = wpm.WebParts[webPartId];

            if (webPart == null)
            {
                webPart = CreateWebPart(wpm, webPartId, webPartFileName,
                    zoneId, ref zoneIndex);
            }
            else
            {
                // HACK: webPart.Zone appears to always be null (SharePoint bug?)
                if (webPart.Zone == null)
                {
                    Logger.LogDebug(
                        CultureInfo.InvariantCulture,
                        "Unable to check zone for Web Part ({0})"
                            + " because webPart.Zone is null.",
                        webPartId);
                }
                else if (webPart.Zone.ID != zoneId)
                {
                    Logger.LogDebug(
                        CultureInfo.InvariantCulture,
                        "Moving Web Part ({0}) to zone ({1})...",
                        webPartId,
                        zoneId);

                    wpm.MoveWebPart(webPart, zoneId, zoneIndex);
                    wpm.SaveChanges(webPart);

                    Logger.LogInfo(
                        CultureInfo.InvariantCulture,
                        "Successfully moved Web Part ({0}) to zone ({1}).",
                        webPartId,
                        zoneId);
                }
            }

            return webPart;
        }

        /// <summary>
        /// Finds the first Web Part with the specified title.
        /// </summary>
        /// <remarks>
        /// This is useful in cases where the Web Part ID is not specified
        /// when it was added to the collection of Web Parts (thus generating
        /// a random identifier and making SPLimitedWebPartCollection[id]
        /// effectively useless).
        /// </remarks>
        /// <param name="wpm">The Web Part manager for the page.</param>
        /// <param name="webPartTitle">The title of the Web Part to find.</param>
        /// <returns>A Web Part object (if one is found), otherwise null.</returns>
        public static WebPart FindWebPartByTitle(
            SPWebPartPages.SPLimitedWebPartManager wpm,
            string webPartTitle)
        {
            if (wpm == null)
            {
                throw new ArgumentNullException("wpm");
            }

            if (string.IsNullOrEmpty(webPartTitle) == true)
            {
                throw new ArgumentException(
                    "The Web Part title must be specified.",
                    "webPartTitle");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Attemping to find Web Part ({0}) on page ({1})...",
                webPartTitle,
                wpm.ServerRelativeUrl);

            for (int i = 0; i < wpm.WebParts.Count; i++)
            {
                WebPart webPart = wpm.WebParts[i];

                if (string.Compare(
                        webPart.Title,
                        webPartTitle,
                        StringComparison.OrdinalIgnoreCase) == 0)
                {
                    Logger.LogDebug(
                        CultureInfo.InvariantCulture,
                        "Found Web Part ({0}) on page ({1}).",
                        webPartTitle,
                        wpm.ServerRelativeUrl);

                    return webPart;
                }
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Unable to find Web Part ({0}) on page ({1}).",
                webPartTitle,
                wpm.ServerRelativeUrl);

            return null;
        }

        private static SPFile GetWebPartFileFromGallery(
            SPSite site,
            string webPartFilename)
        {
            if (site == null)
            {
                throw new ArgumentNullException("site");
            }

            if (webPartFilename == null)
            {
                throw new ArgumentNullException("webPartFilename");
            }

            webPartFilename = webPartFilename.Trim();
            if (string.IsNullOrEmpty(webPartFilename))
            {
                throw new ArgumentException(
                    "The Web Part filename must be specified.",
                    "webPartFilename");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Getting Web Part file ({0}) from site ({1}) gallery...",
                webPartFilename,
                site.Url);

            SPList wpGallery = site.GetCatalog(SPListTemplateType.WebPartCatalog);

            string camlQuery = "<Where><Eq><FieldRef Name=\"FileLeafRef\" />"
                + "<Value Type=\"File\">" + webPartFilename + "</Value></Eq></Where>";

            SPQuery query = new SPQuery();
            query.Query = camlQuery;

            SPListItemCollection webParts = wpGallery.GetItems(query);
            if (webParts.Count == 0)
            {
                string message = string.Format(
                    CultureInfo.InvariantCulture,
                    "Web Part ({0}) was not found in the Web Part Gallery.",
                    webPartFilename);
                
                throw new ArgumentException(
                    message,
                    "webPartFilename");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Found Web Part file ({0}) in site ({1}) gallery.",
                webPartFilename,
                site.Url);

            Debug.Assert(webParts.Count == 1);
            return webParts[0].File;
        }

        /// <summary>
        /// Ensures that a Content Editor Web Part exists on a page
        /// and is configured with the specified settings.
        /// </summary>
        /// <param name="wpm">The Web Part manager for the page.</param>
        /// <param name="webPartId">The unique identifier for the Web Part.</param>
        /// <param name="zoneId">The zone the Web Part should reside in.</param>
        /// <param name="zoneIndex">The index of the Web Part relative to other
        /// Web Parts in the zone. If the Web Part is created, the index is
        /// incremented.</param>
        /// <param name="htmlContent">The content to display within the Web
        /// Part.</param>
        /// <param name="title">The title of the Web Part.</param>
        /// <returns>The new or existing Web Part on the page.</returns>        
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1045:DoNotPassTypesByReference",
            Justification="The zoneIndex is incremented when the Web Part is"
                + " added.")]
        public static SPWebPartPages.ContentEditorWebPart
            EnsureContentEditorWebPart(
                SPWebPartPages.SPLimitedWebPartManager wpm,
                string webPartId,
                string zoneId,
                ref int zoneIndex,
                string htmlContent,
                string title)
        {
            if (wpm == null)
            {
                throw new ArgumentNullException("wpm");
            }

            if (webPartId == null)
            {
                throw new ArgumentNullException("webPartId");
            }

            webPartId = webPartId.Trim();
            if (string.IsNullOrEmpty(webPartId))
            {
                throw new ArgumentException(
                    "The Web Part ID must be specified.",
                    "webPartId");
            }

            if (zoneId == null)
            {
                throw new ArgumentNullException("zoneId");
            }

            zoneId = zoneId.Trim();
            if (string.IsNullOrEmpty(zoneId))
            {
                throw new ArgumentException(
                    "The Web Part zone must be specified.",
                    "zoneId");
            }

            if (htmlContent == null)
            {
                throw new ArgumentNullException("htmlContent");
            }

            htmlContent = htmlContent.Trim();
            if (string.IsNullOrEmpty(htmlContent))
            {
                throw new ArgumentException(
                    "The HTML content must be specified.",
                    "htmlContent");
            }

            // Note: title is optional

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Ensuring Content Editor Web Part ({0}) on page ({1})...",
                webPartId,
                wpm.ServerRelativeUrl);

            SPWebPartPages.ContentEditorWebPart webPart =
                (SPWebPartPages.ContentEditorWebPart) EnsureWebPart(
                    wpm,
                    webPartId,
                    "MSContentEditor.dwp",
                    zoneId,
                    ref zoneIndex);

            if (webPart.Content == null
                || webPart.Content.InnerText != htmlContent
                || webPart.Title != title
                || webPart.ChromeType != PartChromeType.None)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Updating properties of Content Editor Web Part ({0}) on"
                        + " page ({1})...",
                    webPartId,
                    wpm.ServerRelativeUrl);

                // To bring content into the Content Editor Web Part,
                // we need to use an XML Document
                XmlDocument xmlDoc = new XmlDocument();
                XmlElement xmlElement = xmlDoc.CreateElement("HtmlContent");
                xmlElement.InnerText = htmlContent;
                webPart.Content = xmlElement;

                webPart.Title = title;
                webPart.ChromeType = PartChromeType.None;

                wpm.SaveChanges(webPart);

                Logger.LogInfo(
                    CultureInfo.InvariantCulture,
                    "Successfully updated properties of Content Editor Web Part"
                        + " ({0}) on page ({1}).",
                    webPartId,
                    wpm.ServerRelativeUrl);
            }
            else
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "The Content Editor Web Part ({0}) on page ({1}) is already"
                        + " configured with the expected settings.",
                    webPartId,
                    wpm.ServerRelativeUrl);
            }

            return webPart;
        }
    }
}

Most of the code in the SharePointWebPartHelper class is very straightforward. The most interesting hack -- er, I mean workaround -- in the code is to handle the scenario where the zoneIndex is sometimes mysteriously incremented by more than one, but you can read more about this in a previous post -- if you are really interested.

Using the SharePointWebPartHelper class couldn't be easier. For a simple example, consider the scenario that I mentioned earlier about adding a Login Form Web Part to a page:

            // Configure Web Parts
            SPWebPartPages.SPLimitedWebPartManager wpm =
                publicWeb.GetLimitedWebPartManager(
                    page.Url,
                    PersonalizationScope.Shared);

            using (wpm)
            {
                string zoneId = "LeftColumnZone";
                int zoneIndex = 0;

                LoginFormWebPart loginForm =
                    (LoginFormWebPart) SharePointWebPartHelper.EnsureWebPart(
                        wpm,
                        "LoginForm",
                        "Fabrikam_LoginForm.webpart",
                        zoneId,
                        ref zoneIndex);

                loginForm.DisplayRememberMe = false;
                wpm.SaveChanges(loginForm);

                // HACK: Avoid memory leak in SPLimitedWebPartManager
                if (wpm.Web != null)
                {
                    wpm.Web.Dispose();
                }
            }

Just be sure to avoid the memory leak in SPLimitedWebPartManager, like I show in the code example above.

Here's another code example where I configured a bunch of Web Parts on a search results page:

namespace Fabrikam.Web.Search.Configuration
{
    /// <summary>
    /// Exposes static methods for configuring the <b>Fabrikam.Web.Search</b>
    /// feature. This class cannot be inherited.
    /// </summary>
    /// <remarks>
    /// All methods of the <b>FeatureConfigurator</b> class are static and can
    /// therefore be called without creating an instance of the class.
    /// </remarks>  
    [CLSCompliant(false)]
    [SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true)]
    public sealed class FeatureConfigurator
    {
        ...

        private static void ConfigureSearchResultsPageWebParts(
            SPWeb searchWeb,
            PublishingPage searchResultsPage,
            string defaultSearchResultsPageUrl,
            string scopeName,
            string additionalQueryTerms,
            string contextualScopeUrl)
        {
            SPWebPartPages.SPLimitedWebPartManager wpm =
                searchWeb.GetLimitedWebPartManager(
                    searchResultsPage.Url, PersonalizationScope.Shared);

            using (wpm)
            {
                #region Zone1

                string zoneId = "Zone1";
                int zoneIndex = 0;

                SharePointWebPartHelper.EnsureContentEditorWebPart(
                    wpm,
                    "startNewSearchHeading",
                    zoneId,
                    ref zoneIndex,
                    "<h2 class='accent3'>Start a new search</h2>",
                    "Start New Search Heading");

                SearchBoxEx searchBox =
                    (SearchBoxEx)SharePointWebPartHelper.EnsureWebPart(
                        wpm,
                        "searchBox",
                        "SearchBox.dwp",
                        zoneId,
                        ref zoneIndex);

                searchBox.DropDownModeEx = DropDownModesEx.HideScopeDD;
                searchBox.SearchResultPageURL = defaultSearchResultsPageUrl;
                searchBox.Width = string.Empty;

                wpm.SaveChanges(searchBox);

                SharePointWebPartHelper.EnsureContentEditorWebPart(
                    wpm,
                    "refineSearchHeading",
                    zoneId,
                    ref zoneIndex,
                    "<h2 class='accent3'>Refine your search</h2>",
                    "Refine Search Heading");

                FabrikamSearchBoxWebPart searchScopeOptions =
                    (FabrikamSearchBoxWebPart)SharePointWebPartHelper.EnsureWebPart(
                        wpm,
                        "searchScopeOptions",
                        "FabrikamSearchBox.webpart",
                        zoneId,
                        ref zoneIndex);

                searchScopeOptions.ShowSearchBox = false;
                wpm.SaveChanges(searchScopeOptions);

                ConfigureFacetedSearchWebParts(
                    wpm,
                    zoneId,
                    ref zoneIndex,
                    scopeName);

                #endregion
                
                #region Zone2

                zoneId = "Zone2";
                zoneIndex = 0;

                SharePointWebPartHelper.EnsureWebPart(
                    wpm,
                    "searchTabs",
                    "FabrikamListBoundTabStrip.webpart",
                    zoneId,
                    ref zoneIndex);

                SharePointWebPartHelper.EnsureWebPart(
                    wpm,
                    "searchSummary",
                    "SearchSummary.dwp",
                    zoneId,
                    ref zoneIndex);

                bool showBestBetsWebPart = false;

                if (string.Compare(
                    searchResultsPage.Name,
                    "Results.aspx",
                    StringComparison.OrdinalIgnoreCase) == 0)
                {
                    showBestBetsWebPart = true;
                }
                else if (string.Compare(
                    searchResultsPage.Name,
                    "AllSitesResults.aspx",
                    StringComparison.OrdinalIgnoreCase) == 0)
                {
                    showBestBetsWebPart = true;
                }

                if (showBestBetsWebPart == true)
                {
                    ConfigureBestBetsWebPartForSiteSearch(
                        wpm,
                        zoneId,
                        ref zoneIndex);
                }

                SearchStatsWebPart searchStats =
                    (SearchStatsWebPart) SharePointWebPartHelper.EnsureWebPart(
                        wpm,
                        "searchStats",
                        "searchStats.dwp",
                        zoneId,
                        ref zoneIndex);

                searchStats.DisplayResponseTime = false;
                wpm.SaveChanges(searchStats);
                
                SharePointWebPartHelper.EnsureWebPart(
                    wpm,
                    "searchPaging1",
                    "SearchPaging.dwp",
                    zoneId,
                    ref zoneIndex);

                ConfigureCoreResultsWebPartsForSiteSearch(
                    wpm,
                    zoneId,
                    ref zoneIndex,
                    scopeName);

                SharePointWebPartHelper.EnsureWebPart(
                    wpm,
                    "searchPaging2",
                    "SearchPaging.dwp",
                    zoneId,
                    ref zoneIndex);

                if (additionalQueryTerms != null
                    || contextualScopeUrl != null)
                {
                    ConfigureCoreResultsExtensionWebPart(
                        wpm,
                        additionalQueryTerms,
                        contextualScopeUrl,
                        zoneId,
                        ref zoneIndex);
                }

                #endregion
                
                // HACK: Avoid memory leak in SPLimitedWebPartManager
                if (wpm.Web != null)
                {
                    wpm.Web.Dispose();
                }
            }
        }
    }
}

Even though I didn't add any comments to this (rather lengthy) method, I think you'll find it is actually very easy to understand simply by reading the method names and parameters -- or at least that's what I hope for the people that are actually maintaining this code from my previous project ;-)

I hope you find the SharePointPublishingHelper and SharePointWebPartHelper classes as useful as I have over the past several years.

Be sure to stay tuned, because there are more SharePoint helper classes to come, but my daughter just woke up and now I need to go make some pancakes!

Event ID 10016, KB 920783, and the WSS_WPG Group

Jeremy Jameson — Sat, 17 Oct 2009 11:33:00 GMT

If you've ever deployed Windows SharePoint Services (WSS) v3 or Microsoft Office SharePoint Server (MOSS) 2007 in a least privilege configuration, you have undoubtedly encountered errors similar to the following in your Windows event log:

The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID {61738644-F196-11D0-9953-00C04FD919C1} to the user TECHTOOLBOX\svc-sharepoint-dev SID (S-1-5-21-3914637029-2275272621-3670275343-1145) from address LocalHost (Using LRPC). This security permission can be modified using the Component Services administrative tool.

Specifically, I am referring to Event ID 10016 and 10017.

While these errors are apparently benign, it is still rather annoying to have your event logs flooded with this kind of "noise." [Personally, I very much like to see servers come up "clean" after a reboot (meaning no new errors or warnings in the event log). This is especially true when you use something like Systems Center Operations Manager (SCOM) to monitor your servers, and one or more members of the Operations team receives notification whenever an error occurs on a Production server.]

There are lots of sources out on the Internet that provide the steps necessary to resolve these errors (essentially you need to grant the SharePoint service account Local Activation permission to the IIS WAMREG Admin Service).

This week, I was glad to discover that Microsoft has a KnowledgeBase article (KB 920783) describing the errors and the associated workaround. [This KB article appears to have been published over two years ago, but honestly, I was completely unaware of it until just yesterday when I was building out the Production MOSS 2007 environment on my latest project.]

However, I see two problems with KB 920783:

First, it doesn't mention Microsoft Office SharePoint Server (MOSS) 2007 -- which might explain why I never came across this KB before when searching the Internet. Sure, we all know that MOSS 2007 is built on top of WSS v3 and thus KB articles that apply to "Microsoft Windows SharePoint Services 3.0" almost always apply to "Microsoft Office SharePoint Server 2007", but if you don't specifically state that, you shouldn't expect Bing or Google to return the KB article when searching for something like "MOSS 10016" or "Microsoft Office SharePoint Server 10016". Right?
Second, and much more important, the KB article instructs you to:

"type the domain user account that you specified as the Windows SharePoint Services 3.0 service account"

I definitely don't recommend doing that -- unless you just like making more work for yourself than necessary. Instead, you should specify the local WSS_WPG group rather than a specific user. This is what I've been doing for the last couple of years so trust me, it works.

The reason why I recommend using the "Windows SharePoint Services Worker Process Group" instead of individual users is that you are likely (or at least hopefully) using different service accounts for your SharePoint farm (e.g. TECHTOOLBOX\svc-sharepoint) and for the application pools behind your Web applications (e.g. TECHTOOLBOX\svc-web-fabrikam). Furthermore, if you are doing a large enterprise deployment of SharePoint, you likely have multiple Web applications (e.g. SSP, My Sites, etc.) that each utilize a separate service account.

Rather than granting the Local Activation permission to each of these service accounts individually, it is much more effective to just grant the permission to the WSS_WPG group instead.

Note

When creating a new Web application, SharePoint automatically adds the corresponding service account to the local WSS_WPG group on each SharePoint server in your farm.

[Update (2009-10-29):

Matt McEvoy contacted me last Monday regarding the fact that I didn't specify the WSS_ADMIN_WPG group -- only the WSS_WPG group.

Ugh...that will teach me to try to recall something like this from memory. When I was writing this blog post, I mistakenly thought that the service account for the SharePoint farm was added to both WSS_ADMIN_WPG and WSS_WPG. However, this isn't the case.

Therefore you need to be sure to apply the steps in KB 920783 using both groups if you want to rid your event logs of these errors once and for all. Again, this is assuming you are using least privilege accounts -- which I certainly hope you are.

Thanks, Matt, for pointing out my omission.]

KB 896861 and "Microsoft Fix it"

Jeremy Jameson — Wed, 14 Oct 2009 12:25:00 GMT

A couple of days ago, I rebuilt one of my development VMs for Microsoft Office SharePoint Server (MOSS) 2007. As I've noted in a previous post, I've found that I need to apply one of the registry hacks described in the following KnowledgeBase article:

You receive error 401.1 when you browse a Web site that uses Integrated Authentication and is hosted on IIS 5.1 or a later version

http://support.microsoft.com/kb/896861

When I browsed to KB 896861 this week, I noticed that it has been updated since the last time I saw it. Apparently, this has become such a common issue that Microsoft now provides an automated way to fix the issue:

Fix this problem
Microsoft Fix it 50306

This was the first time I've seen a reference to Microsoft Fix it, and I have to say that I like it -- and for more than just the cute moniker and button image, too!

Bug in MOSS 2007 FBA with Insufficient Permissions in SQL Server

Jeremy Jameson — Mon, 12 Oct 2009 13:58:00 GMT

A couple of weeks ago I was setting up Forms-Based Authentication (FBA) on my new development VM for Microsoft Office SharePoint Server (MOSS) 2007, and I spent a few hours troubleshooting why I couldn't add a custom role ("Authenticated Users") to a SharePoint site.

Just like I'd done a couple of times before, I meticulously followed Steve Peschka's article on MSDN -- Forms Authentication in SharePoint Products and Technologies (Part 1) -- that provides lots of details with regards to ensuring you have all the right Web.config entries (connection strings, membership element, etc.) necessary to make FBA work.

I'd also granted the following permissions in SQL Server to the service account for the application pool corresponding to my Web application:

aspnet_Membership_BasicAccess
aspnet_Roles_BasicAccess

Like I said before, I had previously configured FBA in MOSS 2007 for a couple of previous projects, but this time, in addition to actually authenticating users (i.e. "membership"), I also needed to restrict access to certain areas of the site based on groups of FBA users (i.e. "roles").

However, when I tried adding my custom Authenticated Users role to a SharePoint group, the only thing SharePoint would recognize was NT AUTHORITY\authenticated users. I even tried typing in the fully qualified role name (FabrikamSqlRoleProvider:Authenticated Users), but SharePoint didn't recognize that either.

The really frustrating part is that I could add users from my ASP.NET membership database to SharePoint groups, I just couldn't add roles. This certainly made me think that I had the necessary Web.config entries.

I then fired up SQL Server Profiler to verify that SharePoint was at least trying to query my ASP.NET database when adding a role to a SharePoint group. Sure enough, I saw the following statement in the profiler trace:

exec dbo.aspnet_Roles_RoleExists @ApplicationName=N'Fabrikam Portal',@RoleName=N'Authenticated Users'

Hmmm...from this I could tell that SharePoint was definitely attempting to validate the custom role. I then attached the debugger and set it to break on all exceptions (the approach I frequently use when I am completely baffled by something).

That's when I found the problem, namely a SqlException:

The EXECUTE permission was denied on the object 'aspnet_Roles_RoleExists', database 'FabrikamPortal', schema 'dbo'.

Ugh...it turns out the aspnet_Roles_RoleExists stored procedure is by default only granted EXECUTE permission to the aspnet_Roles_ReportingAccess database role within SQL Server. Unfortunately, SharePoint was simply "swallowing" that SqlException and assuming the role simply did not exist. I don't know about you, but I consider a "swallowed exception" like this to be a bug. Others may disagree, but that's my opinion.

The lesson learned here is that when using Forms-Based Authentication and the out-of-the-box ASP.NET membership and role providers, your service account needs to be added to the following database roles in your ASP.NET database:

aspnet_Membership_BasicAccess
aspnet_Membership_ReportingAccess
aspnet_Roles_BasicAccess
aspnet_Roles_ReportingAccess

The reason you should add it the aspnet_Membership_ReportingAccess database role -- in addition to the aspnet_Roles_ReportingAccess database role -- is that the sprocs that allow you to do partial matching on user names (e.g. aspnet_Membership_FindUsersByName) are by default only granted EXECUTE permission to the aspnet_Membership_ReportingAccess database role (not aspnet_Membership_BasicAccess).

In other words, when I said earlier that I could add FBA users to a SharePoint group when my service account was a member of the aspnet_Membership_BasicAccess database role, that only worked because I typed in the full username -- which gets validated using the aspnet_Membership_GetUserByName sproc (which is granted EXECUTE permission to the aspnet_Membership_BasicAccess database role).

By the way, here's some script to automatically create a user in SQL Server for the service account and add it to the necessary database roles:

USE [FabrikamPortal]
GO

CREATE USER [TECHTOOLBOX\svc-web-fabrikam-dev]
FOR LOGIN [TECHTOOLBOX\svc-web-fabrikam-dev]
WITH DEFAULT_SCHEMA=[dbo]
GO

EXEC sp_addrolemember N'aspnet_Membership_BasicAccess', N'TECHTOOLBOX\svc-web-fabrikam-dev'
GO
EXEC sp_addrolemember N'aspnet_Membership_ReportingAccess', N'TECHTOOLBOX\svc-web-fabrikam-dev'
GO
EXEC sp_addrolemember N'aspnet_Roles_BasicAccess', N'TECHTOOLBOX\svc-web-fabrikam-dev'
GO
EXEC sp_addrolemember N'aspnet_Roles_ReportingAccess', N'TECHTOOLBOX\svc-web-fabrikam-dev'
GO

You'll obviously need to replace the name of your ASP.NET database and service account accordingly.

Isn't it fun conforming to the principle of least privilege?!

Introducing the SharePointPublishingHelper Class

Jeremy Jameson — Fri, 09 Oct 2009 15:19:00 GMT

In my previous post, I described a utility to import pages into Microsoft Office SharePoint Server (MOSS) 2007 from an Excel input file.

Aside from the code to read data from the input file into a DataSet, the main work is performed by the SharePointPublishingHelper class (which was written long before the ImportPages.exe utility). This class is simply a collection of helper methods for the SharePoint publishing API.

SharePointPublishingHelper makes it really easy to:

Attempt to retrieve a reference to a PublishingPage by specifying an SPWeb and the page URL name (e.g. default.aspx)
Ensure an expected page exists in a site (creating a new page as necessary)
Check if a page is checked out to anyone
Check if a page is checked out by the current user
Ensure a page is checked out to the current user (since you can't make any changes to a page that isn't checked out to you)
Ensure a page is associated with a specific page layout (and corresponding content type)
Publish a page (correctly handling page libraries that are configured for approval as well as those that are not configured for approval)
Delete a page by specifying the page URL name

Here's the code for the SharePointPublishingHelper class:

using System;
using System.Diagnostics;
using System.Globalization;
using System.Security.Permissions;

using Microsoft.SharePoint;
using Microsoft.SharePoint.Publishing;
using Microsoft.SharePoint.Security;

using Fabrikam.Demo.CoreServices.Logging;

namespace Fabrikam.Demo.CoreServices.SharePoint
{
    /// <summary>
    /// Exposes static methods for commonly used helper functions for the
    /// SharePoint Publishing API. This class cannot be inherited.
    /// </summary>
    /// <remarks>
    /// All methods of the <b>SharePointPublishingHelper</b> class are static
    /// and can therefore be called without creating an instance of the class.
    /// </remarks>
    [CLSCompliant(false)]
    [SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true)]
    public sealed class SharePointPublishingHelper
    {
        private SharePointPublishingHelper() { } // all members are static

        /// <summary>
        /// Deletes the specified publishing page in the site (if it exists).
        /// </summary>
        /// <param name="web">The site containing the publishing page.</param>
        /// <param name="pageUrlName">The URL name of the page (e.g.
        /// "default.aspx").</param>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1054:UriParametersShouldNotBeStrings",
            Justification = "Page URL name is only the last part of the local"
                + " path of the page URL.")]
        public static void DeletePage(
            SPWeb web,
            string pageUrlName)
        {
            if (web == null)
            {
                throw new ArgumentNullException("web");
            }
            else if (string.IsNullOrEmpty(pageUrlName) == true)
            {
                throw new ArgumentException(
                    "The page URL name must be specified.",
                    "pageUrlName");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Deleting page ({0}) from Web ({1})...",
                pageUrlName,
                web.Url);

            PublishingPage page = FindPublishingPage(web, pageUrlName);
            
            if (page == null)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "The page specified to delete ({0})"
                        + " does not exist in the Web ({1}).",
                    pageUrlName,
                    web.Url);

            }
            else
            {
                string absolutePageUrl =
                        page.ListItem.Web.Url + '/' + page.Url;
                
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Deleting page: {0}...",
                    absolutePageUrl);

                page.ListItem.Delete();

                Logger.LogInfo(
                    CultureInfo.InvariantCulture,
                    "Successfully deleted page: {0}.",
                    absolutePageUrl);
            }
        }

        /// <summary>
        /// Ensures a page exists and is configured according with the
        /// specified settings.
        /// </summary>
        /// <param name="web">The site containing the publishing page.</param>
        /// <param name="pageUrlName">The URL name of the page (e.g.
        /// "default.aspx").</param>
        /// <param name="pageTitle">The title of the page.</param>
        /// <param name="pageLayoutUrl">The URL of the page layout for the
        /// page (e.g. /_catalogs/masterpage/WelcomeLinks.aspx).</param>
        /// <returns>The new or existing publishing page.</returns>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1054:UriParametersShouldNotBeStrings",
            Justification = "Page URL name is only the last part of the local"
                + " path of the page URL. Page layout URL is a server-relative"
                + " URL (e.g. /_catalogs/masterpage/WelcomeLinks.aspx).")]
        public static PublishingPage EnsurePage(
            SPWeb web,
            string pageUrlName,
            string pageTitle,
            string pageLayoutUrl)
        {
            return EnsurePage(
                web,
                pageUrlName,
                pageTitle,
                pageLayoutUrl,
                null);
        }

        /// <summary>
        /// Ensures a page exists and is configured according with the
        /// specified settings.
        /// </summary>
        /// <param name="web">The site containing the publishing page.</param>
        /// <param name="pageUrlName">The URL name of the page (e.g.
        /// "default.aspx").</param>
        /// <param name="pageTitle">The title of the page.</param>
        /// <param name="pageLayoutUrl">The URL of the page layout for the
        /// page (e.g. /_catalogs/masterpage/WelcomeLinks.aspx).</param>
        /// <param name="pageDescription">The description of the page (optional).</param>
        /// <returns>The new or existing publishing page.</returns>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1054:UriParametersShouldNotBeStrings",
            Justification = "Page URL name is only the last part of the local"
                + " path of the page URL. Page layout URL is a server-relative"
                + " URL (e.g. /_catalogs/masterpage/WelcomeLinks.aspx).")]
        public static PublishingPage EnsurePage(
            SPWeb web,
            string pageUrlName,
            string pageTitle,
            string pageLayoutUrl,
            string pageDescription)
        {
            if (web == null)
            {
                throw new ArgumentNullException("web");
            }
            else if (pageUrlName == null)
            {
                throw new ArgumentNullException("pageUrlName");
            }
            else if (string.IsNullOrEmpty(pageUrlName) == true)
            {
                throw new ArgumentException(
                    "The page URL name must be specified.",
                    "pageUrlName");
            }
            else if (pageTitle == null)
            {
                throw new ArgumentNullException("pageTitle");
            }
            else if (string.IsNullOrEmpty(pageTitle) == true)
            {
                throw new ArgumentException(
                    "The page title must be specified.",
                    "pageTitle");
            }
            else if (pageLayoutUrl == null)
            {
                throw new ArgumentNullException("pageLayoutUrl");
            }
            else if (string.IsNullOrEmpty(pageLayoutUrl) == true)
            {
                throw new ArgumentException(
                    "The page layout URL must be specified.",
                    "pageLayoutUrl");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Ensuring page ({0}) in Web ({1})...",
                pageUrlName,
                web.Url);

            PublishingSite pubSite = new PublishingSite(web.Site);
            PageLayout layout = pubSite.PageLayouts[pageLayoutUrl];

            if (layout == null)
            {
                string message = string.Format(
                    CultureInfo.InvariantCulture,
                    "Invalid page layout specified ({0}).",
                    pageLayoutUrl);

                throw new ArgumentException(
                    message,
                    "pageLayoutUrl");
            }

            PublishingWeb pubWeb = PublishingWeb.GetPublishingWeb(web);
            PublishingPageCollection pages = pubWeb.GetPublishingPages();

            PublishingPage page =
                pages[pubWeb.PagesListName + "/" + pageUrlName];

            if (page == null)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Creating page ({0}/{1}/{2})...",
                    web.Url,
                    pubWeb.PagesListName,
                    pageUrlName);

                page = pages.Add(pageUrlName, layout);
                page.Title = pageTitle;

                if (string.IsNullOrEmpty(pageDescription) == false)
                {
                    page.Description = pageDescription;
                }

                page.ListItem.SystemUpdate(false);

                Logger.LogInfo(
                    CultureInfo.InvariantCulture,
                    "Successfully created page: {0}/{1}.",
                    web.Url,
                    page.Url);
            }
            else
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "The specified page ({0}/{1}) already exists.",
                    web.Url,
                    page.Url);
            }

            return page;
        }

        /// <summary>
        /// Ensure the specified page is configured with the specified content
        /// type and page layout.
        /// </summary>
        /// <param name="page">The page to check (and update as necessary).</param>
        /// <param name="pageLayoutUrl">The URL of the page layout for the
        /// page (e.g. /_catalogs/masterpage/WelcomeLinks.aspx).</param>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1054:UriParametersShouldNotBeStrings",
            Justification = "Page layout URL is a server-relative"
                + " URL (e.g. /_catalogs/masterpage/WelcomeLinks.aspx).")]
        public static void EnsurePageContentTypeAndLayout(
            PublishingPage page,
            string pageLayoutUrl)
        {
            if (page == null)
            {
                throw new ArgumentNullException("page");
            }

            if (string.IsNullOrEmpty(pageLayoutUrl) == true)
            {
                throw new ArgumentException(
                    "The page layout must be specified.",
                    "pageLayoutUrl");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Ensuring content type and page layout ({0}) for page"
                    + " ({1}/{2})...",
                pageLayoutUrl,
                page.ListItem.Web.Url,
                page.Url);

            PublishingSite site = new PublishingSite(page.PublishingWeb.Web.Site);
            PageLayout layout = site.PageLayouts[pageLayoutUrl];
            
            SPContentTypeId contentTypeId = layout.AssociatedContentType.Id;
            
            // HACK: The content type associated with each "Pages" library
            // actually gets a unique ContentTypeId (it is appended with a
            // unique identifier for the list -- thus inheriting from the
            // content type associated with the page layout).
            SPContentType pageContentType = null;
            
            foreach (SPContentType listContentType in
                page.PublishingWeb.PagesList.ContentTypes)
            {
                if (listContentType.Parent.Id == contentTypeId)
                {
                    pageContentType = listContentType;
                    break;
                }
            }
            
            if (pageContentType == null)
            {
                string message = string.Format(
                    CultureInfo.InvariantCulture,
                    "The content type ({0}) associated with the page layout"
                        + " ({1}) is not enabled on the list ({2}/{3}).",
                    layout.AssociatedContentType.Name,
                    pageLayoutUrl,
                    page.ListItem.Web.Url,
                    page.PublishingWeb.PagesListName);

                throw new ArgumentException(message, "pageLayoutUrl");
            }

            if ((SPContentTypeId)page.ListItem["ContentTypeId"] !=
                    pageContentType.Id
                || page.Layout.Name != layout.Name)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Updating content type and page layout ({0}) for page"
                        + " ({1}/{2})...",
                    pageLayoutUrl,
                    page.ListItem.Web.Url,
                    page.Url);

                page.ListItem["ContentTypeId"] = pageContentType.Id;
                page.Layout = layout;
                page.Update();

                Logger.LogInfo(
                    CultureInfo.InvariantCulture,
                    "Successfully updated content type and page layout ({0})"
                        + "  for page: {1}/{2}.",
                    pageLayoutUrl,
                    page.ListItem.Web.Url,
                    page.Url);
            }
            else
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "The page ({0}/{1}) is already configured with the"
                        + " specified content type and page layout ({2}).",
                    page.ListItem.Web.Url,
                    page.Url,                    
                    pageLayoutUrl);
            }
        }

        /// <summary>
        /// Ensures the specified page is checked out to the current user.
        /// </summary>
        /// <param name="page">The publishing page to check out.</param>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Naming",
            "CA1702:CompoundWordsShouldBeCasedCorrectly",
            MessageId = "ToMe")]
        public static void EnsurePageIsCheckedOutToMe(
            PublishingPage page)
        {
            if (page == null)
            {
                throw new ArgumentNullException("page");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Ensuring page ({0}/{1}) is checked out to the current"
                    + " user...",
                page.ListItem.Web.Url,
                page.Url);

            bool isPageCheckedOut = IsPageCheckedOut(page);

            if (isPageCheckedOut == true)
            {
                bool isPageCheckedOutToMe = IsPageCheckedOutToMe(page);

                if (isPageCheckedOutToMe == true)
                {
                    Logger.LogDebug(
                        CultureInfo.InvariantCulture,
                        "The page ({0}/{1}) is already checked out to the"
                            + " current user.",
                        page.ListItem.Web.Url,
                        page.Url);

                    return;
                }

                string message = string.Format(
                    CultureInfo.InvariantCulture,
                    "Cannot check out the page "
                        + " ({0}/{1}) because it is already checked"
                        + " out to somebody else.",
                    page.ListItem.Web.Url,
                    page.Url);

                throw new InvalidOperationException(message);
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Checking out page ({0}/{1})...",
                page.ListItem.Web.Url,
                page.Url);

            page.CheckOut();

            Logger.LogInfo(
                CultureInfo.InvariantCulture,
                "Successfully checked out page: {0}/{1}.",
                page.ListItem.Web.Url,
                page.Url);
        }

        /// <summary>
        /// Finds the specified publishing page in the site.
        /// </summary>
        /// <param name="web">The site containing the publishing page.</param>
        /// <param name="pageUrlName">The URL name of the page (e.g.
        /// "default.aspx").</param>
        /// <returns>The specified publishing page, or null if the page does not
        /// exist.</returns>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1054:UriParametersShouldNotBeStrings",
            Justification = "Page URL name is only the last part of the local"
                + " path of the page URL.")]
        public static PublishingPage FindPublishingPage(
            SPWeb web,
            string pageUrlName)
        {
            if (web == null)
            {
                throw new ArgumentNullException("web");
            }
            else if (string.IsNullOrEmpty(pageUrlName) == true)
            {
                throw new ArgumentException(
                    "The page URL name must be specified.",
                    "pageUrlName");
            }

            PublishingWeb pubWeb = PublishingWeb.GetPublishingWeb(web);
            PublishingPageCollection pages = pubWeb.GetPublishingPages();

            PublishingPage page = pages[pubWeb.PagesListName + "/" + pageUrlName];
            return page;
        }

        /// <summary>
        /// Determines if the specified page is approved.
        /// </summary>
        /// <param name="page">The page to determine the status of.</param>
        /// <returns>True if the specified page is approved (or if the
        /// corresponding library is not configured for approval), otherwise
        /// false.</returns>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1011:ConsiderPassingBaseTypesAsParameters")]
        public static bool IsPageApproved(
            PublishingPage page)
        {
            if (page == null)
            {
                throw new ArgumentNullException("page");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Checking if page ({0}/{1}) is approved...",
                page.ListItem.Web.Url,
                page.Url);

            bool isApproved = false;

            if (page.ListItem.ModerationInformation == null)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Pages library ({0}/{1}) is not configured for approval.",
                    page.ListItem.Web.Url,
                    page.PublishingWeb.PagesListName);

                Debug.Assert(page.ListItem.Versions != null);
                Debug.Assert(page.ListItem.Versions.Count > 0);

                if (page.ListItem.Versions[0].Level == SPFileLevel.Published)
                {
                    isApproved = true;
                }
            }
            else
            {

                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Pages library ({0}/{1}) is configured for approval.",
                    page.ListItem.Web.Url,
                    page.PublishingWeb.PagesListName);

                isApproved =
                    (page.ListItem.ModerationInformation.Status ==
                        SPModerationStatusType.Approved);
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "The page ({0}/{1} is {2}.",
                page.ListItem.Web.Url,
                page.Url,
                isApproved ? "approved" : "not approved");

            return isApproved;
        }

        /// <summary>
        /// Determines if the specified page is checked out.
        /// </summary>
        /// <param name="page">The page to determine the status of.</param>
        /// <returns>True if the specified page is checked out, otherwise false.</returns>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1011:ConsiderPassingBaseTypesAsParameters")]
        public static bool IsPageCheckedOut(
            PublishingPage page)
        {
            if (page == null)
            {
                throw new ArgumentNullException("page");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Checking if page ({0}/{1}) is checked out...",
                page.ListItem.Web.Url,
                page.Url);

            bool isCheckedOut =
                (page.ListItem.File.CheckOutStatus != SPFile.SPCheckOutStatus.None);

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "The page ({0}/{1}) is {2}.",
                page.ListItem.Web.Url,
                page.Url,
                isCheckedOut ? "checked out" : "not checked out");

            return isCheckedOut;
        }

        /// <summary>
        /// Determines if the specified page is checked out to the current user.
        /// </summary>
        /// <param name="page">The page to determine the status of.</param>
        /// <returns>True if the specified page is checked out to the current
        /// user, otherwise false.</returns>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1011:ConsiderPassingBaseTypesAsParameters"),
        System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Naming",
            "CA1702:CompoundWordsShouldBeCasedCorrectly",
            MessageId = "ToMe")]
        public static bool IsPageCheckedOutToMe(
            PublishingPage page)
        {
            if (page == null)
            {
                throw new ArgumentNullException("page");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Checking if page ({0}/{1}) is checked out to current user...",
                page.ListItem.Web.Url,
                page.Url);

            bool isCheckedOut = (page.ListItem.File.CheckOutStatus
                != SPFile.SPCheckOutStatus.None);

            if (isCheckedOut == false)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "The page ({0}/{1}) is not checked out.",
                    page.ListItem.Web.Url,
                    page.Url);

                return false;
            }

            bool isCheckedOutToMe = (page.ListItem.File.CheckedOutBy.LoginName
                == page.ListItem.Web.CurrentUser.LoginName);

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "The page ({0}/{1}) is {2} to the current user.",
                page.ListItem.Web.Url,
                page.Url,
                isCheckedOutToMe ? "checked out" : "not checked out");

            return isCheckedOutToMe;
        }

        /// <summary>
        /// Publishes the specified page (making it generally available for viewing).
        /// </summary>
        /// <param name="page">The page to publish.</param>
        /// <param name="comments">Comments to associate with the publishing action.</param>
        [System.Diagnostics.CodeAnalysis.SuppressMessage(
            "Microsoft.Design",
            "CA1011:ConsiderPassingBaseTypesAsParameters")]
        public static void PublishPage(
            PublishingPage page,
            string comments)
        {
            if (page == null)
            {
                throw new ArgumentNullException("page");
            }

            Logger.LogDebug(
                CultureInfo.InvariantCulture,
                "Publishing page ({0}/{1})...",
                page.ListItem.Web.Url,
                page.Url);

            if (page.ListItem.ModerationInformation == null)
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Pages library ({0}/{1}) is not configured for approval."
                        + " Checking in page...",
                    page.ListItem.Web.Url,
                    page.PublishingWeb.PagesListName);

                page.ListItem.File.CheckIn(
                    comments,
                    SPCheckinType.MajorCheckIn);
            }
            else
            {
                Logger.LogDebug(
                    CultureInfo.InvariantCulture,
                    "Pages library ({0}/{1}) is configured for approval."
                        + " Scheduling page for publication...",
                    page.ListItem.Web.Url,
                    page.PublishingWeb.PagesListName);

                page.Schedule(comments);
            }

            Logger.LogInfo(
                CultureInfo.InvariantCulture,
                "Successfully published page: {0}/{1}.",
                page.ListItem.Web.Url,
                page.Url);
        }
    }
}

In addition to the SharePointPublishingHelper class, I often rely on several other custom SharePoint "helper" classes, such SharePointListHelper and SharePointWebPartHelper. These classes evolved over time by refactoring common code that was previously repeated in numerous places. In other words, as Scott Hanselman likes to say...a developer should always try to write DRY code (meaning, don't repeat yourself).

Importing Pages into MOSS 2007 from an Excel File

Jeremy Jameson — Thu, 08 Oct 2009 16:54:00 GMT

In my previous post, I briefly introduced the concept of a utility to import pages into Microsoft Office SharePoint Server (MOSS) 2007 from an Excel input file. This can be very useful for Development and Test environments (where you frequently rebuild the sites during the development process), as well as when migrating legacy Web sites to a Production environment -- or whenever a "quick and dirty" process is deemed sufficient for the business needs.

I've done a number of MOSS 2007 migration projects over the last several years and developed a number of tools to migrate content into SharePoint. About a month ago, I utilized some existing code I had previously written to import pages from a simple Excel workbook -- rather than some other data source, such as a legacy database. The Excel workbook simply contains one or more worksheets, where each worksheet represents a particular group of pages with similar columns/fields. Each worksheet must have the following columns:

WebUrl
PageUrlName
PageLayout
Title

Additional columns (such as Page Content) can also be specified -- including custom columns for content types that are not included out-of-the-box with MOSS 2007.

Note that I am not advocating installing the full Microsoft Office suite -- or even just Excel -- on your SharePoint servers just for the sake of importing a few hundred (or perhaps a few thousand) pages into your site. Rather, you would simply need to install the following on one of the servers in your farm:

2007 Office System Driver: Data Connectivity Components
http://www.microsoft.com/downloads/details.aspx?familyid=7554F536-8C28-4598-9B72-EF94E038C891&displaylang=en

Preferably, you would install this only on a backend "job/index" server that isn't servicing any user requests directly and then run the ImportPages.exe utility on that server -- or at least, that's what I thought originally.

The problem is that in order to utilize the currently available OLEDB providers (which are 32-bit), you have to force the process to run as 32-bit (by setting the platform target to x86). However, as noted in my previous post, you can't utilize the SharePoint API from a 32-bit process running on an x64 environment. Quite the paradox indeed.

The solution that I came up with is to enhance the ImportPages.exe utility to support an Excel input file as well as a simple XML file that contains a serialized DataSet.

In other words, importing pages from Excel in an x64 environment requires a two-step process:

Convert the Excel input file into a simple XML file that contains a serialized DataSet. Note that this can be done on any environment (i.e. it does not necessarily have to be done on one of the SharePoint servers). In other words, you could have someone create the Excel input file on his or her laptop and then convert it to a DataSet XML file.
Import the pages from the DataSet XML file. Note that this step must be performed on one of the SharePoint servers in the farm.

To convert an Excel file to a DataSet XML file:

ConvertToDataSet.exe Sample.xslx

To import pages from the generated DataSet XML file:

ImportPages.exe http://fabrikam Sample.xml

Here is the code for the ConvertToDataSet.exe utility:

using System;
using System.Data;
using System.IO;

using Fabrikam.Demo.CoreServices;

namespace Fabrikam.Demo.Tools.ConvertToDataSet
{
    /// <summary>
    /// Main class for the ConvertToDataSet.exe utility.
    /// </summary>
    class Program
    {
        private Program() { } // all members are static

        private static void Main(
            string[] args)
        {
            if (args.Length != 1)
            {
                Console.Error.WriteLine(
                    "Usage: ConvertToDataSet"
                    + " {filename}");

                Environment.Exit(1);
            }

            string inputFileName = args[0];

            try
            {
                string fileExtension = Path.GetExtension(inputFileName);

                if (string.Compare(
                    fileExtension,
                    ".xml",
                    StringComparison.OrdinalIgnoreCase) == 0)
                {
                    throw new ArgumentException(
                        "Cannot convert an XML file to a DataSet.");
                }

                string outputFileName = Path.ChangeExtension(inputFileName, "xml");

                DataSet data = DataSetHelper.LoadFromFile(inputFileName);

                data.WriteXml(outputFileName);

                Console.WriteLine(
                    "DataSet serialized to file: {0}",
                    outputFileName);

                Environment.Exit(0);
            }
            catch (Exception ex)
            {
                Console.Error.WriteLine(ex.Message);

#if DEBUG
                Console.Error.WriteLine("(stack trace):");
                Console.Error.WriteLine(ex.StackTrace);
#endif

                Environment.Exit(1);
            }
        }
    }
}

Note that the bulk of the code has been refactored into my DataSetHelper class:

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.OleDb;
using System.Globalization;
using System.IO;

namespace Fabrikam.Demo.CoreServices
{
    /// <summary>
    /// Exposes static methods for commonly used helper functions for
    /// the <see cref="System.Data.DataSet" /> class.
    /// This class cannot be inherited.
    /// </summary>
    /// <remarks>
    /// All methods of the <c>DataSetHelper</c> class are static and can
    /// therefore be called without creating an instance of the class.
    /// </remarks>
    public sealed class DataSetHelper
    {
        private DataSetHelper() { } // all members are static

        /// <summary>
        /// Loads a DataSet from the specified file.
        /// </summary>
        /// <remarks>The input file may be an Excel file (.xls or .xlsx) or a
        /// simple CSV file (i.e. comma-separated values).</remarks>
        /// <param name="fileName">The filename (including the path) from which
        /// to load the data.</param>
        /// <returns>A <see cref="System.Data.DataSet" /> containing the data
        /// read from the file.</returns>
        public static DataSet LoadFromFile(
            string fileName)
        {
            if (fileName == null)
            {
                throw new ArgumentNullException("fileName");
            }
            else if (string.IsNullOrEmpty(fileName) == true)
            {
                throw new ArgumentException(
                    "The file name must be specified.",
                    "fileName");
            }

            FileInfo inputFileInfo = new FileInfo(fileName);

            string connectionString = GetOleDbConnectionString(inputFileInfo);

            OleDbConnection connection = new OleDbConnection(connectionString);
            connection.Open();

            string[] tableNames = GetTableNames(inputFileInfo, connection);

            DataSet data = new DataSet();
            data.Locale = CultureInfo.InvariantCulture;

            foreach (string tableName in tableNames)
            {
                DataTable table = data.Tables.Add(tableName);

                string commandText = string.Format(
                    CultureInfo.InvariantCulture,
                    "SELECT * FROM [{0}]",
                    tableName);

                OleDbCommand selectCommand = new OleDbCommand(
                    commandText,
                    connection);

                OleDbDataAdapter adapter = new OleDbDataAdapter(selectCommand);

                adapter.Fill(table);
            }

            return data;
        }

        private static string GetOleDbConnectionString(
            FileInfo inputFileInfo)
        {
            string connectionString = null;

            if (string.Compare(
                inputFileInfo.Extension,
                ".xls",
                StringComparison.OrdinalIgnoreCase) == 0)
            {
                connectionString =
                    @"Provider=Microsoft.Jet.OLEDB.4.0;"
                    + "Data Source=" + inputFileInfo.FullName + ";"
                    + "Extended Properties=\"Excel 8.0;HDR=YES;\"";
            }
            else if (string.Compare(
                inputFileInfo.Extension,
                ".xlsx",
                StringComparison.OrdinalIgnoreCase) == 0)
            {
                connectionString =
                    @"Provider=Microsoft.ACE.OLEDB.12.0;"
                    + "Data Source=" + inputFileInfo.FullName + ";"
                    + "Extended Properties=\"Excel 12.0;HDR=Yes;ReadOnly=true;"
                    + "IMEX=1;\"";
            }
            else
            {
                string folder = inputFileInfo.DirectoryName;

                connectionString =
                    @"Provider=Microsoft.Jet.OLEDB.4.0;"
                    + "Data Source=" + folder + ";"
                    + "Extended Properties=\"text;HDR=YES;FMT=Delimited;\"";

            }

            return connectionString;
        }

        private static string[] GetTableNames(
            FileInfo inputFileInfo,
            OleDbConnection connection)
        {
            bool isExcelFile = false;

            if (string.Compare(
                inputFileInfo.Extension,
                ".xls",
                StringComparison.OrdinalIgnoreCase) == 0)
            {
                isExcelFile = true;
            }
            else if (string.Compare(
                inputFileInfo.Extension,
                ".xlsx",
                StringComparison.OrdinalIgnoreCase) == 0)
            {
                isExcelFile = true;
            }

            if (isExcelFile == true)
            {
                // Get all of the Table names from the Excel workbook 
                DataTable worksheetTables = connection.GetOleDbSchemaTable(
                    OleDbSchemaGuid.Tables,
                    null);

                List<string> tableNames = new List<string>();

                for (int i = 0; i < worksheetTables.Rows.Count; i++)
                {
                    string tableName =
                        (string)worksheetTables.Rows[i]["TABLE_NAME"];

                    if (string.Compare(
                        tableName,
                        "_xlnm#_FilterDatabase",
                        StringComparison.OrdinalIgnoreCase) == 0)
                    {
                        continue;
                    }

                    tableNames.Add(tableName);
                }

                return tableNames.ToArray();
            }
            else
            {
                string[] tableNames = new string[1];
                tableNames[0] = inputFileInfo.Name;
                return tableNames;
            }
        }
    }
}

Similarly, the bulk of the code for the ImportPages.exe utility has been refactored into the PageImporter and SharePointPublishingHelper classes.

Here's the code for the PageImporter class:

using System;
using System.Data;
using System.Diagnostics;
using System.Globalization;
using System.IO;

using Microsoft.SharePoint;
using Microsoft.SharePoint.Publishing;

using Fabrikam.Demo.CoreServices;
using Fabrikam.Demo.CoreServices.Logging;
using Fabrikam.Demo.CoreServices.SharePoint;

namespace Fabrikam.Demo.Tools.SharePoint.ImportPages
{
    /// <summary>
    /// Exposes static methods for importing pages into a SharePoint site.
    /// This class cannot be inherited.
    /// </summary>
    /// <remarks>
    /// All methods of the <c>PageImporter</c> class are static and can
    /// therefore be called without creating an instance of the class.
    /// </remarks>    
    class PageImporter
    {
        private PageImporter() { } // all members are static 

        /// <summary>
        /// Imports pages into the specified SharePoint site using data from
        /// the specified input file.
        /// </summary>
        /// <param name="siteUrl">The URL of the SharePoint site.</param>
        /// <param name="fileName">The filename (including the path) from which
        /// to load the data.</param>
        public static void Import(
            string siteUrl,
            string fileName)
        {
            DataSet data = DataSetHelper.LoadFromFile(fileName);

            using (SPSite site = new SPSite(siteUrl))
            {
                Import(site, data);
            }
        }

        private static void Import(
            SPSite site,
            DataSet data)
        {
            foreach (DataTable table in data.Tables)
            {
                Import(site, table);
            }
        }

        private static void Import(
            SPSite site,
            DataTable table)
        {
            SPWeb web = null;

            foreach (DataRow row in table.Rows)
            {
                string webUrl = (string)row["WebUrl"];
                string pageUrlName = (string)row["PageUrlName"];

                try
                {
                    EnsureReferenceToExpectedWeb(site, ref web, webUrl);

                    Import(web, row);
                }
                catch (ArgumentNullException ex)
                {
                    Logger.LogWarning(
                        CultureInfo.InvariantCulture,
                        "Error importing page ({0}) into Web ({1}) - {2}.",
                        pageUrlName,
                        webUrl,
                        ex.Message);
                }
                catch (FileNotFoundException ex)
                {
                    Logger.LogWarning(
                        CultureInfo.InvariantCulture,
                        "Error importing page ({0}) into Web ({1}) - {2}.",
                        pageUrlName,
                        webUrl,
                        ex.Message);
                }
            }

            if (web != null)
            {
                web.Dispose();
            }
        }

        private static void EnsureReferenceToExpectedWeb(
            SPSite site,
            ref SPWeb web,
            string expectedWebUrl)
        {
            Debug.Assert(string.IsNullOrEmpty(expectedWebUrl) == false);

            if (web == null)
            {
                web = site.OpenWeb(expectedWebUrl);
            }
            else if (string.Compare(
                web.ServerRelativeUrl,
                expectedWebUrl,
                StringComparison.OrdinalIgnoreCase) != 0)
            {
                web.Dispose();

                web = site.OpenWeb(expectedWebUrl);
            }
        }

        private static void Import(
            SPWeb web,
            DataRow row)
        {
            string pageUrlName = (string)row["PageUrlName"];

            string pageLayoutUrl =
                "_catalogs/masterpage/" + (string)row["PageLayout"];

            string pageTitle = null;
            if (row.IsNull("Title") == false)
            {
                pageTitle = (string)row["Title"];

                if (pageTitle.Length == 255)
                {
                    Logger.LogWarning(
                        CultureInfo.InvariantCulture,
                        "The title specified for the page ({0}) may have been"
                            + " truncated, since it is exactly 255 characters.",
                        pageUrlName);
                }
            }

            PublishingPage page = SharePointPublishingHelper.EnsurePage(
                web,
                pageUrlName,
                pageTitle,
                pageLayoutUrl);

            bool skipPageConfiguration =
                CheckIfPageImportShouldBeSkipped(
                    page);

            if (skipPageConfiguration == true)
            {
                return;
            }

            SharePointPublishingHelper.EnsurePageIsCheckedOutToMe(
                page);

            page.Title = pageTitle;

            foreach (DataColumn column in row.Table.Columns)
            {
                string columnName = column.ColumnName;

                if (string.Compare(
                    columnName,
                    "WebUrl",
                    StringComparison.OrdinalIgnoreCase) == 0)
                {
                    continue;
                }
                else if (string.Compare(
                    columnName,
                    "PageUrlName",
                    StringComparison.OrdinalIgnoreCase) == 0)
                {
                    continue;
                }
                else if (string.Compare(
                    columnName,
                    "PageLayout",
                    StringComparison.OrdinalIgnoreCase) == 0)
                {
                    continue;
                }
                else if (string.Compare(
                    columnName,
                    "Title",
                    StringComparison.OrdinalIgnoreCase) == 0)
                {
                    continue;
                }

                if (row.IsNull(columnName) == false)
                {
                    page.ListItem[columnName] = row[columnName];
                }
            }

            page.Update();

            SharePointPublishingHelper.PublishPage(
                page,
                "Published by Fabrikam.Demo.Tools.ImportPages.PageImporter.");
        }

        private static bool CheckIfPageImportShouldBeSkipped(
            PublishingPage page)
        {
            if (SharePointPublishingHelper.IsPageApproved(page))
            {
                Logger.LogInfo(
                    CultureInfo.InvariantCulture,
                    "Skipping import of page ({0}/{1}) because the page"
                        + " is approved.",
                    page.ListItem.Web.Url,
                    page.Url);

                return true;
            }

            bool isPageCheckedOut =
                SharePointPublishingHelper.IsPageCheckedOut(
                    page);

            bool isPageCheckedOutToMe =
                SharePointPublishingHelper.IsPageCheckedOutToMe(
                    page);

            if (isPageCheckedOut == true
                && isPageCheckedOutToMe == false)
            {
                Logger.LogWarning(
                    CultureInfo.InvariantCulture,
                    "Skipping import of page"
                        + " ({0}/{1}) because the page is already checked"
                        + " out to somebody else.",
                    page.ListItem.Web.Url,
                    page.Url);

                return true;
            }

            return false;
        }
    }
}

I'll cover the details of the SharePointPublishingHelper class in a separate post.