Mark Arend : Development

Debugging web parts and other SharePoint custom code

markarend — Tue, 09 Sep 2008 15:34:00 GMT

Here are some tips for debugging your SharePoint web parts and other custom code.

1. How do you debug assemblies that are in the GAC?

2. Is there an easy way of attaching the debugger to the right process?

Debugging assemblies that live in the GAC

When debugging an assembly that lives in the GAC, you have to copy the symbol file (.pdb) from your project to the GAC. The problem is, when you open a Windows file explorer to the GAC directory, C:\Windows\assembly, it doesn't behave like a folder that you can drag-and-drop into. You must copy using another method.

To view the GAC as typical folder structure using file explorer, enter %systemroot%\assembly\gac_msil in the Run... dialog box under the start menu. This folder opens the part of the GAC that contains managed code assemblies (from this location in file explorer, you can click the folder-up button to see other parts of the GAC).

From %systemroot%\assembly\gac_msil, locate the folder having the name of your assembly, and open that. Open the folder in there having the correct version number. Finally you’ll see the dll. To debug, drag the .pdb file from your bin\debug directory into that folder. You can also use an xcopy command to do this, now that you see the full path to the dll location. This path will change if you modify your assembly's version number. Note that whenever you upgrade your solution, the .pdb file is removed from that folder, so you have to copy it there again to debug again.

Of course you want to make sure to build in debug mode. Some projects are set up with dependencies on the \release\ directory, which I think is a bad idea, but it's a fact of life sometimes. If you don't want to change your build to Debug, you can quickly change your Release build to emit debug symbols: open the properties page for the project—make sure Configuration is Active (Release)—and on the Build tab, check “Define DEBUG constant” and uncheck “Optimize code” (you may also have to click the Advanced button and select "full" symbols). Now building in release mode will include debug symbols. Be sure to change these settings back once you’re done.

Attaching the debugger to the right process

A number of people have asked me about the "Attach Debugger" menu item they see me use. This is a community feature that's been around for more than a year, but it's good to refresh people's memory of this extremely useful tool.

A SharePoint product team blog entry discusses the Attach Debugger feature that you can install for SharePoint pages. It has a link to the CodePlex project SharePoint 2007 Features that you can install to enable this feature on the Site Actions menu of your sites:

Extremely helpful! And as a bonus, it seems faster than doing it from Visual Studio's Attach Debugger command.

I recommend you look at the SharePoint 2007 Features project, it's got lots of goodies besides this.

Technorati Tags: SharePoint Development

Modifying MySite Owner Security

markarend — Thu, 10 Apr 2008 07:57:00 GMT

When someone creates a MySite, they are automatically given full control of that site. More accurately: a MySite is a site collection; the creator becomes the first & only Site Collection Administrator; the creator is made a member of the Owners group.

Some people feel that this is too much control, and they'd like to limit it in some way. My first piece of advice in this situation is: don't. MySites are built with dependencies on the permissions and identity of the owner, the person who created the site. Changing this can introduce undesirable or confusing behavior. Then, if requirements later change, after hundreds or thousands of people have created their MySite, more code must be written to effect those changes. But if you insist on going down this path, I found a way to meet this seemingly simple request. The answer is most certainly not simple.

Basically, you have to follow Steve Peschka’s approach described in the blog entry customizing My Sites as the framework for doing this. Then, add code to that scheme that knows how to change the security. There are a few added challenges:

You must do two things: 1) change the creator’s group membership and 2) remove them as a Site Collection Administrator.
To give the creator of the MySite specific privileges, you typically want to create a custom privilege level and add the user into that group. The benefit is that you can tweak this to allow/prevent actions at a fairly granular level, such as being able to block the ability to add subsites.
The creator of the MySite is the only Site Collection Administrator, so you can’t remove this person from this role until you first put someone else in there. For best governance, this should be a user account that’s not really associated with an individual, but shared among trusted admins. Unfortunately it must be a user account, not a group.
There seems to be a timer job that fixes up the title of the MySite to match the name of the site administrator… so you need some code that resets this back every time the user opens the site.

Why can’t we just change the MySite site definition (template)? Two reasons:

Modifying the MySite site definition is “unsupported” by Microsoft. If you did it, then wanted help with problems related to it, the support engineer would make a best effort, but could not guarantee a fix. Steve’s article talks more about this.
Moot point! User security is not stored in the site definition, so even if it were supported to modify it, there would be nothing to do.

Below are code samples that address the challenges listed above. Your job would be to integrate them with Steve Peschka’s code. Sorry I can’t give a more complete solution. This is one task that is very difficult to accomplish in SharePoint, and the methods described are the only way we know of doing it in a supported way.

(BTW, I use lots of try...catch blocks here to get the most granular logging. You can implement Utility.LogMessage any way you like; I use MSDN's Trace Log Example to write to the ULS logs, and CodePlex's Log Viewer Feature to read them.)

public void RestrictMySiteOwner()
{
    SPUserInfo originalOwnerInfo = new SPUserInfo();
    SPUserInfo newOwnerInfo = new SPUserInfo();

    // Store the original owner in site properties so we can restore it later if necessary
    try
    {
        StoreValueInSiteProperty(_curWeb, "OriginalOwner", MakeUserInfoString(originalOwnerInfo));
        Utility.LogMessage("Stored original administrator in site property.");
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't store original administrator in site property.  " + ex.Message);
    }

    // Switch out the primary site administrator with a pre-defined one.
    try
    {
        newOwnerInfo.LoginName = "DOMAIN\MySitesAdmin";
        newOwnerInfo.Email = "mysitesadmin@customer.com";
        newOwnerInfo.Name = "MySites domain manager";
        newOwnerInfo.Notes = "Owner of all MySites";
        originalOwnerInfo = ReplacePrimarySiteAdministrator(_curWeb, newOwnerInfo);
        Utility.LogMessage("Switched primary administrator.");
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't replace primary administrator.  " + ex.Message);
    }

    // Add the My Site creator to a group with limited permissions to control what is permissible on the site
    try
    {
        ChangeCurrentUserPermission(_curWeb, "Restricted Owner");
        Utility.LogMessage("Changed current user permissions.");
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't change current user permissions.  " + ex.Message);
    }

    // Reset the name of the site back to original owner (was changed by switching primary site admin)
    try
    {
        SetSiteTitle(_curWeb, originalOwnerInfo.Name);
        Utility.LogMessage("Reset site title.");
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't reset site title.  " + ex.Message);
    }
}


protected SPUserInfo ReplacePrimarySiteAdministrator(SPWeb site, SPUserInfo newAdminInfo)
{
    SPUser originalOwner = null;
    SPUserInfo originalOwnerInfo = new SPUserInfo();
 
    SPSecurity.RunWithElevatedPrivileges(delegate()
    {
        // Get the parent site collection
        SPSite sitecollection = site.Site;

        originalOwner = sitecollection.Owner;
        originalOwnerInfo = Utility.GetUserInfo(originalOwner);

        if (newAdminInfo.LoginName != originalOwnerInfo.LoginName)
        {
            try
            {
                // Add new admin to Full Control group
                SPRoleDefinition admins = site.RoleDefinitions["Full Control"];
                if (admins != null)
                {
                    SPRoleAssignment roleAssignment = 
                        new SPRoleAssignment(newAdminInfo.LoginName, newAdminInfo.Email, 
                        newAdminInfo.Name, newAdminInfo.Notes);
                   SPRoleDefinitionBindingCollection roleDefBindings = 
                        roleAssignment.RoleDefinitionBindings;
                    roleDefBindings.Add(admins);
                    site.RoleAssignments.Add(roleAssignment);
                    //site.Update(); // Don't need to do this
                }

                // Reset owner of site collection to new admin
                sitecollection.Owner = site.Users[newAdminInfo.LoginName];
                sitecollection.Owner.Update();

                //change the userprofile guid of the site
                sitecollection.Properties["urn:schemas-microsoft-com:sharepoint:portal:profile:userprofile_guid"] = toProfile.ID.ToString().Replace("{", string.Empty).Replace("}", string.Empty); 
                sitecollection.Properties.Update(); 

            }
            catch (SPException ex)
            {
                // We may get here if the user running this thread is the same as the 
                // aministrator we're trying to remove.
                Utility.LogMessage("Error: can't replace primary administrator.  " + ex.Message);
            }
        }
    });

    return originalOwnerInfo;
}


protected void ChangeCurrentUserPermission(SPWeb site, string roleDefinitionName)
{
    SPUser curUser = site.CurrentUser;
    string curUserName = curUser.LoginName;

    SPRoleDefinition roleDefFull = null;         // Full control privilege level
    SPRoleDefinition roleDefContributor = null;  // Contributor privilege level
    SPRoleDefinition roleDefLimitedOwner = null; // Our new privilege level
            
    // Create new role definition
    try
    {
        // Get existing role definitions
        roleDefFull = site.RoleDefinitions["Full Control"];
        roleDefContributor = site.RoleDefinitions["Contribute"];

        // Create custom role definition
        roleDefLimitedOwner = new SPRoleDefinition();
        roleDefLimitedOwner.Name = roleDefinitionName;
        roleDefLimitedOwner.Description = "Permission settings for the owner of this MySite.";
        site.FirstUniqueRoleDefinitionWeb.RoleDefinitions.Add(roleDefLimitedOwner);
        site.FirstUniqueRoleDefinitionWeb.Update();

        // Get the new role definition again... makes sure the Update took all the way or something.
        roleDefLimitedOwner = site.FirstUniqueRoleDefinitionWeb.RoleDefinitions[roleDefinitionName];

        // Set permissions of new role... start from Contributor as base.
        roleDefLimitedOwner.BasePermissions = roleDefContributor.BasePermissions;
        // To ADD a permission, use Bitwise-Or assignment:       permissions |= permission
        // To REMOVE, use Bitwise-And assignment to Complement:  permissions &= ~permission
        roleDefLimitedOwner.BasePermissions |= SPBasePermissions.CancelCheckout;
        roleDefLimitedOwner.BasePermissions &= ~SPBasePermissions.AddDelPrivateWebParts;
        roleDefLimitedOwner.BasePermissions &= ~SPBasePermissions.UpdatePersonalWebParts;
        roleDefLimitedOwner.Update();
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't create new permission group \"" 
            + roleDefinitionName + "\". " + ex.Message);
    }
            
    // Add owner to new role definition
    try
    {
        SPRoleAssignment assignment = new SPRoleAssignment(curUser);
        if (roleDefLimitedOwner != null)
        {
            assignment.RoleDefinitionBindings.Add(roleDefLimitedOwner);
        }
        else
        {
            //Workaround: for now we just make owner a contributor
            assignment.RoleDefinitionBindings.Add(roleDefContributor);
        }
        site.RoleAssignments.Add(assignment);
        //site.Update(); // Don't need to do this
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't add user \"" + curUserName 
            + "\" to new permission group \"" + roleDefinitionName + "\".  " + ex.Message);
    }

    // Remove owner from Full Control
    try
    {
        SPRoleAssignment userRoleAssignment = UserRoleAssignment(site, curUser, roleDefFull);
        if (userRoleAssignment != null)
        {
            userRoleAssignment.RoleDefinitionBindings.Remove(roleDefFull);
            userRoleAssignment.Update();
        }
        //site.Update(); // Don't need to do this
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't remove user \"" + curUserName 
            + "\" from default permission groups.  " + ex.Message);
    }
}
 
 
private SPRoleAssignment UserRoleAssignment(SPWeb site, SPUser curUser, SPRoleDefinition curRoleDef)
{
    foreach (SPRoleAssignment roleAssignment in site.RoleAssignments)
    {
        if (roleAssignment.Member.ID == curUser.ID)
        {
            foreach (SPRoleDefinition roleDefinition in roleAssignment.RoleDefinitionBindings)
            {
                if (roleDefinition.Id == curRoleDef.Id)
                {
                    return roleAssignment;
                }
            }
        }
    }
    return null;
}


protected void StoreValueInSiteProperty(SPWeb site, string propertyName, string propertyValue)
{
    if (site.Properties.ContainsKey(propertyName))
        site.Properties[propertyName] = propertyValue;
    else
        site.Properties.Add(propertyName, propertyValue);
    site.Properties.Update();
}


protected void SetSiteTitle(SPWeb site, string title)
{
    try
    {
        site.Title = title;
        //site.Update(); // Don't need to do this
    }
    catch (Exception ex)
    {
        Utility.LogMessage("Error: can't set site title to \"" + title + "\".  " + ex.Message);
    }
}

Technorati Tags: MySite,SharePoint Security,SharePoint Development

New Design and Build Guide for MOSS 2007

markarend — Wed, 12 Mar 2008 21:36:56 GMT

This is a topic near and dear to my heart. I've spent many hours helping customers plan and execute a wide range of customizations for MOSS 2007.

Now, the ECM Team (Enterprise Content Management) has published an exhaustive paper on these topics. At least, I was exhausted reading it! No, really, it's a great guide and covers so many areas that normally don't even get touched on.

Download: http://go.microsoft.com/fwlink/?LinkId=110087.

Announcement and request for feedback:
http://blogs.msdn.com/ecm/archive/2008/03/11/announcing-design-and-build-sites-for-office-sharepoint-server-2007.aspx.

Topics

Customization approaches (Basic, Intermediate, Advanced - Author centric, Advanced - Developer centric)
Site Elements (Authored elements, Developed elements)
Environments (Dev, Config mgt, Integration, Authoring, Pilot, Production)
Tools & Processes
Development team roles (Architect, Developer, Designer, Author, Tester, Admin)
Customization Scenarios (Corporate extranet, Enterprise internet)
Upgrading from SPS 2003 to MOSS 2007 (addressing customizations)
Upgrading from CMS 2002 to MOSS 2007 (pointers to resources)
Deployment (Authored, Developed)
Solution packages
Roadmaps (Sites & Subsites, Page design, Document mgt, InfoPath, Workflows, BI, Records mgt)

This is a tremendously valuable resource, go get it!

Technorati Tags: Development,MOSS 2007

Test With Correct Privileges

markarend — Fri, 07 Mar 2008 00:12:13 GMT

When developing web parts and other components with code, it's important to remember that operations performed by code may require certain privileges. During development, the code runs in the security context of the developer, who is usually a server administrator. This means all operations are permitted, so access problems don't arise. During testing and production, however, the security context of the code may be different because it's being invoked by testers and end-users.

Therefore, all components should be tested with several end-user accounts: one would be the developer’s account which is usually a server admin; that’s how most people do their unit testing. But tests should also be made with an account of minimal privilege, say Visitor. Some components might be doing things with the object model that will only work when an administrator invokes it. If this turns out to be the case, a technique called “elevation of privilege” can be used to overcome it. See MSDN article Running Commands with Elevated Privileges in Windows SharePoint Services 3.0 for an explanation of doing this.

What code needs to be tested using different privileges?

1. Code that's invoked as the result of an end-user action. An example is a custom web part that is displayed on a page. The end-user opens the page and this is what invokes the web part code under the user's security context. A counter-example is timer job code that is invoked by SharePoint.

2. Code that uses the SharePoint API. This is typically where security errors arise. If a component does not use the SharePoint API, then chances are it will not need to elevate its privilege. To be entirely sure, it should be tested at minimum privilege, but this criterion can be used to prioritize testing of other components higher.

How can a developer test with different accounts?

The simplest approach is to create three accounts "Test Visitor," "Test Member," and "Test Owner" in Active Directory and then to add these accounts to matching groups on the test sites.

First, the developer must decide in what security contexts the component is supposed to work. For example, most end-user web parts should work for users signed on as Test Visitor. On the other hand, an stsadm extension is only invoked by a server admin and should not be tested with a minimum privilege account.

"Sign on as different user" can be used to choose one of the test accounts as appropriate; however, in some cases, this approach does not seem to “stick” based on browser state or browsing behavior (e.g. browsing to My Site when browser is configured for automatic logon). The best way to ensure that the desired account is used is to configure the browser to always prompt for credentials, then to open a new browser window to begin testing. Then, re-run all unit tests that exercise the component directly (not, say, the deployment of the component).

Technorati Tags: Minimum privilege,Web part security,Code Access Security

Using Reflector to See SharePoint's "Source" Code

markarend — Fri, 29 Feb 2008 00:49:00 GMT

Important Note: Customers are advised to review the end user license agreement (EULA) for the relevant Microsoft products before using this tool. This blog entry does not authorize or endorse any use of Microsoft products that is not addressed specifically by the product's EULA.

Do you sometimes use a "reflector" program to examine SharePoint source code? This can be necessary to:

Understand out how SharePoint is doing something, to find what elements can be customized.
Understand what members to override in classes you create that inherit from SharePoint classes.

The de facto standard program for doing this is Lutz Roeder's Reflector. The MSDN article on ten must-have tools has a great description of how helpful it can be and how to use it.

One thing the article doesn't tell you is where to find SharePoint's assemblies; fortunately it's not that hard. Click "Open" in Reflector, then with a little bit of browsing, most people easily find the dlls in the ISAPI directory of the "12-hive" (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI).

Open one or more of these dlls, then browse the classes to see how SharePoint's put together… the Disassemble tool is exceedingly useful! Note: the code you see is reconstructed from MSIL, into the language of your choice (C#, VB, Delphi, C++ or Chrome)… the product teams didn't really use all those goto's!

However, some SharePoint namespaces seem to be missing or incomplete when viewed in Reflector. This had me baffled for quite some time, until I found the following.

Some SharePoint assemblies are stored in the _app_bin directory of the web application instead of the ISAPI directory in the 12-hive. Presumably, the reason for distributing certain code into the web application's root is related to something like stability, security or scalability. But the important point is if you're using Reflector to look at SharePoint code, you may need to look here as well.

This is especially true for a namespace like Microsoft.SharePoint.ApplicationPages. Two assemblies contribute members to this namespace: 12\ISAPI\Microsoft.SharePoint.dll and Inetpub\...\_app_bin\Microsoft.SharePoint.ApplicationPages.dll. If you just reflect the one in ISAPI, you'll see very few members; the ones you typically want to look at are in _app_bin!

SharePoint DLL locations

Here's a list of all the SharePoint assemblies and where they appear.

12\ISAPI

Microsoft.Office.Excel.Server.Udf.dll
Microsoft.Office.Excel.Server.WebServices.dll
Microsoft.Office.Policy.dll
Microsoft.Office.Server.dll
Microsoft.Office.Server.Search.dll
microsoft.office.workflow.tasks.dll

Microsoft.SharePoint.dll
microsoft.sharepoint.portal.dll
Microsoft.SharePoint.Portal.SingleSignon.dll
Microsoft.SharePoint.Portal.SingleSignon.Security.dll
Microsoft.SharePoint.Publishing.dll
Microsoft.SharePoint.Search.dll
Microsoft.SharePoint.Security.dll
microsoft.sharepoint.WorkflowActions.dll

Not .NET assemblies (can't reflect):
OWSSVR.DLL
SHTML.DLL
SSOCLI.DLL

Inetpub\wwwroot\wss\VirtualDirectories\<web app>\_app_bin

Microsoft.Office.DocumentManagement.Pages.dll
Microsoft.Office.officialfileSoap.dll
Microsoft.Office.Policy.Pages.dll
Microsoft.Office.SlideLibrarySoap.dll
Microsoft.Office.Workflow.Pages.dll
Microsoft.Office.WorkflowSoap.dll

Microsoft.SharePoint.ApplicationPages.dll

stssoap.dll

Obfuscation tips

Sometimes, not too often, when Reflector is disassembling source code for you, it will show "This item is obfuscated and can not be translated" instead of code. I think the product team did this for some of the more "sensitive" areas of the code (such as LogInAsAnotherUser) and also on a bit of a random basis, perhaps to prevent fully reverse engineering the product. In any case, by changing the translation language to IL in Reflector, you might get enough information from the MSIL code to answer the question that drove you there in the first place. For example, sometimes I just need to know what modules are being called, or what strings are being used to find other resources in the 12-hive; these are still visible in MSIL.

Important Note: Customers are advised to review the end user license agreement (EULA) for the relevant Microsoft products before using this tool.

Technorati Tags: SharePoint,MOSS+2007,Development

SharePoint Lists as DataTables

markarend — Sat, 26 Jan 2008 01:50:00 GMT

You may know that SPList can return a DataTable type: SPList.Items.GetDataTable(). And if you've worked with this, you may know that the DataTable you get is not databound to the list. This means you can't update the List from changes you make to the DataTable. In addition, if the List is empty, the DataTable you get back won't have any schema information. I don't know if that's a bug, but it's certainly a pain.

I wrote a code sample to overcome these problems by providing a new type: ListDataTable. Instantiate an object of this type with the URL of a list, and it behaves like a DataTable connected to the list. You can perform Insert, Update and Delete queries on the ListDataTable, then just call Update() to save those changes back to the list.

Download: http://code.msdn.microsoft.com/ListDataTable.

Important Guidelines

"Wow, I can use SharePoint like a database? Even making join relationships between lists?" Hold on, there, Skippy. Yes, it's possible, but maybe not so desirable. Consider what you want to accomplish and why. You can use Excel spreadsheets like a database with joined tables, too, but it's not usually a good idea. SharePoint and Excel are designed for user interaction, not serving data to applications... so this approach may not scale arbitrarily.

Recommendation 1: use the ListDataTable type to help you manipulate data in SharePoint lists with the power and flexibility of queries.

Recommendation 2: Don't forget Microsoft Access... it's a fantastic tool for manipulating SharePoint lists. On a list's DataSheet view, click the long, thin toolbar tab to the right of the datasheet. This opens options for linking to Access. Or, from Access, open your SharePoint lists directly. You can really clean up data in big lists in a hurry with this approach.

And Sahil Malik has another technique for treating lists like tables: creating a join using a DataView web part from SharePoint Designer 2007.

Scripting STSADM commands

markarend — Thu, 08 Nov 2007 18:02:00 GMT

This posting contains tips 'n tricks for using STSADM commands within batch files.

Note, the primary Microsoft reference for STSADM: TechNet article Stsadm command-line tool (Office SharePoint Server).

Testing results

STSADM returns 0 as ERRORLEVEL in DOS when it succeeds, or -1 when it fails. To test for success, IF %ERRORLEVEL% EQU 0 can be used.

Redirecting output

When scripting stsadm, sometimes it’s desired to redirect output to a file, using the “>” operator. This works as expected, but if stsadm displays error messages from executing its requested operation, the messages still go to the screen and not into the file. This is because DOS has two standard output devices, stdout and stderr. The > operator redirects stdout only. Fortunately, we can add another operator that will redirect stderr to stdout so that all messages go into the output file:

stsadm -o addsolution -filename MySoln.wsp >output.txt 2>&1

>& is a command redirection operator. Stderr is handle 2, and stdout is handle 1. Therefore, 2>&1 means redirect stderr to stdout. Since we've already redirected stdout to output.txt, then all messages will now go into this file.

See Microsoft article Using command redirection operators for further applications of this and related operators.

Finally, remember that >> will append messages from stdout to the specified file instead of overwriting; >>output.txt 2>&1 works to append all messages and errors.

Asynchronous commands

Some stsadm commands create a timer job to execute the operation so that it may be executed on multiple farm servers. Commands that use the following modifiers may behave this way:

-immediate Use a timer job to execute the operation on all applicable servers immediately… the job will start within one minute.

–time <time> Use a timer job to execute the operation on all applicable servers at the specified <time>.

-local Execute the operation on the local server immediately; return only after operation completes.

When using -immediate or -time, stsadm will return after creating the timer job. However, the operation itself will take a little while longer to finish its execution. This can become a problem in scripts such as

stsadm -o retractsolution -name mysolution.wsp -immediate

stsadm -o deletesolution -name mysolution.wsp

Here, the second command will fail because the retractsolution operation will not have been completed on all servers yet. The simplest fix that can be recommended is to ask for human intervention:

stsadm -o retractsolution -name mysolution.wsp -immediate

echo Wait for mysolution.wsp to be retracted across farm

pause

stsadm -o deletesolution -name mysolution.wsp

Note that the stsadm command execadmsvcjobs might seem to offer a clever solution, because it will cause outstanding deployments to complete synchronously. However, this command affects the local server only, so the same situation still exists on a multi-server farm.

Customer stsadm scenario… with a challenge

Many customers have requirements to deploy required configurations to their farm; this typically means they must deploy to several environments such as Dev, UAT and Production. They want to script these configurations whenever possible so that the process can be automated. Stsadm supplies operations to set some configurations, but not all. In addition, some of the ones that it does not cover can be very tedious to configure manually, such as search scopes, profile properties and audience definitions.

In these situations, creating an stsadm extension is a great approach to setting these configurations in an automated way. It matches the approach of scripting default stsadm commands to execute in different environments. Also, stsadm extensions can be deployed as other customizations are deployed, using SharePoint Solution files.

A flexible way to design these extensions to supply an import/export pair of operations. This way, a developer can configure their SharePoint instance to have the appropriate configurations, then use the stsadm extension to export the configuration settings as an xml file. On the UAT and Production farms, the stsadm extension can then be used to import from the xml file to make the appropriate configuration settings. This approach allows the process to be maintained throughout production, usually without requiring the developer’s involvement for further changes.

One challenge with this is deploying and executing the stsadm extension. These two operations should be performed as separate steps, because the deployment must be fully completed before attempting to execute. On a local server, a script can ensure the deployment step is complete by using the -local argument, but on a farm with multiple servers, the deployment must use -immediate, which executes as soon as possible, but asynchronously.

To do both operations in a script, it is necessary to wait for deployment explicitly, as shown in the following example:

stsadm -o addsolution -filename ImportProfileProperties.wsp

stsadm -o deploysolution -name ImportProfileProperties.wsp -immediate -allowgacdeployment

echo Wait for ImportProfileProperties.wsp to be deployed across farm

pause

stsadm -o importprofileproperties -filename RequiredProperties.xml

XSL Tips

markarend — Thu, 08 Nov 2007 17:45:00 GMT

When writing XSL, it is often desirable to be able to see the raw XML that you're trying to transform. This is especially helpful when customizing XSL for web parts such as content query. The red text below show how to modify the ContentQueryMain.xsl file to do this for a Content Query web part. This can be used in other web parts that utilize XSL transforms also.

<xsl:template match="/">
    <textarea rows="20" cols="100">
        <xsl:apply-templates/>
    </textarea>
    <xsl:call-template name="OuterTemplate" />
/xsl:template>
<xsl:template match="node()|@*">
    
        <xsl:copy>
        
        <xsl:apply-templates select="node()|@*"/>
    </xsl:copy>
</xsl:template>

MOSS Search UI Architecture - Introduction

markarend — Fri, 06 Jul 2007 23:13:00 GMT

To modify SharePoint’s default search behaviors, it’s important to understand how the user interface elements for search work together to specify a query, execute a search and display the results. The following high-level overview shows how searches are executed and displayed.

1. The search box near the upper right of most pages is a control that accepts search keywords and scope from the user, and passes them to the search results page using a query string and HTTP GET.

2. The search controls on the advanced search page accept keywords, scopes, languages and properties from the user, and passes them to the search results page using form variables and HTTP POST.

3. The search results page calls an internal class with the inputs it receives, which builds the query, issues the query against the index and receives the results. The internal class is implemented as a singleton: Microsoft.Office.Server.Search.WebControls.SearchResultHiddenObject

4. Web parts on the search results page retrieve their portion of the results from the internal class to render their specialty: core results, high confidence results, page number, summary or statistics.

This architecture provides a very powerful and flexible way to configure different types of search results. Unfortunately it is restricted to Keyword Syntax queries and is not possible to take advantage of the search engine's ability to use SQL Syntax queries, without development effort. The development effort could be rather significant, because the internal class that performs the keyword syntax search is not available for custom class derivation.

For the form fields used from the Advanced Search page, here's a table of form fields that I came across. It was constructed using reflector and reverse engineering of search and search result pages.

Forms Param	Used for	Legal values and comments
ASB_OT_	?	?
ASB_TQS_AndQ_tb	AND query text	Url encoded string
ASB_TQS_PhraseQ_tb	Phrase query text	Url encoded string
ASB_TQS_OrQ_tb	OR query text	Url encoded string
ASB_TQS_NotQ_tb	NOT query text	Url encoded string
ASB_SS_scb_<index>_<scopeidnumber>	Site Scope	<index> is the selected index number value (0 based) <scopeidnumber> is an integer value identifier the selected scope When submitted the selected properties are included and the values are "on". If a property is not used then the form data is not included.
ASB_SS_lcb_<number>_<lanuagecode>	Language scope	<number> is index number of language (0 based) <languagecode> is an know number identifier for language. All the language codes can be found in the Properties property of the advanced search web part. Forms data is not there if none are selected
ASB_SS_rtlb	Document type scope	Legal values from adv search box are: "default "= all "documents"=Documents "worddocuments"=Word documents "exceldocuments"=Excel documents "presentations"=Presentations
ASB_PS_plb_<propnumber>	Property name	<propnumber> runs from 0 to 5 Contains "(pick property)" if there is no content
ASB_PS_olb_<propnumber>	Property operator	<propnumber> runs from 0 to 5 Contains "contains" if there is no content
ASB_PS_pvtb_<propnumber>	Property value	<propnumber> runs from 0 to 5 Is null if there is no content
ASB_PS_lolb_<propnumber>	Property operator (between expressions)	And/Or boolean expressions expected Form data is included if the property expression is followed by a new property expression
ASB_TextDT_Props	Text properties	#;# seperated field
ASB_DateTimeDT_Props	DateTime properties	#;# seperated field
ASB_ResType_Query	Type of search	FullTextQuery, if form data is available otherwise Keyword query
ASB_BS_SRCH_1	The search button	Search

Parsing multi-value fields - MultiChoice, Lookup, User, URL. Rules for the ";#" delimiter.

markarend — Tue, 29 May 2007 19:39:00 GMT

For certain types of columns (aka fields), SharePoint stores several values in one field, using a delimiter to separate the different values.

Column Type	Items	Delimiter	Example
Choices, configured as Display choices using Checkboxes (allow multiple selections)	All selected choices	;#	;#First;#Second;#Third;#
Lookup	Item index, lookup value	;#	1;#First
Person or Group	Item index, user value	;#	42;#Mark Arend
Hyperlink or Picture	URL, text link	,	http://microsoft.com, Microsoft

Whenever parsing’s involved, I like to use regular expressions, even if I’m just extracting a single substring. This way, I can maintain all my parsing instructions in one place, making it easy to fix problems like missing a boundary case in my original understanding of the parse.

Below are three regular expressions that parse the above column types. I usually make these static members of a class named Util, so I can call them easily throughout my code. The code samples include commented examples of using each regular expression.

/// <summary>

/// Regular expression to isolate multi-choice values

/// </summary>

/// <example>

// System.Text.RegularExpressions.Match choice;

// string strField;

// foreach (SPListItem item in list.Items)

// {

// strField = (string)item["MultiChoiceField"];

// output.Append(item.Title + ": " + strField + "<BR>");

// if (strField != null)

// {

// choice = Util.rexMultiChoiceField.Match(strField);

// while (choice.Success)

// {

// output.Append("- " + choice.Result("$1") + "<BR>");

// choice = choice.NextMatch();

// }

///// </example>

internal static System.Text.RegularExpressions.Regex rexMultiChoiceField =

new System.Text.RegularExpressions.Regex(@"#(.+?);",

System.Text.RegularExpressions.RegexOptions.Compiled);

/// <summary>

/// Regular expression to isolate an ID ("$1") or lookup value ("$2")

/// Use this for "Lookup" fields, or for "Person or Group" fields

/// </summary>

/// <example>

/// string LookupField, LookupId, LookupValue;

/// if (Util.rexLookupField.Match(LookupField).Success)

/// {

/// LookupId = Util.rexLookupField.Match(LookupField).Result("$1");

/// LookupValue = Util.rexLookupField.Match(LookupField).Result("$2");

/// }

/// </example>

internal static System.Text.RegularExpressions.Regex rexLookupField =

new System.Text.RegularExpressions.Regex(@"(\d+);#(.*)$",

System.Text.RegularExpressions.RegexOptions.Compiled);

/// <summary>

/// Regular expression to isolate a URL ("$1") or its Description ("$2")

/// </summary>

/// <example>

/// string UrlField, UrlPath, UrlName;

/// if (Util.rexUrlField.Match(UrlField).Success)

/// {

/// UrlPath = Util.rexUrlField.Match(UrlField).Result("$1");

/// UrlName = Util.rexUrlField.Match(UrlField).Result("$2");

/// }

/// </example>

internal static System.Text.RegularExpressions.Regex rexUrlField =

new System.Text.RegularExpressions.Regex(@"^(.*), +(.*)$",

System.Text.RegularExpressions.RegexOptions.Compiled);

See also the SPListItem Class discussion in the SDK for additional samples. If you know of better details regarding these delimited value fields, please post replies. Thanks!!

Visual Studio Extensions (VSe) Deploy Failed (for SharePoint Solution)

markarend — Wed, 16 May 2007 17:32:00 GMT

When using Visual Studio 2005 extensions for Windows SharePoint Services 3.0 to deploy a solution, you may see "Deploy started" then Deploy failed" in the status window and the error "'Object reference not set to an instance of an object." in the Error List window. Typical behavior is that the solution deployment works at first, possibly even for many deployments in a row, then at some point this error begins to appear and you can't deploy the solution any more.

A workaround that fixes this problem for me is as follows:

1. Close Visual Studio 2005.

2. Uninstall (retract and remove) all deployed solutions. You only need to uninstall the ones you want to redeploy. The quickest way to uninstall is to open a command window to the directory of your Project\bin\debug folder and enter setup /uninstall. This runs faster than the solution management page in central administration (that one's performed by a timer job, which has a "now" setting but still seems to take a minute to launch itself).

3. IISRESET. Oh, iisreset, what would we do without you?

4. Open you solution in VS2005 again and Deploy Solution from the Build menu, or you can Deploy <project> to get just the ones you uninstalled in step 2.

If this doesn't work for you, a discussion at http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1165399&SiteID=1 may give some additional clues. This theorizes the Object reference error stems from the deployment code trying to enumerate features that have a null definition property, and this prompted me to try the above steps. These have worked several times now for me, so I'm hoping this is the case.

Good luck!

SPSchedule.FromString(recurrenceValue) - syntax/format for recurrence value

markarend — Wed, 17 Jan 2007 00:15:00 GMT

The SPSchedule class allows one to specify repeating schedules for timer jobs using derived classes such as SPMinuteSchedule, SPHourlySchedule, etc. Those are pretty easy to figure out from the SDK (http://msdn2.microsoft.com/en-us/library/microsoft.sharepoint.spschedule.aspx).

SPSchedule also provides a FromString(recurrenceValue) method that allows you to specify the same kinds of schedules from a text string, but allowable formats for the recurrenceValue string are not well documented. Here are legal formats I’ve discovered through testing.

Formats Used Below

Format	Description	Allowable values
m	minutes	0-59
s	seconds	0-59
ddd	abbreviated name of day of week	mon, tue, wed, thu, fri, sat, sun
d	numeric day of month	1-31
mmm	abbreviated name of month	jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec
HH:MM:SS	Time of day as hours, minutes, seconds using 24 hour-clock.	00:00:00 - 23:59:59

Minute Schedule

Syntax	Description
every Interval minutes	Repeat every Interval minutes (starting at 0 seconds)
every Interval minutes at Begin	Repeat every Interval minutes starting at Begin seconds
every Interval minutes between Begin and End	Repeat every Interval minutes starting between Begin and End seconds

Parameter	Format	Sample
Interval	m	every 15 minutes
Begin	s	every 15 minutes at 0
End	s	every 15 minutes between 0 and 15

Hourly schedule

Syntax	Description
hourly at Begin	Repeat every hour starting at Begin minutes
hourly between Begin and End	Repeat every hour starting between Begin and End minutes

Parameter	Format	Sample
Begin	m	hourly at 1
End	m	hourly between 1 and 10

Daily schedule

Syntax	Description
daily at Begin	Repeat every day starting at Begin time
daily between Begin and End	Repeat every day starting between Begin and End times

Parameter	Format	Sample
Begin	HH:MM:SS	daily at 09:00:00
End	HH:MM:SS	daily between 09:00:00 and 17:30:00

Weekly schedule

Syntax	Description
weekly at Begin	Repeat every week starting at Begin day and time
weekly between Begin and End	Repeat every week starting between Begin and End day and times

Parameter	Format	Sample
Begin	ddd HH:MM:SS	weekly at mon 09:00:00
End	ddd HH:MM:SS	weekly between mon 09:00:00 and fri 17:30:00

Monthly schedule

Syntax	Description
monthly at Begin	Repeat every month starting at Begin day of month and time
monthly between Begin and End	Repeat every month starting between Begin and End day of month and times

Parameter	Format	Sample
Begin	d HH:MM:SS	monthly at 1 09:00:00
End	d HH:MM:SS	monthly between 1 09:00:00 and 15 17:30:00

Yearly schedule

Syntax	Description
yearly at Begin	Repeat every year starting at Begin month, day of month and time
yearly between Begin and End	Repeat every year starting between Begin and End month, day of month and times

Parameter	Format	Sample
Begin	mmm d HH:MM:SS	yearly at jan 1 09:00:00
End	mmm d HH:MM:SS	yearly between feb 1 09:00:00 and apr 15 17:30:00

CAML Query for Publishing Pages

markarend — Tue, 02 Jan 2007 19:57:00 GMT

The "Scheduling Start Date" and "Scheduling End Date" on Page content types created by the Publishing feature work great in default views of Pages libraries: SharePoint filters by these dates automatically. But when you're using code to return the items from a Pages library--e.g. list.GetItems(query)--you must filter them explicitly by constructing the appropriate query.

One necessary fact may not be clear: these two date fields use a null value to indicate special dates "Immediately" and "Never" respectively (I know, the SDK says that special date values are used, but it doesn't behave this way). So, a CAML query to return all Pages by valid Scheduling dates (named Publishing dates internally), including Immediately and Never, is as follows:

<Where>

<And>

<Or>

<Leq>

</Value>

</Leq>

</IsNull>

</Or>

<Or>

<Geq>

</Value>

</Geq>

</IsNull>

</Or>

</And>

</Where>

Note, Content Query Web Part filters--the filters that you see in the task pane when you modify the web part--don’t work with these dates. But they wouldn't be that useful even if they did work, since you'd probably need four filters and only three are provided. (Also note, changing the number of filters would be extremely difficult.)

So if you want to use CQWP in conjunction with these dates, you have two choices. Of course you can use the above technique to filter by the dates using the CAML query. But this means your filters will be disabled for other purposes, and your users may not want that.

The other choice is to use these dates in the way they were intended! Anything that displays a link to a document in SharePoint, including the CQWP, will obey these publishing dates for users that don’t have the “Edit Items” permission. Since many of your users will probably need this permission, the way to set it up is to use a separate site for the items you want to show according to publishing dates. The "News" site that's created as a subsite of the portal is an excellent example of this. Now, on the News site, put everyone in your organization into the "Visitor" group, and only add news editors into the "Member" group. Then, editors will be able to see items outside of their publication dates, so they can edit them, but everyone else will only see articles that are within the publishing dates. This includes when they are displayed in the CQWP, without having to use CAML queries.