This post is to solicit your opinion about whether and how Microsoft should change the presentation of C# property declarations in SDK topics to take into account certain changes in the 3.0 version of C#.

Every property of a managed class has a reference topic in the product’s offline SDK and a corresponding MSDN page. Near the top of the page is the declaration signature of the class, in a monospaced font, as it would look in each of several languages. For example, the SPList.AllowRssFeeds topic has the following for Visual Basic and C#:

VB: Public ReadOnly Property AllowRssFeeds As Boolean

C#: public bool AllowRssFeeds { get; }

And for SPList.AlertTemplate, the topic has this:

VB: Public Property AlertTemplate As SPAlertTemplate 

C#: public SPAlertTemplate AlertTemplate { get; set; }

In reality, nearly all managed code at Microsoft is written in C#. The syntax examples for other languages only show, approximately, what the declaration would have looked like if the class had been written in that language. Notice that whether or not the property is read-only can be conveyed in the VB with that language's ReadOnly keyword. But C#'s little-known readonly keyword can be applied to fields, not properties. So to indicate whether or not a property is read-only, the declaration adds "{ get; set; }", for read/write properties, or just "{ get; }", for read-only properties.

This system has worked well even though the C# declarations were not literally correct syntax. Indeed, they worked well partly because they were not correct syntax. Since they were not valid declarations, there has been no danger that readers would think that the pseudo-declaration indicated anything about how the get and set accessors were implemented under the hood. Thus, the programming design principle of “information hiding” is preserved.

But with C# 3.0, a declaration such as this one:

A:

public type SomeProperty { get; set; }

is valid code and it would compile “as is.”  It is equivalent to the following code. In fact, it is transformed into the following code on an early pass of the compiler.

B:

private type someField;

public type SomeProperty

{

    get { return someField; }

    set { someField = value; }

}

The C# 3.0 syntax for a read-only wrapper property simply adds a “private” access modifier to the set accessor:

C:

public type SomeOtherProperty { get; private set; }

which compiles to:

D:

private type someOtherField;

public type SomeOtherProperty

{

    get { return someField; }

}

( Note by the way, that since both A and B compile to identical IL code, no code analysis tool, such as .Net Reflector or Visual Studio’s Object Browser, can determine whether the property’s source code originated as A or B. A parallel point applies to C and D.)

The problem this creates for the property topics in our SDKs is that our presentation of read/write properties now can be mistaken for the actual implementation, especially by new developers who are coming to Microsoft managed code development after the release of C# 3.0 and are not familiar with the history of the syntax conventions on these pages.

Such a misunderstanding can, in turn, lead developers to make mistaken assumptions about the property’s get and set accessors. Our current way of presenting a read/write property declaration in our SDKs implies, since the release of C# 3.0, that the property is a simple wrapper property. This, in turn, implies several things that may be false. Among them are:

• The set accessor does no validation on the input.
• The set and get accessors do not directly throw (or catch) any exceptions.
• The get accessor is simply returning a field value as distinct from calculating a value from multiple fields.

Two proposals have been made for how C# property declarations should look going forward.

Explicit Labeling Proposal:

Under this proposal, the read/write or read-only character of a property would be explicitly stated in a different font from the monospaced font of the declaration:

public type SomeProperty [read/write]

public type SomeOtherProperty [read-only]

Put the ‘Pseudo’ Back in 'Pseudo-Code' Proposal:

Under this proposal, the tradition of using pseudo-code to indicate the writeable status would be preserved, but the braces and semi-colons would be removed so that the presentation is once again not legal code.

public type SomeProperty get set

public type SomeOtherProperty get

The Office developer documentation team would like you to vote on these proposals. One way is to add a comment to this post. If you are an MVP member of the Office Content Publishing Collaboration site, you also have the option of voting on this survey page.

You may also, of course, vote to leave the property declarations just as they are, or make an alternate proposal.

One final point: Although this survey is sponsored by Office Developer Documentation, the issue is relevant to all managed code SDKs (and to the way that Visual Studio’s Object Browser presents properties). Although it will take time for all branches of the company to align on a single policy, you should vote as though you were helping to set policy for all divisions of Microsoft. It is likely that, eventually, all the affected Microsoft teams will settle on a consistent manner of presenting C# properties. In other words, do not vote against a proposal merely because it is inconsistent with the current policy of some other SDK or tool.  

(Cross-posted from my individual work blog.)

For the last several weeks, I’ve been experimenting with using mind maps, both as a way of brainstorming about developer content, and as a possible way of presenting content in a less-structured way than a typical (linear) table of contents. In regards to that last goal, I really liked how mind maps let you graphically lay out a content area, and draw non-linear relationships between various nodes. I could see how this could be a much more intuitive, discoverable way of presenting content, especially for a technology as complex and interrelated as Windows SharePoint Services. I was using an application called MindManager (from Mindjet) to create my mind maps, and was generally pretty happy with the results.

There was just one problem: in order to share a fully-functioning map with anyone else, they needed to have the same software (or at least a downloadable viewer) installed. Sure, Mind Manager let me save a given mind map as a image map or static PDF, but each of those formats ruled out one of the main selling points on the mind maps themselves: the ability to collapse/expand various nodes on the map, and thereby control the level of information you were looking at. Without that interactive ability, it really seemed like the value of using mind maps as content/resource maps was fairly restricted.

Perhaps MindJet thought the same thing, because the new version of MindManager now lets you save your mind maps as interactive PDF files. Now anyone who can view PDFs can also interact with your maps, and customize the view by collapsing/expanding the nodes of their choice.

So I’ve created an interactive mind map as another way of presenting some of the information we currently have on MSDN, and I’ve love to know what people think of it. Basically, I’ve taken some of the content links present on the Workflow Resource Center and restructured them as a mind map. Rather than present the content based on content type or source, as the Resource Center does, I’ve tried to present the content grouped to present a workflow development overview, as well as by developer tool. Icons next to each link display what type of content I’m linking to (SDK topic, blog, multimedia, etc.); the text color denotes whether the link applies just to Windows SharePoint Services (green text) or Office SharePoint Server as well (blue text).

I’m hoping you’ll take a few minutes and download the map, play with it, and let me know what you think, even if you don’t have any interest in workflow development. More than just does this specific map meet your needs, I’ve love to hear whether you think using such maps in general is a useful/worthwhile way to present developer documentation. Leave a comment after the post or ping me directly through the blog; I’d love to hear from you.

(One minor drawback I can see is how much saving as an interactive PDF bloats the size of the file: this download, of a fairly small map, is 1.5M.)

Now if we could just get them to add a ‘Save as Silverlight’ feature…

One of the things I've wanted to do on this blog is give people a clearer view into how we actually produce developer documentation for SharePoint; the processes we use, the decisions we make, and what factors influence those decisions. (The Office client developer docs team have done several great posts over on the Office Client Developer Content blog around similar topics.) With that in mind, when someone asked me the other day what we actually do with those comments people enter for our content on MSDN, I figured it was worth answering here.

The short answer is: we collect, classify, and prioritize them for action, pretty much like a software product team would triage bugs entered against their product.

And here's the long answer:

For each comment, when we initially receive it we assign a status, which is further broken down by a sub-status value as well. This enables us to separate out and focus on the comments for which some action can be taken, and group those comments in terms of what’s being requested. The status and sub-status values we use include:

·         New

This is the status automatically assigned to comments that haven’t been classified yet.

·         In progress

Comments that contain actionable content. The comment has been initially triaged and prioritized.

o   Code sample

User wants a code sample to illustrate what the topic covers.

o   Incorrect/Incomplete info

User points out that there is incorrect or incomplete information in the topic. (Not surprisingly, these are usually our first priority to fix.)

o   More Info

User is asking for more information to be added to the topic.

o   More Research Needed

We need to do more research on the comment before we can accurately categorize it.

o   New Topic Requested

User is asking for a new topic, separate from the topic where they added the comment.

·         No Action

o   Negative

Comments that are negative but not specific enough to be actionable.

o   Noise

Unintelligible comments, such as random typing.

o   Positive

Positive comment for which no action is necessary.

·         Off-topic

Comments that don't apply to the topic itself, but might be actionable at another level.

o   MSDN Issue

Comment refers to a wider MSDN issue.

o   Noise

Comments that are intelligible, but have nothing to do with the topic ("I like cheese, do you like cheese?")

o   Product Issue

Comment refers to a product issue, such as whether the user likes the feature the topic describes, rather than the documentation itself.

o   SDK Issue

Comment refers to a larger issue concerning our content set, that might be present in the topic.

·         Spam

Exactly what you'd expect: foreign financial scams, discounted prescriptions, offers to refinance our property at 1 Microsoft Way.

·         Fixed

The actionable content of the comment has been addressed. For sub-status, we leave the value at what is was when the comment was In progress.

Once we’ve got them classified, we can properly triage those comments that require action, prioritizing them along with the other content feedback we get from newsgroups, blogs, MVPs, MSDN Community Content comments, internal and external partner groups, and various other channels.

Now that I’ve outlined what we do with the comments users enter, let’s take a quick look at what those comments actually say. So what are people telling us about the WSS SDK? Taking a look at the comments entered for the WSS 3.0 SDK since its publication online, here's how they currently break out, according to status:

As you can see, 46% of the comments actually require no action beyond the initial triage. Of the actionable comments, about two-thirds are still in some state of being worked on. Like I said, these comments are prioritized in with all the content suggestions we receive from other channels as well, so that our documentation team is working on the highest priority work items, regardless of the path it took to get to us.

Looking a little closer at the comments still in progress, here's what you get when you break down those comments by sub-status:

Users are overwhelmingly (78% of the total) asking for more information (52%) or code samples (26%). This is in line with feedback we’ve been getting through other channels, and is something we’ve been actively working to address.

When you include the comments we've fixed, here are the classifications for all actionable comments we've received to date:

One other metric we periodically look at is the trend of the positive and negative comments over time. I'm really happy to see that as we periodically republish the SDK with new, expanded, and revised material, positive comments have tended to increase at a faster rate than negative ones:

We do some additional data analysis on comments, but that’s the basics. So next time you’re looking at developer documentation on MSDN, take a moment to enter a comment to let us know how we’re doing. I guarantee we’re on the other end, listening.

(cross-posted from Enterprise Search Blog) 

Are you trying to think of ways to make your Search Server site more interactive and graphical? You can use Federation as a way to enhance the functionality of your search result pages.

The Federated Search Web Part makes it possible to display more than results from OpenSearch (1.0/1.1) sites on your search results page. The Search Server 2008 SDK explains how to include results from SQL Server database queries and search sites that do not expose XML feeds (such as Atom or RSS). In both scenarios, you do this by means of a "connector," a light-weight interface that sends queries to a given location or database, places the results into a structured XML document, and sends that XML to a Federated Search Web Part.

This sample demonstrates how you can extend the connector concept to Web service requests other than basic search queries. It shows how to use a connector to pass an address string to Microsoft's MapPoint Web service in order to obtain the latitude and longitude coordinates for that address. Once you have those coordinates, displaying a Microsoft Virtual Earth map requires only the addition of some Javascript to your Federated Location definition file. See this Codeplex project for the sample code (along with a sample Federated location definition file) and an explanation of how to implement it on your own site.

This sample is part of the Search Community Toolkit.

 Jim Crowley
Programming Writer
Microsoft Corp

Introduction

If you have a custom content component that you want to be included in Windows SharePoint Server 3.0 backups and restores, you must represent the component with a class that implements the IBackupRestore interface. This post explains how to do that. There is a complete example following the procedures. This post assumes that you are familiar with my earlier post: Programming with the Windows SharePoint Services Backup/Restore Object Model.

Note: Unless explicitly stated otherwise, all classes and interfaces referred to in this post are in the Microsoft.SharePoint.Administration.Backup namespace. Classes that you will create are in bold pink. Also, this post refers at times to the “reference topic” for this or that method/property for supplementary information. As of the date of this posting these reference topics in the most recent versions of the WSS SDK (1.3), in either download or MSDN form, did not yet contain this new material. Look for another update soon. The additional information is not necessary for using this post as a learning exercise or implementing the example it contains.

Your class does not have to derive from Microsoft.SharePoint.Administration.SPPersistedObject but if your content is a database, we recommend that you derive your class from either Microsoft.SharePoint.Administration.SPDatabase or from Microsoft.SharePoint.Administration.SPContentDatabase. Both of the latter classes are derived from SPPersistedObject and both implement IBackupRestore. Therefore, you will have default implementations of members of IBackupRestore that you can use when appropriate.

You can create as many types of IBackupRestore classes you want and, if you want, they can be nested as a tree of component classes. But the highest class in any such tree must derive (directly or indirectly) from SPPersistedObject object and must be a child of Microsoft.SharePoint.Administration.SPFarm. If your content class is not a child of any other custom content class, it must derive (directly or indirectly) from SPPersistedObject object and must be a child of SPFarm.

If your class derives from a class that already implements IBackupRestore object (whether or not it derives from SPPersistedObject), and you want to replace an inherited implementation of an IBackupRestore member, your class declaration should explicitly reference IBackupRestore like this:

[C#]

public class MyClass : SPPersistedObject, IBackupRestore

Your "override" of any IBackupRestore member should explicitly include "IBackupRestore" in the member name and it should not include the public keyword. The following is an example:

[C#]

UInt64 IBackupRestore.DiskSizeRequired { ... }

Alternatively, if the implementation of the member in the parent class used the virtual or override keywords, you can use the override keyword in your implementation like this:

[C#]

public override UInt64 DiskSizeRequired { ... }

Warning: Do not hide the inherited member implementation by redeclaring the member either with or without the new keyword ([new] public UInt64 DiskSizeRequired { ... }). In the procedure below, the member signatures are written as they would be for a class that does not derive from a class that already implements IBackupRestore. Be sure to change them to the required pattern if your class does derive from such a parent.

If your class derives from SPPersistedObject, let the Id and Name properties of that class serve as the implementation of the IBackupRestore.Id and IBackupRestore.Name properties. You may override the properties, but do not create a second implementation of either of them. Your class should have just one Name and one Id property.

Procedures

Development Advice

The following tips may be helpful as you develop your custom content classes particularly because you will probably be creating objects and adding them to the farm multiple times as you work.

Example

The following code implements a custom content component that represents a single Web.config file on a front end server. Replace the TestSite part of the file path in the constructor implementation with a directory name from your test server. The compiled assembly must be strong-named and installed in the GAC.

In the example following the class implementation, there is the code for a simple console application that will register the component as a child of the farm or delete it from the farm.

using System;

using System.IO;

using System.Collections.Generic;

using Microsoft.SharePoint.Administration;

using Microsoft.SharePoint.Administration.Backup;

namespace MyCompany.SharePoint.Administration

{

    public class CriticalFiles : SPPersistedObject, IBackupRestore

    {

        public CriticalFiles() { }

        public CriticalFiles(String componentName, SPPersistedObject parent)

                   : base(componentName, parent)

        {

            String pathOfFile = @"C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\TestSite\Web.config";

            FrontEndFilePaths.Add(pathOfFile);

        }

        [Persisted]

        private const Int32 NUMBER_OF_FILES_TO_BACK_UP = 1;

        [Persisted]

        private List<String> FrontEndFilePaths = new List<String>(NUMBER_OF_FILES_TO_BACK_UP);

        public Boolean CanSelectForBackup

        {

            get { return true; }

            set { }

        }

        public Boolean CanSelectForRestore

        {

            get { return true; }

            set { }

        }

        public Boolean CanRenameOnRestore

        {

            get { return false; }

        }

        public UInt64 DiskSizeRequired

        {

            get

            {

                UInt64 total = 0;

                List<FileInfo> FrontEndFiles = new List<FileInfo>(NUMBER_OF_FILES_TO_BACK_UP);

                foreach (String path in FrontEndFilePaths)

                {

                    FileInfo file = new FileInfo(path);

                    FrontEndFiles.Add(file);

                }

                foreach (FileInfo file in FrontEndFiles)

                {

                    total = total + (UInt64)file.Length;

                }

                return total;

            }

        }

        public void AddBackupObjects(SPBackupRestoreObject parent)

        {

            if (parent == null)

            {

                throw new ArgumentNullException("parent");

            }

            SPBackupRestoreObject self = parent.AddChild(this);

            self.Information.SetParameter(SPBackupRestoreObject.SPTypeName, this.GetType());

            self.Information.SetParameter(SPBackupRestoreObject.SPDescription, "The critical files on all front end servers.");

        }

        public Boolean OnAbort(Object sender, SPBackupRestoreInformation args)

        {

            return true;

        }

        public Boolean OnPrepareBackup(Object sender, SPBackupInformation args)

        {

            if (args == null)

            {

                throw new ArgumentNullException("args");

            }

            args.SetParameter(SPBackupRestoreObject.SPName, this.Name);

            return true;

        }

        public Boolean OnBackup(Object sender, SPBackupInformation args)

        {

            if (args == null)

            {

                throw new ArgumentNullException("args");

            }

            Boolean successSignal = true;

            foreach (String path in FrontEndFilePaths)

            {

                FileInfo file = new FileInfo(path);

                try

                {

                    String mappedFileName = args.GenerateFileMapping(file.Name);

                    file.CopyTo(args.Location + @"\" + mappedFileName, true);

                    args.Log(SPBackupRestoreLogSeverity.Verbose, "Backed up " + file.Name + " in (" + mappedFileName + ")");

                }

                catch (Exception e)

                {

                    args.Log(SPBackupRestoreLogSeverity.Verbose, file.Name + " not backed u " + e.Message);

                    successSignal = false;

                }

            }

            args.CurrentProgress = 50;

            return successSignal;

        }

        public Boolean OnBackupComplete(Object sender, SPBackupInformation args)

        {

            if (args == null)

            {

                throw new ArgumentNullException("args");

            }

            args.CurrentProgress = 100;

            return true;

        }

        public Boolean OnPreRestore(Object sender, SPRestoreInformation args)

        {

            if (args == null)

            {

                throw new ArgumentNullException("args");

            }

            return true;

        }

        public Boolean OnRestore(Object sender, SPRestoreInformation args)

        {

            if (args == null)

            {

                throw new ArgumentNullException("args");

            }

            // If the CriticalFiles object was deleted from the farm after it was

            // backed up, restore it to the configuration database.

            CriticalFiles cf = SPFarm.Local.GetChild<CriticalFiles>(this.Name);

            if (cf == null)

            {

                this.Update();

                args.Log(SPBackupRestoreLogSeverity.Verbose, this.Name + " added back to configuration database.");

            }

            Boolean successSignal = true;

            // TODO: The following loop restores files to the local server. If there are

            //       multiple front end servers, your code must iterate through all of

            //       SPFarm.Local.Servers and restore the same files to every server whose

            //       Role property is SPServerRole.WebFrontEnd

            foreach (String path in FrontEndFilePaths)

            {

                FileInfo backupCopy = new FileInfo(path);

                String mappedFileName = args.ReverseFileMapping(backupCopy.Name);

                FileInfo file = new FileInfo(args.Location + @"\" + mappedFileName);

                try

                {

                    file.CopyTo(path, true);

                    args.Log(SPBackupRestoreLogSeverity.Verbose, "Restored " + backupCopy.Name);

                }

                catch (Exception e)

                {

                    args.Log(SPBackupRestoreLogSeverity.Verbose, file.Name + " not restored: " + e.Message);

                    successSignal = false;

                }

            }

            args.CurrentProgress = 50;

            return successSignal;

        }

        public Boolean OnPostRestore(Object sender, SPRestoreInformation args)

        {

            if (args == null)

            {

                throw new ArgumentNullException("args");

            }

            args.CurrentProgress = 100;

            return true;

        }

    }

}

Description

The following is a console application that will add or delete your content object from the configuration database.

using System;

using System.Collections.Generic;

using System.Text;

using Microsoft.SharePoint.Administration;

using Microsoft.SharePoint.Administration.Backup;

namespace MyCompany.SharePoint.Administration

{

    class Program

    {

        static void Main(string[] args)

        {

            CriticalFiles cf = SPFarm.Local.GetChild<CriticalFiles>("Critical Front End Files");

            if (cf == null)

            {

                Console.WriteLine("There is no CriticalFiles object in the configuration database.");

                Console.Write("Enter 'A' to add it. Press Return to do nothing:");

                String response = Console.ReadLine();

                if (response == "A")

                {

                    CriticalFiles myCriticalFiles = new CriticalFiles("Critical Front End Files", SPFarm.Local);

                    myCriticalFiles.Update();

                }

            }

            else

            {

                Console.WriteLine("There is a CriticalFiles object in the configuration database.");

                Console.Write("Enter 'D' to delete it. Press Return to do nothing:");

                String response = Console.ReadLine();

                if (response == "D")

                {

                    cf.Delete();

                }

            }

        }// end Main

    }// end Program

}

See Also

Programming with the Windows SharePoint Services Backup/Restore Object Model

How to: Programmatically Back Up Content

How to: Programmatically Restore Content

Introduction

This post describes how to back up and restore individual site collections programmatically.

Note: Unless explicitly stated otherwise, all classes referred to in this post are in the Microsoft.SharePoint.Administration or Microsoft.SharePoint namespaces (not Microsoft.SharePoint.Administration.Backup).

Procedures                         

Example

The following example shows a simple way to programmatically back up or restore a site collection. You will need to replace all the System.Guid values with actual values from your deployment and replace the placeholder values in the Backup and Restore methods with actual URLs and paths from your deployment.

// Get a reference to the Web application publishing

// Web service.

SPFarm myFarm = SPFarm.Local;

SPServiceCollection myServices = myFarm.Services;

Guid serviceID = new Guid("21d91b29-5c5b-4893-9264-4e9c758618b4");

SPWebService webPubService = (SPWebService)myServices[serviceID];

// Get a reference to the Web application that hosts the

// site collection.

SPWebApplicationCollection myApps = webPubService.WebApplications;

Guid appID = new Guid("10ea4e6f-ae37-4909-b04f-f516c066bc37");

SPWebApplication myApp = myApps[appID];

// As alternative to the preceding three lines, you can use

// the following when you know the URL of the Web application:

//     SPWebApplication myApp = SPWebApplication.Lookup(url_of_Web_app)

// Get a reference to the Web application's collection of

// site collections.

SPSiteCollection mySiteCols = myApp.Sites;

// Back up a specified site collection.

mySiteCols.Backup(@"htt//Server/sites/MySiteCollection", @"\\OtherServer\WSSBackups\SiteCollections\BackupOfMySiteCollection", true);

// Restoring the site collection is identical to the preceding

// code except that the "Restore" is used in place of "Backup".

//

// mySiteCols.Restore(@"htt//Server/sites/MySiteCollection", @"\\OtherServer\WSSBackups\SiteCollections\BackupOfMySiteCollection", true);

Comments

The Microsoft.SharePoint.SPSite class does not implement Backup.IBackupRestore and the SPSiteCollection.Backup and SPSiteCollection.Restore methods do not use the facilities of the Microsoft.SharePoint.Administration.Backup namespace. This means that records of backups and restorations of site collections are not kept in a history file (spbrtoc.xml) in the backup directory. Similarly, backup and restoration data is not stored in spbackup.xml or sprestore.xml files, neither are these site collection operations logged in spbackup.log or sprestore.log files.

If you want to do any kind of logging of backups and restorations of site collection operations, you will have to program your own system. Writing to the system-created spbrtoc.xml, spbackup.xml, sprestore.xml, spbackup.log, and sprestore.log files is not supported in Windows SharePoint Services 3.0. Neither is moving them, deleting them, or renaming them. However, you can create files that merge data from the system-created files with data from your site collection backups and restorations.

See Also

Programming with the Windows SharePoint Services Backup/Restore Object Model

How to: Programmatically Back Up Content

How to: Programmatically Restore Content

How to: Create a Content Class That Can Be Backed Up and Restored

Introduction

This post explains how to create an application that restores from a backup a content component, including a custom content component, of a Windows SharePoint Services 3.0 farm. The post assumes that you are familiar with my previous posts Overview of Backing Up and Restoring Data in Windows SharePoint Services and Programming with the Windows SharePoint Services Backup/Restore Object Model.

Note: Unless specified otherwise, all classes referred to in this post are in the Microsoft.SharePoint.Administration.Backup namespace. Classes that you create are in bold pink.

Procedures

Example

The following code shows how to program a restoration of a content component. Replace the placeholder \\Server\WSSBackups with the path of your backup location. The runtime will automatically find the most recent backup at that location.

using System;

using Microsoft.SharePoint.Administration;

using Microsoft.SharePoint.Administration.Backup;

namespace MyCompany.SharePoint.Administration.Backup

{

    class Restore

    {

        static void Main(string[] args)

        {

            // Create the restore settings.

            SPRestoreSettings settings = SPBackupRestoreSettings.GetRestoreSettings(@"\\Server\WSSBackups", "Overwrite");

            // Identify the content component to restore.

            Console.Write("Enter name of component to restore (default is whole farm):");

            settings.IndividualItem = Console.ReadLine();

            // Set optional operation parameters.

            settings.IsVerbose = true;

            settings.UpdateProgress = 10;

            // Create the restore operation and return its ID.

            Guid restore = SPBackupRestoreConsole.CreateBackupRestore(settings);

            SPBackupRestoreObject node = EnsureUniqueValidComponentName(settings, ref restore);

            if (node != null)

            {

                // Set the restore as the active job and run it.

                if (SPBackupRestoreConsole.SetActive(restore) == true)

                {

                    if (SPBackupRestoreConsole.Run(restore, node) == false)

                    {

                        // Report "error" through your UI.

                        String error = SPBackupRestoreConsole.Get(restore).FailureMessage;

                        Console.WriteLine(error);

                    }

                }

                else

                {

                    // Report through your UI that another backup

                    // or restore operation is underway.

                    Console.WriteLine("Another backup or restore operation is already underway. Try again when it ends.");

                }

                // Clean up the operation.

                SPBackupRestoreConsole.Remove(restore);

                Console.WriteLine("Restore attempt complete. Press Enter to continue.");

                Console.ReadLine();

            }

        }// end Main

        private static SPBackupRestoreObject EnsureUniqueValidComponentName(SPBackupRestoreSettings settings, ref Guid operationGUID)

        {

            SPBackupRestoreObjectCollection list = SPBackupRestoreConsole.FindItems(operationGUID, settings.IndividualItem);

            SPBackupRestoreObject component = null;

            if (list.Count <= 0)

            {

                Console.WriteLine("There is no component with that name. Run again with a new name.");

                Console.WriteLine("Press Enter to continue.");

                Console.ReadLine();

            }

            else if (list.Count > 1)  // The component name specified is ambiguous. Prompt user to be more specific.

            {

                Console.WriteLine("More than one component matches the name you entered.");

                Console.WriteLine("Run again with one of the following:");

                for (int i = 0; i < list.Count; i++)

                {

                    Console.WriteLine("\t{0}", list[i].ToString());

                }

                Console.WriteLine("Press Enter to continue.");

                Console.ReadLine();

            }

            else

            {

                component = list[0];

            }

            return component;

        }// end EnsureUniqueValidComponentName

    }// end Restore class

}// end namespace

 See Also

Programming with the Windows SharePoint Services Backup/Restore Object Model

How to: Programmatically Back Up Content

Introduction

This post explains how to program a backup of a content component, including custom components, in a Windows SharePoint Services 3.0 farm. The post assumes that you are familiar with my previous posts Overview of Backing Up and Restoring Data in Windows SharePoint Services and Programming with the Windows SharePoint Services Backup/Restore Object Model.

Note: Unless specified otherwise, all classes referred to in this post are in the Microsoft.SharePoint.Administration.Backup namespace. Classes that you create are in bold pink.

Procedures

Example

The following code shows how to program a backup of a content component.

using System;

using Microsoft.SharePoint.Administration;

using Microsoft.SharePoint.Administration.Backup;

namespace MyCompany.SharePoint.Administration.Backup

{

    class Backup

    {

        static void Main(string[] args)

        {

            // Identify the location for the backup storage.

            Console.Write("Enter full UNC path to the directory where the backup will be stored:");

            String backupLocation = Console.ReadLine();

            // Create the backup settings.

            SPBackupSettings settings = SPBackupRestoreSettings.GetBackupSettings(backupLocation, "Full");

            // Identify the content component to backup.

            Console.Write("Enter name of component to backup (default is whole farm):");

            settings.IndividualItem = Console.ReadLine();

            // Set optional operation parameters.

            settings.IsVerbose = true;

            settings.UpdateProgress = 10;

            settings.BackupThreads = 10;

            // Create the backup operation and return its ID.

            Guid backup = SPBackupRestoreConsole.CreateBackupRestore(settings);

            // Ensure that user has identified a valid and unique component.

            SPBackupRestoreObject node = EnsureUniqueValidComponentName(settings, ref backup);

            // Ensure that there is enough space.

            Boolean targetHasEnoughSpace = false;

            if (node != null)

            {

                targetHasEnoughSpace = EnsureEnoughDiskSpace(backupLocation, backup, node);

            }

            // If there is enough space, attempt to run the backup.

            if (targetHasEnoughSpace)

            {

                // Set the backup as the active job and run it.

                if (SPBackupRestoreConsole.SetActive(backup) == true)

                {

                    if (SPBackupRestoreConsole.Run(backup, node) == false)

                    {

                        // Report "error" through your UI.

                        String error = SPBackupRestoreConsole.Get(backup).FailureMessage;

                        Console.WriteLine(error);

                    }

                }

                else

                {

                    // Report through your UI that another backup

                    // or restore operation is underway.

                    Console.WriteLine("Another backup or restore operation is already underway. Try again when it ends.");

                }

                // Clean up the operation.

                SPBackupRestoreConsole.Remove(backup);

                Console.WriteLine("Backup attempt complete. Press Enter to continue.");

                Console.ReadLine();

            }

        }// end Main

        private static SPBackupRestoreObject EnsureUniqueValidComponentName(SPBackupRestoreSettings settings, ref Guid operationGUID)

        {

            SPBackupRestoreObjectCollection list = SPBackupRestoreConsole.FindItems(operationGUID, settings.IndividualItem);

            SPBackupRestoreObject component = null;

            if (list.Count <= 0)

            {

                Console.WriteLine("There is no component with that name. Run again with a new name.");

                Console.WriteLine("Press Enter to continue.");

                Console.ReadLine();

            }

            else if (list.Count > 1)  // The component name specified is ambiguous. Prompt user to be more specific.

            {

                Console.WriteLine("More than one component matches the name you entered.");

                Console.WriteLine("Run again with one of the following:");

                for (int i = 0; i < list.Count; i++)

                {

                    Console.WriteLine("\t{0}", list[i].ToString());

                }

                Console.WriteLine("Press Enter to continue.");

                Console.ReadLine();

            }

            else

            {

                component = list[0];

            }

            return component;

        }// end EnsureUniqueValidComponentName

        private static Boolean EnsureEnoughDiskSpace(String location, Guid backup, SPBackupRestoreObject node)

        {

            UInt64 backupSize = SPBackupRestoreConsole.DiskSizeRequired(backup, node);

            UInt64 diskFreeSize = 0;

            UInt64 diskSize = 0;

            Boolean hasEnoughSpace = true;

            try

            {

                SPBackupRestoreConsole.DiskSize(location, out diskFreeSize, out diskSize);

            }

            catch

            {

                diskFreeSize = diskSize = UInt64.MaxValue;

            }

            if (backupSize > diskFreeSize)

            {

                // Report through your UI that there is not enough disk space.

                Console.WriteLine("{0} bytes of space is needed but the disk hosting {1} has only {2}.", backupSize, location, diskFreeSize);

                Console.WriteLine("Please try again with a different backup location or a smaller component.");

                hasEnoughSpace = false;

            }

            else if (backupSize == UInt64.MaxValue || diskFreeSize == 0)

            {

                // Report through your UI that it cannot be determined whether there is enough disk space.

                Console.WriteLine("Cannot determine if that location has enough disk space.");

                Console.WriteLine("Please try again with a different backup location or a smaller component.");

                hasEnoughSpace = false;

            }

            return hasEnoughSpace;

        }// end EnsureEnoughDiskSpace

    }// end Backup class

}// end namespace

See Also

Programming with the Windows SharePoint Services Backup/Restore Object Model

Introduction

This post describes the architecture of the backup and restore object model in Windows SharePoint Services 3.0 and provides some advice about how to program against it. Your backup application can be an independent application or an extension of the stsadm.exe command-line tool. For more information about how to extend stsadm, see How to: Extend the STSADM Utility.

Note: All classes in this post are in the Microsoft.SharePoint.Administration.Backup namespace unless specified otherwise.

The Backup and Restore Object Model

At the top of the object model is the SPBackupRestoreConsole class. (See ①, in Figure 1.) It provides an umbrella operations manager responsible for queuing backup and restore jobs, starting jobs, and persisting a history of jobs in the form of xml files. An object of this class is "just underneath the surface" of the UI of a backup application. Its members are mainly static methods and properties that provide hooks for the UI of the backup application. Both the Central Administration application and the stsadm.exe command-line tool use these hooks.

There is always just one object of this class and it is created the first time one of its members is called. It remains in memory until the next time Internet Information Server (IIS) is reset; however, its memory footprint is small.

Figure 1: The primary objects in the backup/restore hierarchy.

Each particular backup or restore operation is represented by an object of the class SPBackupRestoreConsoleObject that is created by the SPBackupRestoreConsole.CreateBackupRestore method of the console object. (See ②, in Figure 1, the queued backup and restore operations.) The properties of one of these lightweight objects hold information about the operation such as whether it is a backup or a restore, the location of the backup files, the backup or restore method, the current stage of the operation, and the tree of content components that his being backed up or restored. These objects also have an SPBackupRestoreConsoleObject.Id property of type System.Guid that serves as a handle for the object that can be passed to the methods of SPBackupRestoreConsole. For example, SPBackupRestoreConsole.Run is passed the ID of the operation (that is; the SPBackupRestoreConsoleObject object) that the console is to run next. Finally, each SPBackupRestoreConsoleObject has a SPBackupRestoreConsoleObject.Settings property that holds a persisting object that contains a reusable pattern of backup (or restore) settings. This object also identifies the content component that the operation is to back up or restore. The settings objects are discussed in more detail near the end of this section.

While the SPBackupRestoreConsoleObject objects represent operations, the content components themselves are represented by SPBackupRestoreObject objects. (See ③, in Figure 1.) These objects can be nested with the SPBackupRestoreObject.Children property. Therefore, each one represents a tree of one or more content components. For example, a Web application would be represented by a by SPBackupRestoreObject object that would have each of the Web application's content databases as child SPBackupRestoreObject objects. The tree of components that are the subject a particular backup or restore operation is internally linked to the SPBackupRestoreConsoleObject object that represents the operation. You can get a reference to the topmost SPBackupRestoreObject object in the tree by passing the ID of the SPBackupRestoreConsoleObject object to the SPBackupRestoreConsole.GetRoot method.

A SPBackupRestoreObject object is a container for two critical types of objects:

·         In its SPBackupRestoreObject.Information property the SPBackupRestoreObject object holds either a SPBackupInformation or a SPRestoreInformation object. These kinds of objects hold information about how to back up or restore a particular component and methods that can be called internally by SPBackupRestoreConsoleObject and SPBackupRestoreObject as part of an operation. More specifically, these classes hold methods and properties whose implementation should never change regardless of what kind of content component is being represented. Hence, they are sealed and deriving a new class from their parent, SPBackupRestoreInformation, is not supported.

·         In its SPBackupRestoreObject.IBackupRestore property the SPBackupRestoreObject object holds an object that implements the IBackupRestore interface (and might also inherit from Microsoft.SharePoint.Administration.SPPersistedObject). (See ④, in Figure 1.) Implementation of the latter interface turns an object into something that can potentially be backed up and restored. Like an SPBackupRestoreInformation object, an IBackupRestore object contains members that provide information and methods needed for backups and restorations. But the members of IBackupRestore must be given different implementations depending on the kind of content component that is being represented. Most importantly, the implementation includes event handlers for an operation's events, such as IBackupRestore.OnBackup and IBackupRestore.OnRestore.

Besides acting as a container, SPBackupRestoreObject is a helper class that provides easier interaction between the operation objects (SPBackupRestoreConsoleObject), on the one hand, and the component information objects (SPBackupRestoreInformation and IBackupRestore), on the other.

The final major classes are SPBackupSettings and SPRestoreSettings. An object of one of these two types is passed to the SPBackupRestoreConsole.CreateBackupRestore method when an operation is created. (See ⑤, in Figure 1.) Each of these objects holds settings for use in operations; most importantly, they identify, in their SPBackupRestoreSettings.IndividualItem property, the content component that is the subject of the backup or restore operation. (See ⑥, in Figure 1.)

Programming Advice

Ways of Customization

Most of the critical classes in the backup restore object model are sealed (NotInheritable in Visual Basic). Moreover, although the following classes are not sealed, deriving from them is not supported. In each case, you must use the existing derived classes:

·         SPBackupRestoreInformation. (Existing derived classes: SPBackupInformation and SPRestoreInformation.)

·         SPBackupRestoreSettings. (Existing derived classes: SPBackupSettings and SPRestoreSettings.)

Accordingly, there are just two primary points of customization:

·         You can create your own high level application, and UI, that will use the static methods of the operations console — an SPBackupRestoreConsole object — to manage and run backup and restore jobs. For more information about how to do this see How to: Programmatically Back Up Content and How to: Programmatically Restore Content.

·         You can create a class that implements the IBackupRestore interface and might or might not also derive the Microsoft.SharePoint.Administration.SPPersistedObject class. For more information about how to create a custom content class, see How to: Create a Content Class That Can Be Backed Up and Restored.

Permissions

Code that backs up content must run in the user context of a farm administrator. Code that restores must run in the context of a user who is both a farm administrator and an administrator on all the front-end servers. The user should also have read and write permissions for the backup location.

Failure Logging

If a backup or restore operation fails, details about the failure will be logged in spbackup.log or sprestore.log in the backup location.

Backups and Restores of Site Collections

Aside from custom content types that you create by implementing IBackupRestore, the smallest content object that you can back up and restore with the classes in the Microsoft.SharePoint.Administration.Backup namespace is a content database. To programmatically backup or restore individual site collections, use Microsoft.SharePoint.Administration.SPSiteCollection.Backup(System.String,System.String,System.Boolean) and Microsoft.SharePoint.Administration.SPSiteCollection.Restore(System.String,System.String,System.Boolean). For more information on backing up and restoring site collections, see How to: Programmatically Backup and Restore a Single Site Collection.

Introduction

In my last two posts I discussed how third party clients can most efficiently synchronize with Windows SharePoint Services 3.0 In this article I look into extending the “Connect To…” menu item. The “Connect To…” menu item is extensible so that any client can make use of it.

Extending the “Connect To” menu item

Client applications must do two things in order for the Connect To menu item to be displayed:

1.      Register the stssync:// protocol handler

2.      Install an ActiveX control that returns the application name and an icon filename

More information about registering protocol handlers is available from http://msdn.microsoft.com/workshop/networking/pluggable/overview/appendix_a.asp.

The ActiveX control must have a ProgID of SharePoint.Stssynchandler.3 and implement the following methods:

HRESULT GetStssyncAppNameForType([in] BSTR pbstrType,  [out, retval] BSTR *pbstrAppName);

HRESULT GetStssyncIconName([out, retval] BSTR *pbstrIconName);

Before displaying the menu item, SharePoint calls the GetStssyncAppNameForType() method in the ActiveX control and passes in the current list’s sync type.  Each list has a default sync type (accessed in the object model at SPList.SyncType) based on its base type, base template, and whether it allows content types. This type can be overridden as an attribute of the list template definition in a feature.

Typically, the ActiveX control checks the sync type against a list of supported types and returns the name of the application to display in the menu item, or null if the type is not supported – in which case the menu item is not displayed.

If a non-null application name is returned, SharePoint then calls the control’s GetStssyncIconName() method to get the filename of an icon to display in the menu item.  The filename returned is used in the context menu for document library folders, while the “menu” prefix is added to the filename to get the image used in the toolbar menu item.  Thus, clients should install two images in the images path of each SharePoint web front end – foo.gif and menufoo.gif.

If the browser does not support Active X, (Windows Internet Explorer 5 and later), we display the “Connect To…” menu with a default icon and assume that the stssync protocol is supported.

When the button is clicked, JavaScript redirects the browser to an stssync:// URL with the right context parameters.  The URL is formatted as follows:

stssync://sts/?ver=1.1&type=Type&cmd=add-folder&base-url=WebUrl&list-url=ListUrl&guid=ListId&site-name=SiteName&list-name=ListName

Parameter Descriptions

Document libraries may add two parameters to the end of the stssync:// URL when syncing a folder:

&folder-url=FolderUrl&folder-id=FolderId

Acknowledgements

Once again, I would like to acknowledge Matt Swann (Microsoft Corporation) and Bill Snead (Microsoft Corporation) for their gracious help in technical reviews of this material.

 See Also

http://blogs.msdn.com/sharepointdeveloperdocs/archive/2008/01/21/synchronizing-with-windows-sharepoint-services-part-1.aspx

http://blogs.msdn.com/sharepointdeveloperdocs/archive/2008/01/22/synchronizing-with-windows-sharepoint-services-part-2.aspx

Introduction

This post provides some basic facts about backing up and restoring data in Windows SharePoint Services 3.0. It will serve as background information for several developer-oriented posts that I will create in the next couple of weeks.

Note: Unless explicitly stated otherwise, all classes and members referenced in this post are in the Microsoft.SharePoint.Administration.Backup namespace.

What Can Be Backed Up and What Can Be Restored

There are six types of content components built-in to Windows SharePoint Services 3.0 that can be backed up and restored through either the Central Administration application's UI, the Stsadm.exe Command-line Tool or a custom application that uses the Windows SharePoint Services 3.0 backup and restore object model.

·         Site collections, each of which might contain multiple Web sites.

·         Content databases, each of which might contain multiple site collections.

·         Web applications, each of which might contain multiple content databases.

·         Content publishing "Web services," each of which might contain multiple Web applications.

Note: This refers to content publishing "Web services" (which are really partitions of content) that are represented in the object model by Microsoft.SharePoint.Administration.SPWebService objects. It does not refer to the functional Web services in the more common sense of "Web service," such as the Alerts (websvcAlerts) service or the Meetings (websvcMeetings) service. For more information about content publishing "Web services," see Server and Site Architecture: Object Model Overview and The High Level Object Model of Windows SharePoint Sevices 3.0 (to be published on MSDN).

·         Search Windows service including its databases and indexes.

Note: This does not refer to the Search Web service websvcSPSearch.

·         A whole Windows SharePoint Services farm.

If an enhanced functionality product such as Office SharePoint Server 2007 has been installed in addition to Windows SharePoint Services 3.0 and that product includes Shared Service Providers, then they can also be backed up and restored with Windows SharePoint Services 3.0.

In addition, you can create new types of content objects that can be backed up and restored by implementing the IBackupRestore interface.

Limitations

There are some limitations on what can be backed up and restored through either the Central Administration application's UI, the stsadm command line utility or a custom application that uses the Windows SharePoint Services 3.0 backup and restore object model.

·         You cannot backup (or restore) an individual Web site, list, or list item except by backing up (or restoring) the entire site collection to which it belongs.

·         You cannot back up a Windows SharePoint Services farm's configuration database or the content database of the Central Administration application except by backing up the whole farm.

·         You cannot restore a farm's configuration database or the content database of the Central Administration application. The backups of these components that are included in a backup of a farm provide a snapshot of these components at the time of the backup. Such snapshots might be useful for troubleshooting because they can be used to compare with the present state of the components using SQL Server tools. (There is an exception to this point: you can restore the configuration database and the content database of the Central Administration application if you are using the Volume Shadow Copy Service – VSS - of Windows Server 2003/2008).

·         You cannot back up the Internet Information Server (IIS) metabase.

The following kinds of content cannot be backed up with either the Central Administration application's UI or the stsadm command line utility, but you can create custom backup solutions with the Windows SharePoint Services SDK that include these types of content.

·         Registry keys.

·         Files that live on the front-end servers; that is, outside any content database, such as certain master pages, .ascx files, web.config files, and other configuration files.

Types of Backups and Restorations

Backups of a given component can be either full or incremental. In the latter case, only parts of the component that have changed since the last full backup are backed up.

Note: A Windows SharePoint Services Search index cannot be incrementally backed up. If a search index is included in an incremental backup job, the index will get a full backup.

Restorations can either overwrite the original backup source or they can be to a new location. This means that the backup and restore functionality in Windows SharePoint Services 3.0 can also be used as a method of migrating content components.

Two Kinds of Custom Backup Applications

There are two ways to use the Windows SharePoint Services object model to create custom backup applications.

The Main Backup and Restore Object Model

You can create a backup and restore application by using the main backup and restore object model, sometimes called "catastrophic" backup/restore. This is located mainly in the Microsoft.SharePoint.Administration.Backup namespace; but backups and restores of individual site collections are performed with the Microsoft.SharePoint.Administration.SPSiteCollection.Backup() and Microsoft.SharePoint.Administration.SPSiteCollection.Restore() methods.

Interface to the Volume Shadow Copy Service

Windows SharePoint Services 3.0 deployments can also take advantage of the Volume Shadow Copy Service (VSS) in Windows Server 2003/2008. Windows SharePoint Services 3.0 includes a Windows SharePoint Services VSS Writer service that will create shadow copies of native and custom content in the deployment. The service contains a VSS writer that will write shadow copies of all native Windows SharePoint Services 3.0 databases and all custom databases. Non-database custom components can also be registered with the service by using the SPVssComponentDefinition and SPVssDiscoveryHelper classes. It is also necessary that you create a VSS writer for any such non-database custom components.

Note: With the VSS service, you can target only the whole farm or individual content databases for database shadow copy. Individual Web applications and individual content publishing "Web services" cannot be set for shadow copying independently of making a shadow copy of the whole farm. (For information about the meaning of "content publishing Web service" see the first note of this post.)

Introduction

In my last blog post I introduced GetListItemChangesSinceToken and discussed how using GetListItemChangesSinceToken can make synchronization more efficient. In this post I'll talk some more about synchronization; take a quick look at GetList and UpdateListItems, and property bags. I'll finish up by discussing conflict detection, and performance best practices.

Other Web Services

In addition to GetListItemChangesSinceToken, Lists.asmx defines several other web services which allow clients to query for changes and make updates. There are several good examples in the Lists web service topic in the Windows SharePoint Services SDK.

GetList

GetList returns field schemas and other list properties.  Clients typically call this before syncing a new list, and then parse the response to match WSS fields to their client-side representation. 

If the field exists as an out-of-box site column, it can be matched by its field ID.  In other cases, the field's internal name can be referenced.

UpdateListItems

UpdateListItems adds, modifies, or deletes list items.  Clients typically call this to keep server items in sync with changes made on the client.

Request

public SoapXml.SoapXmlElement UpdateListItems(string listName, SoapXml.SoapXmlElement updates)

This is the format of the updates parameter:

<Batch [update options]>

   <Method ID="X" Cmd="CMD">

      <Field Name="InternalName">VALUE</Field>

   </Method>

</Batch>

A batch is a collection of methods, each of which specifies the following value for the Cmd attribute:

List of batch methods

Note   Setting ModerationStatus must be done using a Cmd=Moderate call. Attempting to set ModerationStatus using Cmd=Update or Cmd=New has no effect. For more information about ListItem, refer to the SDK.

The ID attribute of the Method tag is only used to correlate the method in the batch with the right item in the result.

Update Options

Fields

In updates and deletes, the ID field needs to be supplied in order to identify the item.  If the update is being made on a document library, the FileRef field is also required to identify the document being updated.

When updating an item, only the changed fields need to be supplied.

When adding or changing certain types of items, certain fields may be required. 

Clients may update the property bag in two ways:

Note   There is no way for a client to delete individual name/value pairs in the property bag. See Property Bag below for more information.

Response

The return value is:

<Results>.

   <Result ID="X,CMD">

      <ErrorCode>HR</ErrorCode>

      <ID>ID</ID>

      <z:row ... />

   </Result>

</Results>

Property Bag

Property bags are mechanisms for developers to add their custom data to corresponding objects inside of SharePoint.  Property bags let developers transform simple lists into rich data stores, and webs into full applications. The property bag is a virtual container that can store almost any typed of value. The SPListItem Properties property returns a property bag for the specified object.

Note   If you use anything besides a String, int, or DateTime for the value, you will get a SPUnsupportedPropertyDataTypeException, with the message of "Only String, int, and DateTime datatypes can be used as the value in Properties."

A call to the Update method on the object persists the values set in the property bag. All values can be stored and retried by using Web service methods.

You must send the entire property bag, you cannot update just part of it.

For more information about property bags in Windows SharePoint Services, refer to http://msdn2.microsoft.com/en-us/library/ms480101.aspx.

Fields and Properties

SharePoint has very extensible list schemas.  A client that has fixed content types (calendars, tasks, etc) needs to be able to relate its own item properties to specific fields on the list.  Typically this is done by calling GetList to get the field schema before a client syncs to a SharePoint list for the first time.

Item properties in a client are best matched to fields on the server by the field GUID in the list schema.  The internal name of SharePoint fields can also be used to associate client-side properties with their server-side counterparts.  If a client cannot find an appropriate field on the server, it should store the data in the property bag for other clients to consume.  The client should cache these associations so that it doesn't have to fetch the list schema on every call.

GetListItemChangesSinceToken is a very fast call when no changes were done on the list. GetList is not quite as fast.

When a client calls GetListItemChangesSinceToken, it will receive a new list schema if the schema has changed. Unfortunately, if the schema change indicates a change in the internal name of one of the fields which are requested by name, the results of the call must be discarded. This is because the field you requested by name may not be included in the result set.

There are a few snags with respect to client use of the property bag.  If, when making an update, a client doesn't know which particular property has changed, it must update every property (even the ones not set) so that it is sure to clear the ones recently emptied.

Note   A field with an empty value is equivalent to an absent field; while an empty-value property is not.

If a new field is added to the list, values stored by the client in an equivalent property are not automatically promoted into the field value. The client must decide whether to respect the value in the property and the value in the field, possibly losing user data in the process.

Document Sync

Clients should use HTTP/DAV to sync document content. Although some SharePoint web methods support document content fetch and update (for attachments), this is not the preferred way of transferring binary document content. Because SOAP is based on XML, binary documents need to be encoded into an XML-compliant form. This is generally accomplished using hex encoding, which roughly doubles the bandwidth used.

A core field in document libraries (and also in generic lists in wssversion3short) is FileRef, which is basically a combination of two other fields associated with columns in the SQL table: DirName (server path to the containing folder) and LeafName (name of the document).

Document libraries also have two other fields computed from FileRef - ServerUrl and EncodedAbsUrl.

One of these can be used to reference a document.

On a generic list, another field (Attachments) is used to determine if the item has attachments.

Although there is a separate method to determine what these attachments are, by default there is no way to quickly determine if any attachments of an item have changed. This is what the IncludeAttachmentUrls and IncludeAttachmentVersion query options are for.

When these options are used, the value of this field should contain; #[AttachmentUrl];#[AttachmentGuid],[AttachmentVersion] for each attachment. The client can compare these values with what they stored to determine which attachments need to be re-fetched.

Because document contents can be quite big, a client can support a header-only mode that lets the user decide which document contents should be off-lined. MaxBulkDocumentSyncSize is a property that can be set on the server to guide the client when to automatically sync all contents.

Note    Some generic lists have content that can be quite large. Discussion Boards are an example. The content of a discussion item is stored in a couple of rich text fields. The header-only concept could also be used here by separate the call for the generic fields from a separate GetListItems or call for the contents. We have no examples of the best way doing this and we have not analyzed it for performance.

Conflict Detection

Besides performance considerations, discussed below, this is arguably the most important sync topic: How to detect and deal with an object that was modified on both sides.

Fetch-Send-Refresh

It is better to have conflicts detected and resolved by the client. The client can better detect that the changes didn't actually conflict, it can raise an alert for the user to manually correct the conflict, and it can store a copy of the changes applied by the local user in the user's storage.

Thus it is best if a sync operation consists first of fetching the changed data from the server, then detecting any conflicts with changes in the client copy, and finally uploading those changes if there is no conflict.

SharePoint objects may have some business logic applied at the point of the update. Because of that, the UpdateListItems method returns the updated values of all the fields and properties of the updated items.

owshiddenversion Field

wssversion3short uses this field to detect conflicts. If the field value is not supplied on update, the server will overwrite any changes. A client should always supply it on update to prevent data loss. This number should be whatever the server last sent.

This is used so that the server can tell if you're updating a stale copy of the item.  For example, a client syncs an item and gets a value of '2' for this attribute.  Someone changes the title of the item on the server, so the value increments to become 3.  When the client sends a change with value '2', the server complains because the item has been modified since the client last requested it.

When there is a conflict, the server will return a TP_E_VERSIONCONFLICT (0x81020015) error and the current contents of the item.

vti_versionhistory Property

The hidden version field is sufficient for simple conflict detection when all clients are synchronizing with a central server however; peer to peer synchronization presents further challenges. You want to avoid raising unnecessary conflicts when the change was synchronized by a peer client.

This situation may also happen in a non-peer-to-peer scenario. If a client successfully uploads a change to the server but does not receive an acknowledgment (response from UpdateListItems), the client needs a way to know that its changes were uploaded on next sync.

ETag DAV Header

Document and Attachment fetches and updates are done through HTTP/DAV. For that protocol, we have a separate mechanism for conflict detection. In every http get of a file (be it a document in a list, an attachment or a page outside a list) we return an ETag, which is supposed to be another blob that contains a guid and a version number. When uploading a document with http put, a client should request that the ETag matches the one supplied.

Note   Version history is not supported for this protocol.

Attachments

Although updating attachments can be done through HTTP/DAV, adding attachments requires using the AddAttachment method from the lists.asmx web service which takes a binary array and returns the URL of the attachment.

For more information about AddAttachment , refer to lists.asmx web service.

Performance

There are some performance issues to keep in mind when you are dealing with syncing with a server.

Latency

The amount of time it takes for an action to complete is what is important to a user. Also, there are usually limits on the amount of time allowed to process a request on a database, on a front end and by the entire request, so extremely long requests can turn into denied requests. Even despite this, you want to be able to give the users feedback on the action.  This is why paging is required.

Using the row limit property on GetListItemChangesSinceToken to limit the amount of data requested each time is crucial for the above reasons, but it should be clear that it will also increase the total amount of time to complete a sync process.

Throughput

Obviously, reducing the total amount of cycles required to process a request helps performance by reducing latency. However, with multiple clients, it is more important to reduce the adverse effects one client has on the others. Most of the time, it is easier, cleaner, safer and more effective to make the server do some processing than to implement the same processing on a multiple number of clients. However, to increase throughput, it is almost always better to do that work on the client. Although the server will likely be a lot more powerful, the client will likely have more available CPU time. A sync client should make the data request to the server be as little and simple as possible.

Bandwidth

We target high-bandwidth scenarios, but even then, it is important to try to minimize the amount of data sent across the wire.

It a client is not going to require a piece of information, it should avoid requesting it.

Paging

When performing a full sync (no change token), the client should request a maximum number of items returned per page using the rowLimit parameter. If the filtered number of items in the list is greater than that number, the server will return a ListItemCollectionPositionNext attribute to be used to request the next page.

We will only return the current change token of the list on the first page to prevent the loss of any changes being made to the first page. The client should store the change token from the first page for a subsequent incremental sync.

Secondary pages will also not include list and global properties like permissions, alternate urls and TTL.

rowLimit is also supported on incremental syncs (change token supplied), but on an incremental sync this will limit the processing of our internal change log, and we have an internal limit of 100. Although the client can be sure the number of items returned will never be greater than that limit, in certain circumstances not all changes may have been synchronized even if the number of items returned is smaller than the limit. This is because we will stop processing the change log as soon as we reach a number of updates equal to the limit. When that is the case, we return the MoreChanges attribute to indicate there are more changes in the change log. Instead of waiting for the next sync period, the client should request more changes immediately using the returned change token.

The limit works this way on incremental sync for a few reasons:

• We have an internal limit of 100 because we don't have a modified time index we can use to filter the items returned and SQL has a limit of 160 on the number of ors in a query and starts performing badly close to that number. 100 gives us a potential extra 60 as part of the filter requested by the client.
• We could have made several separate SQL queries, but that would imply supporting all ordering and filtering on the middle tier.

Because of this, we needed to return a change token that is not current, so that extra changes could be processed on a separate call. We could have still looked at the entire change log to better determine the latest point at which the number of items returned would be smaller than the limit, but even this wouldn't be accurate without filtering on the middle tier.

Filtering and Ordering

Filtering is a way to allow the user to only get a certain set of items in a list. The two most common usages of this are for folder sync, where the user only gets the items inside a folder, and for certain Group Board scenarios where the user only gets the items associated with him/her.

Filtering can be done using the contains parameter or the query parameter. Contains is more restrictive since it is basically the Where clause of a SharePoint CAML query, while query is the full query. Contains is safer to use because we can optimize certain scenarios. Query is more powerful and flexible, but the caller must understand its performance effects.

A client should avoid filtering by a non-indexed column. Otherwise, fetching a page will require a scan of the entire list until it finds the number of items requested.

A client should also avoid requesting an order unless the column of the order is indexed. Otherwise, fetching a page will at a minimum require a sort on the entire filtered dataset.

Finally, if the filter is not on the same indexed column of the order, then SQL may still scan the entire list to avoid sorting the filtered dataset.

An incremental sync has an implicit filter. We will request items with a certain ID. In this case, the client should never order by something other than ID. Filtering by something else is OK, for the dataset is restricted to a maximum of 100.

Filtering by folder can be done using the Folder query option, but the list should be ordered by the FileLeafRef. For a recursive query, it should first be ordered by FileDirRef as well.

There is also a way to filter by multiple folders using something like

"<Or><BeginsWith><FieldRef Name="FileRef"/><Value Type="Note">Shared Documents/folder1/</Value></BeginsWith><BeginsWith><FieldRef Name="FileRef"/><Value Type="Note">Shared Documents/folder2/</Value></BeginsWith></Or>".

This will synchronize the full contents of folder1 and folder2.

The client should use this in the contains parameter and add the following query option:

"<OptimizeFor>FolderUrls</OptimizeFor> "

This will make sure the SQL query is optimized appropriately by ordering it by FileDirRef, FileLeafRef and constraining the right columns.

Conclusion

The most efficient way to synchronize with Windows SharePoint Services is to download only those items that have changed since the last synchronization occurred. In wssversion3short this can be done by calling the GetListItemChangesSinceToken Web method.

GetListItemChangesSinceToken allows clients to track changes on a list.  Changes, including deleted items, are returned along with a token that represents the moment in time when those changes were requested. By including this token the next time you call GetListItemChangesSinceToken, the server looks for only those changes that have occurred since the token was generated.

We discussed considerations that the developer must keep in mind to obtain the best possible performance.

Acknowledgements

See Also

http://blogs.msdn.com/sharepointdeveloperdocs/archive/2008/01/21/synchronizing-with-windows-sharepoint-services-part-1.aspx

See Also

Introduction

There may be occasions where you will need to synchronize third party programs with Windows SharePoint Services. GetListItemChangesSinceToken, introduced in Windows SharePoint Services version 3.0, makes this synchronization much more efficient.

This post explores using GetListItemChangesSinceToken to synchronize with Windows SharePoint Services version 3.0. The article details the queryOptions used by GetListItemChangesSinceToken.

The follow on post will discuss conflict detection issues; and document best practices for clients who want to synchronize with calendar items, documents, and tasks in Windows SharePoint Services version 3.0.

Please forgive the length of this entry, the various tables and descriptions I've included will be in future updates of the SDK.

Synchronization Using GetListItemChangesSinceToken

You can use GetList to download a list; however, the most efficient way to synchronize information is to download only those items that have changed since the last synchronization occurred. In Windows SharePoint Services version 3.0 this can be done by calling the GetListItemChangesSinceToken Web method.

Sending a GetListItemChangesSinceToken request without including a token returns the list schema, the full list contents and a token. The token represents the moment when those changes were requested. By including this token the next time you call GetListItemChangesSinceToken, the server returns only those changes that have occurred since the token was generated.

If the list schema itself has changed, GetListItemChangesSinceToken returns the entire list schema, the full list contents, and a token.

#GetListItemChangesSinceToken Request and Response

Here is a listing of the GetListItemChangesSinceToken Request and Response and an explanation of the very useful queryOptions parameter.

Request

public SoapXml.SoapXmlElement GetListItemChangesSinceToken(

            string listName,

            string viewName,

            SoapXml.SoapXmlElement query,

            SoapXml.SoapXmlElement viewFields,

            string rowLimit,

            SoapXml.SoapXmlElement queryOptions,

            string changeToken,

            SoapXml.SoapXmlElement contains)

Query Options Parameter

The queryOptions element can contain a variety of tags which modify the query.  For Boolean options the default is FALSE, only a value of TRUE (must be uppercase) enables them.

Available Query Option Tags

Examples of ExpandUserField

<ExpandUserField>FALSE</ExpandUserField> looks like:

ows_Author="1;#Admin AdminName"

<ExpandUserField>TRUE</ExpandUserField> looks like:

 ows_Author="1;#Admin AdminName,#login\name,#email@address,#sip@address,#Admin AdminName "

Default Query Options

If the queryOptions parameter is absent, the following default options are used:

<RecurrenceOrderBy>TRUE</RecurrenceOrderBy>

<ViewAttributes Scope="Recursive" />

<DateInUtc>TRUE</DateInUtc>

<IncludePermissions>TRUE</IncludePermissions>

<IncludeAttachmentUrls>TRUE</IncludeAttachmentUrls>

<IncludeAttachmentVersion>TRUE</IncludeAttachmentVersion>

<RecurrencePatternXMLVersion>v3</RecurrencePatternXMLVersion>

<ExpandUserField>TRUE</ExpandUserField>

<MeetingInstanceID>-1</MeetingInstanceID>

Note   Clients should always specify a query option; failing to do so will negatively impact scale performance.

Response

A GetListItemChangesSinceToken response is delivered in the following format:

<listitems [list and global properties] [namespace declarations]>

  <Changes [MoreChanges="TRUE"] [LastChangeToken="X"]>

    [Changes]

  </Changes>

      <rs:data ItemCount="N" [ListItemCollectionPositionNext="X"]>

  [Data]

  </rs:data>

</listitems>

Note   X is an opaque token used to determine the page of items to return.  Like the changeToken value, this should never be parsed or constructed.

Namespace declarations

Any XML namespaces used below are declared here.

List and global properties

These include configuration properties, alternate URL information, and list permissions.

List and global properties table

Changes

Changes consist of a list of change events that need to be specially handled by the client.

These are taken from our internal change log when a change token is supplied. For a full synchronization (no change token) we still return the Changes tag with the current change token.

The limit on the number of updates returned from a change token is 100. The MoreChanges attribute indicates that the last change token is not current. More changes were done on the list and the client should call this method again with the new change token.

In both of the cases above, the client should ignore other changes and do a full reconciliation of the list.

Data

The ItemCount attribute contains the number of items returned. Each Item is returned as a <z:row> tag.

A ListItemCollectionPositionNext attribute is returned only for a full sync (no change token) when a row limit parameter was used to limit the number of items returned in one call. This type of paging is blocked on an incremental sync by restricting the number of updates processed from the change log to the row limit if smaller than our internal maximum.

The contents of the items are returned as attributes with an "ows_" prefix and the internal name of the field. The values are encoded to be valid XML attribute values. A set of fields is always returned.

Some fields are marked as required and returned if the IncludeMandatoryColumns option is set. If <ViewFields /> is sent, all fields are returned.

When Properties="TRUE" is set in the ViewFields tag, the property bag (MetaInfo field) is separated into an attribute per property. These have a prefix of "ows_MetaInfo_".

Most of the column values are simply converted from their internal representation, while others are constructed specially for a client. Several attributes are not represented by a field in the list schema.

Table: List of attributes not represented by a field in the list schema.

See http://blogs.msdn.com/sharepoint/archive/2007/05/14/understanding-the-sharepoint-calendar-and-how-to-export-it-to-ical-format.aspx for more details on this XML

Acknowledgements

I would like to acknowledge the following persons for their gracious help in technical reviews for this article: Matt Swann (Microsoft Corporation), Bill Snead (Microsoft Corporation).

See Also

I will be continuing this discussion in part 2.

In my post last week I talked about customizing alert notifications and alert templates. Sometimes you need to go further and create an alert handler.  This week I’d like to share a code sample from Rashid Aga, one of our escalation engineers.  His sample demonstrates how to intercept and modify alert mails using the IAlertNotifyHandler interface.

Problem:

==============

There are issues with particular field names like ItemName which will be truncated in an alert email at 70 characters. There can also be situations where you want to embed additional content in the email or change the layout and email appearance altogether.

Solution:

==============

Make use of the IAlertNotifyHandler interface to intercept the email and modify it.

Steps:

I have included the sample code below along with the steps to set up the scenario. I have formatted the output of my code to resemble the default alert template emails as close as possible, you can customize it further to suit your needs.

1      Create a class project that inhertits from the IAlertNotifyHandler interface. Include the Microsoft.SharePoint and Microsoft.SharePoint.Utilities namespaces in the project.

                    This is the code for the class:

By now you’ve no doubt heard that SP1 for Windows SharePoint Services 3.0, Office SharePoint Server 2007, and Office SharePoint Designer are now available. (And if you haven’t heard, go here to get all the important details before you download and install the Service Packs.)

For a complete listing of the documentation available for SharePoint SP1, check out the What’s New in SharePoint Server 2007 Service Pack 1 page on MSDN, which includes information resources for both WSS and MOSS SP1.

I’m happy to announce that as part of the SP1 push, we’ve again updated the WSS and MOSS SDKs on MSDN:

·         Windows SharePoint Services 3.0 SDK (December 2007 Refresh)

·         Office SharePoint Server 2007 SDK (December 2007 Refresh)

Here’s a quick run-down of additions and updates we’ve made to the WSS SDK:

New and Updated Material in the Windows SharePoint Services 3.0 SP1 SDK

New conceptual sections and topics that detail new features in Windows SharePoint Services 3.0 SP1 include:

·      ASP.NET AJAX in Windows SharePoint Services

This section provides overview information, installation instructions, and programming examples that introduce you to how Microsoft ASP.NET AJAX interacts with Web Parts. Topics include:

·         Overview: ASP.NET AJAX and Web Parts in Windows SharePoint Services 3.0

·         Installing ASP.NET 2.0 AJAX Extensions 1.0 in Windows SharePoint Services Version 3.0

·         Walkthrough: Creating a Basic ASP.NET AJAX-enabled Web Part

Other new conceptual sections and topics include:

·      Workflow Actions Schema Overview

The section discusses how to use the Workflow Actions schema to create and deploy custom workflow actions. The section includes reference topics for each of the 14 elements in the schema, as well as an example .ACTIONS file:

·         .ACTIONS File Example (WorkflowActions)

·      Alerts in Windows SharePoint Services

This section provides 8 new topics of overview information and programming tasks to help you work with Windows SharePoint Services 3.0 Alerts.

Significantly revised sections and topics include:

·      Custom Field Types

This section has been expanded with nine new topics, including the following new walkthrough:

·         Walkthrough: Creating a Custom Field Type

In addition, reference material for all base field-rendering control types, and most derived types, in Microsoft.SharePoint.WebControls has been expanded.

·      Mobile Development

This section has been expanded with 6 new conceptual topics, including the following new procedural and walkthrough topics:

·         How to: Customize Mobile Home Pages

·         How to: Customize Mobile List View and Form Pages

·         How to: Customize Field Rendering on Mobile Pages

·         Walkthrough: Customizing Item Titles on a Mobile Display Form

·         Walkthrough: Creating a Custom Field Rendering Control for Mobile Pages

In addition, reference material for all types in the Microsoft.SharePoint.MobileControls namespace has been added or expanded. The types in this namespace are now fully documented.

·      Working with Site Templates and Definitions

Several topics have been expanded.

·      All types in the Microsoft.SharePoint namespace that connected to auditing are now fully documented.

·      More than 80 code samples rewritten to adhere to the best coding practices as described in the following articles:

·         Best Practices: Common Coding Issues When Using the SharePoint Object Model

·         Best Practices: Using Disposable Windows SharePoint Services Objects

·      Greatly expanded reference material for 39 types in the Microsoft.SharePoint.Utilities namespace.

This update also includes numerous other updates and revisions to existing SDK content.

And as for the SharePoint Server 2007 SDK, we’ve added and revised quite a bit of material there as well:

New and Updated Material in the SharePoint Server 2007 SP1 SDK

New and Improved Features in Office SharePoint Server 2007 SP1

·         Support for Custom HTTP and SOAP Headers

New Conceptual Topics and How Tos

·         SharePoint Products and Technologies Customization Best Practices

·         Modeling Web Service Systems

·         Modeling Database Systems

·         Document Converter Framework Sample

·         Working with Menus and Navigation Objects

·         Modifying Navigation Settings Through the UI