Eric White's Blog : SharePoint

Fifty SharePoint 2010 End User / Site Administrator Training Videos

EricWhite — Thu, 07 Jan 2010 19:38:00 GMT

Point8020 Limited has posted fifty SharePoint 2010 end user / site admin training videos. Good videos.

Using the SharePoint 2010 Client Object Model with Open XML

EricWhite — Tue, 01 Dec 2009 13:46:00 GMT

The SharePoint 2010 Managed Client Object Model is a set of .NET managed libraries that allow us to write code for client computers to work with many of the common objects in our SharePoint sites. Programs running on the client can add and remove lists, add, update, and delete list items, modify documents in document libraries, create sites, manage permissions of items, add and remove web parts from a page, and much more.

Open XML, the document format of Office 2007 and 2010, is an ISO/IEC standard (IS29500), which describes the internals of word-processing, spreadsheet, and presentation documents. Open XML files are stored per the Open Packaging Conventions specification (Part 2 of IS29500). They are essentially ZIP files that contain XML parts inside. The Open XML SDK is a .NET managed library that makes it easy to write programs to create, modify, or query Open XML documents. At the end of this article, I’ve listed a number of resources to help you get started using the Open XML SDK 2.0.Contents

Note: This blog post is an MSDN article that I've written. It will be published sometime in the near future. It provides you with what you need to start working with documents in document libraries using the client object model. Thanks to Michael Cheng for technically reviewing this post.

Overview

Retrieving a Document from a Document Library

Modifying a Document in a Document Library

Uploading a Document to a Document Library

Creating a New Document in Memory and Adding it to a Document Library

Processing all Documents in a Library

Creating an Open XML Word-Processing Document from a SharePoint Wiki

Conclusion

Additional Resources

Overview

Using these two technologies together enables some interesting scenarios:

You can write a client-side application that creates a word-processing document or presentation from data retrieved elsewhere, and automatically uploads this document to a document library in SharePoint.
You can write an application to query all documents in a document library, extracting information from each document.
You can write an application to process all documents in a document library, making the same modifications to each one. One interesting possibility is that you can remove all comments and personal information, and accept all tracked changes in each document in the library.
You can extract the contents of a SharePoint wiki, and construct an Open XML document from those contents.

This article builds on the information presented in Using the SharePoint 2010 Managed Client Object Model. Read that post if you are not familiar with how SharePoint client object model works.

Retrieving a Document from a Document Library

There are two basic operations when working with the client object model and Open XML — retrieving a document, and saving a document. Following is a small example that retrieves a document from a document library, and prints the text of the first paragraph.

This article uses Windows console applications for the sample code, but you can use the same approach with other application types.

To build the example, you need to add references to four assemblies. Two assemblies are necessary for accessing the SharePoint Managed Client Object Model: Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll. Two assemblies are necessary for using the Open XML SDK: DocumentFormat.OpenXml.dll and WindowsBase.dll. The post Using the SharePoint 2010 Managed Client Object Model contains detailed instructions on building a console application that uses the SharePoint 2010 client object model.

To build the examples presented in this post, you will also need to download and install the Open XML SDK. The Open XML SDK documentation contains instructions on building Open XML SDK applications. Download the Open XML SDK 2.0

using System;

using System.IO;

using System.Linq;

using DocumentFormat.OpenXml.Packaging;

using DocumentFormat.OpenXml.Wordprocessing;

using Microsoft.SharePoint.Client;

// The following using directive is used to resolve the ambiguity

// between the System.IO.File and Microsoft.SharePoint.Client.File

// classes.

using ClientOM = Microsoft.SharePoint.Client;

class Program

{

static private void CopyStream(Stream source, Stream destination)

{

byte[] buffer = new byte[32768];

int bytesRead;

{

bytesRead = source.Read(buffer, 0, buffer.Length);

destination.Write(buffer, 0, bytesRead);

} while (bytesRead != 0);

}

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List sharedDocumentsList = clientContext.Web.Lists

.GetByTitle("Shared Documents");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml =

@"<View>

<Query>

<Where>

<Eq>

</Eq>

</Where>

</Query>

</View>";

ListItemCollection listItems = sharedDocumentsList.GetItems(camlQuery);

clientContext.Load(sharedDocumentsList);

clientContext.Load(listItems);

clientContext.ExecuteQuery();

if (listItems.Count == 1)

{

ClientOM.ListItem item = listItems[0];

Console.WriteLine("FileLeafRef: {0}", item["FileLeafRef"]);

Console.WriteLine("FileDirRef: {0}", item["FileDirRef"]);

Console.WriteLine("FileRef: {0}", item["FileRef"]);

Console.WriteLine("File Type: {0}", item["File_x0020_Type"]);

FileInformation fileInformation =

ClientOM.File.OpenBinaryDirect(clientContext, (string)item["FileRef"]);

using (MemoryStream memoryStream = new MemoryStream())

{

CopyStream(fileInformation.Stream, memoryStream);

using (WordprocessingDocument doc =

WordprocessingDocument.Open(memoryStream, false))

{

var firstParagraph = doc

.MainDocumentPart

.Document

.Body

.Elements<Paragraph>()

.FirstOrDefault();

if (firstParagraph != null)

{

string text = firstParagraph.InnerText;

Console.WriteLine(text);

}

else

Console.WriteLine("Document doesn't contain any paragraphs.");

}

else

Console.WriteLine("Document not found.");

}

[Download Code]

This example produces output similar to the following:

FileLeafRef: Test.docx

FileDirRef: /Shared Documents

FileRef: /Shared Documents/Test.docx

File Type: docx

This is the text of the first paragraph.

This example prints the fields that we’ll often use when working with documents in a document library:

The FileLeafRef field contains the name of the file.

The FileDirRef field contains the document library name. If the file is in a folder, FileDirRef will also contain the name of the folder.

The FileRef field contains the full name, including the document library name, any folders, and the name of the file.

The CAML query looks for an item in the document library where the FileLeafRef field has a value of Test.docx, and sets the RowLimit element of the View element to one.

camlQuery.ViewXml =

@"<View>

<Query>

<Where>

<Eq>

</Eq>

</Where>

</Query>

</View>";

Because the example sets the RowLimit element to one, we know that the query will return a ListItemCollection object with either zero or one items in it, so we can test if listItems.Count == 1 to determine if the query found the document or not.

To retrieve the file, we use the Microsoft.SharePoint.Client.File.OpenBinaryDirect method to return an object of type Microsoft.SharePoint.Client.FileInformation.

FileInformation fileInformation =

ClientOM.File.OpenBinaryDirect(clientContext, (string)item["FileRef"]);

The FileInformation object contains a reference to a stream, which we can use to stream the file to our application. However, that stream is not a resizable stream. The WordprocessingDocument.Open method takes a stream as an argument, but that stream must be a resizable stream. Therefore, we have to create a MemoryStream object, and then copy from the stream returned by the FileInformation.Stream property into the MemoryStream object. If we try to open the WordprocessingDocument object from the stream returned by the FileInformation.Stream property, the Open XML SDK will throw an exception, System.IO.IOException, with the message “Cannot open package because FileMode or FileAccess value is not valid for the stream.”

You may have noticed that it was not necessary to call the ClientContext.ExecuteQuery method before accessing the FileInformation.Stream property. This is because the File.OpenBinaryDirect method does not participate in the normal network communication of the client object model. In the article Using the SharePoint 2010 Managed Client Object Model, I explained how the client side of the client object model bundles requests as XML, the XML is sent to the server, which fulfills the requests, and then sends the response in JSON. If you were to upload files using the XML and JSON, the client object model would need to convert the binary Open XML document to Base64 encoding. Using this encoding is less efficient than directly sending the binary data. Microsoft recommends that where possible to directly send binary data. This will result in a more efficient application and less network traffic.

Once we have a MemoryStream object, which is resizable if created with the constructor that takes no parameters, then we can open the document using the Open XML SDK, and process it as usual.

The MemoryStream class contains a constructor that takes a byte array, which is then subsequently used as the backing store for the stream. However, that constructor returns a memory stream that is not resizable, so we can never use that constructor to create a memory stream for use with the Open XML SDK.

Modifying a Document in a Document Library

We can modify the previous example so that it alters the document, and places the altered document back into the document library.

using System;

using System.IO;

using System.Linq;

using DocumentFormat.OpenXml.Packaging;

using DocumentFormat.OpenXml.Wordprocessing;

using Microsoft.SharePoint.Client;

using ClientOM = Microsoft.SharePoint.Client;

class Program

{

static private void CopyStream(Stream source, Stream destination)

{

byte[] buffer = new byte[32768];

int bytesRead;

{

bytesRead = source.Read(buffer, 0, buffer.Length);

destination.Write(buffer, 0, bytesRead);

} while (bytesRead != 0);

}

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List sharedDocumentsList = clientContext.Web.Lists

.GetByTitle("Shared Documents");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml =

@"<View>

<Query>

<Where>

<Eq>

</Eq>

</Where>

</Query>

</View>";

ListItemCollection listItems =

sharedDocumentsList.GetItems(camlQuery);

clientContext.Load(sharedDocumentsList);

clientContext.Load(listItems);

clientContext.ExecuteQuery();

if (listItems.Count == 1)

{

ClientOM.ListItem item = listItems[0];

Console.WriteLine("FileLeafRef: {0}", item["FileLeafRef"]);

Console.WriteLine("FileDirRef: {0}", item["FileDirRef"]);

Console.WriteLine("FileRef: {0}", item["FileRef"]);

Console.WriteLine("File Type: {0}", item["File_x0020_Type"]);

FileInformation fileInformation =

ClientOM.File.OpenBinaryDirect(clientContext,

(string)item["FileRef"]);

using (MemoryStream memoryStream = new MemoryStream())

{

CopyStream(fileInformation.Stream, memoryStream);

using (WordprocessingDocument doc =

WordprocessingDocument.Open(memoryStream, true))

{

// Insert a new paragraph at the beginning of the

//document.

doc.MainDocumentPart.Document.Body.InsertAt(

new Paragraph(

new Run(

new Text("Newly inserted paragraph."))), 0);

}

// Seek to beginning before writing to the SharePoint server.

memoryStream.Seek(0, SeekOrigin.Begin);

// SaveBinaryDirect replaces the document on the SharePoint

// server.

ClientOM.File.SaveBinaryDirect(clientContext,

(string)item["FileRef"], memoryStream, true);

}

else

Console.WriteLine("Document not found.");

}

[Download Code]

Notice that we passed true to the WordprocessingDocument.Open method, because we are going to modify the document. Once we have the document open using the Open XML SDK, we can modify it as usual.

After exiting the scope created by the statement that opens the document (by calling the WordprocessingDocument.Open method), the MemoryStream object will contain the modified document. Before we can stream it back to the server, we need to seek to the beginning of the stream. Then, we call the File.SaveBinaryDirect method, which places the altered document back into the document library.

Uploading a Document to a Document Library

The next operation that we want to cover is uploading an existing document to a document library. We can upload a document by calling the File.SaveBinaryDirect method, passing the server relative URL, and a stream. The following example uploads a new document to the Shared Documents folder:

using System;

using System.IO;

using DocumentFormat.OpenXml.Packaging;

using DocumentFormat.OpenXml.Wordprocessing;

using Microsoft.SharePoint.Client;

// The following directive is used to avoid ambiguity between the

// System.IO.File and Microsoft.SharePoint.Client.File classes.

using ClientOM = Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

using (FileStream fileStream =

new FileStream("NewDocument.docx", FileMode.Open))

ClientOM.File.SaveBinaryDirect(clientContext,

"/Shared Documents/NewDocument.docx", fileStream, true);

}

[Download Code]

Creating a New Document in Memory and Adding it to a Document Library

There are two common ways that Open XML developers create documents:

Use an existing document as a template, open it, modify it in some desired way, and then save it to the desired location.
Create a document from scratch. Often, this is done with the help of DocumentReflector, a tool that comes with the Open XML SDK 2.0. This tool enables you to generate the C# code to create any Open XML document.

In either case, you can create the new document in a MemoryStream object. The following example reads a document into a byte array, creates a resizable memory stream using the MemoryStream constructor that takes no arguments, writes the byte array to the MemoryStream, alters that document in memory, then uploads the newly altered document to the Shared Documents library.

To run this example, create a new word-processing document named Template.docx, initialize it with any content you want (including no content at all), and place it in the bin directory where the example will run. The example inserts a new paragraph at the beginning of the document before uploading it to the document library.

using System;

using System.IO;

using DocumentFormat.OpenXml.Packaging;

using DocumentFormat.OpenXml.Wordprocessing;

using Microsoft.SharePoint.Client;

using ClientOM = Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

byte[] byteArray = System.IO.File.ReadAllBytes("Template.docx");

using (MemoryStream memoryStream = new MemoryStream())

{

memoryStream.Write(byteArray, 0, (int)byteArray.Length);

using (WordprocessingDocument doc =

WordprocessingDocument.Open(memoryStream, true))

{

// Insert a new paragraph at the beginning of the

//document.

doc.MainDocumentPart.Document.Body.InsertAt(

new Paragraph(

new Run(

new Text("Newly inserted paragraph."))), 0);

}

string fileUrl = "/Shared Documents/NewDocumentFromTemplate.docx";

memoryStream.Seek(0, SeekOrigin.Begin);

ClientOM.File.SaveBinaryDirect(clientContext, fileUrl, memoryStream, true);

}

[Download Code]

Processing all Documents in a Library

Sometimes you want to iterate through all documents in a library, performing the same query or modification on each. The document library may contain folders, each containing documents, and possibly sub-folders. The following example shows how to use the Scope=’Recursive’ attribute of the View element to process all documents in a library, including documents that are in sub-folders.

If you are processing all documents in a large document library that contains more than 2000 documents, you will want to implement a paging algorithm, as detailed in Using the SharePoint 2010 Managed Client Object Model. If you are building an interactive application that does this, you may want to use asynchronous processing, as detailed in the section of that article entitled Asynchronous Processing.

using System;

using System.IO;

using System.Linq;

using DocumentFormat.OpenXml.Packaging;

using DocumentFormat.OpenXml.Wordprocessing;

using Microsoft.SharePoint.Client;

using ClientOM = Microsoft.SharePoint.Client;

class Program

{

static private void CopyStream(Stream source, Stream destination)

{

byte[] buffer = new byte[32768];

int bytesRead;

{

bytesRead = source.Read(buffer, 0, buffer.Length);

destination.Write(buffer, 0, bytesRead);

} while (bytesRead != 0);

}

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List sharedDocumentsList = clientContext.Web.Lists

.GetByTitle("Shared Documents");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml =

@"<View Scope='Recursive'>

<Query>

<Where>

<Eq>

</Eq>

</Where>

</Query>

</View>";

ListItemCollection listItems =

sharedDocumentsList.GetItems(camlQuery);

clientContext.Load(listItems);

clientContext.ExecuteQuery();

foreach (var item in listItems)

{

Console.WriteLine("FileLeafRef: {0}", item["FileLeafRef"]);

Console.WriteLine("FileDirRef: {0}", item["FileDirRef"]);

Console.WriteLine("FileRef: {0}", item["FileRef"]);

FileInformation fileInformation =

ClientOM.File.OpenBinaryDirect(clientContext,

(string)item["FileRef"]);

using (MemoryStream memoryStream = new MemoryStream())

{

CopyStream(fileInformation.Stream, memoryStream);

using (WordprocessingDocument doc =

WordprocessingDocument.Open(memoryStream, true))

{

var firstParagraph = doc.MainDocumentPart.Document

.Body.Elements<Paragraph>().FirstOrDefault();

if (firstParagraph != null)

{

string text = firstParagraph.InnerText;

Console.WriteLine(" First paragraph text: {0}",

text);

}

else

Console.WriteLine(

"Document doesn't contain any paragraphs.");

}

Console.WriteLine();

}

[Download Code]

When processing all Open XML documents in a document library or in a folder, it is convenient to use a CAML query to filter for documents of the desired document type, “docx” in this example. The emphasized code in the CAML query in the example shows how to do this.

The PowerTools for Open XML project contains a library to accept all tracked revisions for a word-processing document. It is an easy modification to this example to create a program that will iterate through all documents in a document library, accepting changes for each. You can find the RevisionAccepter class under the Downloads tab at http://www.codeplex.com/powertools.

Creating an Open XML Word-Processing Document from a SharePoint Wiki

Using the Open XML SDK along with the client object model allows us to do some innovative things. Sometimes a SharePoint wiki user might have the need to distribute an Open XML word-processing document to people who don’t have access to that wiki. Those people may be customers or vendors who don’t have access to the corporate network, but have a need for the information contained in the wiki. It would be convenient to convert a SharePoint wiki to an Open XML document that you can send to anyone who has Microsoft Word 2007 or 2010. We can build an example program that does this using the client object model.

We will use the altChunk capabilities of Open XML to accomplish this. Open XML WordprocessingML provides the capability for importing external content through the altChunk element. To use altChunk, you do the following:

You create a new part in the package. The part can have a number of content types, including entire Open XML documents, HTML, or plain text. We’ll use the ability to import HTML in this example.
You store the content that you want to import into the part.
When you create the part, you create a relationship from the main document part to the new alternative format part.
You add a w:altChunk element at the location where you want to import the alternative format content. The r:id attribute of the w:altChunk element identifies the chunk to import. The w:altChunk element is block-level content, which means that you can insert it in the document anywhere you can insert a paragraph (w:p) element.

This example extracts the HTML for each topic in the wiki, and inserts each page into its own part, and imports each part into the document using w:altChunk.

One important point to make about altChunk is that it is used only for importing content. If you open the document using Word 2007 or Word 2010 and save it, the newly saved document will not contain the alternative format content part, nor the altChunk markup that references it. Microsoft Word 2007 or Microsoft Word 2010 will convert the HTML to WordprocessingML markup such as paragraph elements (w:p), run elements (w:r), and text elements (w:t).

First, I’ll show a small example that extracts the contents of each wiki page as HTML:

using System;

using System.Xml.Linq;

using Microsoft.SharePoint.Client;

class RetrieveListItems

{

static void Main()

{

ClientContext clientContext = new ClientContext("http://intranet.contoso.com");

List oList = clientContext.Web.Lists.GetByTitle("Eric's Wiki");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml = @"<View/>";

ListItemCollection listItems = oList.GetItems(camlQuery);

clientContext.Load(

listItems,

items => items.Include(

item => item["FileRef"],

item => item["WikiField"]));

clientContext.ExecuteQuery();

foreach (ListItem oListItem in listItems)

{

Console.WriteLine("FileRef: {0}", oListItem["FileRef"]);

XElement e = XElement.Parse((string)oListItem["WikiField"]);

Console.WriteLine(e);

Console.WriteLine("====================================");

}

[Download Code]

Change the URL that is passed to the ClientContext constructor to the URL of your SharePoint site. Change the title that is passed to the ClientContext.ListCollection.GetByTitle method to the title of a wiki in your SharePoint site.

This example uses LINQ to XML to format (indent) the XHTML that we retrieve from the wiki. When you run this example, you will see something like the following:

FileRef: /Erics Wiki/AnotherPage.aspx

<tbody>

<div>

<p>This is some content in another page.</p>

<p>

</p>

<p></p>

</div>

</td>

</tr>

</tbody>

</table>

<span id="layoutsData" style="display:none">false,false,1</span>

</div>

====================================

FileRef: /Erics Wiki/This is a longer title of a page.aspx

<tbody>

<div>

<p>This page contains some content we will extend in the future.</p>

<p></p>

</div>

</td>

</tr>

</tbody>

</table>

<span id="layoutsData" style="display:none">false,false,1</span>

</div>

As you can see, the contents of each page in the wiki are contained in a <div> element. Inside the <div> element is a table element that the wiki uses for layout. The element that we’re really interested in is the <div> element inside the table cell that has a class attribute with the value of ms-rte-layoutszone-outer. When we construct the HTML to insert in the alternative format import part, we will insert that inner <div> element into the <body> element of the HTML document. The example will then write the newly constructed HTML into the alternative content part.

This conversion does not convert wiki links to bookmarks and links within the document. In addition, there are certain characters that can be inserted in the wiki page that don’t convert properly. It is a proof-of-concept, and is intended to show the possibilities of using the client object model with Open XML. It is not intended to solve all issues associated with conversion of a SharePoint wiki to an Open XML document.

Here is the example to extract the contents of all pages of a SharePoint wiki, and create a word-processing document from them.

using System;

using System.Linq;

using System.IO;

using System.Xml.Linq;

using Microsoft.SharePoint.Client;

using ClientOM = Microsoft.SharePoint.Client;

using DocumentFormat.OpenXml.Packaging;

using DocumentFormat.OpenXml.Wordprocessing;

class RetrieveListItems

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List oList = clientContext.Web.Lists.GetByTitle("Eric's Wiki");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml = @"<View/>";

ListItemCollection listItems = oList.GetItems(camlQuery);

clientContext.Load(

listItems,

items => items.Include(

item => item["FileLeafRef"],

item => item["WikiField"]));

clientContext.ExecuteQuery();

System.IO.File.Delete("WikiDocument.docx");

System.IO.File.Copy("EmptyDocument.docx", "WikiDocument.docx");

int altChunkIdCounter = 1;

int blockLevelCounter = 0;

bool first = true;

using (WordprocessingDocument myDoc =

WordprocessingDocument.Open("WikiDocument.docx", true))

{

MainDocumentPart mainPart = myDoc.MainDocumentPart;

mainPart.Document.Body.RemoveAllChildren<Paragraph>();

foreach (ClientOM.ListItem listItem in listItems)

{

string wikiPageTitleAspx = (string)listItem["FileLeafRef"];

string wikiPageTitle = wikiPageTitleAspx.Substring(

0, wikiPageTitleAspx.Length - 5);

if (first)

{

first = false;

mainPart.Document.Body.InsertAt(new Paragraph(

new ParagraphProperties(

new RunProperties(

new RunFonts()

{

AsciiTheme = ThemeFontValues.MajorHighAnsi,

HighAnsiTheme = ThemeFontValues.MajorHighAnsi

new Bold(),

new Color()

{

Val = "1F497D",

ThemeColor = ThemeColorValues.Text2

new FontSize() { Val = 28 },

new FontSizeComplexScript() { Val = 28 }),

new ParagraphStyleId() { Val = "Heading1" }),

new Run(

new RunProperties(

new RunFonts()

{

AsciiTheme = ThemeFontValues.MajorHighAnsi,

HighAnsiTheme = ThemeFontValues.MajorHighAnsi

new Bold(),

new Color()

{

Val = "1F497D",

ThemeColor = ThemeColorValues.Text2

new FontSize() { Val = 28 },

new FontSizeComplexScript() { Val = 28 }),

new Text(wikiPageTitle))),

blockLevelCounter++);

}

else

{

mainPart.Document.Body.InsertAt(new Paragraph(

new Run(

new Break() { Type = BreakValues.Page })),

blockLevelCounter++);

mainPart.Document.Body.InsertAt(new Paragraph(

new ParagraphProperties(

new RunProperties(

new RunFonts()

{

AsciiTheme = ThemeFontValues.MajorHighAnsi,

HighAnsiTheme = ThemeFontValues.MajorHighAnsi

new Bold(),

new Color()

{

Val = "1F497D",

ThemeColor = ThemeColorValues.Text2

new FontSize() { Val = 28 },

new FontSizeComplexScript() { Val = 28 }),

new ParagraphStyleId() { Val = "Heading1" }),

new Run(

new RunProperties(

new RunFonts()

{

AsciiTheme = ThemeFontValues.MajorHighAnsi,

HighAnsiTheme = ThemeFontValues.MajorHighAnsi

new Bold(),

new Color()

{

Val = "1F497D",

ThemeColor = ThemeColorValues.Text2

new FontSize() { Val = 28 },

new FontSizeComplexScript() { Val = 28 }),

new Text(wikiPageTitle))),

blockLevelCounter++);

}

XElement wikiField = XElement.Parse((string)listItem["WikiField"]);

XElement div = wikiField.Descendants("div").Where(e =>

(string)e.Attribute("class") == "ms-rte-layoutszone-inner")

.FirstOrDefault();

string html = String.Format(

"<html><head/><body>{0}</body></html>",

div != null ? div.ToString() : "");

string altChunkId = String.Format("AltChunkId{0}",

altChunkIdCounter++);

AlternativeFormatImportPart chunk =

mainPart.AddAlternativeFormatImportPart(

AlternativeFormatImportPartType.Xhtml, altChunkId);

using (Stream chunkStream = chunk.GetStream(FileMode.Create,

FileAccess.Write))

using (StreamWriter stringStream = new StreamWriter(chunkStream))

stringStream.Write(html);

AltChunk altChunk = new AltChunk();

altChunk.Id = altChunkId;

mainPart.Document.Body.InsertAt(altChunk,

blockLevelCounter++);

}

mainPart.Document.Save();

}

[Download Code]

It provides quite a bit of functionality in less than 130 lines of code. Here is what one of the pages in the SharePoint wiki looks like:

The same page, as shown in Word 2010 looks like this:

Conclusion

The SharePoint 2010 Managed Client Object Model enables developers to write powerful client-side applications that work with SharePoint 2010 Foundation Server and SharePoint 2010 Server. The Open XML SDK provides the ability to create, modify, and query Open XML documents. When used together, they enable new ways of using Open XML documents to collaborate and share information.

Additional Resources

This article combines the use of two technologies: the Open XML SDK, and the SharePoint 2010 Managed Client Object Model. The first place to start with Open XML is the Open XML Developer Center on MSDN. There is a lot of content there, including MSDN articles, How To videos, and links to lots of blog posts. In particular, the following will help you get started with the Open XML SDK 2.0:

Download the Open XML SDK 2.0

Creating Documents by Using the Open XML Format SDK 2.0 (Part 1 of 3)

Creating Documents by Using the Open XML Format SDK 2.0 (Part 2 of 3)

Creating Documents by Using the Open XML Format SDK 2.0 (Part 3 of 3)

There are a lot of resources to help you get started using the SharePoint 2010 Managed Client Object Model. First, read in Using the SharePoint 2010 Managed Client Object Model. In addition, the following are good resources for the client object model:

Using the Client APIs

SharePoint 2010 Developer Center

How to: Work with Web Sites

How to: Work with Users and Groups

How to: Break Role Assignment Inheritance

How to: Work with User Custom Actions

Using the SharePoint 2010 Managed Client Object Model

EricWhite — Fri, 20 Nov 2009 08:07:00 GMT

The SharePoint 2010 Client Object Model is a very cool feature of SharePoint 2010 Foundation Server that enables developers to write applications that access SharePoint data from a .NET application running on a client computer, from a Silverlight application, and from JavaScript running client-side in a SharePoint web part. The client object model enables this without requiring the developer to write or install code on the SharePoint server. The client object model makes it possible to build new categories of applications that integrate with Microsoft SharePoint Foundation 2010, including writing .NET applications, rich interactive web parts, and Silverlight applications.

Note: This blog post is an MSDN article, Using the SharePoint 2010 Managed Client Object Model, that I've written. It provides you with what you need to start working with the client object model. Many thanks to Michael Cheng of the client object model team for technically reviewing this post, and providing me with many suggestions that improved the post.

This post provides the basics on using the Client Object Model. The post Using the SharePoint 2010 Client Object Model with Open XML provides additional information around uploading and downloading files from a document library, as well as examples that modify Open XML documents using the Open XML SDK.

Using the Managed Client Object Model

How It Works

Creating a Windows Console Managed Client Object Model Application

The Managed Client Object Model

Object Identity

Trimming Result Sets

Creating and Populating a List

Using CAML to Query a List

Filtering the Child Collection returned by LoadQuery using LINQ

Using the LoadQuery Method

Increasing Performance by Nesting Includes in LoadQuery

Filtering the Child Collection returned by LoadQuery using LINQ

Updating Client Objects

Deleting Client Objects

Discovering the Schema for Fields

Accessing Large Lists

Asynchronous Processing

Other Resources

Introduction

A team leader has created a SharePoint team site with numerous lists that are necessary to manage her team’s mission. She has the need to do ad-hoc bulk modifications to these lists – perhaps updating assignments and estimates based on an Open XML spreadsheet, or moving items from one SharePoint list to another. She wants to write a small custom application to help her manage this. A software company that sells a traditional rich client application wants to integrate SharePoint document libraries and lists into their application, and they want this integration to be seamless, or even invisible to their users. A SharePoint developer wants to build a rich web part for SharePoint that brings list data into their custom Ajax web code. He also wants to build an even richer Silverlight application that does the same thing.

What do these people have in common? They can make use of the SharePoint Managed Client Object Model to accomplish their goals. The SharePoint Managed Client Object Model is a capability of SharePoint 2010 that allows us to write client-side code to work with all of the common objects in our SharePoint sites. Programs running on the client can add and remove lists, add, update, and delete list items, modify documents in document libraries, create sites, manage permissions of items, add and remove web parts from a page, and much more.

Prior to the availability of the Managed Client Object Model, these people didn’t have a lot of options. They could use the SharePoint Web Services, but this can be a fairly difficult task. If the SharePoint Web Services didn’t provide the capabilities that they needed, they could write server-side code to provide a new web service (an even more difficult task). Some IT departments disallow server-side code, or allow only code that is written by the IT department, so sometimes that isn’t even an option. The SharePoint Client Object Model enables new types of applications, and makes it much easier for developers to write client-side code that interacts with their SharePoint data.

Using the Managed Client Object Model

To use the Managed Client Object Model (sometimes referred to as the client object model), you write .NET managed code that uses an API that is similar to the SharePoint Foundation Server-Side Object Model that developers use on the server. The client object model has classes for accessing site collection information, site information, list and list item information, and much more.

In the case of web parts, you use an ECMAScript programming interface that is similar to the .NET API. For Silverlight, you use a subset of the API that is available through .NET on the client. While much of the information presented in this article is relevant to the ECMAScript and Silverlight APIs, this article will focus primarily on using the client object model from a .NET client application.

The client object model consists of two assemblies containing five namespaces. If you look at the classes available in those namespaces, you will see that there are very, very many of them. Don’t worry; many of those classes are used internally by the object model. We’re interested only in a subset of them, primarily classes that have direct counterparts to some familiar classes in the SharePoint Foundation Server-Side Object Model. The following table shows a few of the classes and their corresponding classes in the server object model:

Client	Server
Microsoft.SharePoint.Client.ClientContext	Microsoft.SharePoint.SPContext
Microsoft.SharePoint.Client.Site	Microsoft.SharePoint.SPSite
Microsoft.SharePoint.Client.Web	Microsoft.SharePoint.SPWeb
Microsoft.SharePoint.Client.List	Microsoft.SharePoint.SPList
Microsoft.SharePoint.Client.ListItem	Microsoft.SharePoint.SPListItem
Microsoft.SharePoint.Client.Field	Microsoft.SharePoint.SPField

You’ll notice that the client object model uses the same legacy naming pattern for site collections and sites as the server object model. The Site class represents site collections, and the Web class represents sites. My preference for using these classes is to name the variables so that the variable name indicates whether it is a site collection or a site, even though we must use the Site and Web classes to declare them:

ClientContext clientContext = new ClientContext(siteUrl);

Site siteCollection = clientContext.Site;

Web site = clientContext.Web;

How It Works

An application that uses SharePoint data will interact with the API in a number of ways – call methods and get the return values, pass a CAML query and get the results, and set/get properties. After you use the API to accomplish a specific task the client object model bundles up all of these uses of the API into some XML that it sends to the SharePoint server. The server receives this request, and makes appropriate calls into the object model on the server, collects the responses, and forms them into Java Script Object Notation (JSON), and sends that JSON back to the client object model. The client object model parses the JSON and presents the results to the application as .NET objects (or JavaScript objects in the case of ECMAScript). The following diagram shows these interactions:

It is important to note that you control when the client object model initiates the process of sending the XML to the server and receiving the JSON back from the server. We’ll see how shortly.

The bundling of multiple method calls into a single round trip to the server are dictated by the realities of network speed, network latency, and desired performance characteristics. If the client object model sent data to the server with every method request, the performance of the system, and the increased network traffic would make the system unworkable.

As I mentioned, we must explicitly control when the client object model bundles method calls and sends a request to the server. As part of this process, before initiating the round-trip to the server, we must explicitly specify what data we want to retrieve from the server. This is the biggest difference between the client object model and the server object model. But once you understand the model, it is pretty easy. The easiest way to start understanding the difference is to see a simple application.

Creating a Windows Console Managed Client Object Model Application

This article uses Windows console applications for the sample code, but you can use the same approach with other application types.

To build the application, you need to add references to two assemblies, Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll. Installing SharePoint installs these assemblies on the SharePoint server. The two assemblies are located at:

%ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\ISAPI

Note: For the pre-release versions of SharePoint 2010 Foundation Server and SharePoint 2010 Server, you must install the assembles as described here. After the release, the procedure for installing the SharePoint 2010 Managed Client Object Model assemblies will change. I will update this post and the MSDN article with the new procedures around the time of the final release of SharePoint Server 2010.

Copy those two assemblies and place them in a convenient location on your development client computer. You will need to browse to those assemblies to add references to them when you are setting up projects that use the client object model.

To Build the Application

Start Microsoft Visual Studio 2010.

On the File menu, point to New, and then click Project.

In the New Project dialog box, in the Recent Template pane, expand Visual C#, and click Windows.

To the right of the Recent Template pane, Click Console Application.

By default, Visual Studio creates a project that targets the .NET Framework 4, but we must target the .NET Framework 3.5. From the list at the upper part of the File Open Dialog, Select .NET Framework 3.5.

In the Name box, type a name for your project, such as FirstClientApiApplication.

In the Location box, type the location where you want to place the project.

Click OK to create the solution

Add References to the Microsoft.SharePoint.Client and Microsoft.SharePoint.Client.Runtime Assemblies

The classes that you use in a client object model application are located in Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll. As I mentioned, before adding those references, you need to copy those assemblies from the SharePoint 2010 Server to your client development computer.

On the Project menu, click Add Reference to open the Add Reference Dialog box.

Select the Browse tab, navigate to the location where you placed Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll, select both DLLs, and click OK.

Add the Sample Code to the Solution

In Visual Studio, replace the contents of the Program.cs source file with the following code:

using System;

using Microsoft.SharePoint.Client;

class DisplayWebTitle

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

Web site = clientContext.Web;

clientContext.Load(site);

clientContext.ExecuteQuery();

Console.WriteLine("Title: {0}", site.Title);

}

[Download Code]

Replace the URL in the ClientContext constructor with the URL to your SharePoint site. Build and run the solution. The example prints the title of the site.

Just as with the SharePoint Foundation Server-Side Object Model, you create a context for the SharePoint site that you want to access. You can then retrieve a reference to the site from the context.

The call to ClientContext.ExecuteQuery causes the client object model to send the request to the server. There will not be any network traffic until the application calls ClientContext.ExecuteQuery.

An important point to make about this example is that the call to the Load method doesn’t actually load anything. Instead, it informs the client object model that when the application calls the ClientContext.ExecuteQuery method, you want to load the property values of the siteCollection object.

This is the model that all interactions with the server take:

You inform the client object model about the operations that you want to take. This includes accessing the values of properties of objects (for example, objects of the List, ListItem, and Web classes), CAML queries that you want to run, and objects such as ListItem objects that you want to insert, update or delete.
You then call ClientContext.ExecuteQuery. No network traffic takes place until you call ExecuteQuery. Until that point, your application is only registering its requests.

Now that you have seen a simple example that demonstrates the nature of client object model applications, i.e. you first set up a query, then you execute the query, which causes the client object model to send traffic on a round-trip to the server, let’s take a detailed look at the model, why it is designed the way it is, and how you code applications using the model.

The Managed Client Object Model

There are a number of aspects of the client object model that we must examine. There are some specific approaches that the client object model takes to minimize network traffic. There are powerful ways that we can construct queries. There are techniques that we can use to increase performance on the server. We need to see how to create, update, and delete client objects. There are approaches that we can take to work with very large lists. But before we dive into any of these subjects, we need to examine the issue of object identity.

Object Identity

The key idea behind object identity is that client objects refer to their corresponding SharePoint object both before and after calling ExecuteQuery, and they continue to refer to that same SharePoint object through multiple calls to ExecuteQuery.

This means that in the process of setting up the query, the client object model will return objects to us that we can use further in setting up the query before calling ExecuteQuery. This allows us to write more complex queries before starting the round-trip to the server. We can do more interesting things in a single query, and eliminate network traffic.

The following example gets the Announcements list object, and then gets all items of that list (using the simplest possible CAML query).

using System;

using Microsoft.SharePoint.Client;

class Program

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List list = clientContext.Web.Lists.GetByTitle("Announcements");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml = "<View/>";

ListItemCollection listItems = list.GetItems(camlQuery);

clientContext.Load(list);

clientContext.Load(listItems);

clientContext.ExecuteQuery();

foreach (ListItem listItem in listItems)

Console.WriteLine("Id: {0} Title: {1}", listItem.Id,

listItem["Title"]);

}

[Download Code]

Notice the sequence in this example:

It first gets a Microsoft.SharePoint.Client.List object using the ListCollection.GetByTitle method. Remember, this List object has no data in it; it won’t have data in any of its properties until the application calls the ExecuteQuery method.
It then calls the GetItems method on the list object, even though that list object hasn’t been populated with data.
It finally calls the Load method on both the list and listItems objects, and then calls the ExecuteQuery method.

The key point about this is that the client object model remembers that the list object is the one that the application initialized using the ListCollection.GetByTitle method, and that the client object model should execute the CAML query on that same list object after the list object has been retrieved from the SharePoint database. Any class that derives from Microsoft.SharePoint.Client.ClientObject has these semantics.

And, as mentioned, we can continue to use client objects to setup further queries after calling the ExecuteQuery method. In the following example, the code loads the list and calls the ExecuteQuery method. It then uses that list client object to call the List.GetItems method, and then calls ExecuteQuery again. The list object retained its identity through the call to ExecuteQuery.

using System;

using Microsoft.SharePoint.Client;

class Program

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List list = clientContext.Web.Lists

.GetByTitle("Announcements");

clientContext.Load(list);

clientContext.ExecuteQuery();

Console.WriteLine("List Title: {0}", list.Title);

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml = "<View/>";

ListItemCollection listItems = list.GetItems(camlQuery);

clientContext.Load(listItems);

clientContext.ExecuteQuery();

foreach (ListItem listItem in listItems)

Console.WriteLine("Id: {0} Title: {1}",

listItem.Id, listItem["Title"]);

}

[Download Code]

Some properties and methods return objects or value types that do not derive from the Microsoft.SharePoint.Client.ClientObject class. We benefit from using client object identity to access methods and properties only when those methods and properties return client objects or collections of them. For instance, some classes, such as FieldUrlValue and FieldLookupValue derive from the ClientValueObject class, and we can’t make use of properties that return those types until after the call to the ExecuteQuery method. Some properties return .NET types such as string or integer, and we also can’t use properties or methods that return those until after the call to the ExecuteQuery method. Since we can’t use the values of any properties until those values have been populated in the ExecuteQuery call, we can’t, for instance, find an item in a list, and use the value of one of that item’s fields to select items in a further query. If we try to use a property before it has been populated by the ExecuteQuery method, the client object model will throw a PropertyOrFieldNotInitializedException.

Important note: Client object identity is valid only for a single ClientContext object. If we initialize another ClientContext object to the same SharePoint site, we can’t use client objects from one client context with the other one.

We’ll make use of object identity behavior in a number of the examples that I present in this article.

This example doesn’t do any error handling. If the Announcements list doesn’t exist, the client object model will throw an exception in the call to the ExecuteQuery method. You should be prepared to catch exceptions when you write code that may fail if you ask for objects that may not exist.

Trimming Result Sets

SharePoint 2010 Server is often deployed in organizations with many thousands of users. When building an application that accesses SharePoint 2010 Server across the network, it makes sense to build it so that that it uses the least amount of network traffic. There are a number of ways that the client object model helps us do this. The simplest approach is to use lambda expressions to specify exactly which properties the client object model should return to the application.

The following example shows how to specify that when the client object model loads the site object, it needs to load only the Title and Description properties. This will reduce the size of the JSON response from the server back to the client object model.

using System;

using Microsoft.SharePoint.Client;

class Program

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

Web site = clientContext.Web;

clientContext.Load(site,

s => s.Title,

s => s.Description);

clientContext.ExecuteQuery();

Console.WriteLine("Title: {0} Description: {1}",

site.Title, site.Description);

}

[Download Code]

If we don’t include these lambda expressions in the call to the Load method, then by default it loads a much larger number of properties (but not all of them). The first two examples called Load without specifying which properties to load, so the JSON packet that the server returned was somewhat larger than it needed to be. While in these small examples, it doesn’t make much difference, when loading thousands of list items, carefully specifying the required properties will reduce network traffic.

This use of lambda expressions provides a means for us to specify a list of our desired .NET properties to the Load method. But reducing network traffic isn’t the only benefit we derive from the client object model’s use of lambda expressions. Later on in this article, we’ll see how we can filter result sets using lambda expressions.

Next I'll show an example that creates a list and add some data to it. Then we’ll have some sample data to work with for the rest of this article.

Creating and Populating a List

The following example creates a list, and adds a couple of fields and a few items to it.

using System;

using Microsoft.SharePoint.Client;

class Program

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

Web site = clientContext.Web;

// Create a new list.

ListCreationInformation listCreationInfo =

new ListCreationInformation();

listCreationInfo.Title = "Client API Test List";

listCreationInfo.TemplateType = (int)ListTemplateType.GenericList;

List list = site.Lists.Add(listCreationInfo);

// Add fields to the list.

Field field1 = list.Fields.AddFieldAsXml(

@"<Field Type='Choice'

DisplayName='Category'

Format='Dropdown'>

<Default>Specification</Default>

<CHOICE>Specification</CHOICE>

<CHOICE>Development</CHOICE>

<CHOICE>Documentation</CHOICE>

</CHOICES>

</Field>",

true, AddFieldOptions.DefaultValue);

Field field2 = list.Fields.AddFieldAsXml(

@"<Field Type='Number'

DisplayName='Estimate'/>",

true, AddFieldOptions.DefaultValue);

// Add some data.

ListItemCreationInformation itemCreateInfo =

new ListItemCreationInformation();

ListItem listItem = list.AddItem(itemCreateInfo);

listItem["Title"] = "Write specs for user interface.";

listItem["Category"] = "Specification";

listItem["Estimate"] = "20";

listItem.Update();

listItem = list.AddItem(itemCreateInfo);

listItem["Title"] = "Develop proof-of-concept.";

listItem["Category"] = "Development";

listItem["Estimate"] = "42";

listItem.Update();

listItem = list.AddItem(itemCreateInfo);

listItem["Title"] = "Write test plan for user interface.";

listItem["Category"] = "Test";

listItem["Estimate"] = "16";

listItem.Update();

listItem = list.AddItem(itemCreateInfo);

listItem["Title"] = "Validate SharePoint interaction.";

listItem["Category"] = "Test";

listItem["Estimate"] = "18";

listItem.Update();

listItem = list.AddItem(itemCreateInfo);

listItem["Title"] = "Develop user interface.";

listItem["Category"] = "Development";

listItem["Estimate"] = "18";

listItem.Update();

clientContext.ExecuteQuery();

}

[Download Code]

In many cases, where it is possible to create a client object, the application can call an Add method that takes as an argument an object that specifies creation information. In this example, you can see the use of the ListCreationInformation class for creating a List object, and the use of the ListItemCreationInformation class for creating a ListItem object. You often will set properties of the creation information class after instantiating it. You can see that the code sets the Title and TemplateType properties of the ListItemCreationInformation object. Note that to create a list, you call the ListCollection.Add method, but to create a ListItem, you call the List.AddItem method. One is on the collection, and the other is on the singleton.

Creating fields in a list also doesn’t use a method named Add that takes a FieldCreationInformation object as an argument, because when we create fields, we are not really creating an instance of the Microsoft.SharePoint.Client.Field class; we are creating an instance of a class that derives from Field. There are many, many options available for those derived classes, and this would significantly complicate the design of a FieldCreationInformation class; for this reason, the client object model doesn’t include such a class. Instead, the simplest way to create a field is to specify a little bit of XML that defines the field, and pass that XML to the FieldCollection.AddFieldAsXml method. There is a FieldCollection.Add method that we can use to create a field, but instead of taking a FieldCreationInformation object, it takes another Field object as a parameter that it uses as a prototype for the field to be created. This is useful in some scenarios.

In the Discovering the Schema for Fields section of this article, I’ll show you an easy way to discover the XML that you need to specify for fields that you want to create.

It’s important to note that, of course, no objects are actually added to the SharePoint database until the application calls ExecuteQuery.

There is one more item of interest in this example. Notice that after calling the List.AddItem method, the example sets three indexed properties. We are setting the values of the fields that were just added to the list. After setting these properties, the application must call the ListItem.Update method, informing the client object model that those objects have been modified. The client object model will not work properly if you don’t do so. We will see the use of the Update method in further examples, when I show how to modify existing client objects.

Now that we have some data, let’s discover some interesting ways to query and modify it.

Using CAML to Query a List

The following example shows how to query the list that we created in the last example using CAML. This example prints the Development items from our test list.

using System;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List list = clientContext.Web.Lists

.GetByTitle("Client API Test List");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml =

@"<View>

<Query>

<Where>

<Eq>

<Value Type='Text'>Development</Value>

</Eq>

</Where>

</Query>

</View>";

ListItemCollection listItems = list.GetItems(camlQuery);

clientContext.Load(

listItems,

items => items

.Include(

item => item["Title"],

item => item["Category"],

item => item["Estimate"]));

clientContext.ExecuteQuery();

foreach (ListItem listItem in listItems)

{

Console.WriteLine("Title: {0}", listItem["Title"]);

Console.WriteLine("Category: {0}", listItem["Category"]);

Console.WriteLine("Estimate: {0}", listItem["Estimate"]);

Console.WriteLine();

}

[Download Code]

This example produces the following output:

Title: Develop proof-of-concept.

Category: Development

Estimate: 42

Title: Develop user interface.

Category: Development

Estimate: 18

You may notice that there is a difference between the lambda expressions that we specify in this example and the lambda expressions in the example presented in the Trimming Result Sets section. We must use the ClientObjectQueryableExtension.Include extension method to specify the properties that we want to load for each item in the collection that we’re loading. The items parameter of the lambda expression is of type ListItemCollection, which of course doesn’t contain an indexed property that allows us to specify which properties to load for items in the collection. Instead, we call the Include extension method, which allows us to specify which parameters of that child collection to load. Parameters to lambda expressions in the Include extension method are of the type of the items of the collection, so this allows us to specify the properties that we want to load for each item in the collection.

Again, it isn’t entirely necessary to understand the exact semantics of this use of lambda expressions. Just remember two coding idioms:

If you are requesting that the client object model load certain properties of a client object (not a collection of them), then specify the properties in the lambda expression that you place directly in the Load method:

clientContext.Load(site,

s => s.Title,

s => s.Description);

If you are requesting that the client object model load certain properties of each of the items in a collection of client objects, then use the Include extension method, and pass the lambda expressions that specify your desired properties to the Include method:

clientContext.Load(

listItems,

items => items

.Include(

item => item["Title"],

item => item["Category"],

item => item["Estimate"]));

Note: I'm currently working on a blog post on exactly how lambda expressions work in the client object model. Now THAT is going to be a geeky post! J

Filtering the Child Collection returned by Load using LINQ

Because the ClientObjectQueryableExtension.Include extension method returns IQueryable<T>, we can chain from the Include method into the IQueryable<T>.Where extension method. This provides a succinct way to filter the result set. We should only use this capability when querying client object collections other than collections of ListItem objects, because while we could use this technique to filter collections of ListItem objects, the use of CAML will result in better performance. This is important enough to say again:

Never use the IQueryable<T>.Where extension method when querying ListItem objects. The reason is that under the covers, the client object model first evaluates the result of the CAML query, retrieves the results, and then filters the resulting collection using LINQ. If you filter an extremely large list using LINQ instead of CAML, behind the scenes, the client object model will attempt to retrieve all items in the list before filtering with LINQ and will either be issuing queries that will take too much in the way of system resources, or perhaps will not succeed, and the reason won't be evident unless you know how the client object model works internally. So we need to continue to use CAML when querying list items. There are some interesting ways that we can make CAML easier to use. But that's another blog post.

The following example queries the client object model for all lists that are not hidden. Notice that we must include a using directive for System.Linq.

using System;

using System.Linq;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

ListCollection listCollection = clientContext.Web.Lists;

clientContext.Load(

listCollection,

lists => lists

.Include(

list => list.Title,

list => list.Hidden)

.Where(list => ! list.Hidden)

);

clientContext.ExecuteQuery();

foreach (var list in listCollection)

Console.WriteLine(list.Title);

}

[Download Code]

On my computer, this example produces the following output:

Announcements

Calendar

Client API Test List

Content and Structure Reports

Customized Reports

Eric's ToDo List

Eric's Wiki

Form Templates

Links

Reusable Content

Shared Documents

Site Assets

Site Collection Documents

Site Collection Images

Site Pages

Style Library

Tasks

Team Discussion

Workflow Tasks

Using the LoadQuery Method

The LoadQuery method is similar in functionality to the Load method, except that in certain circumstances, the client object model can process the queries more efficiently and use memory more efficiently. And it also allows for a more flexible programming style.

LoadQuery has different semantics than the Load method. Whereas the Load method populates the client object (or client object collection) with data from the server, the LoadQuery method populates and returns an entirely new collection. This means that you can query the same object collection multiple times and retain separate result sets for each query. For instance, you can query for all items in a project list that are assigned to a certain person, and separately query for all items that have an estimated hours that is greater than a certain threshold, and access both result sets simultaneously. It also allows you to let these collections go out of scope, and thereby become eligible for garbage collection. Collections that you load using the Load method can be garbage collected only when the client context variable itself goes out of scope. Other than these distinctions, the LoadQuery method is very similar to the Load method.

The following example uses LoadQuery to retrieve a list of all the lists in the site.

using System;

using System.Collections.Generic;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

Web site = clientContext.Web;

ListCollection lists = site.Lists;

IEnumerable<List> newListCollection = clientContext.LoadQuery(

lists.Include(

list => list.Title,

list => list.Id,

list => list.Hidden));

clientContext.ExecuteQuery();

foreach (List list in newListCollection)

Console.WriteLine("Title: {0} Id: {1}",

list.Title.PadRight(40), list.Id.ToString("D"));

}

[Download Code]

Notice that the LoadQuery method returns an entirely new list collection that we can iterate through. The new list collection has a type of IEnumerable<List> instead of ListCollection.

There is one aspect of the semantics of LoadQuery that you need to pay attention to. In the above example, the original lists variable doesn’t have its property values populated after ExecuteQuery returns. If you want that list to be populated, you must explicitly call Load on it, specifying which properties you want loaded.

Increasing Performance by Nesting Includes in LoadQuery

When calling the LoadQuery method, we can specify multiple levels of properties to load. This allows the client object model to optimize its access to the SharePoint server internally by reducing the number of times the client object model must do a round-trip to the SharePoint server to retrieve the data that you need. The following query retrieves all lists from the site, and all fields from each list. It then prints them to the console, indicating if each list or field is hidden.

using System;

using System.Collections.Generic;

using Microsoft.SharePoint.Client;

class Program

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

IEnumerable<List> lists = clientContext.LoadQuery(

clientContext.Web.Lists.Include(

list => list.Title,

list => list.Hidden,

list => list.Fields.Include(

field => field.Title,

field => field.Hidden)));

clientContext.ExecuteQuery();

foreach (List list in lists)

{

Console.WriteLine("{0}List: {1}",

list.Hidden ? "Hidden " : "", list.Title);

foreach (Field field in list.Fields)

Console.WriteLine(" {0}Field: {1}",

field.Hidden ? "Hidden " : "",

field.Title);

}

[Download Code]

This approach allows the server portion of the client object model to be more efficient than if the application first loaded a list of lists, and then loaded fields for each list.

Filtering the Child Collection returned by LoadQuery using LINQ

The LoadQuery method takes an object of type IQueryable<T> as its parameter, and this allows us to write LINQ queries instead of CAML to filter the results. This example returns a collection of all document libraries that are not hidden.

using System;

using System.Linq;

using System.Collections.Generic;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

ListCollection listCollection = clientContext.Web.Lists;

IEnumerable<List> hiddenLists = clientContext.LoadQuery(

listCollection

.Where(list => !list.Hidden &&

list.BaseType == BaseType.DocumentLibrary));

clientContext.ExecuteQuery();

foreach (var list in hiddenLists)

Console.WriteLine(list.Title);

}

[Download Code]

Updating Client Objects

Updating client objects using the client object model is pretty simple. We retrieve the object(s), alter properties, call the Update method for each object we change, and then call the ExecuteQuery method. The following example modifies items in the Client API Test List, increasing the estimate for all development items by 50% (a common operation):

using System;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List list = clientContext.Web.Lists

.GetByTitle("Client API Test List");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml =

@"<View>

<Query>

<Where>

<Eq>

<Value Type='Text'>Development</Value>

</Eq>

</Where>

</Query>

</View>";

ListItemCollection listItems = list.GetItems(camlQuery);

clientContext.Load(

listItems,

items => items.Include(

item => item["Category"],

item => item["Estimate"]));

clientContext.ExecuteQuery();

foreach (ListItem listItem in listItems)

{

listItem["Estimate"] = (double)listItem["Estimate"] * 1.5;

listItem.Update();

}

clientContext.ExecuteQuery();

}

[Download Code]

Deleting Client Objects

Deleting client objects is just as easy. However, there is one very important dynamic around deleting client objects from a client object collection. We can’t simply iterate through the collection, deleting objects. As soon as we delete the first object, it causes the iterator of the client object collection to malfunction. The iterator may throw an exception, or it may quietly finish but not visit all items in the collection. Instead, we need to materialize the collection into a List<T> using the ToList method, and then iterate through that list, deleting the client objects.

The following example deletes the test items from our Client API Test List. It shows using the ToList method to materialize the collection before we iterate through it:

using System;

using System.Linq;

using System.Collections.Generic;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext = new ClientContext("http://intranet.contoso.com");

List list = clientContext.Web.Lists.GetByTitle("Client API Test List");

CamlQuery camlQuery = new CamlQuery();

camlQuery.ViewXml =

@"<View>

<Query>

<Where>

<Eq>

</Eq>

</Where>

</Query>

</View>";

ListItemCollection listItems = list.GetItems(camlQuery);

clientContext.Load(

listItems,

items => items.Include(

item => item["Title"]));

clientContext.ExecuteQuery();

foreach (ListItem listItem in listItems.ToList())

listItem.DeleteObject();

clientContext.ExecuteQuery();

}

[Download Code]

The following code snippet shows the incorrect approach:

clientContext.Load(

listItems,

items => items.Include(

item => item["Title"]));

clientContext.ExecuteQuery();

// the following line doesn’t include the call to ToList

foreach (ListItem listItem in listItems)

listItem.DeleteObject();

clientContext.ExecuteQuery();

Finally, just to clean up the Client API Test List, here is an example that deletes the list and its items:

using System;

using Microsoft.SharePoint.Client;

class DisplayWebTitle

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

clientContext.Web.Lists.GetByTitle("Client API Test List")

.DeleteObject();

clientContext.ExecuteQuery();

}

[Download Code]

Discovering the Schema for Fields

As promised, this section shows an easy way to discover the XML schema that you need to know to create the fields that you want to create in a list. First, go into your SharePoint site, and create a list that contains columns that are configured as you want them. Then you can then use the small example below to output the XML that will create those fields.

The following example prints the field schemas for the fields that I added to the Client API Test List:

using System;

using System.Linq;

using System.Xml.Linq;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List list = clientContext.Web.Lists

.GetByTitle("Client API Test List");

clientContext.Load(list);

FieldCollection fields = list.Fields;

clientContext.Load(fields);

clientContext.ExecuteQuery();

foreach (var f in fields)

{

XElement e = XElement.Parse(f.SchemaXml);

string name = (string)e.Attribute("Name");

if (name == "Category" || name == "Estimate")

{

e.Attributes("ID").Remove();

e.Attributes("SourceID").Remove();

e.Attributes("ColName").Remove();

e.Attributes("RowOrdinal").Remove();

e.Attributes("StaticName").Remove();

Console.WriteLine(e);

Console.WriteLine("===============");

}

[Download Code]

When you run this after creating the list using the example program in the section Creating and Populating a List , it produces the following output:

<Default>Specification</Default>

<CHOICE>Specification</CHOICE>

<CHOICE>Development</CHOICE>

<CHOICE>Documentation</CHOICE>

</CHOICES>

</Field>

===============

===============

The example removes attributes that you don’t need for creating the field.

Accessing Large Lists

SharePoint development guidelines indicate that you should not attempt to retrieve more than 2000 items in a single query. If this is a possibility in your application, consider using the RowLimit element in your CAML queries to limit the amount of data that the client object model retrieves for your application. Sometimes you must access all items in a list that may contain more than 2000 items. If you must do so, then best practice is to page through the items 2000 at a time. This section presents an approach to paging using the CamlQuery.ListItemCollectionPosition property.

using System;

using System.Linq;

using Microsoft.SharePoint.Client;

class Program

{

static void Main()

{

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

List list = clientContext.Web.Lists

.GetByTitle("Client API Test List");

// First, add 20 items to Client API Test List so that there are

// enough records to show paging.

ListItemCreationInformation itemCreateInfo =

new ListItemCreationInformation();

for (int i = 0; i < 20; i++)

{

ListItem listItem = list.AddItem(itemCreateInfo);

listItem["Title"] = String.Format("New Item #{0}", i);

listItem["Category"] = "Development";

listItem["Estimate"] = i;

listItem.Update();

}

clientContext.ExecuteQuery();

// This example shows paging through the list ten items at a time.

// In a real-world scenario, you would want to limit a page to

// 2000 items.

ListItemCollectionPosition itemPosition = null;

while (true)

{

CamlQuery camlQuery = new CamlQuery();

camlQuery.ListItemCollectionPosition = itemPosition;

camlQuery.ViewXml =

@"<View>

</ViewFields>

</View>";

ListItemCollection listItems = list.GetItems(camlQuery);

clientContext.Load(listItems);

clientContext.ExecuteQuery();

itemPosition = listItems.ListItemCollectionPosition;

foreach (ListItem listItem in listItems)

Console.WriteLine(" Item Title: {0}", listItem["Title"]);

if (itemPosition == null)

break;

Console.WriteLine(itemPosition.PagingInfo);

Console.WriteLine();

}

[Download Code]

This example produces the following output:

Item Title: Write specs for user interface.

Item Title: Develop proof-of-concept.

Item Title: Write test plan for user interface.

Item Title: Validate SharePoint interaction.

Item Title: Develop user interface.

Item Title: New Item #0

Item Title: New Item #1

Item Title: New Item #2

Item Title: New Item #3

Item Title: New Item #4

Paged=TRUE&p_ID=10

Item Title: New Item #5

Item Title: New Item #6

Item Title: New Item #7

Item Title: New Item #8

Item Title: New Item #9

Item Title: New Item #10

Item Title: New Item #11

Item Title: New Item #12

Item Title: New Item #13

Item Title: New Item #14

Paged=TRUE&p_ID=20

Item Title: New Item #15

Item Title: New Item #16

Item Title: New Item #17

Item Title: New Item #18

Item Title: New Item #19

Asynchronous Processing

If you are building an application that needs to attach to SharePoint sites that may or may not be available, or if you need to regularly invoke queries that may take a long time, you should consider using asynchronous processing. This will allow you to continue to be responsive to your user while the query executes in a separate thread. In your main thread, you can set a timer to let you know if the query has taken longer than your desired threshold, you can keep the user posted with the status of the query, and when the query finally completes, you can present the user with the results.

The ECMAScript version of the client object model and the Silverlight version (when it modifies the user interface) use asynchronous processing. The topic Data Retrieval Overview in the SharePoint SDK contains examples of how to use asynchronous processing using ECMAScript and Silverlight.

When building a traditional .NET application, such as a Windows Forms or WPF application, you may want to use asynchronous processing. The following example uses the BeginInvoke method to execute a query asynchronously. You will notice that the code passes a statement lambda expression to BeginInvoke, which makes it convenient to code this pattern, because the statement lambda expression can reference automatic variables in the method that contains it. You can see that the statement lambda expression has access to the newListCollection variable. (C# closures make the language 'just work' the way we expect it to.)

using System;

using System.Collections.Generic;

using Microsoft.SharePoint.Client;

class Program

{

static void Main(string[] args)

{

AsynchronousAccess asynchronousAccess = new AsynchronousAccess();

asynchronousAccess.Run();

Console.WriteLine("Before exiting Main");

Console.WriteLine();

Console.WriteLine("In a real application, the application can");

Console.WriteLine("continue to be responsive to the user.");

Console.WriteLine();

Console.ReadKey();

}

class AsynchronousAccess

{

delegate void AsynchronousDelegate();

public void Run()

{

Console.WriteLine("About to start a query that will take a long time.");

Console.WriteLine();

ClientContext clientContext =

new ClientContext("http://intranet.contoso.com");

ListCollection lists = clientContext.Web.Lists;

IEnumerable<List> newListCollection = clientContext.LoadQuery(

lists.Include(

list => list.Title));

AsynchronousDelegate executeQueryAsynchronously =

new AsynchronousDelegate(clientContext.ExecuteQuery);

executeQueryAsynchronously.BeginInvoke(

arg =>

{

Console.WriteLine("Long running query has completed.");

foreach (List list in newListCollection)

Console.WriteLine("Title: {0}", list.Title);

}, null);

}

[Download Code]

The example produces the following output:

About to start a query that will take a long time.

Before exiting Main

In a real application, the application can

continue to be responsive to the user.

Long running query has completed.

Title: Announcements

Title: Cache Profiles

Title: Calendar

Title: Client API Test List

Title: Content and Structure Reports

Title: Content type publishing error log

Title: Converted Forms

Title: Customized Reports

Title: Eric's ToDo List

Title: Eric's Wiki

Title: Form Templates

Title: Links

Other Resources

The SharePoint SDK contains some good resources and example code for the client object model that will be useful in a variety of scenarios:

SharePoint 2010 Developer Center

How to: Work with Web Sites

How to: Work with Users and Groups

How to: Break Role Assignment Inheritance

How to: Work with User Custom Actions

How to: Work with WebParts on a Page

Setting Up a Basic ASPX Page for ECMAScript

Using the Silverlight Object Model

Availability of Public Office 2010 and SharePoint 2010 Betas

EricWhite — Thu, 19 Nov 2009 00:43:00 GMT

Today at PDC, we announced the availability of our public Office 2010 and SharePoint 2010 betas. The next few months are simply going to be fun. We can finally talk about all the cool new features, and especially from my point of view, the new stuff for developers. It's a whole new world. We're making this beta available in seven languages - English, Spanish, Japanese, Simplified Chinese, Russian, French and German.

Gray Knowlton has a fun post on the Office 2010 and SharePoint 2010 PDC Keynote. Gave me a sense of being there. J And Erika Ehrli covered the beta release nicely as well.

Here's some links to places to get started:

Download Microsoft Office Professional Plus 2010 Beta

Project 2010 Beta

Project Server 2010 Beta

SharePoint Designer 2010 Beta (32-bit)

SharePoint Designer 2010 Beta (64-bit)

SharePoint Foundation 2010 Beta

SharePoint Server 2010 Beta

Visio 2010 Beta

Visual Studio 2010 Beta 2

The new developer centers are live:

Office 2010 Beta Developer Center

SharePoint 2010 Beta Developer Center

Visual Studio 2010 Beta Developer Center

The Office Developer Documentation team made an announcement that is important for any developers who are getting started in 2010 development. If you want to make sure that you are viewing the most up-to-date help for the Office 2010 client applications, read about the Developer Help Updates for Office 2010 Beta.

SharePoint 2010 and Office 2010 Developer Training Courses Launched on Channel 9

EricWhite — Tue, 10 Nov 2009 06:46:00 GMT

Channel 9 launched new developer training courses for SharePoint 2010 and Office 2010. I know the folks who produced this, and personally seen a lot of it. This is good stuff. The training consists of extensive recordings from top MVPs/experts on how to develop with both SharePoint and Office 2010. As a developer, I'm personally particularly enthused about three developer aspects of SharePoint 2010: Client Object Model, Sandboxed Solutions, and LINQ to SharePoint, but there's a lot more there than that. If you haven't seen what the buzz is about, check it out.

SharePoint 2010 Developer Training: SharePoint 2010 has evolved into a first-class developer platform. Visual Studio 2010 integration is fantastic.

Office 2010 Developer Training: There are a number of new features and UI extensions that you'll want to know about.

Also while I'm at it, want to let you know about Windows Server 2008 R2 training. I really appreciate W2008 R2.

Generating Documents from SharePoint Lists using Open XML Content Controls

EricWhite — Wed, 14 Oct 2009 03:37:00 GMT

[Blog Map]

It's often the case that a department manager needs to regularly send a nicely formatted status report to her general manager or that a team leader needs to send a weekly status report to a number of interested parties. To collaborate with others in their organizations, both the manager and the team leader can maintain status information in SharePoint lists. The question for developers is how to include the information in the lists in a document such as a status report.

I've written an article entitled Generating Documents from SharePoint with Open XML Content Controls, published in the October issue of MSDN Magazine that details how you can put together a simple, flexible, and powerful system for generating Open XML word-processing documents using SharePoint lists as sources for data in tables of word-processing documents.

The key aspect of a successful approach is to make it so that the folks who need to generate reports can do so without involving the services of a software developer. We can adopt an approach where the user can configure the sources of data for tables in a document using content controls. The user can surround a table with a content control, and set the content control tag to the name of the SharePoint list. The user can insert content controls into cells in a nicely formatted table, and set the tags to the names of the columns of the SharePoint list. We then have enough information to retrieve the necessary data from the SharePoint list (or lists) and generate a document that contains that data. If the department manager or team leader subsequently adds a new column to their SharePoint list, they can add a new column to their table, insert a new content control, set the tag, and thereafter, their reports will contain data from the new column in the SharePoint list.

To make this clear, a typical SharePoint list looks like this:

The template Open XML word-processing document might look like the following. Notice that there is a content control surrounding the entire table, with a tag and title of "Team Members", and there is a content control in a table cell with tag and title of "TeamMemberName". If we were to put the insertion point in the Role cell, you would see that there is a content control there. There are similar content controls around and in the Current Work Items table.

The resulting generated report looks like this:

One aspect of the approach that I took is that I abstracted the operations involving the content controls into a ContentControlManager class. You can call a method, GetContentControls, which returns some XML that describes the content controls in the document. In the example that I present with the article, GetContentControls returns the following XML:

</Table>

</Table>

</ContentControls>

This gives us the information that we need to use the SharePoint object model to retrieve the necessary data. After retrieving that data, we can construct a small XML tree that looks like this:

<Row>

<Field Name="TeamMemberName"

Value="Bob" />

<Field Name="Role"

Value="Developer" />

</Row>

<Row>

<Field Name="TeamMemberName"

Value="Susan" />

<Field Name="Role"

Value="Program Manager" />

</Row>

<Row>

<Field Name="TeamMemberName"

Value="Jack" />

<Field Name="Role"

Value="Test" />

</Row>

</Table>

<Row>

<Field Name="ItemName"

Value="Learn SharePoint 2010" />

<Field Name="Description"

Value="This should be fun!" />

<Field Name="EstimatedHours"

Value="80" />

<Field Name="AssignedTo"

Value="All" />

</Row>

<Row>

<Field Name="ItemName"

Value="Finalize Import Module Specification" />

<Field Name="Description"

Value="Make sure to handle all document formats." />

<Field Name="EstimatedHours"

Value="35" />

<Field Name="AssignedTo"

Value="Susan" />”

</Row>

<Row>

<Field Name="ItemName"

Value="Write Test Plan" />

<Field Name="Description"

Value="Include regression testing items." />

<Field Name="EstimatedHours"

Value="20" />

<Field Name="AssignedTo"

Value="Jack" />

</Row>

</Table>

</ContentControls>

We can pass that XML to the SetContentControls method in the ContentControlManager class, which will modify the document so that the tables in the document are populated with the data from the XML. This is useful functionality in its own right, and could be used in other scenarios than generating documents from SharePoint lists.

The MSDN article contains a download that contains the ContentControlManager class, as well as the SharePoint code to populate tables with data from SharePoint lists.

Office Developer Conference moving to SharePoint Conference 2009

EricWhite — Mon, 01 Jun 2009 19:56:00 GMT

Gray Knowlton (Group Product Manager for Office) has posted news that Office Developer Conference (ODC) will not take place this year. Instead, the ODC content will be included within the SharePoint Conference.

From his post:

As you may have seen at PDC, TechEd or elsewhere, Office 2010 is on its way. To help you get ready, Office 2010 for Developers will be highlighted at the upcoming SharePoint Conference (October 2009, Las Vegas, NV) and TechEd conferences around the world in 2009 and 2010.

NET: Office Developer Conference will not take place this year; instead we are including the Office Developer Conference content within the SharePoint Conference. If you are an attendee of Office Developer Conference in the past, we strongly recommend you come see us at the SharePoint Conference in October, where we’ll cover Office client development in depth. Be sure to sign up for the Technical Preview as well!

We are optimizing our show presence for developers seeking opportunities to build on the Office platform, which includes Office client applications, SharePoint, Exchange and Communicator. By adding the ODC track to the 2009 SharePoint conference, we can provide better exposure to those seeking to develop solutions across the platform.

LINQ / Open XML at TechEd 2009

EricWhite — Tue, 05 May 2009 01:44:00 GMT

I’ll be presenting a talk on LINQ and Open XML at TechEd 2009. The session is “OFC403 Developing Office Client Solutions Using LINQ and Open XML”. I’ll be presenting on Thursday, 5/14, at 4:30PM in Room 411. Here is the abstract for the talk:

In this session, learn how to leverage LINQ with version 1 and 2 of the Open XML SDK to create, modify, and query Open XML documents. Version 1 works well with LINQ to XML, and Version 2 provides a strongly-typed document object model (also LINQ friendly) that makes it easier to work with XML parts. Both approaches benefit from the conciseness of LINQ and the power of functional construction. In addition, we talk about and demonstrate tools that you can use to develop Open XML solutions faster and better. This session is very technical; knowledge of generics and LINQ are prerequisites to get the most from this session.

I’ve presented this talk before, at TechEd in Barcelona last fall, and at an internal Microsoft conference. In general, folks like this talk, and have given me high scores. However, in both previous presentations of this talk, one or two attendees criticize the talk because I spent some amount of time talking about LINQ and why you want to use it.

I feel conflicted about how to present the material in this talk. I always take an informal survey at the beginning of the talk, and 50% of the attendees haven’t really used LINQ, nor know too much about functional programming. I feel that without a little introduction to LINQ, these people might not appreciate the benefits that LINQ and functional programming offer to us. I certainly understand the sentiments of the LINQ / functional programming experts who attend – they basically want to see the tricks and patterns that I’ve developed for working with Open XML, and feel that the time spent on LINQ is wasted for them.

After a fair amount of reflection, I feel that more people are better served if I speak about LINQ for a few minutes before we dive into its application with Open XML, so I am again going to do the short intro to LINQ – I’ll limit this intro to 15 minutes, I promise! My goal in this intro is not to bring people up to speed on LINQ, but instead point out the basic principles that underlie it – specifically, how to craft pure functional transforms.

In the LINQ portion of the talk, as well as when I talk about using it with Open XML, I will be occasionally pointing out pages on my blog that contain more information. My goal here isn’t to increase traffic on my blog J, instead, it allows me to talk about concepts in as short amount of time possible, and attendees can refer to my blog to fill in the technical details. I want to cover as much information as possible, and this is an effective way, I believe.

Staffing the Office / SharePoint Booth

I’ll also be at the Office / SharePoint booths at the following times:

· Wednesday, May 13, 3:00 PM – 6:00 PM

· Thursday, May 14, 10:45 AM – 1:45 PM

· Friday, May 15, 10:00 AM - 1:15 PM

I really enjoy meeting customers and Microsoft employees who are using Office development technologies, including Open XML and SharePoint. Please feel free to come by the booth at the above times and introduce yourself! I’d love to hear about your development efforts, and am happy to help in any way that I can. I'll have a little time available for 1:1 consultations - if you'd like to chat in a side meeting about your development efforts, email me through the EMAIL link above.

Links – April 29, 2009

EricWhite — Wed, 29 Apr 2009 21:08:00 GMT

Kirk Evans , an architect evangelist at Microsoft, has posted five cool videos on SharePoint development:

Johann Granados, from Staff DotNet, has written some interesting posts on using the Open XML SDK v2:

Two Upcoming Presentations on LINQ and Open XML

EricWhite — Thu, 15 Jan 2009 04:12:00 GMT

At TechReady8, I’ll be presenting a 400 level course “Developing Office Client Solutions using LINQ and Open XML”. TechReady8 is a Microsoft employee only event, but non-Microsoft folks in the Seattle area can come see the same talk at the .NET Developers Association of Redmond meeting on January 19^th. This talk will be at 7:00PM, in Microsoft Building 40, Steptoe Room #1450. You can find more information at www.dotnetda.com.

Here is the abstract for the talk:

Developing Office Client Solutions using LINQ and Open XML (400)

There are a variety of approaches to working with Open XML documents using LINQ to XML, and each approach is optimal for certain situations. The most common pattern is to use pure functional transforms for retrieving data from documents. An interesting alternative approach uses LINQ to XML annotations to do document centric transforms; this approach uses LINQ to XML in the style of XSLT. Sometimes the best approach for certain tasks is in-place tree modification ala DOM, but there are pitfalls. Highlights of the talk: changing the basic approach to one of transformations instead of algorithms, an explanation and justification for functional purity, and a discussion of the development thought process when writing pure functional transforms. You'll see lots of demos, including querying word processing and spreadsheet documents, along with a natural syntax for querying Excel tables. In addition, you'll be introduced to tools that you can use to develop Open XML solutions more efficiently and reliably. Because this is a very technical session, knowledge of generics and LINQ are suggested.

More SharePoint and Silverlight Samples

EricWhite — Thu, 15 Jan 2009 04:09:00 GMT

Steve Fox and Paul Stubbs have posted a couple more samples to the SharePoint and Silverlight Blueprint Codeplex site. These are upgraded versions of the Colleague Viewer and the Custom Navigation (from Silverlight Beta 2). There are now four samples updated for Silverlight 2 RTM/SharePoint 2007:

1. Hello World

2. Slider Control

3. Colleague Viewer

4. Custom Navigation

You can find all of the blueprints here: http://www.codeplex.com/SL4SP.

These samples will offer developers a great starting point for integrating Silverlight and SharePoint. Paul and Steve also recently completed the following book, which will be published on February 23^rd : http://www.amazon.com/Professional-Microsoft-SharePoint-Development-Silverlight/dp/0470434007. It covers a number of Silverlight integrations with SharePoint and offers some patterns for developers to follow. Developers can also go to www.mssharepointdeveloper.com for additional SharePoint development resources—including Silverlight and SharePoint labs and content.

Modifying Open XML Documents that are in SharePoint Document Libraries using Web Services

EricWhite — Fri, 09 Jan 2009 04:54:00 GMT

[Blog Map]

When using the Open XML SDK with SharePoint web services, one of the most basic operations is to get a document from a document library using web services, modify it using the Open XML SDK (and LINQ to XML), and save it back to the document library. This post describes how to do this, and provides a sample in C#.

It is simple to extend this sample to iterate through all documents in a library, apply some changes to each one, and save them back. In an upcoming post, I’ll present a sample to ‘sanitize’ (remove comments, accept revisions, and remove personal information) all documents in a document library. This is pretty useful. I keep a library of documents that I send externally as needed, and it’s always best to not have personal information embedded in the documents. By running this upcoming sample, I can regularly check to make sure that the document library is clean, even if other folks are editing documents in the library.

For a brief tutorial on SharePoint web services, see “Getting Started with SharePoint (WSS) Web Services using LINQ to XML”. For this example, you need to add two references to web services (both Lists and Copy). The procedure for adding a reference to the Copy web service is the same as adding a reference to the Lists web service.

This code uses the Open XML SDK. Remember to add a reference to the Open XML SDK assembly. This code uses V1 of the SDK. It should work with V2 CTP but I haven't tried it.

The code references the System.IO.FileFormatException class, which is in the WindowsBase assembly, so add a reference to it.

This code uses the technique of converting XmlNode to XElement (and back again), as detailed in “Convert XElement to XmlNode (and Convert XmlNode to XElement)”, so that we can use LINQ to XML instead of XmlDocument.

One important aspect of the code is that you retrieve the document as a byte array:

ModifyDoc.CopyWebService.FieldInformation[] fields;

byte[] byteArray;

copy.GetItem(url, out fields, out byteArray);

After retrieving the byte array, you can write the byte array to a MemoryStream, and use the MemoryStream to open an in-memory Open XML document. After modifying the in-memory document, you can convert it back to a byte array and serialize back to the SharePoint document library. The technique is described in the post, “Working with In-Memory Open XML Documents”.

Here is the code to serialize it back to the SharePoint document library:

string[] urls = { url };

ModifyDoc.CopyWebService.CopyResult[] copyResults;

copy.CopyIntoItems(url, urls, fields, mem.ToArray(), out copyResults);

Now that we’ve covered these basics, in the near future, I'll show using SharePoint web services and the Open XML SDK to do some more interesting stuff.

Here is the complete listing (the code is added as an attachment to this post):

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml;

using System.Xml.Linq;

using System.IO;

using DocumentFormat.OpenXml.Packaging;

namespace ModifyDoc

{

public static class MyExtensions

{

public static XDocument GetXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.Annotation<XDocument>();

if (xdoc != null)

return xdoc;

using (StreamReader sr = new StreamReader(part.GetStream()))

using (XmlReader xr = XmlReader.Create(sr))

xdoc = XDocument.Load(xr);

part.AddAnnotation(xdoc);

return xdoc;

}

public static void PutXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.GetXDocument();

if (xdoc != null)

{

// Serialize the XDocument object back to the package.

using (XmlWriter xw =

XmlWriter.Create(part.GetStream

(FileMode.Create, FileAccess.Write)))

{

xdoc.Save(xw);

}

public static string StringConcatenate(

this IEnumerable<string> source)

{

return source.Aggregate(

new StringBuilder(),

(s, i) => s.Append(i),

s => s.ToString());

}

public static XElement GetXElement(this XmlNode node)

{

XDocument xDoc = new XDocument();

using (XmlWriter xmlWriter = xDoc.CreateWriter())

node.WriteTo(xmlWriter);

return xDoc.Root;

}

public static XmlNode GetXmlNode(this XElement element)

{

using (XmlReader xmlReader = element.CreateReader())

{

XmlDocument xmlDoc = new XmlDocument();

xmlDoc.Load(xmlReader);

return xmlDoc;

}

class Program

{

static void Main(string[] args)

{

string documentLibraryName = "Open XML Documents";

string documentName = "Test.docx";

XNamespace s = "http://schemas.microsoft.com/sharepoint/soap/";

XNamespace rs = "urn:schemas-microsoft-com:rowset";

XNamespace z = "#RowsetSchema";

// Make sure that you use the correct namespace, as well as the correct reference

// name. The namespace (by default) is the same as the name of the application

// when you created it. You specify the reference name in the Add Web Reference

// dialog box.

// Namespace Reference Name

// | |

// V V

ModifyDoc.ListsWebService.Lists lists =

new ModifyDoc.ListsWebService.Lists();

// Fix Namespace and Reference Name for the Copy web service too

ModifyDoc.CopyWebService.Copy copy =

new ModifyDoc.CopyWebService.Copy();

// Update the following URL to point to the Lists web service for

// your SharePoint site.

lists.Url = "http://localhost/_vti_bin/Lists.asmx";

lists.Credentials = System.Net.CredentialCache.DefaultCredentials;

copy.Credentials = System.Net.CredentialCache.DefaultCredentials;

XElement listCollection = lists.GetListCollection().GetXElement();

// get the node for the library that we want

XElement library = listCollection

.Elements(s + "List")

.Where(l => (string)l.Attribute("Title") == documentLibraryName)

.FirstOrDefault();

if (library == null)

{

Console.WriteLine("Library {0} doesn't exist.", documentLibraryName);

Environment.Exit(0);

}

// get the ID of the library

string libId = (string)library.Attribute("ID");

XElement item = GetItemByLinkFilename(lists, libId, documentName);

if (item == null)

{

Console.WriteLine("Document {0} doesn't exist.", documentName);

Environment.Exit(0);

}

// get the document from the doc library as a byte array

string url = item.Attribute("ows_EncodedAbsUrl").Value;

ModifyDoc.CopyWebService.FieldInformation[] fields;

byte[] byteArray;

copy.GetItem(url, out fields, out byteArray);

// create a memory stream from the byte array

using (MemoryStream mem = new MemoryStream())

{

mem.Write(byteArray, 0, (int)byteArray.Length);

try

{

// create a WordprocessingDocument from the memory stream

using (WordprocessingDocument wordDoc =

WordprocessingDocument.Open(mem, true))

{

XNamespace w =

"http://schemas.openxmlformats.org/wordprocessingml/2006/main";

// modify the document as necessary

// for this example, we'll insert a simple paragraph at the

// beginning of the document

XDocument doc = wordDoc.MainDocumentPart.GetXDocument();

doc.Element(w + "document")

.Element(w + "body")

.AddFirst(

new XElement(w + "p",

new XElement(w + "r",

new XElement(w + "t", "Hello, there")

)

);

// write the XDocument back into the Open XML document

wordDoc.MainDocumentPart.PutXDocument();

}

// use the Copy web service to save the document back to the

// document library.

string[] urls = { url };

ModifyDoc.CopyWebService.CopyResult[] copyResults;

copy.CopyIntoItems(url, urls, fields, mem.ToArray(), out copyResults);

}

catch (System.IO.FileFormatException e)

{

// document is invalid

Console.WriteLine(e);

Environment.Exit(0);

}

private static XElement GetItemByLinkFilename(

ModifyDoc.ListsWebService.Lists lists, string libId,

string documentName)

{

XNamespace z = "#RowsetSchema";

// get the XElement for the row that contains info about the document

// that we want to modify

XElement queryOptions = new XElement("QueryOptions",

new XElement("Folder"),

new XElement("IncludeMandatoryColumns", false)

);

XElement viewFields = new XElement("ViewFields");

XElement item = lists.GetListItems(libId, "", null,

viewFields.GetXmlNode(), "", queryOptions.GetXmlNode(), "")

.GetXElement()

.Descendants(z + "row")

.Where(i => (string)i.Attribute("ows_LinkFilename") == documentName)

.FirstOrDefault();

return item;

}

Getting Started with SharePoint Web Services using LINQ to XML

EricWhite — Tue, 06 Jan 2009 03:52:00 GMT

[Blog Map]

Web services are one of the most effective and convenient ways for casual developers to access SharePoint lists and document libraries. They have a reputation for being a bit difficult; LINQ to XML can make them easier to use. This post presents the basic steps for getting started with Windows SharePoint Services Web Services using LINQ to XML (either C# or VB).

In several upcoming posts, I’ll present some more information about using web services, as well as more notes on the code presented in this post. There are some interesting and easy ways to explore SharePoint web services using LINQ to XML. However, before addressing more involved scenarios, I want to make sure that you can get started easily.

First, a few points about SharePoint web services. They can be disabled for a SharePoint site, but by default, they are enabled. In most cases, it is easy to use them; if you have permissions to manually create and modify documents for a site, you probably can use web services to do so too.

One important feature of SharePoint that enables effective collaboration is the My Site feature. For an introduction to your My Site, see this Office Online article. Chances are, if you have a My Site in your company, you can use web services to programmatically access it. In an upcoming project, I plan to use the SharePoint collaboration features to make managing the project easier. Then I'll write a little C# program that collects information from lists and document libraries (using web services) and produces a nicely formatted status report as an Open XML Wordprocessing document, and then places the report into an appropriate document library.

Here are the steps:

Create a new Windows Console application (either C# or VB).

Important note: Pay attention to the name that you give your application, as the classes created to access the SharePoint web services will be in the namespace of your application. For this example, name the application SPWebServicesExample.

Add a reference to your SharePoint web service. Select Project, Add Service Reference…

Click the Advanced button:

Click the ‘Add Web Reference’ button:

In the Add Web Reference dialog box, enter the URL of the ‘Lists’ web service. Web services in SharePoint are located in the ‘_vti_bin’ directory under the relevant site. If you are using web services to access your My Site, then enter the URL: ‘http://my/sites/your-alias- here/_vti_bin/lists.asmx’. If you are using web services to access a SharePoint site located at http://ourteamsite, the URL will be ‘http://ourteamsite/_vti_bin/lists.asmx’.

Click the Go button to confirm the URL. If you have entered a valid URL, then this dialog box tells you that it found a web service. The dialog box will be populated with the various operations available in the web service.

Change the web reference name to something relevant. For accessing the Lists web service, a good web reference name is ‘ListsWebService’.

Click the Add Reference button to add this reference to your project.

Following is a small example to get all of the lists and list items in a site. The example creates a small XML "report" that contains relevant data on the lists and items. In addition to the listing on this page, I've attached the code to this post.

In general, when using this code as boilerplate, you have to remember to update three things (all highlighted in the listing below):

The namespace for your web service proxy class (two occurrences)
The web reference name (two occurrences)
The URL of the SharePoint site that you want to access

Note that if you used the names suggested earlier, you will not need to update the first two items, as the web service proxy will already be set to “SPWebServicesExample” and the web reference name will already be set to “ListsWebService”.

There are a couple of notes to make about the following code:

The code to create the report uses the LINQ to XML idiom that I detailed in the blog post titled ''Writing Succinct Code to move XElement and XAttribute Objects from One LINQ to XML Tree to Another".
When printing the "Report" XML tree to the console, the code uses the technique that I described in "Align Attributes when Formatting XML using LINQ to XML". To make it more convenient to convert an XML tree to a string where attributes are aligned, in this example I wrote an extension method, ToStringAlignAttributes.
To convert the LINQ to XML trees to XmlNode (and back), the codes uses the approach from "Convert XElement to XmlNode (and Convert XmlNode to XElement) ".

It is more convenient to use LINQ to XML instead of XmlDocument for accessing web services. You can write simpler, easier to read code when creating the XML that you pass as arguments to operations. In particular, for this purpose, VB XML literals rock! And the code to query the XML returned by the web service is much easier to write, in my opinion.

I'll be blogging more about how to use these web services. There's a lot that you can do with them. I am especially enthused about taking advantage of what you can do with Open XML and the Open XML SDK. Stay tuned...

C#

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml;

using System.Xml.Linq;

namespace SPWebServicesExample

{

public static class MyExtensions

{

public static XElement GetXElement (this XmlNode node)

{

XDocument xDoc = new XDocument();

using (XmlWriter xmlWriter = xDoc.CreateWriter())

node.WriteTo(xmlWriter);

return xDoc.Root;

}

public static XmlNode GetXmlNode (this XElement element)

{

using (XmlReader xmlReader = element.CreateReader())

{

XmlDocument xmlDoc = new XmlDocument();

xmlDoc.Load(xmlReader);

return xmlDoc;

}

public static string ToStringAlignAttributes(this XElement element)

{

XmlWriterSettings settings = new XmlWriterSettings();

settings.Indent = true;

settings.OmitXmlDeclaration = true;

settings.NewLineOnAttributes = true;

StringBuilder stringBuilder = new StringBuilder();

using (XmlWriter xmlWriter = XmlWriter.Create(stringBuilder, settings))

element.WriteTo(xmlWriter);

return stringBuilder.ToString ();

}

class Program

{

static void Main(string[] args)

{

XNamespace s = "http://schemas.microsoft.com/sharepoint/soap/";

XNamespace rs = "urn:schemas-microsoft-com:rowset";

XNamespace z = "#RowsetSchema";

// Make sure that you use the correct namespace, as well as the correct reference

// name. The namespace (by default) is the same as the name of the application

// when you created it. You specify the reference name in the Add Web Reference

// dialog box.

// Namespace Reference Name

// | &nb sp; |

// V &nb sp; V

SPWebServicesExample.ListsWebService.Lists lists =

new SPWebServicesExample.ListsWebService.Lists ();

// Make sure that you update the following URL to point to the Lists web service

// for your SharePoint site.

lists.Url = "http://my/sites/your-alias-here/_vti_bin/Lists.asmx";

//lists.Url = "http://xyzteamsite/_vti_bin/Lists.asmx";

lists.Credentials = System.Net.CredentialCache.DefaultCredentials;

XElement queryOptions = new XElement("QueryOptions",

new XElement("Folder"),

new XElement("IncludeMandatoryColumns", false)

);

XElement viewFields = new XElement("ViewFields");

XElement listCollection = lists.GetListCollection().GetXElement();

XElement report = new XElement("Report",

listCollection

.Elements(s + "List")

.Select(

l =>

{

return new XElement("List",

l.Attribute("Title"),

l.Attribute ("DefaultViewUrl"),

l.Attribute("Description"),

l.Attribute("DocTemplateUrl"),

l.Attribute("BaseType"),

l.Attribute("ItemCount"),

l.Attribute("ID"),

lists.GetListItems((string) l.Attribute("ID"), "", null,

viewFields.GetXmlNode(), "", queryOptions.GetXmlNode(), "")

.GetXElement()

.Descendants(z + "row")

.Select(r =>

new XElement("Row",

r.Attribute("ows_Title"),

r.Attribute("ows_ContentType"),

r.Attribute("ows_FSObjType"),

r.Attribute("ows_Attachments"),

r.Attribute("ows_FirstName"),

r.Attribute("ows_LinkFilename"),

r.Attribute("ows_EncodedAbsUrl"),

r.Attribute("ows_BaseName"),

r.Attribute("ows_FileLeafRef"),

r.Attribute("ows_FileRef"),

r.Attribute("ows_ID"),

r.Attribute("ows_UniqueId"),

r.Attribute("ows_GUID")

)

);

}

)

);

Console.WriteLine (report.ToStringAlignAttributes());

}

VB

Imports System.Xml

Imports System.Text

Module Module1

<System.Runtime.CompilerServices.Extension()> _

Public Function GetXElement (ByRef node As XmlNode) As XElement

dim xDoc as XDocument = new XDocument()

using xmlWriter as XmlWriter = xDoc.CreateWriter()

node.WriteTo(xmlWriter)

End Using

return xDoc.Root

End Function

<System.Runtime.CompilerServices.Extension()> _

Public Function GetXmlNode (ByRef element As XElement) As XmlNode

Using xmlReader as XmlReader = element.CreateReader()

Dim xmlDoc As XmlDocument = New XmlDocument

xmlDoc.Load(xmlReader)

Return xmlDoc

End Using

End Function

<System.Runtime.CompilerServices.Extension()> _

Public Function ToStringAlignAttributes(ByVal element As XElement) As String

Dim settings As XmlWriterSettings = New XmlWriterSettings()

settings.Indent = True

settings.OmitXmlDeclaration = True

settings.NewLineOnAttributes = True

Dim stringBuilder As StringBuilder = New StringBuilder()

Using xmlWriter As XmlWriter = xmlWriter.Create(stringBuilder, settings)

element.WriteTo(xmlWriter)

End Using

Return stringBuilder.ToString()

End Function

Sub Main()

Dim s as XNamespace = "http://schemas.microsoft.com/sharepoint/soap/"

Dim rs as XNamespace = "urn:schemas-microsoft- com:rowset"

Dim z as XNamespace = "#RowsetSchema"

' Make sure that you use the correct namespace, as well as the correct reference name.

' The namespace (by default) is the same as the name of the application when you

' created it. You specify the reference name in the Add Web Reference dialog box.

' Namespace Reference Name

' | |

' V V

Dim lists As SPWebServicesExample.ListsWebService.Lists = _

New SPWebServicesExample.ListsWebService.Lists()

' Update the following URL to point to the Lists web service

' for your SharePoint site.

lists.Url = http://my/sites/your-alias-here/_vti_bin/Lists.asmx

'lists.Url = "http://xyzteamsite/_vti_bin/Lists.asmx";

lists.Credentials = System.Net.CredentialCache.DefaultCredentials

Dim queryOptions = _

<IncludeMandatoryColumns>false</IncludeMandatoryColumns>

</QueryOptions>

Dim viewFields = <ViewFields/>

dim listCollection as XElement = lists.GetListCollection().GetXElement()

Dim report as XElement = _

<%= listCollection _

.Elements(s + "List") _

.Select( Function(l) _

new XElement("List", _

l.Attribute("Title"), _

l.Attribute("DefaultViewUrl"), _

l.Attribute("Description"), _

l.Attribute("DocTemplateUrl"), _

l.Attribute("BaseType"), _

l.Attribute("ItemCount"), _

l.Attribute("ID"), _

lists.GetListItems(l.Attribute("ID"), "", Nothing, _

viewFields.GetXmlNode(), "", queryOptions.GetXmlNode(), "") _

.GetXElement() _

.Descendants(z + "row") _

.Select (Function(r) _

new XElement ("Row", _

r.Attribute("ows_Title"), _

r.Attribute("ows_ContentType"), _

r.Attribute("ows_FSObjType"), _

r.Attribute("ows_Attachments"), _

r.Attribute("ows_FirstName"), _

r.Attribute("ows_LinkFilename"), _

r.Attribute("ows_EncodedAbsUrl"), _

r.Attribute("ows_BaseName"), _

r.Attribute("ows_FileLeafRef"), _

r.Attribute("ows_FileRef"), _

r.Attribute("ows_ID"), _

r.Attribute("ows_UniqueId"), _

r.Attribute("ows_GUID") _

) _

) %>

</Report>

Console.WriteLine (report.ToStringAlignAttributes())

End Sub

End Module

After you build and run this example, your output should look like this. Note that for even a small SharePoint site, this may run several thousand lines long, so you may have to redirect the output to a file or change the screen buffer height on the command prompt window to view it all.

<List

Title="AList"

DefaultViewUrl="/sites/ericwhit/AList/Forms/AllItems.aspx"

Description="AList"

DocTemplateUrl="/sites/ericwhit/AList/Forms/template.dotx"

BaseType="1"

ItemCount="0"

ID="{AA12AA75-2422 -48BB-B41A-267C61D7A54F}" />

<List

Title="Eric White's Wiki"

DefaultViewUrl="/sites/ericwhit/Eric Whites Wiki/Forms/AllPages.aspx"

Description="Wiki"

DocTemplateUrl=""

BaseType="1"

ItemCount="3"

ID="{AA0A2753-5A94 -42C1-BAA7-66DD1409CD3E}">

<Row

ows_ContentType="Wiki Page"

ows_FSObjType="2;#0"

ows_LinkFilename="Home.aspx"

ows_EncodedAbsUrl="http://my/sites/ericwhit/Eric%20Whites%20Wiki/Home.aspx"

ows_BaseName="Home"

ows_FileLeafRef="2;#Home.aspx"

ows_FileRef="2;#sites/ericwhit/Eric Whites Wiki/Home.aspx"

ows_ID="2"

ows_UniqueId="2;#{AA6833E4-98F2-42C4-9355-6DF2453DC7D1}"

ows_GUID="{AAEA7CFF-9FE1-47F9-8C48-94B631289D0F}" />

<Row

...

Modifying Open XML Documents using the SharePoint Object Model

EricWhite — Fri, 12 Dec 2008 09:33:00 GMT

[Blog Map]

When working with Open XML documents from within SharePoint, you may want to open a specific document, modify it in some way, and then save it, either replacing the original document, or saving to a new location. This isn’t very hard, but there are a few issues. This post describes the issues and presents a minimal amount of code that shows how to open an Open XML document from a SharePoint list, modify it, and then save it back to either the same location, or a different one if you desire.

In detail, the technique consists of:

Writing a function that takes an SPFile as an argument. This SPFile object is the DocumentLibrary list item that we want to modify. This factoring allows us to first write this code as a console application. Then, when we have the code debugged, we can easily move the factored code into a SharePoint feature.
Reading the document into a byte array using the SPFile.OpenBinary method.
Creating a resizable MemoryStream object using the parameterless constructor of the MemoryStream class.
Writing the byte array into the MemoryStream.
Opening the document using the Open XML SDK. Trap the System.IO.FileFormatException when opening. The Open XML SDK will throw this exception if the document isn’t valid.
Modifying the document using LINQ to XML and the Open XML SDK. This code is similar to a bunch of other code that I’ve posted on my blog.
Writing the document back to the document library.

This code can either use the Open XML SDK V1 or CTP1 of the Open XML SDK V2.

The code that I present in this post uses the techniques for working with Open XML documents in memory. The blog post, Working With In-Memory Open XML Documents, describes some of the issues, including the issue that we can’t use the MemoryStream constructor that takes a byte array as a parameter, because that constructor creates a non-resizable MemoryStream. The Open XML SDK requires that the stream be resizable, as a modified part that is written back to the package may be larger than the original.

In this example, I use one of the techniques that I’ve found I really like when working with the SharePoint object model. I first develop my code that uses the object model using a console application, factoring the code so that I can easily move the necessary code to a feature. This means that while I’m writing my code and getting it to work properly, I have a very fast edit/compile/debug cycle. Then, when I have all of my properly factored code as I want it, I can assemble my SharePoint feature in no time at all.

Note that one of the exceptions that the Open XML SDK can throw is System.IO.FileFormatException. This exception is defined in the System.IO.Packaging namespace, which is in the WindowsBase assembly. To compile this console application, you must add references to the following three assemblies:

DocumentFormat.OpenXml, which is the Open XML SDK (either V1 or CTP1 of V2)
Microsoft.SharePoint
WindowsBase

Because this is a console application that is using the SharePoint object model, you must run this code as an administrator on the box that hosts the SharePoint site. In my case, I wrote this code inside of the virtual machine that you can find here.

Here is the code that shows how to modify an Open XML document:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.IO;

using System.IO.Packaging;

using System.Xml;

using System.Xml.Linq;

using Microsoft.SharePoint;

using DocumentFormat.OpenXml.Packaging;

public static class LocalExtensions

{

public static XDocument GetXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.Annotation<XDocument>();

if (xdoc != null)

return xdoc;

using (StreamReader sr = new StreamReader(part.GetStream()))

using (XmlReader xr = XmlReader.Create(sr))

xdoc = XDocument.Load(xr);

part.AddAnnotation(xdoc);

return xdoc;

}

public static void PutXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.GetXDocument();

if (xdoc != null)

{

// Serialize the XDocument object back to the package.

using (XmlWriter xw =

XmlWriter.Create(part.GetStream

(FileMode.Create, FileAccess.Write)))

{

xdoc.Save(xw);

}

public static string StringConcatenate(

this IEnumerable<string> source)

{

return source.Aggregate(

new StringBuilder(),

(s, i) => s.Append(i),

s => s.ToString());

}

class Program

{

enum ModifyDocumentResults

{

Success,

InvalidFileFormat

}

static ModifyDocumentResults ModifyDocument(SPFile file)

{

byte[] byteArray = file.OpenBinary();

using (MemoryStream mem = new MemoryStream())

{

mem.Write(byteArray, 0, (int)byteArray.Length);

try

{

using (WordprocessingDocument wordDoc =

WordprocessingDocument.Open(mem, true))

{

XNamespace w =

"http://schemas.openxmlformats.org/wordprocessingml/2006/main";

// modify the document as necessary

// for this example, we'll convert the first paragraph to upper case

XDocument doc = wordDoc.MainDocumentPart.GetXDocument();

XElement firstParagraph = doc

.Element(w + "document")

.Element(w + "body")

.Element(w + "p");

if (firstParagraph != null)

{

string text = firstParagraph

.Descendants()

.Where(n => n.Name == w + "t" || n.Name == w + "ins")

.Select(n => (string)n)

.StringConcatenate();

firstParagraph.ReplaceWith(

new XElement(w + "p",

new XElement(w + "r",

new XElement(w + "t", text.ToUpper()))));

// write the XDocument back into the Open XML document

wordDoc.MainDocumentPart.PutXDocument();

}

// write it back to the document library

// change linkFilename if you want to write to a different location

// than the original.

string linkFilename = file.Item["LinkFilename"] as string;

file.ParentFolder.Files.Add(linkFilename, mem, true);

}

catch (System.IO.FileFormatException)

{

return ModifyDocumentResults.InvalidFileFormat;

}

return ModifyDocumentResults.Success;

}

static void Main(string[] args)

{

SPSite siteCollection = new SPSite("http://localhost");

SPWeb rootWeb = siteCollection.RootWeb;

// find a file to modify - any file will do so long as it's an Open XML

// WordprocessingDocument. For this example, I modify the first document

// in a document library named 'Open XML Documents'. However, the code

// modifies the document only if it has an extension of .docx. Further,

// the code traps exceptions when attempting to open using the Open XML SDK.

// If an exception is thrown, then the document isn't valid.

SPList spList = rootWeb.Lists["Open XML Documents"];

if (spList.ItemCount >= 1)

{

string linkFilename = spList.Items[0]["LinkFilename"] as string;

if (linkFilename != null)

{

if (linkFilename.EndsWith(".docx"))

{

SPFile file = spList.Items[0].File;

ModifyDocument(file);

}

Code is attached.

Eric White's Blog : SharePoint

Fifty SharePoint 2010 End User / Site Administrator Training Videos

Using the SharePoint 2010 Client Object Model with Open XML

Table of Contents

Overview

Retrieving a Document from a Document Library

Modifying a Document in a Document Library

Uploading a Document to a Document Library

Creating a New Document in Memory and Adding it to a Document Library

Processing all Documents in a Library

Creating an Open XML Word-Processing Document from a SharePoint Wiki

Conclusion

Additional Resources

Using the SharePoint 2010 Managed Client Object Model

Table of Contents

Introduction

Using the Managed Client Object Model

How It Works

Creating a Windows Console Managed Client Object Model Application

To Build the Application

Add References to the Microsoft.SharePoint.Client and Microsoft.SharePoint.Client.Runtime Assemblies

The Managed Client Object Model

Object Identity

Trimming Result Sets

Creating and Populating a List

Using CAML to Query a List

Filtering the Child Collection returned by Load using LINQ

Using the LoadQuery Method

Increasing Performance by Nesting Includes in LoadQuery

Filtering the Child Collection returned by LoadQuery using LINQ

Updating Client Objects

Deleting Client Objects

Discovering the Schema for Fields

Accessing Large Lists

Asynchronous Processing

Other Resources

Availability of Public Office 2010 and SharePoint 2010 Betas

SharePoint 2010 and Office 2010 Developer Training Courses Launched on Channel 9

Generating Documents from SharePoint Lists using Open XML Content Controls

Office Developer Conference moving to SharePoint Conference 2009

LINQ / Open XML at TechEd 2009

Staffing the Office / SharePoint Booth

Links – April 29, 2009

Two Upcoming Presentations on LINQ and Open XML

More SharePoint and Silverlight Samples

Modifying Open XML Documents that are in SharePoint Document Libraries using Web Services

Getting Started with SharePoint Web Services using LINQ to XML

Here are the steps:

C#

VB

Modifying Open XML Documents using the SharePoint Object Model