Eric White's Blog : Useful Code Snippets

Generating Documents from SharePoint Lists using Open XML Content Controls

EricWhite — Wed, 14 Oct 2009 03:37:00 GMT

It's often the case that a department manager needs to regularly send a nicely formatted status report to her general manager or that a team leader needs to send a weekly status report to a number of interested parties. To collaborate with others in their organizations, both the manager and the team leader can maintain status information in SharePoint lists. The question for developers is how to include the information in the lists in a document such as a status report.

I've written an article entitled Generating Documents from SharePoint with Open XML Content Controls, published in the October issue of MSDN Magazine that details how you can put together a simple, flexible, and powerful system for generating Open XML word-processing documents using SharePoint lists as sources for data in tables of word-processing documents.

The key aspect of a successful approach is to make it so that the folks who need to generate reports can do so without involving the services of a software developer. We can adopt an approach where the user can configure the sources of data for tables in a document using content controls. The user can surround a table with a content control, and set the content control tag to the name of the SharePoint list. The user can insert content controls into cells in a nicely formatted table, and set the tags to the names of the columns of the SharePoint list. We then have enough information to retrieve the necessary data from the SharePoint list (or lists) and generate a document that contains that data. If the department manager or team leader subsequently adds a new column to their SharePoint list, they can add a new column to their table, insert a new content control, set the tag, and thereafter, their reports will contain data from the new column in the SharePoint list.

To make this clear, a typical SharePoint list looks like this:

The template Open XML word-processing document might look like the following. Notice that there is a content control surrounding the entire table, with a tag and title of "Team Members", and there is a content control in a table cell with tag and title of "TeamMemberName". If we were to put the insertion point in the Role cell, you would see that there is a content control there. There are similar content controls around and in the Current Work Items table.

The resulting generated report looks like this:

One aspect of the approach that I took is that I abstracted the operations involving the content controls into a ContentControlManager class. You can call a method, GetContentControls, which returns some XML that describes the content controls in the document. In the example that I present with the article, GetContentControls returns the following XML:

</Table>

</Table>

</ContentControls>

This gives us the information that we need to use the SharePoint object model to retrieve the necessary data. After retrieving that data, we can construct a small XML tree that looks like this:

<Row>

<Field Name="TeamMemberName"

Value="Bob" />

<Field Name="Role"

Value="Developer" />

</Row>

<Row>

<Field Name="TeamMemberName"

Value="Susan" />

<Field Name="Role"

Value="Program Manager" />

</Row>

<Row>

<Field Name="TeamMemberName"

Value="Jack" />

<Field Name="Role"

Value="Test" />

</Row>

</Table>

<Row>

<Field Name="ItemName"

Value="Learn SharePoint 2010" />

<Field Name="Description"

Value="This should be fun!" />

<Field Name="EstimatedHours"

Value="80" />

<Field Name="AssignedTo"

Value="All" />

</Row>

<Row>

<Field Name="ItemName"

Value="Finalize Import Module Specification" />

<Field Name="Description"

Value="Make sure to handle all document formats." />

<Field Name="EstimatedHours"

Value="35" />

<Field Name="AssignedTo"

Value="Susan" />”

</Row>

<Row>

<Field Name="ItemName"

Value="Write Test Plan" />

<Field Name="Description"

Value="Include regression testing items." />

<Field Name="EstimatedHours"

Value="20" />

<Field Name="AssignedTo"

Value="Jack" />

</Row>

</Table>

</ContentControls>

We can pass that XML to the SetContentControls method in the ContentControlManager class, which will modify the document so that the tables in the document are populated with the data from the XML. This is useful functionality in its own right, and could be used in other scenarios than generating documents from SharePoint lists.

The MSDN article contains a download that contains the ContentControlManager class, as well as the SharePoint code to populate tables with data from SharePoint lists.

Source Code Available: Complete Implementation of 'Accept All Changes (Tracked Revisions) in Open XML Documents'

EricWhite — Mon, 28 Sep 2009 12:36:00 GMT

[Blog Map]

(Update December 21, 2009 - See Accept Revisions in Open XML WordprocessingML Documents for complete semantics of accepting tracked revisions in WordprocessingML documents.)

As most Open XML developers know, Word 2007 has a feature that tracks changes while you are editing a document. These tracked revisions are recorded in Open XML markup. It often is desirable to use the Open XML SDK to process tracked revisions and produce a new Open XML document that contains no tracked revisions. For instance, you may want to accept tracked revisions programmatically while checking in a document to a SharePoint document library, thereby ensuring that no documents in the library contain tracked revisions. Another important use of accepting revisions is before doing a complex transform, such as one from Open XML to XHTML. The transform is significantly easier to write if you first accept all tracked revisions.

Today, I've posted code in the PowerTools for Open XML project on CodePlex that contains a more complete implementation of Open XML SDK code to accept tracked revisions. To download the code, go to CodePlex.com/PowerTools, click on the Downloads tab, and download AcceptRevisions.zip. The code is released under the Microsoft Public License (Ms-PL), which gives you wide latitude in how you use the code.

The use of the code is pretty simple. You call a static method in the RevisionAccepter class with an open WordprocessingDocument. Upon return of the method, the document will contain no tracked revisions.

using (WordprocessingDocument doc =

WordprocessingDocument.Open("DocumentWithRevisions.docx", true))

{

RevisionAccepter.AcceptRevisions(doc);

}

There have been a couple of implementations of accepting tracked revisions. Some time ago, I posted some LINQ to XML code that accepts tracked revisions. In addition, the Open XML Format SDK 2.0 Code Snippets for Visual Studio 2008 contains a snippet for this purpose. Neither of those implementations is complete. For example, you can track revisions while inserting and deleting content controls, and those revisions won't be accepted by either of the above implementations. In addition, neither of the above versions process deleted paragraph marks properly. There are a number of other areas where neither of the above implementations is correct.

Accepting tracked revisions is a non-trivial problem. There are more than 40 markup elements that you must process. There are 23 elements that must simply be removed. There are four elements that must be collapsed (the element is removed, and child elements are promoted in the XML hierarchy to the level of the removed element). These are the easy ones. In addition to these, there are a handful of elements that can be processed in a few lines of code, and a handful that require somewhat extensive processing, although 'extensive' is relative. The RevisionAccepter class contains around 1700 lines of code, which is still in the category of an example program, not a real development effort.

Currently, the PowerShell cmdlet for accepting tracked revisions still uses the old approach. I'll be updating that cmdlet in a couple of months to use the new version. I've posted the AcceptRevisions code so that C# developers can take advantage of it now.

DescendantsAndSelfTrimmed LINQ to XML Axis Method

EricWhite — Wed, 19 Aug 2009 22:01:00 GMT

[Blog Map]

There are some circumstances where I need a variation on the DescendantsAndSelf axis method that allows me to specify that specific elements (and the descendants of those elements) are ‘trimmed’ from the returned collection. One of the things that's great about LINQ to XML is the ease with which we can create specialized axis methods when necessary.

Here is the scenario: Open XML contains paragraphs that contain runs. A run can contain a picture that contains a text box, which itself contains a paragraph. When I want to write a transform for paragraphs that are children elements of the w:body element, I want to disregard any paragraphs that are children of contained text boxes.

To make this clear, consider the following markup:

<body>

<r>

<t>First paragraph.</t>

</r>

<r>

<pict>

<shape>

<r>

<t>Text in a textbox.</t>

</r>

</textbox>

</shape>

</pict>

</r>

<r>

<t>Second paragraph.</t>

</r>

</body>

</document>

In this markup, I want to transform the elements highlighted in yellow, but while processing those elements, I want to completely disregard the element highlighted in green. If children paragraphs of the textbox element need to be transformed in some fashion, I would deal with them separately – perhaps in an embedded recursive transform.

To enable selecting descendant elements in this fashion, I’ve implemented four extension methods. These are variations on the Descendants and DescendantsAndSelf axis methods. The methods are:

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

XName trimName)

Returns a filtered collection of elements that contain this element, and all descendant elements of this element, in document order. Elements that have a matching XName are not included in the collection. Descendant elements of excluded elements are not included in the collection.

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

Func<XElement, bool> predicate)

Returns a filtered collection of elements that contain this element, and all descendant elements of this element, in document order. Elements that are selected by the predicate are not included in the collection. Descendant elements of excluded elements are not included in the collection.

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

XName trimName)

Returns a filtered collection of the descendant elements for this document or element, in document order. Elements that have a matching XName are not included in the collection. Descendant elements of excluded elements are not included in the collection.

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

Func<XElement, bool> predicate)

Returns a filtered collection of elements that contain this element, and all descendant elements of this element, in document order. Elements that are selected by the predicate are not included in the collection. Descendant elements of excluded elements are not included in the collection.

Here is the listing of these extension methods, as well as some example code to exercise them (also attached):

using System;

using System.Collections.Generic;

using System.Linq;

using System.Xml.Linq;

public static class LocalExtensions

{

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

XName trimName)

{

return DescendantsInternalTrimmed(element, e => e.Name == trimName, true);

}

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

Func<XElement, bool> predicate)

{

return DescendantsInternalTrimmed(element, predicate, true);

}

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

XName trimName)

{

return DescendantsInternalTrimmed(container, e => e.Name == trimName, false);

}

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

Func<XElement, bool> predicate)

{

return DescendantsInternalTrimmed(container, predicate, false);

}

private static IEnumerable<XElement> DescendantsInternalTrimmed(this XContainer container,

Func<XElement, bool> predicate, bool includeSelf)

{

if (includeSelf)

{

XElement element = container as XElement;

yield return element;

}

Stack<IEnumerator<XElement>> iteratorStack = new Stack<IEnumerator<XElement>>();

iteratorStack.Push(container.Elements().GetEnumerator());

while (iteratorStack.Count > 0)

{

while (iteratorStack.Peek().MoveNext())

{

XElement currentXElement = iteratorStack.Peek().Current;

if (predicate(currentXElement))

continue;

yield return currentXElement;

iteratorStack.Push(currentXElement.Elements().GetEnumerator());

}

iteratorStack.Pop();

}

class Program

{

static void Main(string[] args)

{

XElement rootElement = XElement.Parse(

@"<document>

<body>

<r>

<t>First paragraph.</t>

</r>

<r>

<pict>

<shape>

<r>

<t>Text in a textbox.</t>

</r>

</textbox>

</shape>

</pict>

</r>

<r>

<t>Second paragraph.</t>

</r>

</body>

</document>");

foreach (var item in rootElement.DescendantsAndSelfTrimmed("textbox"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

foreach (var item in rootElement

.DescendantsAndSelfTrimmed(e => e.Name == "p" && e.Ancestors("p").Any()))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

foreach (var item in rootElement.DescendantsTrimmed("r"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

foreach (var item in rootElement.DescendantsTrimmed(e => e.Name == "r"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

XDocument doc = XDocument.Parse(

@"<document>

<body>

<r>

<t>First paragraph.</t>

</r>

<r>

<pict>

<shape>

<r>

<t>Text in a textbox.</t>

</r>

</textbox>

</shape>

</pict>

</r>

<r>

<t>Second paragraph.</t>

</r>

</body>

</document>");

foreach (var item in doc.DescendantsTrimmed(e => e.Name == "r"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

}

Merging Comments from Multiple Open XML Documents into a Single Document

EricWhite — Tue, 28 Jul 2009 02:14:00 GMT

Microsoft Word 2007 allows you to lock a document, prohibiting users from making changes to content, while allowing them to add comments. If we have multiple documents that have the same content yet different comments, we can merge those comments into a single document. One possible use would be a specification review system. After the specification writer finishes a specification, she could send it to other members of her team for review. As each reviewer returns the specification, she could merge all comments into a single document, making it simpler to integrate those comments. (This example was inspired by an email thread with Sergey Solyanik, and his need for a comment merger for his very cool code (and soon-to-be spec) review system. Also, need to say thanks, Sergey did a code review, and the code is better for it. J) The code to do comment merging is available in a zip file named CommentMerger.zip in the downloads tab at www.codeplex.com/powertools.

Note: The CommentMerger class is part of the Power Tools for Open XML project. In the future, I’m going to build a new PowerShell cmdlet to do comment merging. PowerTools for Open XML is an open source project on CodePlex that makes it easy to create and modify Open XML documents using PowerShell scripts. It’s important to note that Power Tools for Open XML is not a supported Microsoft product and doesn’t necessarily represent future product direction. We think it will serve as inspiration for customers who need to create and modify Open XML documents programmatically. Power Tools for Open XML is published under the Microsoft public license (Ms-PL), which gives you wide latitude in how you use the code.

The CommentMerger.MergeComments method uses the Open XML SDK. The use of the comment merger class is pretty simple: You call the method passing two open WordprocessingDocument objects:

using (WordprocessingDocument destinationDocument =

WordprocessingDocument.Open("Test1a.docx", true))

using (WordprocessingDocument sourceDocument =

WordprocessingDocument.Open("Test1b.docx", false))

{

CommentMerger.MergeComments(destinationDocument, sourceDocument);

}

Upon return, the comments in the source document are merged into the destination document. To merge comments from multiple documents, you can call the function multiple times.

I wrote the CommentMerger.MergeComments method in the pure functional style. All methods are written without side-effects. After initializing, no variables are mutated. For a detailed explanation of this approach, see Recursive Approach to Pure Functional Transformations of XML.

The CommentMerger.MergeComments method is an example of a ‘Common-vocabulary document-centric transform’. For an overview of these types of transforms, see Document-Centric Transforms using LINQ to XML.

Before merging comments, the CommentMerger.MergeComments method validates that the two documents contain the same content. For more info, see Comparing Two Open XML Documents using the Zip Extension Method.

This code is based on the code I presented in Splitting Runs in Open XML Word Processing Document Paragraphs.

This code pre-atomizes XName objects. See A More Robust Approach for Handling XName Objects in LINQ to XML.

The code uses the Open XML SDK.

Recursive Approach to Pure Functional Transformations of XML

EricWhite — Mon, 20 Jul 2009 16:28:00 GMT

[Blog Map]

Writing pure functional transformations a in a recursive style enables us to put together interesting transformations in a very small amount of code. Using some specific techniques that allow us to write this code very concisely, this approach takes advantage of some perhaps obscure semantics of LINQ to XML. I’ve used this approach to write some interesting transformations – it has become my favorite way to write transformations of a certain variety. This post presents a short tutorial in writing these types of transformations.

Identity Transform

First, let’s take a look at the identity transform:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(@"<Root Att1='1'><Child/></Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

In this code, we are taking advantage of the semantics of the XElement constructor that allow us to pass collections of objects to the constructor. The following code creates a new XElement from an existing XElement:

new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

This code, of course, passes two objects to the constructor (in addition to the name) – a collection of attributes from the source element, and a collection of transformed child nodes from the source element. Note that even though we’re passing collections to the constructor, we’re only passing two objects. We’re passing the two variables that contain the iterators for the queries. Those iterators, of course, implement IEnumerable<T>. There are a number of LINQ to XML methods and constructors that can take content, and if we pass an object that implements IEnumerable of some T, LINQ to XML iterates the enumerable and adds each object in the collection to the newly constructed element. This is performed recursively. For detailed semantics of the constructors and methods, see this page in the LINQ to XML documentation.

This is one of the reasons that the signatures of LINQ to XML constructors and methods that take content are defined using object instead of XObject:

public XElement(XName name, params Object[] content);

Because the signature is defined using object, we can pass individual XElement, XAttribute and XNode objects to the constructor, and we can pass collections of them.

Cloning vs. Attaching

The identity transform also takes advantage of the cloning vs. attaching semantics of LINQ to XML. When we pass a newly constructed XElement to the XElement constructor, the new object does not have a parent so it is simply attached to the XElement that is being constructed. But the highlighted line below causes a node from an existing tree to be passed to the XElement constructor. In this case, the XElement constructor notices that the node is already part of an existing tree, and clones it and adds the cloned node to the XElement being constructed.

XElement element = node as XElement;

if (element != null)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

To make this very clear, take a look at the following example:

XElement tree1 = XElement.Parse("<Root><Child1/></Root>");

XElement child1 = tree1.Element("Child1");

XElement child2 = new XElement("Child2");

XElement tree2 = new XElement("Root", child1, child2);

Console.WriteLine(tree2);

In this example, child1 already is part of tree1, so it is cloned and the newly cloned element is added to tree2. The child2 XElement object has no parent so it is simply attached to tree2.

For more information on cloning vs. attaching, see this page in the LINQ to XML documentation.

Note that when an XElement object is cloned by a LINQ to XML constructor or method, of course all attributes and descendant nodes are also cloned.

Removing Nodes from the Newly Cloned Tree

Sometimes when we clone a tree, we want to trim certain elements and attributes and nodes from the newly cloned tree. We can take advantage of the fact that it’s valid to pass null as one of the arguments to the XElement constructor. If one of the arguments to the XElement constructor is null, the XElement constructor simply ignores that argument:

XElement xml = new XElement("Root",

new XElement("Child1"),

null,

new XElement("Child3"));

Console.WriteLine(xml);

This produces the following output:

<Root>

</Root>

Let’s say that we want to trim all Child2 elements from the following tree:

<Root>

</Root>

We can code the transform like this:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return null;

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(@"<Root><Child1/><Child2/><Child3/></Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

These semantics apply regardless of whether null is passed directly to the XElement constructor, or whether null is passed as part of a collection to the XElement constructor.

Such element name comparisons (element.Name == "Child2") are very efficient if we pre-atomize element and attribute names. See this post for more information about atomization of element and attribute names.

Note: for simplicity of demonstration, in these examples, I didn’t bother to pre-atomize XNames, but pre-atomization is important if performance is important.

Replace an Element with another Element

A common operation in transforms is to replace some particular element with an entirely new element. Let’s say that we want to replace the Child2 element with a Child5 element that contains a child element named Text. We want to transform this:

<Root>

</Root>

To this:

<Root>

</Child5>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return new XElement("Child5",

new XElement("Text"));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

Replace an Element with Multiple Elements

Sometimes we may want to replace a single element with multiple elements. We can take advantage of the fact that the behavior of the XElement constructor for collections of objects is, as mentioned, recursive. If one of the objects in the collection is itself a collection of objects then that child collection is itself iterated and all objects are added to the element being constructed.

The following example shows these semantics of the XElement constructor. The example first creates two lists of XElement objects. It then creates a list that contains the two lists. It passes the list of lists to the XElement constructor, and the resulting XElement contains all of the XElement objects in the two lists:

List<XElement> list1 = new List<XElement> {

new XElement("Child1"),

new XElement("Child2")

};

List<XElement> list2 = new List<XElement> {

new XElement("Child3"),

new XElement("Child4")

};

List<List<XElement>> listOfLists = new List<List<XElement>> { list1, list2 };

XElement newElement = new XElement("Root", listOfLists);

Console.WriteLine(newElement);

This produces the following output:

<Root>

</Root>

Let’s say that we want to replace the Child2 element with three Child5 elements. We want to transform this:

<Root>

</Root>

To this:

<Root>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return new List<XElement> {

new XElement("Child5"),

new XElement("Child5")

};

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

This example is coded a little bit artificially – more typically we would want to replace a particular element with a number of elements that are contained in a collection that is returned by some query. Let’s say that we wanted to effectively remove a certain element from the tree, and replace that element with its children elements. We want to transform this:

<Root>

<Child>

</Child>

</Root>

To this:

<Root>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child")

return element.Elements();

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<Root>

<Child>

</Child>

</Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

This is why it’s important that the Transform function returns object instead of XNode or XObject. This is of course an exact parallel to the XElement constructor – the XElement constructor takes object as an argument so that you can pass either an XObject (or objects of derived classes) or a collection of XObject. The Transform function returns object so that you can return either an XObject or a collection of XObject. (There are other reasons that the XElement constructor takes object as a parameter, but these are the relevant reasons for this discussion.)

More Sophisticated Selection of Nodes to Transform

In the above examples, the code selected elements based on the element name. In XSLT, of course, we often write involved XPath expressions to select nodes to transform. Sometimes we want to select nodes for transformation where the node has specific ancestors. Let’s say that we want to transform a paragraph only if the paragraph is in a content control:

<body>

abc

<sdt>

def

</sdt>

</body>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "p" && element.Parent.Name == "sdt")

return new XElement("p", "different content");

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<body>

abc

<sdt>

def

</sdt>

</body>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

This will be efficient – if the node isn’t a paragraph node, then the Boolean expression will be short-circuited, and no extra work will be done. If the node is a paragraph node, then the code consists of following a pointer in the ancestor linked list, and another efficient XName comparison. I've written even more involved node selection code than this - select a paragraph to transform based on contents of the previous paragraph, for example. (Note that if you are selecting nodes in reverse document order, you may want to use an approach that gives good performance.)

Selectively Transforming Descendant Nodes

Sometimes we want to transform just a specific set of the descendant nodes, ignoring everything else. In the following query, the highlighted line serves the same purpose as xsl:apply-templates in an XSLT style sheet. This is the code that sends each of the descendants through the transform function. If we include a call to the Where extension method in that query, we can cause only a subset of the descendants to be sent through the recursive function.

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

Let’s say that we want to perform a transform on the following XML document:

<Root>

<Child>

</Child>

</Root>

We want to transform all of the ’A’ elements to ‘Apple’, the ‘B’ elements to ‘Banana’ and the ‘C’ elements to ‘Carot’, but only for all of the elements where Id equals ‘1’. We want the resulting XML file to look like this:

<Root>

<Child>

</Child>

</Root>

We can code the transform like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "A")

return new XElement("Apple",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "B")

return new XElement("Banana",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "C")

return new XElement("Carot",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "Child")

return new XElement("Child",

element.Attributes(),

element.Elements()

.Where(e => e.Attribute("Id").Value == "1")

.Select(e => Transform(e)));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

Only the XML elements that satisfy the predicate in the Where clause will be sent back through the transform function.

Notice that in this case, I used the Elements axis instead of the Nodes axis. Based on the needs of my transform, I can use any of a variety of axes to specify which nodes will be sent back through the recursive transform function.

Using Multiple Recursive Functions in the Same Transform

These types of transforms are always written in the pure style. In other words, no variables are ever mutated the while doing this transform. The transform always produces a completely new XML tree and never modifies the source tree. This gives us the freedom to call out to other recursive transforms that are also implemented in the pure style.

Let’s say that we’re transforming an Open XML document and we’ve written an interesting transform just for the paragraph element, we can write our transform for the main document part to call the paragraph transform function as appropriate. Because both transforms are written in a pure fashion, we can do this without worrying about introducing bugs associated with side effects in our code.

Following is the source document that we want to transform. This XML is ‘pseudo’ Open XML markup – namespaces are removed and the XML is simplified for the purposes of demonstration.

<body>

<r>

<t>This is a short sentence.</t>

</r>

<r>

<t>This is a different sentence.</t>

</r>

<sdt>

<r>

<t>This is a sentence in a content control.</t>

</r>

</sdt>

</body>

</document>

We want to produce the following document:

<body>

<r>

<t>THIS IS A SHORT SENTENCE.</t>

</r>

<r>

<t>THIS IS A DIFFERENT SENTENCE.</t>

</r>

<r>

<t>THIS IS A SENTENCE IN A CONTENT CONTROL.</t>

</r>

</body>

</document>

We could code the transformation like this:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Xml.Linq;

class Program

{

// This paragraph transform might do something very interesting - perhaps splitting

// runs in a new way or doing sophisticated modifications to styling. In this example,

// the code will transform the text node to upper case.

static object ParagraphTransform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "t")

return new XElement("t", ((string)element).ToUpper());

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => ParagraphTransform(n)));

}

return node;

}

// This transform might do something very interesting with content controls. In this

// case the transform will do something very simple - it removes the content control

// and promotes its children elements to the level of the content control.

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

// call into the recursive transform for the paragraph

if (element.Name == "p")

return ParagraphTransform(element);

// remove the content control

if (element.Name == "sdt")

return element.Nodes().Select(n => Transform(n));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<document>

<body>

<r>

<t>This is a short sentence.</t>

</r>

<r>

<t>This is a different sentence.</t>

</r>

<sdt>

<r>

<t>This is a sentence in a content control.</t>

</r>

</sdt>

</body>

</document>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

Because this transformation was coded in this style, artifacts of Open XML such as custom XML elements and nested content controls are handled automatically without us taking any special care to handle them.

Sometimes it’s useful to write small utility functions for some of the functionality supplied by each recursive transform function. It may be useful to put each transform function and its associated utility functions in a static class.

Normalization of Empty Elements

There is one point regarding empty elements that is worth mentioning – empty elements <Tag></Tag> are normalized to use the empty-element tag <Tag />. This post contains a discussion on empty elements.

I had an unfortunate bike accident last week and broke my right arm, and I’m right handed! I used the voice recognition features of Windows 7 to write this post. These features are great! I don’t think that the process of writing this post was appreciably slower than usual.

Document-Centric Transforms using LINQ to XML

EricWhite — Thu, 09 Jul 2009 21:16:00 GMT

[Blog Map]

When thought of in a certain way, XML documents come in two flavors – data-centric and document-centric. Further, there are two types of document-centric documents. This post presents my thoughts about approaches to various types of document-centric transformations – data-centric to document-centric, document-centric to data-centric, and document-centric to document-centric. Then, I’ll tie my thoughts back to Open XML transformations.

Data-centric to data-centric is, of course, the scenario that LINQ to XML absolutely shines at. There’s been a lot written about this. This post won’t focus on these types of transformations, but instead will give my thoughts on the wrinkle that document-centric XML documents give to transformations.

First, I’ll define what I mean by data-centric and document-centric XML documents.

Data-Centric XML Document

A data-centric XML document contains regular repeating elements. Child elements of a given element might all have the same tag name, or they might not. Typically, child element order doesn’t matter. There are lots of examples of this – many types of transforms of a relational database to XML results in data-centric XML. RSS feeds are another.

Here’s a data-centric XML document:

</Customer>

</Customer>

</Customers>

Document-Centric XML Document

Document-centric XML documents have the characteristic that the child elements of a given element are much less bounded – you might have many child elements of a given name, or you might have none. You might have ‘recursion’ in the hierarchy – element A is a child of element B, which is itself a child of a different element A. A number of examples: Open XML word processing markup, XHTML, and XPS.

I further divide document-centric XML documents into two camps – those that contained mixed content, and those that don’t. Mixed content is a variety of XML where significant text nodes and elements are interspersed. Insignificant text nodes are the white space that provides indenting when formatting XML. Open XML word processing markup doesn’t contain mixed content, whereas XHTML does:

An Open XML paragraph that contains some bold text:

<w:p>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:r>

<w:rPr>

<w:b/>

</w:rPr>

<w:t>def</w:t>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

An XHTML document that contains significant text nodes interspersed with element start and end tags:

<html>

<body>

abcdefghi

</body>

</html>

Types of Transformations

If we’re going to divide the XML world into the categories of data-centric and document-centric, it follows that there are four types of transformations.

Data-Centric => Data Centric

There is a lot to say (and has been said) about these types of transformations. In the LINQ to XML documentation, I included a tutorial on pure functional transformations of XML. I also have a tutorial on my blog on composing queries in the pure functional style.

Data-Centric => Document Centric

These transforms are report writers for databases – take some subset of records, transform to XML, then transform that XML into another form – XPS, for instance. The transform may be based on another source document, the report definition. These types of transforms are straightforward to write in the pure functional style. Based on the simplicity or complexity of the report definition, this type of transform could be a few hundred lines of code or many thousands.

There are also many good examples of transforming data-centric XML to Open XML markup. We may want to transform a collection of records into a table in a word processing document, or into rows and cells in a worksheet.

Document-Centric => Data Centric

We write this type of transform when querying an Open XML document for some aspect of the markup. If we want to retrieve a collection of comments from a document, or if we want a collection of content controls, then we write a query that iterates over certain descendant elements, projecting a regular data structure – perhaps a collection of strings or anonymous types. The query that I develop in my functional programming tutorial is a document-centric => data-centric transform.

Another example is finding all hyper-links in an XHTML document. It is easy to write a LINQ query to retrieve a collection of links and transform the collection to a regular repeating data structure.

Document-Centric => Document-Centric

This is where it starts to get a little more involved. There are a variety of these types of transformations.

Common-vocabulary document-centric transform: Sometimes we want to transform a XML tree to a new tree in the same vocabulary. For example, we can transform an Open XML document into a new Open XML document with modified contents - comments are removed or revisions accepted. Another example - replace content controls with other markup based on the contents of the content controls.

Different-vocabulary document-centric transform: Sometimes we want to transform from one document-centric vocabulary to another one – Open XML => XHTML, or XHTML => Open XML. With this type of transform, the ease with which we write the transform is directly related to whether the two vocabularies have a similar structure. For instance, there is much that is parallel between Open XML and XHTML. There is a body element. The body contains paragraphs and tables. Tables contain rows, which contain cells. Tables can contain other tables in cells.

XSLT works well for these types of transformations – you write a pattern to match a node, and then supply the transformation for just that node. In the case of XSLT, you can indicate to the transform engine to ‘continue processing rules for child elements’, so that you can specify the transforms for those child elements in their own rules. If you are aware of Flat OPC, it is pretty easy to process Open XML documents using XSLT.

Some time ago, I write a post on an approach for using LINQ to XML annotations for doing this type of transform. In that post, I was proving out whether you could write document-centric transforms using LINQ to XML in a style similar to XSLT. It’s easy to read the code to specify the transform if you read LINQ code easily, but there are obvious problems with the approach, not the least of which is that annotating a tree in that fashion might have performance issues if you are working with too large of a tree.

Even though Open XML and XHTML have similar structures, there are places where the structures are not parallel, and in those cases, you still must jump through hoops. In XSLT, this often means generating intermediate trees to use in subsequent transforms. I’ve seen XSLT transforms where the first thing the developer did was to transform the tree to a new tree with new attributes on elements – the purpose of the attributes was to aid further transforms. If using the LINQ to XML approach that uses annotations, you must deal with the same issues – parts of the transformation are expressed as nice mappings between a pattern that matches nodes and the subsequent transform of those nodes, and parts of the transformation deals with abstractions that often must be explained in comments. It’s just more complicated to do these types of transformations.

Open XML Document-Centric Transforms

There are lots of examples of interesting common-vocabulary document-centric Open XML transforms. Removing comments and accepting revisions are two, but there are many others.

I've needed to write a number of these over the last couple of years. Because I know the size of documents that I potentially need to process (>2 million nodes), I rejected the annotations approach for simple transforms of Open XML documents. For performance reasons, it just wouldn’t fly.

I also rejected using XSLT – I really don’t want to step out into another language. XSLT is an attractive approach if you already have an XSLT transform written, or if you are particularly fluent in XSLT. You must deal with converting the OPC (Zip) file to the Flat OPC format, but this is easy. But when I’m writing little examples that show how to do something interesting in Open XML, XSLT isn’t appropriate.

So, for instance, for the code to accept tracked revisions, I opted for the tree-modification approach. This isn’t idea from a functional programming purist’s point of view, but it performs well in the real world. You have to be careful when coding, but no big deal.

Recursive Approach to Transforms

Lately, I’ve been writing more of these types of transforms in a recursive style. The gist of this technique is that you write a recursive function to clone a tree, and while cloning, you trim nodes, or transform nodes, or whatever.

This approach has good performance, and it is appealing in that when you are writing a more complicated recursive transform, you can write it in terms of other simpler recursive transforms. The code should be written with no side effects, and if so, transforms are easy to write and debug.

This approach has a draw-back. It’s somewhat harder to intuitively see the mapping between the pattern that matches a node, and the transform for that pattern. However, we don’t lose this entirely. For example, we may want to write a recursive transform of an XHTML document to Open XML. Here is the XHTML document:

<html>

<body>

abcdefghi

</body>

</html>

We want to transform it to this document:

<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" >

<w:body>

<w:p>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:r>

<w:rPr>

<w:b/>

</w:rPr>

<w:t>def</w:t>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

</w:body>

</w:document>

We can write the recursive transform like this:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object XHtmlToOpenXml(XNode node)

{

XNamespace w = "http://schemas.openxmlformats.org/wordprocessingml/2006/main";

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "html")

return new XElement(w + "document",

new XAttribute(XNamespace.Xmlns + "w", w.NamespaceName),

new XElement(w + "head", ""),

element.Elements().Select(e => XHtmlToOpenXml(e)));

if (element.Name == "body")

return new XElement(w + "body",

element.Elements().Select(e => XHtmlToOpenXml(e)));

if (element.Name == "p")

return new XElement(w + "p",

element.Nodes().Select(n => XHtmlToOpenXml(n)));

if (element.Name == "b")

return new XElement(w + "r",

new XElement(w + "rPr",

new XElement(w + "b")),

new XElement(w + "t",

element.Value));

}

XText t = node as XText;

if (t != null)

return new XElement(w + "r",

new XElement(w + "t", t.Value));

// ignore all other nodes

return null;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<html>

<body>

abcdefghi

</body>

</html>");

Console.WriteLine(XHtmlToOpenXml(root));

}

In the above transform, the code highlighted in yellow serves the same purpose as the XPath pattern to match in an XSLT template. The code highlighted in green is the “sequence constructor”. The expressions element.Nodes().Select(n => XHtmlToOpenXml(n)) and element.Elements().Select(e => XHtmlToOpenXml(e)) serves the same purpose as the xsl:apply-templates element in an XSLT template.

Key to understanding this transform is that selectively, we can cause every node/element to be passed to this method. But we can also trim descendant nodes if we like, sending only a subset back through this method.

I initially started talking about this approach in a post that described manually cloning XML trees. The code is short and easy to understand.

I used this approach for code to normalize an XML tree. It performs well. Of the approaches that I could have taken for coding the sample, it was by far the easiest.

I also used this approach for the code to split runs in paragraphs. Again, it was the easiest way for me to write the code.

For more information about writing these types of transformations, see Recursive Approach to Pure Functional Transformations of XML.

This certainly isn’t the last word. This is what has been on my mind lately, so wanted to blog it before I forgot about it.

I’m fascinated by XML document transformation, primarily because of the power it gives me. The ability to spin out an Open XML document in a couple hundred lines of code opens up a lot of interesting scenarios. Generating documents server-side in SharePoint or a web application allows us to use documents to make it easier for people to communicate. Document-centric transforms are key in these scenarios.

Comparing Two Open XML Documents using the Zip Extension Method

EricWhite — Sun, 05 Jul 2009 19:09:00 GMT

[BlogMap]

(Note: this is a post on comparing two Open XML word processing documents. For a post on comparing two XML documents, see Equality Semantics of LINQ to XML Trees.)

Sometimes we want to compare two word processing documents to see if they contain the same content. I’m working on a blog post to merge comments from multiple Open XML documents into a single document. This is based on a feature in Word 2007 that allows you to lock a document and prevent changes to content, yet allows users to add comments to the document. However, we don’t want to attempt to merge comments if the documents don’t contain the same content.

One Approach to Comparing Two Documents for Equivalency

If two documents contain exactly the same content, they will have the same number of paragraphs, tables, content controls, and custom XML markup, and more, and these elements will occur in the same order, and have the same content. However, two paragraphs may contain the same content yet their XML representation may be very different if one has a comment and the other does not – the paragraph with the comment may have its runs split differently. I’ve written a previous post that examines run splitting in detail, and contains a method to report where the run splits are, and a method that splits runs based on a list of split locations.

The following markup shows a very simple paragraph. We can see the paragraph element, the run element, and the text element.

<w:p>

<w:r>

<w:t>abcdefghi</w:t>

</w:r>

</w:p>

If we select “def” in the above text, and add a comment, the markup changes to look like this:

<w:p>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:commentRangeStart w:id="0"/>

<w:r>

<w:t>def</w:t>

</w:r>

<w:commentRangeEnd w:id="0"/>

<w:r>

<w:rPr>

<w:rStyle w:val="CommentReference"/>

</w:rPr>

<w:commentReference w:id="0"/>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

We can write a query that returns a collection of a very specific subset of the elements in the XML document. This is the subset of elements that won’t change if the contents of the document don’t change. This query consists of all elements in the document except:

w:commentRangeStart and w:commentRangeEnd – these elements will be added when the user adds comments to a document. Most commonly, these elements occur under the paragraph element (to be trimmed, see below), but its valid for these elements to be children of the body elements, so we should trim them.
w:proofErr – this element is added automatically by Word when there are spelling or grammar errors, and has no effect on content. Word can (and will) add this element even though the document is locked for editing with the exception of being able to add comments. Therefore, we want to trim this element from the collection returned by a query that we’re going to use to determine document equivalency.
Finally, we want to eliminate all of the descendants of paragraphs from the query, as these elements can change quite a bit even if the contents of the document don’t change. Instead, we want to write a bit of code to determine whether two paragraphs are equivalent.

Here is the query that returns a collection of the elements that we’re interested in:

XDocument xDoc1 = doc1.MainDocumentPart.GetXDocument();

var doc1Elements = xDoc1

.Descendants()

.Where(e => e.Name != W.commentRangeStart &&

e.Name != W.commentRangeEnd &&

e.Name != W.proofErr &&

!e.Ancestors(W.p).Any());

We can query two word processing documents, and if the elements in the returned collection are not in the exact same order, then the documents are different. And if corresponding paragraphs contain the same content, per whatever algorithm that we define, then we can say that the documents contain the same content. In the example that I present in this post, I validate paragraph equivalency by checking actual textual content, disregarding formatting changes for runs within the paragraph. In my case, this is good enough, as the transformation that I wrote (and will present in an upcoming post) that moves comments from one document to another will work properly if the paragraphs have the same text.

The above query works just fine for documents that contain tables, content controls, and custom XML. The markup for tables, content controls, and custom XML contains paragraphs, and the paragraphs will be in the same order and have the same content if the documents are equivalent.

We could change this query easily enough to define document equality in just about any way we want. If we want to disregard bookmarks, it’s easy enough to remove them from the results of the query.

The above query is not the most efficient way to do this – more efficient would be to write an iterator that goes through the Descendants axis and trims appropriately. But queries show intent in a better way, and in my informal testing, the above query performs well enough as is for many scenarios.

Now that we’ve defined the query that will return the elements that won’t change if the document content doesn’t change, we can define another query that determines if two queries, evaluated on two Open XML documents, contain the same items, and in the same order. This is a job for the Zip extension method, coming with C# 4.0.

The Zip Extension Method

The Zip extension method processes two sequences, matching up each item in one sequence with a corresponding item in another sequence. While this method won’t be part of the framework until C# 4.0, a simple implementation that we can use with C# 3.0 is trivial:

public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(

this IEnumerable<TFirst> first,

IEnumerable<TSecond> second,

Func<TFirst, TSecond, TResult> func)

{

var ie1 = first.GetEnumerator();

var ie2 = second.GetEnumerator();

while (ie1.MoveNext() && ie2.MoveNext())

yield return func(ie1.Current, ie2.Current);

}

Note: Bart De Smet has a great explanation of the Zip extension method, as well as this example of the implementation of it. That post also has a good explanation of how iterators work, using IL to explain them. With regards to iterators, it’s also useful to read the section 8.14 in the C# 3.0 specification.

Using the Zip Extension Method

If we have one sequence that contains names, and another sequence that contains ages, and we know that the two sequences contain corresponding elements, we can project a new collection of anonymous objects:

string[] names = new[] { "Jim", "Bob", "Susan" };

int[] ages = new[] { 50, 35, 41 };

var q = names.Zip(ages, (name, age) => new

{

Name = name,

Age = age

});

foreach (var item in q)

Console.WriteLine(item);

When you run this example, you see:

{ Name = Jim, Age = 50 }

{ Name = Bob, Age = 35 }

{ Name = Susan, Age = 41 }

Notice that for the projection, we write a lambda expression that takes two arguments – each pair of corresponding items from the two source collections is passed as arguments to the lambda expression.

The following query uses the Zip extension method to project a collection of Booleans indicating if the element or paragraph is equivalent:

IEnumerable<bool> correspondingElementEquivalency = doc1Elements.Zip(doc2Elements, (e1, e2) =>

{

if (e1.Name != e2.Name)

return false;

// determine if two paragraphs contain the same content

if (e1.Name == W.p && (GetParagraphText(e1) != GetParagraphText(e2)))

return false;

return true;

});

GetParagraph text is defined as:

// return the text of a paragraph with revisions accepted

public static string GetParagraphText(XElement p)

{

return p.Descendants(W.r)

.Where(e => e.Parent.Name != W.del && e.Parent.Name != W.moveFrom)

.Descendants(W.t)

.Select(t => (string)t)

.StringConcatenate();

}

So then, we can use the Any extension method to determine if the documents are equivalent:

return ! correspondingElementEquivalency.Any(e => e != true);

This will be pretty efficient, as it uses lazy evaluation, and the Any extension method will terminate processing as soon as the code determines that the documents are different.

One Final Note

This code doesn’t process math markup – if the two documents contain a math formula, and one of the documents is commented, then this query will report that the documents differ. The structure and approach to take are exactly parallel to the approach that I take with comments in regular paragraphs. Extending this to math markup is another post.

The Code

Following is an example that compares two documents to determine if they have the same content. The code is quite short. Note this uses the Open XML SDK.

using System;

using System.Collections.Generic;

using System.IO;

using System.Linq;

using System.Text;

using System.Xml;

using System.Xml.Linq;

using DocumentFormat.OpenXml.Packaging;

public static class Extensions

{

public static XDocument GetXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.Annotation<XDocument>();

if (xdoc != null)

return xdoc;

using (StreamReader streamReader = new StreamReader(part.GetStream()))

xdoc = XDocument.Load(XmlReader.Create(streamReader));

part.AddAnnotation(xdoc);

return xdoc;

}

public static string StringConcatenate(this IEnumerable<string> source)

{

StringBuilder sb = new StringBuilder();

foreach (string s in source)

sb.Append(s);

return sb.ToString();

}

public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(

this IEnumerable<TFirst> first,

IEnumerable<TSecond> second,

Func<TFirst, TSecond, TResult> func)

{

var ie1 = first.GetEnumerator();

var ie2 = second.GetEnumerator();

while (ie1.MoveNext() && ie2.MoveNext())

yield return func(ie1.Current, ie2.Current);

}

public static class W

{

public static XNamespace w =

"http://schemas.openxmlformats.org/wordprocessingml/2006/main";

public static XName p = w + "p";

public static XName r = w + "r";

public static XName t = w + "t";

public static XName commentRangeStart = w + "commentRangeStart";

public static XName commentRangeEnd = w + "commentRangeEnd";

public static XName proofErr = w + "proofErr";

public static XName del = w + "del";

public static XName moveFrom = w + "moveFrom";

}

class Program

{

// return the text of a paragraph with revisions accepted

public static string GetParagraphText(XElement p)

{

return p.Descendants(W.r)

.Where(e => e.Parent.Name != W.del && e.Parent.Name != W.moveFrom)

.Descendants(W.t)

.Select(t => (string)t)

.StringConcatenate();

}

// returns true if the documents contain the same content, otherwise false

private static bool CompareDocuments(WordprocessingDocument doc1,

WordprocessingDocument doc2)

{

XDocument xDoc1 = doc1.MainDocumentPart.GetXDocument();

XDocument xDoc2 = doc2.MainDocumentPart.GetXDocument();

var doc1Elements = xDoc1

.Descendants()

.Where(e => e.Name != W.commentRangeStart &&

e.Name != W.commentRangeEnd &&

e.Name != W.proofErr &&

!e.Ancestors(W.p).Any());

var doc2Elements = xDoc2

.Descendants()

.Where(e => e.Name != W.commentRangeStart &&

e.Name != W.commentRangeEnd &&

e.Name != W.proofErr &&

!e.Ancestors(W.p).Any());

IEnumerable<bool> correspondingElementEquivalency = doc1Elements

.Zip(doc2Elements, (e1, e2) =>

{

if (e1.Name != e2.Name)

return false;

// determine if two paragraphs contain the same content

if (e1.Name == W.p && (GetParagraphText(e1) != GetParagraphText(e2)))

return false;

return true;

});

return ! correspondingElementEquivalency.Any(e => e != true);

}

static void Main(string[] args)

{

using (WordprocessingDocument doc1 = WordprocessingDocument.Open("Test3a.docx", false))

using (WordprocessingDocument doc2 = WordprocessingDocument.Open("Test3b.docx", false))

{

bool same = CompareDocuments(doc1, doc2);

Console.WriteLine(same);

}

Splitting Runs in Open XML Word Processing Document Paragraphs

EricWhite — Tue, 30 Jun 2009 01:49:00 GMT

[Blog Map]

(July 1, 2009 - Updated TransformRun to be recursive)

In Open XML Word processing document markup, paragraphs contain runs, and runs contain text elements. Sometimes when transforming a document, we may want to split runs differently than in the original document. This post presents a couple of small functions that help us deal with paragraphs and runs – determine the split locations of runs, and to split runs.

Word 2007 has a neat feature where you can lock a document and disallow editing of the content; yet allow the user to add comments. You can send this document for review to a number of users, and after the reviewers return the documents, it would be handy to have some code that merges comments from all documents into a single document. I’m currently working on a blog post that shows how to do this. However, adding a comment to a paragraph can cause runs to be split, which adds a bit of complexity.

Paragraphs, Runs, and Text Elements

The following markup shows a very simple paragraph. We can see the paragraph element, the run element, and the text element.

<w:p>

<w:r>

<w:t>abcdefghi</w:t>

</w:r>

</w:p>

If we select “def” in the above text, and add a comment, the markup changes to look like this:

<w:p>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:commentRangeStart w:id="0"/>

<w:r>

<w:t>def</w:t>

</w:r>

<w:commentRangeEnd w:id="0"/>

<w:r>

<w:rPr>

<w:rStyle w:val="CommentReference"/>

</w:rPr>

<w:commentReference w:id="0"/>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

In this paragraph, we can see the commentRangeStart and commentRangeEnd elements. In addition, we can see a special run that contains information on the styling of the text that is commented. This special run contains a commentReference element.

If we want to programmatically insert a comment into a document, we need to split runs as appropriate so that we can insert commentRangeStart, commentRangeEnd, and the special run that contains commentReference into the paragraph.

Note that a paragraph can be split into runs for a variety of reasons, and that there are a number of other valid child elements of the paragraph element. For example, because the above text isn’t a correctly spelled word, and isn’t a sentence with proper grammar, the markup can include w:proofErr elements:

<w:p>

<w:proofErr w:type="spellStart"/>

<w:proofErr w:type="gramStart"/>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:commentRangeStart w:id="0"/>

<w:r>

<w:t>def</w:t>

</w:r>

<w:commentRangeEnd w:id="0"/>

<w:proofErr w:type="gramEnd"/>

<w:r>

<w:rPr>

<w:rStyle w:val="CommentReference"/>

</w:rPr>

<w:commentReference w:id="0"/>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

<w:proofErr w:type="spellEnd"/>

</w:p>

When splitting runs, we want to honor those existing run splits, and make sure that we don’t disturb those other elements.

As Open XML developers know, content controls and custom XML markup are very powerful features of Open XML. They enable a vast number of scenarios – we can make our documents smarter. However, they add an interesting twist to markup. The element for content controls is w:sdt, which contains another element, w:sdtContent, which contains the contents. This means that runs that we potentially want to split occur at different levels of the XML hierarchy:

<w:p>

<w:r>

<w:t>123</w:t>

</w:r>

<w:sdt>

<w:sdtContent>

<w:r>

<w:t>4567</w:t>

</w:r>

</w:sdtContent>

</w:sdt>

<w:r>

<w:t>890</w:t>

</w:r>

</w:p>

Custom XML markup has the same issue. The following schema defines some custom XML markup:

<?xml version="1.0" encoding="utf-8"?>

<xs:schema attributeFormDefault="unqualified"

elementFormDefault="qualified"

xmlns:xs="http://www.w3.org/2001/XMLSchema">

<xs:element name="Root">

<xs:complexType>

<xs:sequence>

<xs:element name="Child"

type="xs:string" />

</xs:sequence>

</xs:complexType>

</xs:element>

</xs:schema>

When we use this custom schema to add structure to a document, it looks like this:

The markup looks like this:

<w:p>

<w:customXml w:uri="http://northwind.com"

w:element="Root">

<w:r>

<w:t>12</w:t>

</w:r>

<w:customXml w:uri="http://northwind.com"

w:element="Child">

<w:r>

<w:t>34</w:t>

</w:r>

</w:customXml>

<w:r>

<w:t>56</w:t>

</w:r>

</w:customXml>

<w:r>

<w:t>7890</w:t>

</w:r>

</w:p>

We may need to split runs at any level - as a child of the paragraph, as content in a content control, or within custom XML markup. We need to use a recursive transform to do the transform, which then handles this issue nicely.

Determining Run Split Locations

The first piece of functionality that we need is a method to return an array of integers indicating where run splits are. If we are moving comments from one document to another, then we want to find out where the run splits are in the source document so that we can create the same run splits in the destination document.

Here’s the prototype of simple method to do so:

static int[] RunSplitLocations(XElement paragraph)

The following paragraph markup contains three runs:

<w:p>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:commentRangeStart w:id="0"/>

<w:r>

<w:t>def</w:t>

</w:r>

<w:commentRangeEnd w:id="0"/>

<w:r>

<w:rPr>

<w:rStyle w:val="CommentReference"/>

</w:rPr>

<w:commentReference w:id="0"/>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

If we call RunSplitLocations for this paragraph, it returns an array that contains:

Splitting Runs

If we have another document that contains no comments in this paragraph, and we want to split runs so that we can insert a comment on the middle three characters, we can call another method that takes an array of integers to do the splitting:

public static XElement SplitRunsInParagraph(XElement p, int[] positions)

If we have a paragraph with this markup:

<w:p>

<w:r>

<w:t>abcdefghi</w:t>

</w:r>

</w:p>

And we call SplitRunsInParagraph passing an array that contains 0, 3, and 6, it returns a paragraph that looks like this:

<w:p xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main">

<w:r>

<w:t>abc</w:t>

</w:r>

<w:r>

<w:t>def</w:t>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

As I previously mentioned, the paragraph may contain child elements other than runs. SplitRunsInParagraph will leave those other elements in place. Also, a run can contain styling information, which we also want to leave in place.

Now that we have some methods to determine where run splits are, and to create run splits, it will be pretty simple to write a pure functional transform to move comments from one document to another (if the documents contain the exact same content, with the exception of comments).

The Code

The following example contains RunSplitLocations and SplitRunsInParagraph. This code uses a node cloning technique similar to what I presented in this post. In addition, the code uses the pre-atomization approach that I showed in this post. This code implements a pure functional transformation - no side effects anywhere, which will make it easy to use when writing the next transformation.

Here’s the code (also attached):

using System;

using System.Collections.Generic;

using System.IO;

using System.Linq;

using System.Text;

using System.Xml;

using System.Xml.Linq;

using DocumentFormat.OpenXml.Packaging;

public static class Extensions

{

public static XDocument GetXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.Annotation<XDocument>();

if (xdoc != null)

return xdoc;

using (StreamReader streamReader = new StreamReader(part.GetStream()))

xdoc = XDocument.Load(XmlReader.Create(streamReader));

part.AddAnnotation(xdoc);

return xdoc;

}

public static string StringConcatenate(this IEnumerable<string> source)

{

StringBuilder sb = new StringBuilder();

foreach (string s in source)

sb.Append(s);

return sb.ToString();

}

public static class W

{

public static XNamespace w =

"http://schemas.openxmlformats.org/wordprocessingml/2006/main";

public static XName t = w + "t";

public static XName r = w + "r";

public static XName del = w + "del";

public static XName body = w + "body";

public static XName p = w + "p";

public static XName moveFrom = w + "moveFrom";

}

class Program

{

static int GetRunLength(XElement e)

{

return e

.Descendants(W.t)

.Select(t => (string)t)

.StringConcatenate()

.Length;

}

// return the run split locations for all runs in the paragraph

static int[] RunSplitLocations(XElement paragraph)

{

// find the runs that don't have w:del or w:moveFrom as parent elements

var runElements = paragraph

.Descendants(W.r)

.Where(e => e.Parent.Name != W.del && e.Parent.Name != W.moveFrom &&

e.Descendants(W.t).Any());

// determine the run length of each run

var runs = runElements

.Select(r => new

{

RunElement = r,

RunLength = GetRunLength(r)

});

// determine the split locations

var runSplits = runs

.Select(r => runs

.TakeWhile(a => a.RunElement != r.RunElement)

.Select(z => z.RunLength)

.Sum());

return runSplits.ToArray();

}

// if value starts or ends with a space, return xml:space="preserve" attribute

// else return null

static XAttribute XmlSpacePreserved(string value)

{

if (value.Substring(0, 1) == " " || value.Substring(value.Length - 1) == " ")

return new XAttribute(XNamespace.Xml + "space", "preserve");

else

return null;

}

private class RunSplits

{

public XElement RunElement { get; set; }

public int RunLength { get; set; }

public int RunLocation { get; set; }

}

private static object RunTransform(XElement element,

int[] positions, IEnumerable<RunSplits> runSplits)

{

// split runs that have child text elements

if (element.Name == W.r && element.Descendants(W.t).Any())

{

// get text of run

string text = element

.Descendants(W.t)

.Select(t => (string)t).StringConcatenate();

// find run in runSplits

RunSplits rs = runSplits.First(r => r.RunElement == element);

// find list of splits in this run

var splitsInThisRun = positions

.Where(p => p >= rs.RunLocation && p < rs.RunLocation + rs.RunLength);

// adjust splits so that split locations are relative to this run instead of

// relative to the beginning of the paragraph

var splitsIntext = splitsInThisRun

.Select(p => p - rs.RunLocation)

.ToArray();

// project collection of strings that will be in the new, split runs

var splitText = splitsIntext

.Select((p, i) =>

i != splitsIntext.Length - 1 ?

text.Substring(p, splitsIntext[i + 1] - p) :

text.Substring(p)

);

// project collection of runs that will replace the original run

return splitText.Select(r =>

new XElement(W.r,

rs.RunElement.Elements().Where(e => e.Name != W.t),

new XElement(W.t,

XmlSpacePreserved(r),

r)));

}

// clone elements other than runs

// must be recursive to handle custom XML markup and content controls

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n =>

{

XElement e = n as XElement;

if (e != null)

return RunTransform(e, positions, runSplits);

return n;

})

);

}

public static XElement SplitRunsInParagraph(XElement p, int[] positions)

{

// find the runs that don't have w:del or w:moveFrom as parent elements

var runElements = p

.Descendants(W.r)

.Where(e => e.Parent.Name != W.del && e.Parent.Name != W.moveFrom &&

e.Descendants(W.t).Any());

// calculate the run length of each run

var runs = runElements

.Select(r => new

{

RunElement = r,

RunLength = GetRunLength(r)

});

// calculate the location of each split

var runSplits = runs

.Select(r => new RunSplits

{

RunElement = r.RunElement,

RunLength = r.RunLength,

RunLocation = runs

.TakeWhile(a => a.RunElement != r.RunElement)

.Select(z => z.RunLength)

.Sum()

});

// the positions argument contains a list of locations where splits will be added

// to the paragraph. In addition, runs may already be split at various places, and

// we want those splits to remain, so we need to create the complete list of

// locations where we want run splits.

// create ordered union of desired splits and existing splits

int[] allSplits = runSplits

.Select(rs => rs.RunLocation)

.Concat(positions)

.OrderBy(s => s)

.Distinct()

.ToArray();

// transform the paragraph to a new paragraph with new splits in runs

return new XElement(W.p,

p.Elements().Select(e => RunTransform(e, allSplits, runSplits))

);

}

static void Main(string[] args)

{

using (WordprocessingDocument doc1 =

WordprocessingDocument.Open("Test.docx", true))

{

XDocument doc = doc1.MainDocumentPart.GetXDocument();

XElement p = doc.Root.Element(W.body).Element(W.p);

//XElement newPara = SplitRunsInParagraph(p, new[] { 12, 15 });

XElement newPara = SplitRunsInParagraph(p, new[] { 10 });

Console.WriteLine(newPara);

}

Querying LINQ to XML Nodes in Reverse Document Order with Better Performance

EricWhite — Wed, 24 Jun 2009 20:25:00 GMT

[Blog Map]

(Update June 25, 2009 - fixed bugs in event handlers associated with deleting last node and inserting node at beginning of list)

Occasionally I need to query LINQ to XML nodes in reverse document order. I’m currently writing some LINQ to XML queries over Open XML documents where I need to select paragraph nodes based on content in the immediately preceding paragraph. However, nodes in LINQ to XML are forward-linked only. We can see evidence of this in the XNode.NodesBeforeSelf and the XElement.ElementsBeforeSelf methods - these methods return collections of nodes in document order, not reverse document order. This was by design – LINQ to XML was designed to provide great performance for the vast majority of scenarios with the minimum memory footprint possible. The need to process nodes in reverse document order is rare, so the designers of LINQ to XML decided that it was more important to reduce memory footprint than to allow for good performance in the few scenarios that require processing in reverse document order, and of course it was a good decision. But the need does exist.

In my scenario (a functional transform that processes Open XML document revisions), it is possible that I would need to process 80,000 (or more) paragraphs. If we use the XNode.PreviousNode property, we won’t have acceptable performance. There is an easy work-around that provides us the ability to query in reverse document order in a way that performs well.

We define a new class, PreviousNodeAnnotation , that contains one public field, public XNode PreviousNode;.
We add instances of this class as annotations on nodes that we need to query in reverse document order.

In the following small example, I select nodes based on previous node value for a document size of 50,000. The slow version exhibits performance of O(n²). I limited the sample size in the example to 50,000 nodes. When I increased the doc size to 80,000 nodes (the size of one of my documents that I need to query), the execution time of the slow version exceeded my patience. In any case, it is clear that I can’t use XNode.PreviousNode for my scenario.

using System;

using System.Linq;

using System.Xml.Linq;

class PreviousNodeAnnotation

{

public XNode PreviousNode;

public PreviousNodeAnnotation(XNode prev) { PreviousNode = prev; }

}

class Program

{

static int DocumentSize = 50000;

static void SlowPreviousNodeAccess()

{

// create a tree with lots of nodes

XElement root = new XElement("Root",

Enumerable.Range(0, DocumentSize).Select(i => new XElement("Child", i)));

// query for all elements where the previous element has a value of 1000

DateTime start = DateTime.Now;

var q = root

.Elements()

.Where(e =>

{

XElement p = e.PreviousNode as XElement;

return (string)p == "1000";

});

var q2 = q.ToList(); // force iteration

TimeSpan duration = DateTime.Now - start;

Console.WriteLine(duration);

}

static void FastPreviousNodeAccess()

{

// create a tree with lots of nodes

XElement root = new XElement("Root",

Enumerable.Range(0, DocumentSize).Select(i => new XElement("Child", i)));

// initialize previous node annotations

XElement prev = null;

foreach (var item in root.Elements())

{

item.AddAnnotation(new PreviousNodeAnnotation(prev));

prev = item;

}

// query for all elements where the previous element has a value of 1000

DateTime start = DateTime.Now;

var q = root

.Elements()

.Where(e =>

{

XElement p = e

.Annotation<PreviousNodeAnnotation>()

.PreviousNode as XElement;

return (string)p == "1000";

});

var q2 = q.ToList(); // force iteration

TimeSpan duration = DateTime.Now - start;

Console.WriteLine(duration);

}

static void Main(string[] args)

{

FastPreviousNodeAccess();

SlowPreviousNodeAccess();

}

On my old slow laptop, the execution time of these two queries is .015 seconds and 30 seconds, respectively.

We can expand on this technique a bit by declaring three extension methods:

public static XNode PreviousNodeFast(this XNode node)

public static IEnumerable<XNode> NodesBeforeSelfFast(this XNode node)

public static IEnumerable<XElement> ElementsBeforeSelfFast(this XElement element)

It’s convenient to use these methods in queries.

In addition, while I’m partial to pure transforms with no side effects, we can declare two event handlers that keep the previous node annotations in sync when adding or deleting nodes. Here’s an example that includes these extension methods and event handlers:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml.Linq;

class PreviousNodeAnnotation

{

public XNode PreviousNode;

public PreviousNodeAnnotation(XNode prev) { PreviousNode = prev; }

}

public static class Extensions

{

public static XNode PreviousNodeFast(this XNode node)

{

return node.Annotation<PreviousNodeAnnotation>().PreviousNode;

}

public static IEnumerable<XNode> NodesBeforeSelfFast(this XNode node)

{

XNode currentNode = node;

while (true)

{

XNode prevNode = currentNode.PreviousNodeFast();

if (prevNode == null)

yield break;

else

yield return prevNode;

currentNode = prevNode;

}

public static IEnumerable<XElement> ElementsBeforeSelfFast(this XElement element)

{

return NodesBeforeSelfFast(element).OfType<XElement>();

}

class Program

{

static void ValidatePreviousNodes(XElement element)

{

XNode prev = null;

foreach (XNode node in element.Nodes())

{

if (node.PreviousNodeFast() != prev)

{

Console.WriteLine("ERROR: previous nodes are invalid");

Environment.Exit(0);

}

prev = node;

}

Console.WriteLine("Validated");

}

static void Main(string[] args)

{

// create a tree with lots of nodes

XElement root = new XElement("Root",

Enumerable.Range(0, 100).Select(i => new XElement("Child", i)));

// setup previous nodes after tree creation

XElement prev = null;

foreach (var item in root.Elements())

{

item.AddAnnotation(new PreviousNodeAnnotation(prev));

prev = item;

}

// add event handlers to take care of adding / deleting nodes

root.Changed += new EventHandler<XObjectChangeEventArgs>((o, e) =>

{

if (e.ObjectChange == XObjectChange.Add)

{

Console.WriteLine("Add");

XNode node = o as XNode;

// o could be an XAttribute, in which case it's not applicable

if (node != null)

{

node.AddAnnotation(new PreviousNodeAnnotation(node.PreviousNode));

if (node.NextNode != null)

{

node.NextNode.RemoveAnnotations<PreviousNodeAnnotation>();

node.NextNode.AddAnnotation(new PreviousNodeAnnotation(node));

}

});

root.Changing += new EventHandler<XObjectChangeEventArgs>((o, e) =>

{

if (e.ObjectChange == XObjectChange.Remove)

{

Console.WriteLine("Remove");

XNode node = o as XNode;

// o could be an XAttribute, in which case it's not applicable

if (node != null)

{

if (node.NextNode != null)

{

node.NextNode.RemoveAnnotations<PreviousNodeAnnotation>();

node.NextNode

.AddAnnotation(new PreviousNodeAnnotation(node.PreviousNode));

}

});

ValidatePreviousNodes(root);

root.Elements().ElementAt(3).AddAfterSelf(

new XElement("NewChild", 999)

);

ValidatePreviousNodes(root);

root.Nodes().ElementAt(2).Remove();

ValidatePreviousNodes(root);

root.Nodes().ElementAt(3).NodesBeforeSelfFast().Remove();

ValidatePreviousNodes(root);

root.AddFirst(

new XElement("ANode", 1)

);

ValidatePreviousNodes(root);

root.Add(

new XElement("ANode", 2)

);

ValidatePreviousNodes(root);

root.Nodes().Last().NodesBeforeSelfFast().Remove();

ValidatePreviousNodes(root);

root.Add(

new XElement("ANode", 2)

);

ValidatePreviousNodes(root);

root.Add(

new XElement("ANode", 2)

);

root.Add(

new XElement("ANode", 2)

);

root.Add(

new XElement("ANode", 2)

);

ValidatePreviousNodes(root);

root.Nodes().First().Remove();

ValidatePreviousNodes(root);

root.Add(

new XElement("ANode", 2)

);

ValidatePreviousNodes(root);

root.Nodes().Last().Remove();

root.Add(Enumerable.Range(0, 100).Select(i => new XElement("Child", i)));

XElement last = root.Elements().ElementAt(50);

foreach (var item in last.NodesBeforeSelfFast())

{

Console.WriteLine(item);

}

I would personally only define these extension methods in the module where I need good performance of reverse document order queries. It would be messy to have these extension methods in scope for modules that don't set up the annotations.

Code is attached.

Comparison of Navigating Parts between System.IO.Packaging and the Open XML SDK

EricWhite — Mon, 11 May 2009 07:30:00 GMT

Code highlighted in yellow shows navigating from the package to the main document part. Code highlighted in green shows navigating from the main document part to the styles part.

using System;

using System.Linq;

using System.IO;

using System.IO.Packaging;

using System.Xml;

using System.Xml.Linq;

using DocumentFormat.OpenXml.Packaging;

class Program

{

static void SystemIoPackaging()

{

const string fileName = "Test.docx";

const string documentRelationshipType =

"http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument";

const string stylesRelationshipType =

"http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles";

XDocument xDocMainDocument = null;

XDocument xDocStyleDocument = null;

using (Package wdPackage = Package.Open(fileName, FileMode.Open, FileAccess.Read))

{

PackageRelationship docPackageRelationship = wdPackage

.GetRelationshipsByType(documentRelationshipType)

.FirstOrDefault();

if (docPackageRelationship != null)

{

Uri documentUri = PackUriHelper.ResolvePartUri(

new Uri("/", UriKind.Relative), docPackageRelationship.TargetUri);

PackagePart documentPart = wdPackage.GetPart(documentUri);

// Load the document XML in the part into an XDocument instance.

using (Stream documentPartStream = documentPart.GetStream())

using (XmlReader documentPartXmlReader =

XmlReader.Create(documentPart.GetStream()))

xDocMainDocument = XDocument.Load(documentPartXmlReader);

// Find the styles part. There will only be one.

PackageRelationship styleRelation = documentPart

.GetRelationshipsByType(stylesRelationshipType)

.FirstOrDefault();

if (styleRelation != null)

{

Uri styleUri = PackUriHelper.ResolvePartUri(documentUri,

styleRelation.TargetUri);

PackagePart stylePart = wdPackage.GetPart(styleUri);

// Load the style XML in the part into an XDocument instance.

using (Stream stylePartStream = stylePart.GetStream())

using (XmlReader stylePartXmlReader =

XmlReader.Create(stylePartStream))

xDocStyleDocument = XDocument.Load(stylePartXmlReader);

}

Console.WriteLine("The main document part has {0} nodes.",

xDocMainDocument.DescendantNodes().Count());

Console.WriteLine("The style part has {0} nodes.",

xDocStyleDocument.DescendantNodes().Count());

}

static void OpenXmlSdk()

{

const string filename = "Test.docx";

using (WordprocessingDocument wordDoc = WordprocessingDocument.Open(filename, true))

{

MainDocumentPart mainPart = wordDoc.MainDocumentPart;

StyleDefinitionsPart styleDefinitionsPart = mainPart.StyleDefinitionsPart;

Console.WriteLine("The main document part has {0} nodes.",

mainPart.RootElement.Descendants().Count());

Console.WriteLine("The style part has {0} nodes.",

styleDefinitionsPart.RootElement.Descendants().Count());

}

static void Main(string[] args)

{

SystemIoPackaging();

OpenXmlSdk();

}

Creating a Template Open XML Document in Memory

EricWhite — Sat, 28 Feb 2009 18:03:00 GMT

One of the most effective ways to generate Open XML documents, spreadsheets, or presentations is to start with a ‘template’ document and then modify the document. For instance, you can start with a blank word processing document that is set up with your desired styles, and then add paragraphs and tables as necessary. Or you can start with a blank PowerPoint presentation that is set up with your desired theme, and add slides. But this means that you have to have a template document lying around that you can open and modify. This is a bit inconvenient and messy. There is another approach – you can create the template document in memory, modify it, and then serialize it to your desired location.

The gist of this approach is:

Create your desired template document using Office 2007 (or Office 2003 with the compatibility pack installed).
Run a little program that reads the template document into memory as a byte array, converts the byte array to a base64 string, and writes a file that contains a simple C# assignment statement.
Paste this assignment statement into your application. As necessary, you can convert this base64 string to an in-memory Open XML document, modify the document, and serialize it to wherever you want.

If you use this approach, you don’t need to keep a template document around that you can read and modify. Your C# application is self-contained.

Note that you can use this approach to embed any binary file into a C# program. This technique will work, of course, with DOCX, XLSX, or PPTX Open XML documents.

There is a slight performance improvement using this approach – you don’t have to go to disk to read the template document. But more importantly, your application is cleaner, and deployment of your application is simpler. You don’t need to deploy your ‘template’ document – it’s contained right there in your C# code.

This approach is related to the approach of working with in-memory Open XML documents.

Here is a simple C# program to convert a template document to the C# assignment statement. This code uses the LINQ technique of chunking a collection into a list of arbitrarily long chunks:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.IO;

class Program

{

static void Main(string[] args)

{

byte[] byteArray = File.ReadAllBytes("Template.docx");

// the following expression creates the base64String, then chunks

// it to lines of 76 characters long

string base64String = (System.Convert.ToBase64String(byteArray))

.Select

(

(c, i) => new

{

Character = c,

Chunk = i / 76

}

)

.GroupBy(c => c.Chunk)

.Aggregate(

new StringBuilder(),

(s, i) =>

s.Append(

i.Aggregate(

new StringBuilder(),

(seed, it) => seed.Append(it.Character),

sb => sb.ToString()

)

.Append(Environment.NewLine),

s =>

{

s.Length -= Environment.NewLine.Length;

return s.ToString();

}

);

// output a C# assignment statement to create a string that contains the template

// document

string chunkedString =

String.Format("string templateDocumentBase64String = @\"{0}\";", base64String);

File.WriteAllText("chunk.cs", chunkedString);

}

When you run this program, it creates a small file that contains the assignment statement:

string templateDocumentBase64String = @"UEsDBBQABgAIAAAAIQDd/JU3ZgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

... lots of lines elided

ZG9jUHJvcHMvYXBwLnhtbFBLAQItABQABgAIAAAAIQAdnTy9ewEAAPcCAAARAAAAAAAAAAAAAAAA

AO0ZAABkb2NQcm9wcy9jb3JlLnhtbFBLAQItABQABgAIAAAAIQBZ7/GiEgcAAPg5AAAPAAAAAAAA

AAAAAAAAAJ8cAAB3b3JkL3N0eWxlcy54bWxQSwUGAAAAAAsACwDBAgAA3iMAAAAA";

The following code shows how to convert this string to an Open XML document that you can then modify. This code uses the Open XML SDK (either V1 or V2):

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.IO;

using System.Xml;

using System.Xml.Linq;

using DocumentFormat.OpenXml.Packaging;

public static class LocalExtensions

{

public static XDocument GetXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.Annotation<XDocument>();

if (xdoc != null)

return xdoc;

using (StreamReader sr = new StreamReader(part.GetStream()))

using (XmlReader xr = XmlReader.Create(sr))

xdoc = XDocument.Load(xr);

part.AddAnnotation(xdoc);

return xdoc;

}

public static void PutXDocument(this OpenXmlPart part)

{

XDocument xdoc = part.GetXDocument();

if (xdoc != null)

{

// Serialize the XDocument object back to the package.

using (XmlWriter xw = XmlWriter.Create(

part.GetStream(FileMode.Create, FileAccess.Write)))

xdoc.Save(xw);

}

public static class W

{

public static XNamespace w =

"http://schemas.openxmlformats.org/wordprocessingml/2006/main";

public static XName body = w + "body";

public static XName r = w + "r";

public static XName p = w + "p";

public static XName t = w + "t";

}

class Program

{

static void Main(string[] args)

{

using (MemoryStream memoryStream = new MemoryStream())

{

string templateDocumentBase64String = @"UEsDBBQABgAIAAAAIQDd/JU3ZgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

... lots of lines elided

AO0ZAABkb2NQcm9wcy9jb3JlLnhtbFBLAQItABQABgAIAAAAIQBZ7/GiEgcAAPg5AAAPAAAAAAAA

AAAAAAAAAJ8cAAB3b3JkL3N0eWxlcy54bWxQSwUGAAAAAAsACwDBAgAA3iMAAAAA";

char[] base64CharArray = templateDocumentBase64String

.Where(c => c != '\r' && c != '\n').ToArray();

byte[] byteArray =

System.Convert.FromBase64CharArray(base64CharArray,

0, base64CharArray.Length);

memoryStream.Write(byteArray, 0, byteArray.Length);

using (WordprocessingDocument doc = WordprocessingDocument.Open(memoryStream, true))

{

// we now have an open in-memory WordprocessingDocument that we can modify

XDocument xDocument = doc.MainDocumentPart.GetXDocument();

// add a paragraph to the document

XElement body = xDocument.Root.Element(W.body);

body.AddFirst(

new XElement(W.p,

new XElement(W.r,

new XElement(W.t, "Hello World!")

)

);

doc.MainDocumentPart.PutXDocument();

}

// at this point, the MemoryStream contains the modified document.

// We could write it back to a SharePoint document library or serve

// it from a web server.

// in this example, we'll serialize back to the file system to verify

// that the code worked properly.

using (FileStream fileStream = new FileStream("Test.docx",

System.IO.FileMode.Create))

{

memoryStream.WriteTo(fileStream);

}

Code is attached.