Eric White's Blog : XML

DescendantsAndSelfTrimmed LINQ to XML Axis Method

EricWhite — Wed, 19 Aug 2009 22:01:00 GMT

There are some circumstances where I need a variation on the DescendantsAndSelf axis method that allows me to specify that specific elements (and the descendants of those elements) are ‘trimmed’ from the returned collection. One of the things that's great about LINQ to XML is the ease with which we can create specialized axis methods when necessary.

Here is the scenario: Open XML contains paragraphs that contain runs. A run can contain a picture that contains a text box, which itself contains a paragraph. When I want to write a transform for paragraphs that are children elements of the w:body element, I want to disregard any paragraphs that are children of contained text boxes.

To make this clear, consider the following markup:

<body>

<r>

<t>First paragraph.</t>

</r>

<r>

<pict>

<shape>

<r>

<t>Text in a textbox.</t>

</r>

</textbox>

</shape>

</pict>

</r>

<r>

<t>Second paragraph.</t>

</r>

</body>

</document>

In this markup, I want to transform the elements highlighted in yellow, but while processing those elements, I want to completely disregard the element highlighted in green. If children paragraphs of the textbox element need to be transformed in some fashion, I would deal with them separately – perhaps in an embedded recursive transform.

To enable selecting descendant elements in this fashion, I’ve implemented four extension methods. These are variations on the Descendants and DescendantsAndSelf axis methods. The methods are:

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

XName trimName)

Returns a filtered collection of elements that contain this element, and all descendant elements of this element, in document order. Elements that have a matching XName are not included in the collection. Descendant elements of excluded elements are not included in the collection.

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

Func<XElement, bool> predicate)

Returns a filtered collection of elements that contain this element, and all descendant elements of this element, in document order. Elements that are selected by the predicate are not included in the collection. Descendant elements of excluded elements are not included in the collection.

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

XName trimName)

Returns a filtered collection of the descendant elements for this document or element, in document order. Elements that have a matching XName are not included in the collection. Descendant elements of excluded elements are not included in the collection.

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

Func<XElement, bool> predicate)

Returns a filtered collection of elements that contain this element, and all descendant elements of this element, in document order. Elements that are selected by the predicate are not included in the collection. Descendant elements of excluded elements are not included in the collection.

Here is the listing of these extension methods, as well as some example code to exercise them (also attached):

using System;

using System.Collections.Generic;

using System.Linq;

using System.Xml.Linq;

public static class LocalExtensions

{

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

XName trimName)

{

return DescendantsInternalTrimmed(element, e => e.Name == trimName, true);

}

public static IEnumerable<XElement> DescendantsAndSelfTrimmed(this XElement element,

Func<XElement, bool> predicate)

{

return DescendantsInternalTrimmed(element, predicate, true);

}

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

XName trimName)

{

return DescendantsInternalTrimmed(container, e => e.Name == trimName, false);

}

public static IEnumerable<XElement> DescendantsTrimmed(this XContainer container,

Func<XElement, bool> predicate)

{

return DescendantsInternalTrimmed(container, predicate, false);

}

private static IEnumerable<XElement> DescendantsInternalTrimmed(this XContainer container,

Func<XElement, bool> predicate, bool includeSelf)

{

if (includeSelf)

{

XElement element = container as XElement;

yield return element;

}

Stack<IEnumerator<XElement>> iteratorStack = new Stack<IEnumerator<XElement>>();

iteratorStack.Push(container.Elements().GetEnumerator());

while (iteratorStack.Count > 0)

{

while (iteratorStack.Peek().MoveNext())

{

XElement currentXElement = iteratorStack.Peek().Current;

if (predicate(currentXElement))

continue;

yield return currentXElement;

iteratorStack.Push(currentXElement.Elements().GetEnumerator());

}

iteratorStack.Pop();

}

class Program

{

static void Main(string[] args)

{

XElement rootElement = XElement.Parse(

@"<document>

<body>

<r>

<t>First paragraph.</t>

</r>

<r>

<pict>

<shape>

<r>

<t>Text in a textbox.</t>

</r>

</textbox>

</shape>

</pict>

</r>

<r>

<t>Second paragraph.</t>

</r>

</body>

</document>");

foreach (var item in rootElement.DescendantsAndSelfTrimmed("textbox"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

foreach (var item in rootElement

.DescendantsAndSelfTrimmed(e => e.Name == "p" && e.Ancestors("p").Any()))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

foreach (var item in rootElement.DescendantsTrimmed("r"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

foreach (var item in rootElement.DescendantsTrimmed(e => e.Name == "r"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

Console.WriteLine("==========");

XDocument doc = XDocument.Parse(

@"<document>

<body>

<r>

<t>First paragraph.</t>

</r>

<r>

<pict>

<shape>

<r>

<t>Text in a textbox.</t>

</r>

</textbox>

</shape>

</pict>

</r>

<r>

<t>Second paragraph.</t>

</r>

</body>

</document>");

foreach (var item in doc.DescendantsTrimmed(e => e.Name == "r"))

Console.WriteLine("{0}{1}", "".PadRight(item.Ancestors().Count() * 2),

item.Name);

}

XML Element and Attribute Name Guidelines

EricWhite — Wed, 12 Aug 2009 22:09:00 GMT

[Blog Map]

Some time ago, a dev team here at Microsoft asked me to review their XML vocabulary that they had designed. They wanted to know if the element and attribute names in their vocabulary design were good ones.

Note: I suspect that this is one of those religious issues. I’m not suggesting that this is the one and only way to design XML vocabularies – this is just what I do. There may be good reasons to use different rules. (I have broken every rule here, probably. Well, that’s the story of my life – make mistakes, suffer from the effects, and try not to do it again. J)

Once you release some software that uses a specific vocabulary, it tends to be set in stone. Even if there’s a problem with the vocabulary, it often is more problematic to try to change it after the fact than to continue to work around the problem, so designing vocabularies deserves some thought beforehand.

Much of what I’m presenting here derives from the Guidelines for Names from the .NET Framework Design Guidelines. Krzysztof Cwalina and Brad Abrams are the authors of the book Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries. Some important excerpts of that book are published on MSDN for easy reference, but I consider that book to be a ‘must read’ on the list of professional .NET developers.

Pascal Casing For Tags Longer than Three Characters

Use Pascal casing for element and attribute names that are longer than three characters.

There are mainly three schemes used for casing:

Pascal casing: The first letter in the tag and the first letter of each subsequent concatenated word are capitalized. For example:

<ForegroundColor>Black</ForegroundColor>

Lower casing with hyphens: tags are lower-case, and hyphens separate words. XSLT uses this style:

<apply-templates />

<value-of select="title" />

Camel casing: The first letter of a tag is lowercase and the first letter of each subsequent concatenated word is capitalized. For example:

<fatalErrorMessage>Contact your IT support department.</fatalErrorMessage>

I prefer Pascal casing for two reasons:

I find the XML easier to read.
If you use an XML to Object mapper such as LINQ to XSD, then the types that the mapper generates will conform to the .NET Framework Design Guidelines. If you use the LINQ to XML approach of pre-atomization of element and attribute names, those classes will contain public fields that conform to the guidelines.

I find that I’m somewhat happy that XSLT uses lower casing with hyphens, because if I use Pascal casing for my tags, it makes the elements and attributes in sequence constructors a little easier to see:

<xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/XSL/Transform' version='1.0'>

<xsl:template match='/Books'>

<xsl:value-of select='Author/@Name'/>

</Author>

</SelectedAuthors>

</xsl:template>

</xsl:stylesheet>

Capitalization Rules for Acronyms

It’s better to use acronyms in tags only when they are widely known and well understood – HTML, for example. Per the .NET Framework design guidelines, capitalize all letters of a two letter acronym, and capitalize the first letter of acronyms that are three characters or longer.

<Html>false</Html>

Don’t capitalize each word in compound words that are written as a single word. Examples: Endpoint, Lifetime, Diskdrive, Hashtable, and Grandchild.

One word that I have problems remembering how to capitalize is FileName. Some dictionaries don’t define it as a closed-form compound word (but some do). The .NET Framework guidelines capitalize it as FileName.

Don’t Use Abbreviations

Unless you are designing a highly-specialized vocabulary such as Open XML markup, optimize for readability instead of shortness of tags.

One abbreviation that it’s ok to use is the abbreviation for identifier (Id). Use Pascal casing for Id.

Just for fun: The .NET Framework Design Guidelines says that ‘Ok’ is an abbreviation. Actually, it might be an acronym. There are a number of theories on the origin of ‘OK’, but my favorite comes from a slogan during the American Presidential election of 1840. That election resulted in the oldest written usage of 'OK'. The democratic candidate, President Martin Van Buren, was nicknamed 'Old Kinderhook' (after his birthplace in New York State), and his election campaign had a slogan, ‘Old Kinderhook is OK’. Van Buren wasn’t re-elected. But following the .NET Framework design guidelines, I would use Pascal casing for ‘Ok’.

The designers of Open XML had very good reasons for making element and attribute names short – documents can be very long, and an increase in name length can have an impact on performance and the memory that is used by tools, so they were justified in having names such as w:p, w:t, and w:sdt. Unless you are designing one of those highly specialized vocabularies, it’s better not to use single character tags.

Word Selection

Avoid language keywords such as ‘default’, ‘abstract’, ‘break’, and ‘event’. If you use a tool that generates C# or some other language for de-serialization of the XML, or if you use the approach of pre-atomization of XName objects, code that exactly matches the element or attribute name won’t be valid.

For convenience, here are links to the C#, VB.NET, and C++ references, so that it’s easy to validate that you are not using keywords:

C# Language Reference

Visual Basic Reference

C++ Reference

Choose easily readable identifier names. For example, a property named HorizontalAlignment is more readable in English than AlignmentHorizontal. AdjustIndentation is more readable than IndentationAdjust.

Optimize for readability over brevity.

XML Namespaces

I put elements in namespaces, and don’t put attributes in namespaces. Elements need to be in namespaces for a variety of reasons. There are some tools that require namespaces.

But attributes are a different story. For one thing, for all practical purposes, attributes inherit the namespace of their element. You can’t access the attribute without accessing the element, and you can’t get to the element without using its namespace. The designers of XML had to allow for namespaces for attributes because there are some special purpose attributes that sometimes need to be added to elements in an existing vocabulary. The xml:space and xml:lang attributes are examples. By placing these in the special xml namespace, we avoid any possible collisions between names.

Another reason that attributes allow namespaces – I’ve written XSLT transforms where the first operation is to transform the XML into a new tree with new attributes on some elements. The purpose of these attributes is to aid further transforms. By creating these attributes in my own namespace, I can make sure that I avoid name collisions.

This is, I believe, why even if there is a default namespace for elements, attributes are always by default in no namespace.

Sometimes I use LINQ to XML trees as a means of passing more complicated configuration information into a method, or returning the results of a query into SQL or Open XML. In this case, I’m not really using LINQ to XML as XML, but as a hierarchical data store that is LINQ friendly. Of course, in this situation, using XML namespaces would be silly.

There are many more things to consider around designing XML vocabularies, such as when to use attributes vs. when to use child elements, avoiding magic values, etc. This is a much larger discussion, and could fill an entire book, I think.

Recursive Approach to Pure Functional Transformations of XML

EricWhite — Mon, 20 Jul 2009 16:28:00 GMT

[Blog Map]

Writing pure functional transformations a in a recursive style enables us to put together interesting transformations in a very small amount of code. Using some specific techniques that allow us to write this code very concisely, this approach takes advantage of some perhaps obscure semantics of LINQ to XML. I’ve used this approach to write some interesting transformations – it has become my favorite way to write transformations of a certain variety. This post presents a short tutorial in writing these types of transformations.

Identity Transform

First, let’s take a look at the identity transform:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(@"<Root Att1='1'><Child/></Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

In this code, we are taking advantage of the semantics of the XElement constructor that allow us to pass collections of objects to the constructor. The following code creates a new XElement from an existing XElement:

new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

This code, of course, passes two objects to the constructor (in addition to the name) – a collection of attributes from the source element, and a collection of transformed child nodes from the source element. Note that even though we’re passing collections to the constructor, we’re only passing two objects. We’re passing the two variables that contain the iterators for the queries. Those iterators, of course, implement IEnumerable<T>. There are a number of LINQ to XML methods and constructors that can take content, and if we pass an object that implements IEnumerable of some T, LINQ to XML iterates the enumerable and adds each object in the collection to the newly constructed element. This is performed recursively. For detailed semantics of the constructors and methods, see this page in the LINQ to XML documentation.

This is one of the reasons that the signatures of LINQ to XML constructors and methods that take content are defined using object instead of XObject:

public XElement(XName name, params Object[] content);

Because the signature is defined using object, we can pass individual XElement, XAttribute and XNode objects to the constructor, and we can pass collections of them.

Cloning vs. Attaching

The identity transform also takes advantage of the cloning vs. attaching semantics of LINQ to XML. When we pass a newly constructed XElement to the XElement constructor, the new object does not have a parent so it is simply attached to the XElement that is being constructed. But the highlighted line below causes a node from an existing tree to be passed to the XElement constructor. In this case, the XElement constructor notices that the node is already part of an existing tree, and clones it and adds the cloned node to the XElement being constructed.

XElement element = node as XElement;

if (element != null)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

To make this very clear, take a look at the following example:

XElement tree1 = XElement.Parse("<Root><Child1/></Root>");

XElement child1 = tree1.Element("Child1");

XElement child2 = new XElement("Child2");

XElement tree2 = new XElement("Root", child1, child2);

Console.WriteLine(tree2);

In this example, child1 already is part of tree1, so it is cloned and the newly cloned element is added to tree2. The child2 XElement object has no parent so it is simply attached to tree2.

For more information on cloning vs. attaching, see this page in the LINQ to XML documentation.

Note that when an XElement object is cloned by a LINQ to XML constructor or method, of course all attributes and descendant nodes are also cloned.

Removing Nodes from the Newly Cloned Tree

Sometimes when we clone a tree, we want to trim certain elements and attributes and nodes from the newly cloned tree. We can take advantage of the fact that it’s valid to pass null as one of the arguments to the XElement constructor. If one of the arguments to the XElement constructor is null, the XElement constructor simply ignores that argument:

XElement xml = new XElement("Root",

new XElement("Child1"),

null,

new XElement("Child3"));

Console.WriteLine(xml);

This produces the following output:

<Root>

</Root>

Let’s say that we want to trim all Child2 elements from the following tree:

<Root>

</Root>

We can code the transform like this:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return null;

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(@"<Root><Child1/><Child2/><Child3/></Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

These semantics apply regardless of whether null is passed directly to the XElement constructor, or whether null is passed as part of a collection to the XElement constructor.

Such element name comparisons (element.Name == "Child2") are very efficient if we pre-atomize element and attribute names. See this post for more information about atomization of element and attribute names.

Note: for simplicity of demonstration, in these examples, I didn’t bother to pre-atomize XNames, but pre-atomization is important if performance is important.

Replace an Element with another Element

A common operation in transforms is to replace some particular element with an entirely new element. Let’s say that we want to replace the Child2 element with a Child5 element that contains a child element named Text. We want to transform this:

<Root>

</Root>

To this:

<Root>

</Child5>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return new XElement("Child5",

new XElement("Text"));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

Replace an Element with Multiple Elements

Sometimes we may want to replace a single element with multiple elements. We can take advantage of the fact that the behavior of the XElement constructor for collections of objects is, as mentioned, recursive. If one of the objects in the collection is itself a collection of objects then that child collection is itself iterated and all objects are added to the element being constructed.

The following example shows these semantics of the XElement constructor. The example first creates two lists of XElement objects. It then creates a list that contains the two lists. It passes the list of lists to the XElement constructor, and the resulting XElement contains all of the XElement objects in the two lists:

List<XElement> list1 = new List<XElement> {

new XElement("Child1"),

new XElement("Child2")

};

List<XElement> list2 = new List<XElement> {

new XElement("Child3"),

new XElement("Child4")

};

List<List<XElement>> listOfLists = new List<List<XElement>> { list1, list2 };

XElement newElement = new XElement("Root", listOfLists);

Console.WriteLine(newElement);

This produces the following output:

<Root>

</Root>

Let’s say that we want to replace the Child2 element with three Child5 elements. We want to transform this:

<Root>

</Root>

To this:

<Root>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return new List<XElement> {

new XElement("Child5"),

new XElement("Child5")

};

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

This example is coded a little bit artificially – more typically we would want to replace a particular element with a number of elements that are contained in a collection that is returned by some query. Let’s say that we wanted to effectively remove a certain element from the tree, and replace that element with its children elements. We want to transform this:

<Root>

<Child>

</Child>

</Root>

To this:

<Root>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child")

return element.Elements();

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<Root>

<Child>

</Child>

</Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

This is why it’s important that the Transform function returns object instead of XNode or XObject. This is of course an exact parallel to the XElement constructor – the XElement constructor takes object as an argument so that you can pass either an XObject (or objects of derived classes) or a collection of XObject. The Transform function returns object so that you can return either an XObject or a collection of XObject. (There are other reasons that the XElement constructor takes object as a parameter, but these are the relevant reasons for this discussion.)

More Sophisticated Selection of Nodes to Transform

In the above examples, the code selected elements based on the element name. In XSLT, of course, we often write involved XPath expressions to select nodes to transform. Sometimes we want to select nodes for transformation where the node has specific ancestors. Let’s say that we want to transform a paragraph only if the paragraph is in a content control:

<body>

abc

<sdt>

def

</sdt>

</body>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "p" && element.Parent.Name == "sdt")

return new XElement("p", "different content");

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<body>

abc

<sdt>

def

</sdt>

</body>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

This will be efficient – if the node isn’t a paragraph node, then the Boolean expression will be short-circuited, and no extra work will be done. If the node is a paragraph node, then the code consists of following a pointer in the ancestor linked list, and another efficient XName comparison. I've written even more involved node selection code than this - select a paragraph to transform based on contents of the previous paragraph, for example. (Note that if you are selecting nodes in reverse document order, you may want to use an approach that gives good performance.)

Selectively Transforming Descendant Nodes

Sometimes we want to transform just a specific set of the descendant nodes, ignoring everything else. In the following query, the highlighted line serves the same purpose as xsl:apply-templates in an XSLT style sheet. This is the code that sends each of the descendants through the transform function. If we include a call to the Where extension method in that query, we can cause only a subset of the descendants to be sent through the recursive function.

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

Let’s say that we want to perform a transform on the following XML document:

<Root>

<Child>

</Child>

</Root>

We want to transform all of the ’A’ elements to ‘Apple’, the ‘B’ elements to ‘Banana’ and the ‘C’ elements to ‘Carot’, but only for all of the elements where Id equals ‘1’. We want the resulting XML file to look like this:

<Root>

<Child>

</Child>

</Root>

We can code the transform like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "A")

return new XElement("Apple",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "B")

return new XElement("Banana",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "C")

return new XElement("Carot",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "Child")

return new XElement("Child",

element.Attributes(),

element.Elements()

.Where(e => e.Attribute("Id").Value == "1")

.Select(e => Transform(e)));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

Only the XML elements that satisfy the predicate in the Where clause will be sent back through the transform function.

Notice that in this case, I used the Elements axis instead of the Nodes axis. Based on the needs of my transform, I can use any of a variety of axes to specify which nodes will be sent back through the recursive transform function.

Using Multiple Recursive Functions in the Same Transform

These types of transforms are always written in the pure style. In other words, no variables are ever mutated the while doing this transform. The transform always produces a completely new XML tree and never modifies the source tree. This gives us the freedom to call out to other recursive transforms that are also implemented in the pure style.

Let’s say that we’re transforming an Open XML document and we’ve written an interesting transform just for the paragraph element, we can write our transform for the main document part to call the paragraph transform function as appropriate. Because both transforms are written in a pure fashion, we can do this without worrying about introducing bugs associated with side effects in our code.

Following is the source document that we want to transform. This XML is ‘pseudo’ Open XML markup – namespaces are removed and the XML is simplified for the purposes of demonstration.

<body>

<r>

<t>This is a short sentence.</t>

</r>

<r>

<t>This is a different sentence.</t>

</r>

<sdt>

<r>

<t>This is a sentence in a content control.</t>

</r>

</sdt>

</body>

</document>

We want to produce the following document:

<body>

<r>

<t>THIS IS A SHORT SENTENCE.</t>

</r>

<r>

<t>THIS IS A DIFFERENT SENTENCE.</t>

</r>

<r>

<t>THIS IS A SENTENCE IN A CONTENT CONTROL.</t>

</r>

</body>

</document>

We could code the transformation like this:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Xml.Linq;

class Program

{

// This paragraph transform might do something very interesting - perhaps splitting

// runs in a new way or doing sophisticated modifications to styling. In this example,

// the code will transform the text node to upper case.

static object ParagraphTransform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "t")

return new XElement("t", ((string)element).ToUpper());

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => ParagraphTransform(n)));

}

return node;

}

// This transform might do something very interesting with content controls. In this

// case the transform will do something very simple - it removes the content control

// and promotes its children elements to the level of the content control.

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

// call into the recursive transform for the paragraph

if (element.Name == "p")

return ParagraphTransform(element);

// remove the content control

if (element.Name == "sdt")

return element.Nodes().Select(n => Transform(n));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<document>

<body>

<r>

<t>This is a short sentence.</t>

</r>

<r>

<t>This is a different sentence.</t>

</r>

<sdt>

<r>

<t>This is a sentence in a content control.</t>

</r>

</sdt>

</body>

</document>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

Because this transformation was coded in this style, artifacts of Open XML such as custom XML elements and nested content controls are handled automatically without us taking any special care to handle them.

Sometimes it’s useful to write small utility functions for some of the functionality supplied by each recursive transform function. It may be useful to put each transform function and its associated utility functions in a static class.

Normalization of Empty Elements

There is one point regarding empty elements that is worth mentioning – empty elements <Tag></Tag> are normalized to use the empty-element tag <Tag />. This post contains a discussion on empty elements.

I had an unfortunate bike accident last week and broke my right arm, and I’m right handed! I used the voice recognition features of Windows 7 to write this post. These features are great! I don’t think that the process of writing this post was appreciably slower than usual.

Document-Centric Transforms using LINQ to XML

EricWhite — Thu, 09 Jul 2009 21:16:00 GMT

[Blog Map]

When thought of in a certain way, XML documents come in two flavors – data-centric and document-centric. Further, there are two types of document-centric documents. This post presents my thoughts about approaches to various types of document-centric transformations – data-centric to document-centric, document-centric to data-centric, and document-centric to document-centric. Then, I’ll tie my thoughts back to Open XML transformations.

Data-centric to data-centric is, of course, the scenario that LINQ to XML absolutely shines at. There’s been a lot written about this. This post won’t focus on these types of transformations, but instead will give my thoughts on the wrinkle that document-centric XML documents give to transformations.

First, I’ll define what I mean by data-centric and document-centric XML documents.

Data-Centric XML Document

A data-centric XML document contains regular repeating elements. Child elements of a given element might all have the same tag name, or they might not. Typically, child element order doesn’t matter. There are lots of examples of this – many types of transforms of a relational database to XML results in data-centric XML. RSS feeds are another.

Here’s a data-centric XML document:

</Customer>

</Customer>

</Customers>

Document-Centric XML Document

Document-centric XML documents have the characteristic that the child elements of a given element are much less bounded – you might have many child elements of a given name, or you might have none. You might have ‘recursion’ in the hierarchy – element A is a child of element B, which is itself a child of a different element A. A number of examples: Open XML word processing markup, XHTML, and XPS.

I further divide document-centric XML documents into two camps – those that contained mixed content, and those that don’t. Mixed content is a variety of XML where significant text nodes and elements are interspersed. Insignificant text nodes are the white space that provides indenting when formatting XML. Open XML word processing markup doesn’t contain mixed content, whereas XHTML does:

An Open XML paragraph that contains some bold text:

<w:p>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:r>

<w:rPr>

<w:b/>

</w:rPr>

<w:t>def</w:t>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

An XHTML document that contains significant text nodes interspersed with element start and end tags:

<html>

<body>

abcdefghi

</body>

</html>

Types of Transformations

If we’re going to divide the XML world into the categories of data-centric and document-centric, it follows that there are four types of transformations.

Data-Centric => Data Centric

There is a lot to say (and has been said) about these types of transformations. In the LINQ to XML documentation, I included a tutorial on pure functional transformations of XML. I also have a tutorial on my blog on composing queries in the pure functional style.

Data-Centric => Document Centric

These transforms are report writers for databases – take some subset of records, transform to XML, then transform that XML into another form – XPS, for instance. The transform may be based on another source document, the report definition. These types of transforms are straightforward to write in the pure functional style. Based on the simplicity or complexity of the report definition, this type of transform could be a few hundred lines of code or many thousands.

There are also many good examples of transforming data-centric XML to Open XML markup. We may want to transform a collection of records into a table in a word processing document, or into rows and cells in a worksheet.

Document-Centric => Data Centric

We write this type of transform when querying an Open XML document for some aspect of the markup. If we want to retrieve a collection of comments from a document, or if we want a collection of content controls, then we write a query that iterates over certain descendant elements, projecting a regular data structure – perhaps a collection of strings or anonymous types. The query that I develop in my functional programming tutorial is a document-centric => data-centric transform.

Another example is finding all hyper-links in an XHTML document. It is easy to write a LINQ query to retrieve a collection of links and transform the collection to a regular repeating data structure.

Document-Centric => Document-Centric

This is where it starts to get a little more involved. There are a variety of these types of transformations.

Common-vocabulary document-centric transform: Sometimes we want to transform a XML tree to a new tree in the same vocabulary. For example, we can transform an Open XML document into a new Open XML document with modified contents - comments are removed or revisions accepted. Another example - replace content controls with other markup based on the contents of the content controls.

Different-vocabulary document-centric transform: Sometimes we want to transform from one document-centric vocabulary to another one – Open XML => XHTML, or XHTML => Open XML. With this type of transform, the ease with which we write the transform is directly related to whether the two vocabularies have a similar structure. For instance, there is much that is parallel between Open XML and XHTML. There is a body element. The body contains paragraphs and tables. Tables contain rows, which contain cells. Tables can contain other tables in cells.

XSLT works well for these types of transformations – you write a pattern to match a node, and then supply the transformation for just that node. In the case of XSLT, you can indicate to the transform engine to ‘continue processing rules for child elements’, so that you can specify the transforms for those child elements in their own rules. If you are aware of Flat OPC, it is pretty easy to process Open XML documents using XSLT.

Some time ago, I write a post on an approach for using LINQ to XML annotations for doing this type of transform. In that post, I was proving out whether you could write document-centric transforms using LINQ to XML in a style similar to XSLT. It’s easy to read the code to specify the transform if you read LINQ code easily, but there are obvious problems with the approach, not the least of which is that annotating a tree in that fashion might have performance issues if you are working with too large of a tree.

Even though Open XML and XHTML have similar structures, there are places where the structures are not parallel, and in those cases, you still must jump through hoops. In XSLT, this often means generating intermediate trees to use in subsequent transforms. I’ve seen XSLT transforms where the first thing the developer did was to transform the tree to a new tree with new attributes on elements – the purpose of the attributes was to aid further transforms. If using the LINQ to XML approach that uses annotations, you must deal with the same issues – parts of the transformation are expressed as nice mappings between a pattern that matches nodes and the subsequent transform of those nodes, and parts of the transformation deals with abstractions that often must be explained in comments. It’s just more complicated to do these types of transformations.

Open XML Document-Centric Transforms

There are lots of examples of interesting common-vocabulary document-centric Open XML transforms. Removing comments and accepting revisions are two, but there are many others.

I've needed to write a number of these over the last couple of years. Because I know the size of documents that I potentially need to process (>2 million nodes), I rejected the annotations approach for simple transforms of Open XML documents. For performance reasons, it just wouldn’t fly.

I also rejected using XSLT – I really don’t want to step out into another language. XSLT is an attractive approach if you already have an XSLT transform written, or if you are particularly fluent in XSLT. You must deal with converting the OPC (Zip) file to the Flat OPC format, but this is easy. But when I’m writing little examples that show how to do something interesting in Open XML, XSLT isn’t appropriate.

So, for instance, for the code to accept tracked revisions, I opted for the tree-modification approach. This isn’t idea from a functional programming purist’s point of view, but it performs well in the real world. You have to be careful when coding, but no big deal.

Recursive Approach to Transforms

Lately, I’ve been writing more of these types of transforms in a recursive style. The gist of this technique is that you write a recursive function to clone a tree, and while cloning, you trim nodes, or transform nodes, or whatever.

This approach has good performance, and it is appealing in that when you are writing a more complicated recursive transform, you can write it in terms of other simpler recursive transforms. The code should be written with no side effects, and if so, transforms are easy to write and debug.

This approach has a draw-back. It’s somewhat harder to intuitively see the mapping between the pattern that matches a node, and the transform for that pattern. However, we don’t lose this entirely. For example, we may want to write a recursive transform of an XHTML document to Open XML. Here is the XHTML document:

<html>

<body>

abcdefghi

</body>

</html>

We want to transform it to this document:

<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" >

<w:body>

<w:p>

<w:r>

<w:t>abc</w:t>

</w:r>

<w:r>

<w:rPr>

<w:b/>

</w:rPr>

<w:t>def</w:t>

</w:r>

<w:r>

<w:t>ghi</w:t>

</w:r>

</w:p>

</w:body>

</w:document>

We can write the recursive transform like this:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object XHtmlToOpenXml(XNode node)

{

XNamespace w = "http://schemas.openxmlformats.org/wordprocessingml/2006/main";

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "html")

return new XElement(w + "document",

new XAttribute(XNamespace.Xmlns + "w", w.NamespaceName),

new XElement(w + "head", ""),

element.Elements().Select(e => XHtmlToOpenXml(e)));

if (element.Name == "body")

return new XElement(w + "body",

element.Elements().Select(e => XHtmlToOpenXml(e)));

if (element.Name == "p")

return new XElement(w + "p",

element.Nodes().Select(n => XHtmlToOpenXml(n)));

if (element.Name == "b")

return new XElement(w + "r",

new XElement(w + "rPr",

new XElement(w + "b")),

new XElement(w + "t",

element.Value));

}

XText t = node as XText;

if (t != null)

return new XElement(w + "r",

new XElement(w + "t", t.Value));

// ignore all other nodes

return null;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<html>

<body>

abcdefghi

</body>

</html>");

Console.WriteLine(XHtmlToOpenXml(root));

}

In the above transform, the code highlighted in yellow serves the same purpose as the XPath pattern to match in an XSLT template. The code highlighted in green is the “sequence constructor”. The expressions element.Nodes().Select(n => XHtmlToOpenXml(n)) and element.Elements().Select(e => XHtmlToOpenXml(e)) serves the same purpose as the xsl:apply-templates element in an XSLT template.

Key to understanding this transform is that selectively, we can cause every node/element to be passed to this method. But we can also trim descendant nodes if we like, sending only a subset back through this method.

I initially started talking about this approach in a post that described manually cloning XML trees. The code is short and easy to understand.

I used this approach for code to normalize an XML tree. It performs well. Of the approaches that I could have taken for coding the sample, it was by far the easiest.

I also used this approach for the code to split runs in paragraphs. Again, it was the easiest way for me to write the code.

For more information about writing these types of transformations, see Recursive Approach to Pure Functional Transformations of XML.

This certainly isn’t the last word. This is what has been on my mind lately, so wanted to blog it before I forgot about it.

I’m fascinated by XML document transformation, primarily because of the power it gives me. The ability to spin out an Open XML document in a couple hundred lines of code opens up a lot of interesting scenarios. Generating documents server-side in SharePoint or a web application allows us to use documents to make it easier for people to communicate. Document-centric transforms are key in these scenarios.

Empty Elements and Self-Closing Tags

EricWhite — Wed, 08 Jul 2009 02:06:00 GMT

Last January, I blogged about an approach to normalizing LINQ to XML trees. That post is based on another post, Manually Cloning LINQ to XML Trees. In those posts, my code to clone an element would clone a self-closing element (<Tag/>) as self-closing, and an empty element with a start and end tag (<Tag></Tag>) as an element with start and end tag.

But, in fact, this was not necessary – empty elements can be always serialized as self-closing elements – the XML specification states, “The representation of an empty element is either a start-tag immediately followed by an end-tag, or an empty-element tag."

Further, per the specification, “the empty-element tag SHOULD be used, and SHOULD only be used, for elements which are declared EMPTY”. This means that it’s always safe to serialize an empty element as a self-closing element, but sometimes it’s not correct to serialize an empty element with a start and end tag.

Originally, the code to clone an element looked like this:

static XElement CloneElement(XElement element)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n =>

{

XElement e = n as XElement;

if (e != null)

return CloneElement(e);

return n;

}),

(!element.IsEmpty && !element.Nodes().OfType<XText>().Any()) ? "" : null

);

}

I’ve revised both of the above referenced posts to remove the code to exactly serialize empty elements as they were in the source document. The new code looks like this:

static XElement CloneElement(XElement element)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n =>

{

XElement e = n as XElement;

if (e != null)

return CloneElement(e);

return n;

})

);

}

static void Main(string[] args)

{

XElement root = XElement.Parse("<Root></Root>");

Console.WriteLine("Original tree");

Console.WriteLine(root);

Console.WriteLine();

Console.WriteLine("Cloned tree");

XElement rootClone = CloneElement(root);

Console.WriteLine(rootClone);

}

The code is simpler and more correct. When you run this example, it produces:

Original tree

Cloned tree

Thanks to Sean Hederman for pointing this out.

Working with Optional Elements and Attributes in LINQ to XML Queries

EricWhite — Thu, 14 May 2009 07:02:00 GMT

Often XML schemas allow for optional elements and attributes. When you write queries on these elements or attributes, you may be tempted to write code that does lots of testing for null. There is a better way to do this, laid out in this post. I covered this idiom in a previous post, but the main purpose of that post wasn’t to explain this idiom. I’m speaking on using LINQ with Open XML tomorrow at TechEd 2009, and need a better example.

The following XML document is a simplified variation of markup that you can find in Open XML word processing documents:

<body>

<r>

<t>Text of first para.</t>

</r>

<pPr>

</pPr>

<r>

<t>Text of second para.</t>

</r>

</body>

</document>

The first paragraph doesn’t have a <pPr> element, whereas the second does. This is allowable in Open XML word processing documents. The first paragraph has the default style.

Our task is to write a query that returns the style name for each paragraph, but if the paragraph has no style name, then the paragraph has the default style. The code projects a collection of an anonymous type that contains the style name and the text. If the paragraph has the default style, the StyleName is set to null.

The approach where the code tests for null looks like this:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static string GetStyleName(XElement p)

{

XElement pPr = p.Element("pPr");

if (pPr != null)

{

XElement pStyle = pPr.Element("pStyle");

if (pStyle != null)

return (string)pStyle.Attribute("val");

}

return null;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<document>

<body>

<t>Text of first para.</t>

<pPr>

</pPr>

<t>Text of second para.</t>

</body>

</document>");

var paragraphs = root

.Element("body")

.Elements("p")

.Select(p => new

{

StyleName = GetStyleName(p),

Text = (string)p.Element("t")

});

foreach (var item in paragraphs)

Console.WriteLine(item);

}

This works just fine, and yields the expected results:

{ StyleName = , Text = Text of first para. }

{ StyleName = Heading1, Text = Text of second para. }

Beyond making the code harder to read, this approach introduces two additional points of possible failure. If I had neglected to write the code to test for null, my code would throw an exception.

There is another way to write this query, which is to use the Elements and Attributes extension methods that operate on IEnumerable<XElement>.

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<document>

<body>

<t>Text of first para.</t>

<pPr>

</pPr>

<t>Text of second para.</t>

</body>

</document>");

var paragraphs = root

.Element("body")

.Elements("p")

.Select(p => new

{

StyleName = (string)p.Elements("pPr").Elements("pStyle")

.Attributes("val").FirstOrDefault(),

Text = (string)p.Element("t")

});

foreach (var item in paragraphs)

Console.WriteLine(item);

}

This also yields the same results, and doesn’t contain the two points of possible failure.

Here’s how this code works. In the snippet below, the highlighted code evaluates to a collection of XElement objects. Notice that I used the Elements method, not the Element method, even though I know that there could only be zero or one <pPr> elements. The highlighted code returns a collection of either zero or one items.

StyleName = (string)p.Elements("pPr").Elements("pStyle")

.Attributes("val").FirstOrDefault(),

The Elements extension method yields all child elements with the given name for each and every element in the source collection. In the snippet below, the highlighted code will return either one XElement object (if there was a <pPr> element), or an empty collection, if there wasn’t a <pPr> element:

StyleName = (string)p.Elements("pPr").Elements("pStyle")

.Attributes("val").FirstOrDefault(),

Next, the code ‘dots’ into the Attributes extension method. Again, the Attributes extension method is happy to take a collection of elements as source. If an empty collection is passed to the Attributes extension method, it also returns an empty collection:

StyleName = (string)p.Elements("pPr").Elements("pStyle")

.Attributes("val").FirstOrDefault(),

The FirstOrDefault extension method either returns the first element in a collection, or it returns the default value for the type of items in the collection. The default value for all reference types (which XAttribute and XElement are) is null. In this case, FirstOrDefault will either return the one “val” attribute, or it will return null.

Finally we cast this one value (either null or an XAttribute) to string. String is a nullable type, of course, and the explicit conversion operator (the cast operator for XAttribute or XElement) is happy to take null, and return null. If there is no <pPr> element, StyleName will be set to null. If there is a pPr element, a pStyle element, and a val attribute, then StyleName will be set to the value of the attribute.

This is where the other nullable CLR types (int?, bool?, double?, etc.) come in handy. If we want to get the value of an optional element or attribute, and we know that the value is an integer or double, or whatever, instead of casting to string, we can cast to any of the other nullable types. The same semantics apply.

Interestingly, this is also efficient. Here’s why:

The LINQ to XML axes use deferred execution.
FirstOrDefault starts the process of materialization, requesting the first item in the collection from the Attributes extension method.
It, in turn, requests the first item from the Elements(“pStyle”) call.
It, in turn, requests the first element from the XElement.Elements method, which finally yields up an XElement (or returns an empty collection if there isn’t one).
This one element is yielded up to the Elements(“pStyle”) extension method.
This one element is yielded up to the Attributes(“val”) extension method.
The one attribute is yielded up to the FirstOrDefault extension method, which due to its semantics, “short circuits” the query, and never requests another item from its source.

The net result of this is that this idiom is efficient, reduces points of possible failure, and is shorter. Once I started using this idiom regularly, it became quite natural. Once one of the LINQ architects described this idiom, I started using it, and mostly don't write code in the other style. So I’m curious – question for all you LINQ users out there, do you use this idiom?

Computing Deep Hash Codes using LINQ to XML

EricWhite — Sat, 21 Feb 2009 21:20:00 GMT

In some scenarios, it is useful to compute a ‘deep’ hash code using LINQ to XML. If you are writing a server-side application that caches queries expressed in XML, then you may want to store these queries in a hash table. XNode (and XElement) inherit the GetHashCode method from object, which returns a hash code based on object identity, which doesn’t help in these scenarios. However, there is a method, XNodeEqualityComparer.GetHashCode, which computes a deep hash code. It’s part of the LINQ to XML API, but it’s not very discoverable via Intellisense, so some folks don’t know about it.

This is related to XNode.DeepEquals, which compares the values of two nodes, including the values of all descendant nodes.

And of course, before computing your deep hash code, you may want to normalize the XML tree, which eliminates trivial differences between trees that will cause computation of different hash codes for semantically identical trees.

Manually Cloning LINQ to XML Trees

EricWhite — Wed, 28 Jan 2009 03:33:00 GMT

[Blog Map]

(July 1, 2009 - Updated - OK to normalize empty elements to an element with a self-closing tag.)

There are a variety of circumstances where you want to clone a LINQ to XML tree while making modifications to the cloned tree. It’s possible to write a very small amount of recursive code to do this. I’ve posted elsewhere about normalizing a LINQ to XML tree, and in that post, I use the coding approach that I present here.

You can, of course, clone an XML tree using the XElement or XDocument constructors, however you have no control over the cloning process.

The gist of the technique is to write a recursive function that clones an element. In its simplest form, here is the code to clone a tree:

static XElement CloneElement(XElement element)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n =>

{

XElement e = n as XElement;

if (e != null)

return CloneElement(e);

return n;

}

)

);

}

static void Main(string[] args)

{

XElement root = new XElement("Root",

new XAttribute("a", 1),

new XElement("Child", 1),

new XElement("Child", 2),

new XComment("This is a comment.")

);

XElement root2 = CloneElement(root);

Console.WriteLine(root);

Console.WriteLine("==========");

Console.WriteLine(root2);

Console.WriteLine();

}

The default behavior of CloneElement is to normalize an empty element (<Tag></Tag>) to a self-closing element (<Tag/>). This is correct behavior - the XML specification basically states that the two forms of markup are equivalent. Further, if an element is declared to be EMPTY, only a self-closing tag will do, whereas any empty element can be converted to a self-closing tag, so it's always safe to convert to self-closing. This is the default behavior of LINQ to XML.

We can modify the CloneElement method to customize the cloned tree. If, say, we want to remove all namespace attributes and remove all comment nodes, we can write it like this:

static XElement CloneElement(XElement element)

{

return new XElement(element.Name,

element.Attributes().Where(a => !a.IsNamespaceDeclaration),

element.Nodes().Select(n =>

{

if (n is XComment)

return null;

XElement e = n as XElement;

if (e != null)

return CloneElement(e);

return n;

}

)

);

}

static void Main(string[] args)

{

XNamespace aw = "http://www.adventureworks.com";

XElement root = new XElement(aw + "Root",

new XAttribute(XNamespace.Xmlns + "aw", aw.NamespaceName),

new XAttribute("a", 1),

new XElement(aw + "Child", 1),

new XElement(aw + "Child", 2),

new XComment("This is a comment.")

);

XElement root2 = CloneElement(root);

Console.WriteLine(root);

Console.WriteLine("==========");

Console.WriteLine(root2);

}

This produces the following output:

<aw:Root xmlns:aw="http://www.adventureworks.com" a="1">

<aw:Child>1</aw:Child>

<aw:Child>2</aw:Child>

</aw:Root>

==========

</Root>

Two Upcoming Presentations on LINQ and Open XML

EricWhite — Thu, 15 Jan 2009 04:12:00 GMT

At TechReady8, I’ll be presenting a 400 level course “Developing Office Client Solutions using LINQ and Open XML”. TechReady8 is a Microsoft employee only event, but non-Microsoft folks in the Seattle area can come see the same talk at the .NET Developers Association of Redmond meeting on January 19^th. This talk will be at 7:00PM, in Microsoft Building 40, Steptoe Room #1450. You can find more information at www.dotnetda.com.

Here is the abstract for the talk:

Developing Office Client Solutions using LINQ and Open XML (400)

There are a variety of approaches to working with Open XML documents using LINQ to XML, and each approach is optimal for certain situations. The most common pattern is to use pure functional transforms for retrieving data from documents. An interesting alternative approach uses LINQ to XML annotations to do document centric transforms; this approach uses LINQ to XML in the style of XSLT. Sometimes the best approach for certain tasks is in-place tree modification ala DOM, but there are pitfalls. Highlights of the talk: changing the basic approach to one of transformations instead of algorithms, an explanation and justification for functional purity, and a discussion of the development thought process when writing pure functional transforms. You'll see lots of demos, including querying word processing and spreadsheet documents, along with a natural syntax for querying Excel tables. In addition, you'll be introduced to tools that you can use to develop Open XML solutions more efficiently and reliably. Because this is a very technical session, knowledge of generics and LINQ are suggested.

Getting Started with SharePoint Web Services using LINQ to XML

EricWhite — Tue, 06 Jan 2009 03:52:00 GMT

[Blog Map]

Web services are one of the most effective and convenient ways for casual developers to access SharePoint lists and document libraries. They have a reputation for being a bit difficult; LINQ to XML can make them easier to use. This post presents the basic steps for getting started with Windows SharePoint Services Web Services using LINQ to XML (either C# or VB).

In several upcoming posts, I’ll present some more information about using web services, as well as more notes on the code presented in this post. There are some interesting and easy ways to explore SharePoint web services using LINQ to XML. However, before addressing more involved scenarios, I want to make sure that you can get started easily.

First, a few points about SharePoint web services. They can be disabled for a SharePoint site, but by default, they are enabled. In most cases, it is easy to use them; if you have permissions to manually create and modify documents for a site, you probably can use web services to do so too.

One important feature of SharePoint that enables effective collaboration is the My Site feature. For an introduction to your My Site, see this Office Online article. Chances are, if you have a My Site in your company, you can use web services to programmatically access it. In an upcoming project, I plan to use the SharePoint collaboration features to make managing the project easier. Then I'll write a little C# program that collects information from lists and document libraries (using web services) and produces a nicely formatted status report as an Open XML Wordprocessing document, and then places the report into an appropriate document library.

Here are the steps:

Create a new Windows Console application (either C# or VB).

Important note: Pay attention to the name that you give your application, as the classes created to access the SharePoint web services will be in the namespace of your application. For this example, name the application SPWebServicesExample.

Add a reference to your SharePoint web service. Select Project, Add Service Reference…

Click the Advanced button:

Click the ‘Add Web Reference’ button:

In the Add Web Reference dialog box, enter the URL of the ‘Lists’ web service. Web services in SharePoint are located in the ‘_vti_bin’ directory under the relevant site. If you are using web services to access your My Site, then enter the URL: ‘http://my/sites/your-alias- here/_vti_bin/lists.asmx’. If you are using web services to access a SharePoint site located at http://ourteamsite, the URL will be ‘http://ourteamsite/_vti_bin/lists.asmx’.

Click the Go button to confirm the URL. If you have entered a valid URL, then this dialog box tells you that it found a web service. The dialog box will be populated with the various operations available in the web service.

Change the web reference name to something relevant. For accessing the Lists web service, a good web reference name is ‘ListsWebService’.

Click the Add Reference button to add this reference to your project.

Following is a small example to get all of the lists and list items in a site. The example creates a small XML "report" that contains relevant data on the lists and items. In addition to the listing on this page, I've attached the code to this post.

In general, when using this code as boilerplate, you have to remember to update three things (all highlighted in the listing below):

The namespace for your web service proxy class (two occurrences)
The web reference name (two occurrences)
The URL of the SharePoint site that you want to access

Note that if you used the names suggested earlier, you will not need to update the first two items, as the web service proxy will already be set to “SPWebServicesExample” and the web reference name will already be set to “ListsWebService”.

There are a couple of notes to make about the following code:

The code to create the report uses the LINQ to XML idiom that I detailed in the blog post titled ''Writing Succinct Code to move XElement and XAttribute Objects from One LINQ to XML Tree to Another".
When printing the "Report" XML tree to the console, the code uses the technique that I described in "Align Attributes when Formatting XML using LINQ to XML". To make it more convenient to convert an XML tree to a string where attributes are aligned, in this example I wrote an extension method, ToStringAlignAttributes.
To convert the LINQ to XML trees to XmlNode (and back), the codes uses the approach from "Convert XElement to XmlNode (and Convert XmlNode to XElement) ".

It is more convenient to use LINQ to XML instead of XmlDocument for accessing web services. You can write simpler, easier to read code when creating the XML that you pass as arguments to operations. In particular, for this purpose, VB XML literals rock! And the code to query the XML returned by the web service is much easier to write, in my opinion.

I'll be blogging more about how to use these web services. There's a lot that you can do with them. I am especially enthused about taking advantage of what you can do with Open XML and the Open XML SDK. Stay tuned...

C#

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml;

using System.Xml.Linq;

namespace SPWebServicesExample

{

public static class MyExtensions

{

public static XElement GetXElement (this XmlNode node)

{

XDocument xDoc = new XDocument();

using (XmlWriter xmlWriter = xDoc.CreateWriter())

node.WriteTo(xmlWriter);

return xDoc.Root;

}

public static XmlNode GetXmlNode (this XElement element)

{

using (XmlReader xmlReader = element.CreateReader())

{

XmlDocument xmlDoc = new XmlDocument();

xmlDoc.Load(xmlReader);

return xmlDoc;

}

public static string ToStringAlignAttributes(this XElement element)

{

XmlWriterSettings settings = new XmlWriterSettings();

settings.Indent = true;

settings.OmitXmlDeclaration = true;

settings.NewLineOnAttributes = true;

StringBuilder stringBuilder = new StringBuilder();

using (XmlWriter xmlWriter = XmlWriter.Create(stringBuilder, settings))

element.WriteTo(xmlWriter);

return stringBuilder.ToString ();

}

class Program

{

static void Main(string[] args)

{

XNamespace s = "http://schemas.microsoft.com/sharepoint/soap/";

XNamespace rs = "urn:schemas-microsoft-com:rowset";

XNamespace z = "#RowsetSchema";

// Make sure that you use the correct namespace, as well as the correct reference

// name. The namespace (by default) is the same as the name of the application

// when you created it. You specify the reference name in the Add Web Reference

// dialog box.

// Namespace Reference Name

// | &nb sp; |

// V &nb sp; V

SPWebServicesExample.ListsWebService.Lists lists =

new SPWebServicesExample.ListsWebService.Lists ();

// Make sure that you update the following URL to point to the Lists web service

// for your SharePoint site.

lists.Url = "http://my/sites/your-alias-here/_vti_bin/Lists.asmx";

//lists.Url = "http://xyzteamsite/_vti_bin/Lists.asmx";

lists.Credentials = System.Net.CredentialCache.DefaultCredentials;

XElement queryOptions = new XElement("QueryOptions",

new XElement("Folder"),

new XElement("IncludeMandatoryColumns", false)

);

XElement viewFields = new XElement("ViewFields");

XElement listCollection = lists.GetListCollection().GetXElement();

XElement report = new XElement("Report",

listCollection

.Elements(s + "List")

.Select(

l =>

{

return new XElement("List",

l.Attribute("Title"),

l.Attribute ("DefaultViewUrl"),

l.Attribute("Description"),

l.Attribute("DocTemplateUrl"),

l.Attribute("BaseType"),

l.Attribute("ItemCount"),

l.Attribute("ID"),

lists.GetListItems((string) l.Attribute("ID"), "", null,

viewFields.GetXmlNode(), "", queryOptions.GetXmlNode(), "")

.GetXElement()

.Descendants(z + "row")

.Select(r =>

new XElement("Row",

r.Attribute("ows_Title"),

r.Attribute("ows_ContentType"),

r.Attribute("ows_FSObjType"),

r.Attribute("ows_Attachments"),

r.Attribute("ows_FirstName"),

r.Attribute("ows_LinkFilename"),

r.Attribute("ows_EncodedAbsUrl"),

r.Attribute("ows_BaseName"),

r.Attribute("ows_FileLeafRef"),

r.Attribute("ows_FileRef"),

r.Attribute("ows_ID"),

r.Attribute("ows_UniqueId"),

r.Attribute("ows_GUID")

)

);

}

)

);

Console.WriteLine (report.ToStringAlignAttributes());

}

VB

Imports System.Xml

Imports System.Text

Module Module1

<System.Runtime.CompilerServices.Extension()> _

Public Function GetXElement (ByRef node As XmlNode) As XElement

dim xDoc as XDocument = new XDocument()

using xmlWriter as XmlWriter = xDoc.CreateWriter()

node.WriteTo(xmlWriter)

End Using

return xDoc.Root

End Function

<System.Runtime.CompilerServices.Extension()> _

Public Function GetXmlNode (ByRef element As XElement) As XmlNode

Using xmlReader as XmlReader = element.CreateReader()

Dim xmlDoc As XmlDocument = New XmlDocument

xmlDoc.Load(xmlReader)

Return xmlDoc

End Using

End Function

<System.Runtime.CompilerServices.Extension()> _

Public Function ToStringAlignAttributes(ByVal element As XElement) As String

Dim settings As XmlWriterSettings = New XmlWriterSettings()

settings.Indent = True

settings.OmitXmlDeclaration = True

settings.NewLineOnAttributes = True

Dim stringBuilder As StringBuilder = New StringBuilder()

Using xmlWriter As XmlWriter = xmlWriter.Create(stringBuilder, settings)

element.WriteTo(xmlWriter)

End Using

Return stringBuilder.ToString()

End Function

Sub Main()

Dim s as XNamespace = "http://schemas.microsoft.com/sharepoint/soap/"

Dim rs as XNamespace = "urn:schemas-microsoft- com:rowset"

Dim z as XNamespace = "#RowsetSchema"

' Make sure that you use the correct namespace, as well as the correct reference name.

' The namespace (by default) is the same as the name of the application when you

' created it. You specify the reference name in the Add Web Reference dialog box.

' Namespace Reference Name

' | |

' V V

Dim lists As SPWebServicesExample.ListsWebService.Lists = _

New SPWebServicesExample.ListsWebService.Lists()

' Update the following URL to point to the Lists web service

' for your SharePoint site.

lists.Url = http://my/sites/your-alias-here/_vti_bin/Lists.asmx

'lists.Url = "http://xyzteamsite/_vti_bin/Lists.asmx";

lists.Credentials = System.Net.CredentialCache.DefaultCredentials

Dim queryOptions = _

<IncludeMandatoryColumns>false</IncludeMandatoryColumns>

</QueryOptions>

Dim viewFields = <ViewFields/>

dim listCollection as XElement = lists.GetListCollection().GetXElement()

Dim report as XElement = _

<%= listCollection _

.Elements(s + "List") _

.Select( Function(l) _

new XElement("List", _

l.Attribute("Title"), _

l.Attribute("DefaultViewUrl"), _

l.Attribute("Description"), _

l.Attribute("DocTemplateUrl"), _

l.Attribute("BaseType"), _

l.Attribute("ItemCount"), _

l.Attribute("ID"), _

lists.GetListItems(l.Attribute("ID"), "", Nothing, _

viewFields.GetXmlNode(), "", queryOptions.GetXmlNode(), "") _

.GetXElement() _

.Descendants(z + "row") _

.Select (Function(r) _

new XElement ("Row", _

r.Attribute("ows_Title"), _

r.Attribute("ows_ContentType"), _

r.Attribute("ows_FSObjType"), _

r.Attribute("ows_Attachments"), _

r.Attribute("ows_FirstName"), _

r.Attribute("ows_LinkFilename"), _

r.Attribute("ows_EncodedAbsUrl"), _

r.Attribute("ows_BaseName"), _

r.Attribute("ows_FileLeafRef"), _

r.Attribute("ows_FileRef"), _

r.Attribute("ows_ID"), _

r.Attribute("ows_UniqueId"), _

r.Attribute("ows_GUID") _

) _

) %>

</Report>

Console.WriteLine (report.ToStringAlignAttributes())

End Sub

End Module

After you build and run this example, your output should look like this. Note that for even a small SharePoint site, this may run several thousand lines long, so you may have to redirect the output to a file or change the screen buffer height on the command prompt window to view it all.

<List

Title="AList"

DefaultViewUrl="/sites/ericwhit/AList/Forms/AllItems.aspx"

Description="AList"

DocTemplateUrl="/sites/ericwhit/AList/Forms/template.dotx"

BaseType="1"

ItemCount="0"

ID="{AA12AA75-2422 -48BB-B41A-267C61D7A54F}" />

<List

Title="Eric White's Wiki"

DefaultViewUrl="/sites/ericwhit/Eric Whites Wiki/Forms/AllPages.aspx"

Description="Wiki"

DocTemplateUrl=""

BaseType="1"

ItemCount="3"

ID="{AA0A2753-5A94 -42C1-BAA7-66DD1409CD3E}">

<Row

ows_ContentType="Wiki Page"

ows_FSObjType="2;#0"

ows_LinkFilename="Home.aspx"

ows_EncodedAbsUrl="http://my/sites/ericwhit/Eric%20Whites%20Wiki/Home.aspx"

ows_BaseName="Home"

ows_FileLeafRef="2;#Home.aspx"

ows_FileRef="2;#sites/ericwhit/Eric Whites Wiki/Home.aspx"

ows_ID="2"

ows_UniqueId="2;#{AA6833E4-98F2-42C4-9355-6DF2453DC7D1}"

ows_GUID="{AAEA7CFF-9FE1-47F9-8C48-94B631289D0F}" />

<Row

...

Writing Succinct Code to move XElement and XAttribute Objects from One LINQ to XML Tree to Another

EricWhite — Thu, 01 Jan 2009 18:36:00 GMT

[Blog Map]

There is an interesting idiom of LINQ to XML that makes it easy to write short, succinct code when creating one XML tree from another. Many folks are not aware of this idiom, but it is a useful and convenient trick. I documented this idiom in the LINQ to XML documentation, however, it is worthwhile to discuss in a blog post. This post describes this idiom in detail.

As you probably know, one of the overloads of the XElement constructors is a params array, so that you can pass multiple child nodes as parameters to the parent XElement constructor. It is perfectly valid to pass null as one of those arguments, and the XElement constructor will simply ignore that argument. For example, in the following code, the null parameter is ignored:

XElement xmlTree = new XElement("Root",

new XElement("Child", 1),

null,

new XElement("Child", 2)

);

Console.WriteLine(xmlTree);

This example produces the following output:

<Root>

</Root>

The same semantics apply when you pass a collection of XElement objects to the constructor. If one of the items in the collection is null, then the null item is simply ignored.

List<XElement> elementList = new List<XElement>{

new XElement("Child", 1),

null,

new XElement("Child", 3)

};

XElement xmlTree = new XElement("Root", elementList);

Console.WriteLine(xmlTree);

This example outputs:

<Root>

</Root>

And, if you use the XElement.Attribute or XElement.Element method, and pass in the name of an element or attribute that doesn’t exist, the method returns null. Consider the following snippet:

XElement xmlTree = new XElement("Root",

new XAttribute("Att1", 1),

new XAttribute("Att3", 3)

);

if (xmlTree.Attribute("Att2") == null)

Console.WriteLine("Att2 doesn't exist");

else

Console.WriteLine("This won't be printed.");

This example will print “Att2 doesn’t exist”.

We can use these semantics to move attributes and child elements (where they may or may not exist) from one element to another. We want to create a new Root element that contains Att2, Att3, and Att4 (but not Att1). And we want the new Root element to contain Att2, Att3, and Att4 only if they exist in the source Root element. We can write this code as follows:

XElement source = new XElement("Root",

new XAttribute("Att1", 1),

new XAttribute("Att3", 3),

new XAttribute("Att4", 4)

);

// move Att2, Att3, and Att4 to the new element, but only if they exist in the

// source element.

XElement newRoot = new XElement("Root",

source.Attribute("Att2"),

source.Attribute("Att3"),

source.Attribute("Att4")

);

Console.WriteLine(newRoot);

This example will produce the following output:

As you can see in the above example, Att2 was not moved to the new tree because it didn’t exist. And based on the above discussed semantics, LINQ to XML didn’t throw any exceptions.

The XML that the SharePoint web services return sometimes contains elements that have many attributes. I know I'm not interested in many of these attributes when writing code to accomplish a particular task. These uninteresting attributes obfuscate the XML, and make it harder to learn about the XML returned by the web services. One approach that I’ve taken is to create a new ‘Report’ XML tree that contains only the elements and attributes that I’m interested in. The idiom discussed in this post is useful when creating the ‘Report’ XML tree:

lists.GetListItems((string)l.Attribute("ID"), "", null,

viewFields.GetXmlNode(), "", queryOptions.GetXmlNode(), "")

.GetXElement()

.Descendants(z + "row")

.Select(r =>

new XElement("Row",

r.Attribute("ows_Title"),

r.Attribute("ows_ContentType"),

r.Attribute("ows_FSObjType"),

r.Attribute("ows_Attachments"),

r.Attribute("ows_FirstName"),

r.Attribute("ows_LinkFilename"),

r.Attribute("ows_EncodedAbsUrl"),

r.Attribute("ows_BaseName"),

r.Attribute("ows_FileLeafRef"),

r.Attribute("ows_FileRef"),

r.Attribute("ows_ID"),

r.Attribute("ows_UniqueId"),

r.Attribute("ows_GUID")

In this example, the code takes a number of attributes (which may or may not exist) from the “row” element, and moves them to the “Row” element in a new XML tree.

Align Attributes when Formatting XML using LINQ to XML

EricWhite — Tue, 30 Dec 2008 04:54:00 GMT

[Blog Map]

Sometimes an XML element contains a large number of attributes, and the values of the attributes may be long. When written to the console, such lines wrap, making it hard to read the XML. In particular, the XML responses from SharePoint web services often contain many attributes for each element. However, you can use appropriate settings for an XmlWriter so that the attributes are aligned, and each attribute is on a separate line. This post shows how to do this.

XNamespace w = "http://www.northwind.com/examples/2008";

XElement root = new XElement(w + "Root",

new XAttribute(XNamespace.Xmlns + "w", w.NamespaceName),

new XAttribute("RootAtt1", 1),

new XAttribute("RootAtt2", "This is an attribute with a very long value"),

new XAttribute("RootAtt3", 3),

new XElement(w + "Child",

new XAttribute(w + "Att1", 1),

new XAttribute("Att2", 2)

)

);

Console.WriteLine(root);

When this example is run, the console window will look like this:

<w:Root xmlns:w="http://www.northwind.com/examples/2008" RootAtt1="1" RootAtt2="

This is an attribute with a very long value" RootAtt3="3">

<w:Child w:Att1="1" Att2="2" />

</w:Root>

The following example shows how to use XmlWriter to align attributes:

XNamespace w = "http://www.northwind.com/examples/2008";

XElement root = new XElement(w + "Root",

new XAttribute(XNamespace.Xmlns + "w", w.NamespaceName),

new XAttribute("RootAtt1", 1),

new XAttribute("RootAtt2", "This is an attribute with a very long value"),

new XAttribute("RootAtt3", 3),

new XElement(w + "Child",

new XAttribute(w + "Att1", 1),

new XAttribute("Att2", 2)

)

);

XmlWriterSettings settings = new XmlWriterSettings();

settings.Indent = true;

settings.OmitXmlDeclaration = true;

settings.NewLineOnAttributes = true;

using (XmlWriter xmlWriter = XmlWriter.Create(Console.Out, settings))

root.WriteTo(xmlWriter);

This produces the following output:

<w:Root xmlns:w="http://www.northwind.com/examples/2008"

RootAtt1="1"

RootAtt2="This is an attribute with a very long value"

RootAtt3="3">

<w:Child

w:Att1="1"

Att2="2" />

</w:Root>

This is much easier to read if there are many attributes.

Convert XElement to XmlNode (and Convert XmlNode to XElement)

EricWhite — Mon, 22 Dec 2008 18:05:00 GMT

[Blog Map]

Sometimes you want to convert an XmlNode to an XElement and back again. Some programming libraries define methods that take XmlNode objects as parameters. These libraries also may contain properties and methods that return XmlNode objects. However, it is more convenient to work with LINQ to XML instead of the classes in System.Xml (XmlDocument, XmlNode, etc.) This post presents a bit of code to do these conversions.

As an example of where you need these methods, you can do a lot with SharePoint using web services. The proxy classes that wsdl.exe creates contain methods that use classes in System.Xml.

It is convenient to write these conversions as extension methods. When converting to and from classes in System.Xml, the code reads much better when you tack an extension method on the end instead of surrounding the expression with a method call. The following code shows how to call the GetListItems method of the Lists web service using the GetXmlNode and GetXElement extension methods:

XElement queryOptions = new XElement("QueryOptions",

new XElement("Folder", "Open XML Documents"),

new XElement("ViewAttributes", new XAttribute("Scope", "Recursive"))

);

XElement viewFields = new XElement("ViewFields",

new XElement("FieldRef", new XAttribute("Name", "GUID")),

new XElement("FieldRef", new XAttribute("Name", "ContentType")),

new XElement("FieldRef", new XAttribute("Name", "BaseName")),

new XElement("FieldRef", new XAttribute("Name", "Modified")),

new XElement("FieldRef", new XAttribute("Name", "EncodedAbsUrl"))

);

XElement listContent = wsList

.GetListItems(documentLibraryGUID,

null, null, viewFields.GetXmlNode(),

null, queryOptions.GetXmlNode(), webId)

.GetXElement();

To create an XmlNode from an XElement, you create an XmlReader using the XNode.CreateReader method, create an XmlDocument, and load the document using the XmlReader. XmlDocument inherits XmlNode.

To create an XElement from an XmlNode, you create an XDocument, create an XmlWriter using the XContainer.CreateWriter method, write the XmlNode to the XDocument, and return the root element of the XDocument.

The following example contains the extension methods and some code that shows their use:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Xml;

using System.Xml.Linq;

public static class MyExtensions

{

public static XElement GetXElement(this XmlNode node)

{

XDocument xDoc = new XDocument();

using (XmlWriter xmlWriter = xDoc.CreateWriter())

node.WriteTo(xmlWriter);

return xDoc.Root;

}

public static XmlNode GetXmlNode(this XElement element)

{

using (XmlReader xmlReader = element.CreateReader())

{

XmlDocument xmlDoc = new XmlDocument();

xmlDoc.Load(xmlReader);

return xmlDoc;

}

class Program

{

static void Main(string[] args)

{

XElement e = new XElement("Root",

new XElement("Child",

new XAttribute("Att", "1")

)

);

XmlNode xmlNode = e.GetXmlNode();

Console.WriteLine(xmlNode.OuterXml);

XElement newElement = xmlNode.GetXElement();

Console.WriteLine(newElement);

}

Code is attached.

A More Robust Approach for Handling XName Objects in LINQ to XML

EricWhite — Mon, 15 Dec 2008 23:54:00 GMT

[Blog Map]

One of the advantages of V2 of the Open XML SDK is that it provides us with a strongly typed XML document object model (DOM). Because elements are represented by classes, and attributes are represented by properties in the classes, developers are not at risk of misspelling element and attribute names. I was chatting about the Open XML SDK V2 with a customer, who appreciated the advantages of this approach. However, V2 of the SDK is not an option for them, because it won’t be available to use to build a shipping product until the release of the next version of Office, and this doesn’t fit their schedule. There is another approach using V1 of the SDK and LINQ to XML that provides some of the benefits of a strongly typed DOM. The approach consists of declaring a static class with static, initialized XName and XNamespace fields in the class, and then using those XName and XNamespace objects in the LINQ to XML code to create and query XML trees.

By using an approach of automatically generating (from XSD) the static classes that contain the names, it assures that the names and namespaces are not misspelled. In addition, this approach has the benefit of pre-atomization of the XName and XNamespace objects. I’ve written a few posts on the benefits of pre-atomization of XName and XNamespace objects:

Writing Robust LINQ to XML Code that Performs Well

Atomized XName and XNamespace Objects

Preatomization of XName Objects

Pre-atomization can yield performance benefits both when creating and querying XML trees. The static members of the class are initialized at application startup time. After startup, the application will not need to further atomize any XName or XNamespace objects.

There is one additional advantage – you get some level of Intellisense support when you use this approach.

The Open XML Markup has a fair number of element and attribute names – I don’t know the exact count. As a data point, I queried a quite large Open XML document for distinct element and attribute names (the Ecma Open XML Specification Part 4), and counted 809 distinct qualified names in 21 namespaces.

I wondered if pre-atomization of two or three thousand XName objects would present a performance problem on application startup, so I wrote a small program to generate a program that contains 3800 initialized XName objects. I took care to generate names that are representative of the names in the Open XML specification – some are short, some are long, and the names are in a variety of namespaces. When run on a fairly slow laptop (2 ghz, 1 core), the program initialization took approximately 1/2 of a second, which is reasonable. Further on in this post, I present the small program that generates the test program.

The following example shows the approach of declaring a couple of static classes that contains static initialized fields. You can see that code in Main uses the initialized names in the static classes both for the construction of the XML tree, and in a query:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml.Linq;

public static class Pkg

{

public static XNamespace ns_pkg =

"http://schemas.microsoft.com/office/2006/xmlPackage";

public static XName package = ns_pkg + "package";

public static XName part = ns_pkg + "part";

public static XName name = ns_pkg + "name";

public static XName contentType = ns_pkg + "contentType";

public static XName xmlData = ns_pkg + "xmlData";

}

public static class Ext

{

public static XNamespace ns_ext =

"http://schemas.openxmlformats.org/officeDocument/2006/extended-properties";

public static XName Properties = ns_ext + "Properties";

public static XName Template = ns_ext + "Template";

public static XName TotalTime = ns_ext + "TotalTime";

}

class Program

{

static void Main(string[] args)

{

var pkg = new XElement(Pkg.package,

new XAttribute(XNamespace.Xmlns + "pkg",

"http://schemas.microsoft.com/office/2006/xmlPackage"),

new XElement(Pkg.part,

new XAttribute(Pkg.name, "/docProps/app.xml"),

new XAttribute(Pkg.contentType,

"application/vnd.openxmlformats-officedocument.extended-properties+xml"),

new XElement(Pkg.xmlData,

new XElement(Ext.Properties,

new XAttribute("xmlns", Ext.ns_ext),

new XElement(Ext.Template, "Normal.dotm"),

new XElement(Ext.TotalTime, "1")

)

);

var totalTime =

(from e in pkg.Descendants(Ext.TotalTime)

select (int)e)

.First();

Console.WriteLine(totalTime);

}

Here is the program that generates the test program:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml.Linq;

class Program

{

static HashSet<string> usedNames;

static void Gen(string className, string nsAbbr, string ns, int count)

{

Console.WriteLine("public static class {0}", className);

Console.WriteLine("{");

Console.WriteLine(" public static XNamespace {0} = \"{1}\";", nsAbbr, ns);

Console.WriteLine();

Random r = new Random();

for (int i = 0; i < count; i++)

{

string s = "";

// the following loop (using the usedNames HashSet) makes sure that

// the code does not generate duplicate names.

while (true)

{

// make sure that the first character of the name is a letter A-Z

char startChar = (char)('A' + (int)(r.NextDouble() * 26));

Guid g = Guid.NewGuid();

int len = (int)Math.Floor(r.NextDouble() * 12 + 2);

s = startChar.ToString() +

g.ToString()

.Replace("-", "")

.Substring(0, len);

if (usedNames.Add(s))

break;

}

Console.WriteLine(" public static XName {0} = {1} + \"{0}\";", s, nsAbbr);

}

Console.WriteLine("}");

Console.WriteLine();

}

static void Main(string[] args)

{

Console.WriteLine(@"using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml.Linq;

");

usedNames = new HashSet<string>();

Gen("Ns_W", "ns_w",

"schemas.openxmlformats.org/wordprocessingml/2006/main", 1000);

Gen("Ns_VE", "ns_ve",

"http://schemas.openxmlformats.org/markup-compatibility/2006", 200);

Gen("Ns_O", "ns_o",

"urn:schemas-microsoft-com:office:office", 1000);

Gen("Ns_R", "ns_r",

"http://schemas.openxmlformats.org/officeDocument/2006/relationships", 600);

Gen("Ns_M", "ns_m",

"http://schemas.openxmlformats.org/officeDocument/2006/math", 1000);

Console.WriteLine(@"

public class Program

{

public static void Main(string[] args)

{

Console.WriteLine(""Entered Main"");

}

");

}

The test program looks something like this:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Xml.Linq;

public static class Ns_W

{

public static XNamespace ns_w = "schemas.openxmlformats.org/wordprocessingml/2006/main";

public static XName A2dff9d51ac4 = ns_w + "A2dff9d51ac4";

public static XName F362d21f0a = ns_w + "F362d21f0a";

public static XName Ne594dbac67 = ns_w + "Ne594dbac67";

public static XName X4cad35 = ns_w + "X4cad35";

public static XName Ybca171196db = ns_w + "Ybca171196db";

public static XName C41d0325b2028 = ns_w + "C41d0325b2028";

public static XName D3e0eb73ab5a94 = ns_w + "D3e0eb73ab5a94";

public static XName M5df215a698c = ns_w + "M5df215a698c";

public static XName Zebe33a = ns_w + "Zebe33a";

public static XName J8d2563d1a13 = ns_w + "J8d2563d1a13";

// ...

}

public static class Ns_VE

{

public static XNamespace ns_ve = "http://schemas.openxmlformats.org/markup-compatibility/2006";

public static XName Afd026fa849e = ns_ve + "Afd026fa849e";

public static XName F206884417 = ns_ve + "F206884417";

public static XName N867bd48925 = ns_ve + "N867bd48925";

public static XName X801087 = ns_ve + "X801087";

public static XName Yf601615604c = ns_ve + "Yf601615604c";

public static XName C1701240984e3 = ns_ve + "C1701240984e3";

// ...

}

public static class Ns_O

{

public static XNamespace ns_o = "urn:schemas-microsoft-com:office:office";

public static XName A4a22d12b3d2 = ns_o + "A4a22d12b3d2";

public static XName Fa7d35e114 = ns_o + "Fa7d35e114";

public static XName N5154768163 = ns_o + "N5154768163";

public static XName Xdf110a = ns_o + "Xdf110a";

public static XName Yece0763cf81 = ns_o + "Yece0763cf81";

public static XName C5322c3f14297 = ns_o + "C5322c3f14297";

public static XName Dd1d69594fef44 = ns_o + "Dd1d69594fef44";

public static XName Mbfb32a08d32 = ns_o + "Mbfb32a08d32";

public static XName Z509498 = ns_o + "Z509498";

public static XName J16db47997f9 = ns_o + "J16db47997f9";

// ...

}

public static class Ns_R

{

public static XNamespace ns_r = "http://schemas.openxmlformats.org/officeDocument/2006/relationships";

public static XName A9676b45e848 = ns_r + "A9676b45e848";

public static XName F51ab6cb14 = ns_r + "F51ab6cb14";

public static XName Ncfb544f931 = ns_r + "Ncfb544f931";

public static XName X4f67fc = ns_r + "X4f67fc";

// ...

}

public static class Ns_M

{

public static XNamespace ns_m = "http://schemas.openxmlformats.org/officeDocument/2006/math";

public static XName A5993dd93a92 = ns_m + "A5993dd93a92";

public static XName Fa5592d4c1 = ns_m + "Fa5592d4c1";

public static XName N29869d8a94 = ns_m + "N29869d8a94";

public static XName Xb80c69 = ns_m + "Xb80c69";

// ...

}

public class Program

{

public static void Main(string[] args)

{

Console.WriteLine("Entered Main");

}

Code is attached.

Debugging LINQ Queries

EricWhite — Fri, 07 Nov 2008 07:49:00 GMT

[Blog Map]

Debugging LINQ queries can be problematic. One of the reasons is that quite often, you write a large query as a single expression, and you can’t set a breakpoint mid-expression. Writing large queries in expression context is particularly powerful when using functional construction to form XML (or using the strongly typed DOM in Open XML SDK V2). This post presents a little trick that makes it easier to use the debugger with LINQ queries that are written using ‘method syntax’.

The gist of the technique is to insert a call to the Select extension method in the middle of your query. You code the Select so that it projects exactly the same results as its source, but using a statement lambda expression If you place the return statement of the lambda expression on its own line, you can set a breakpoint and examine values as they make their way through the query.

The following is a query to split a string into words, convert the words to lower case, count the number of occurances of each word, and return the ten most used words:

var uniqueWords = text

.Split(' ', '.', ',')

.Where(i => i != "")

.Select(i => i.ToLower())

.GroupBy(i => i)

.OrderByDescending(i => i.Count())

.Select(i => new { Word = i.Key, Count = i.Count() })

.Take(10);

But if you set a breakpoint anywhere on the query, you see this:

Let’s say that you want to examine each group and see the group key and the count of items for each group. You can insert a Select statement, as follows:

var uniqueWords = text

.Split(' ', '.', ',')

.Where(i => i != "")

.Select(i => i.ToLower())

.GroupBy(i => i)

.Select(z => {

return z;

})

.OrderByDescending(i => i.Count())

.Select(i => new { Word = i.Key, Count = i.Count() })

.Take(10);

you can now set a breakpoint on the ‘return z’ statement, and examine each group in turn as they are yielded up by the GroupBy extension method:

You can see that the key is “on”, and that there are four items in the group.

After you are done debugging, you can remove the added call to Select.

Alternatively, you could convert any of the other lambda expressions to a statement lambda expression, format code so that a statement is on its own line, and then set a breakpoint.