Recursive Approach to Pure Functional Transformations of XML

EricWhite — Mon, 20 Jul 2009 16:28:00 GMT

Writing pure functional transformations a in a recursive style enables us to put together interesting transformations in a very small amount of code. Using some specific techniques that allow us to write this code very concisely, this approach takes advantage of some perhaps obscure semantics of LINQ to XML. I’ve used this approach to write some interesting transformations – it has become my favorite way to write transformations of a certain variety. This post presents a short tutorial in writing these types of transformations.

Identity Transform

First, let’s take a look at the identity transform:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(@"<Root Att1='1'><Child/></Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

In this code, we are taking advantage of the semantics of the XElement constructor that allow us to pass collections of objects to the constructor. The following code creates a new XElement from an existing XElement:

new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

This code, of course, passes two objects to the constructor (in addition to the name) – a collection of attributes from the source element, and a collection of transformed child nodes from the source element. Note that even though we’re passing collections to the constructor, we’re only passing two objects. We’re passing the two variables that contain the iterators for the queries. Those iterators, of course, implement IEnumerable<T>. There are a number of LINQ to XML methods and constructors that can take content, and if we pass an object that implements IEnumerable of some T, LINQ to XML iterates the enumerable and adds each object in the collection to the newly constructed element. This is performed recursively. For detailed semantics of the constructors and methods, see this page in the LINQ to XML documentation.

This is one of the reasons that the signatures of LINQ to XML constructors and methods that take content are defined using object instead of XObject:

public XElement(XName name, params Object[] content);

Because the signature is defined using object, we can pass individual XElement, XAttribute and XNode objects to the constructor, and we can pass collections of them.

Cloning vs. Attaching

The identity transform also takes advantage of the cloning vs. attaching semantics of LINQ to XML. When we pass a newly constructed XElement to the XElement constructor, the new object does not have a parent so it is simply attached to the XElement that is being constructed. But the highlighted line below causes a node from an existing tree to be passed to the XElement constructor. In this case, the XElement constructor notices that the node is already part of an existing tree, and clones it and adds the cloned node to the XElement being constructed.

XElement element = node as XElement;

if (element != null)

{

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

To make this very clear, take a look at the following example:

XElement tree1 = XElement.Parse("<Root><Child1/></Root>");

XElement child1 = tree1.Element("Child1");

XElement child2 = new XElement("Child2");

XElement tree2 = new XElement("Root", child1, child2);

Console.WriteLine(tree2);

In this example, child1 already is part of tree1, so it is cloned and the newly cloned element is added to tree2. The child2 XElement object has no parent so it is simply attached to tree2.

For more information on cloning vs. attaching, see this page in the LINQ to XML documentation.

Note that when an XElement object is cloned by a LINQ to XML constructor or method, of course all attributes and descendant nodes are also cloned.

Removing Nodes from the Newly Cloned Tree

Sometimes when we clone a tree, we want to trim certain elements and attributes and nodes from the newly cloned tree. We can take advantage of the fact that it’s valid to pass null as one of the arguments to the XElement constructor. If one of the arguments to the XElement constructor is null, the XElement constructor simply ignores that argument:

XElement xml = new XElement("Root",

new XElement("Child1"),

null,

new XElement("Child3"));

Console.WriteLine(xml);

This produces the following output:

<Root>

</Root>

Let’s say that we want to trim all Child2 elements from the following tree:

<Root>

</Root>

We can code the transform like this:

using System;

using System.Linq;

using System.Xml.Linq;

class Program

{

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return null;

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(@"<Root><Child1/><Child2/><Child3/></Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

These semantics apply regardless of whether null is passed directly to the XElement constructor, or whether null is passed as part of a collection to the XElement constructor.

Such element name comparisons (element.Name == "Child2") are very efficient if we pre-atomize element and attribute names. See this post for more information about atomization of element and attribute names.

Note: for simplicity of demonstration, in these examples, I didn’t bother to pre-atomize XNames, but pre-atomization is important if performance is important.

Replace an Element with another Element

A common operation in transforms is to replace some particular element with an entirely new element. Let’s say that we want to replace the Child2 element with a Child5 element that contains a child element named Text. We want to transform this:

<Root>

</Root>

To this:

<Root>

</Child5>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return new XElement("Child5",

new XElement("Text"));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

Replace an Element with Multiple Elements

Sometimes we may want to replace a single element with multiple elements. We can take advantage of the fact that the behavior of the XElement constructor for collections of objects is, as mentioned, recursive. If one of the objects in the collection is itself a collection of objects then that child collection is itself iterated and all objects are added to the element being constructed.

The following example shows these semantics of the XElement constructor. The example first creates two lists of XElement objects. It then creates a list that contains the two lists. It passes the list of lists to the XElement constructor, and the resulting XElement contains all of the XElement objects in the two lists:

List<XElement> list1 = new List<XElement> {

new XElement("Child1"),

new XElement("Child2")

};

List<XElement> list2 = new List<XElement> {

new XElement("Child3"),

new XElement("Child4")

};

List<List<XElement>> listOfLists = new List<List<XElement>> { list1, list2 };

XElement newElement = new XElement("Root", listOfLists);

Console.WriteLine(newElement);

This produces the following output:

<Root>

</Root>

Let’s say that we want to replace the Child2 element with three Child5 elements. We want to transform this:

<Root>

</Root>

To this:

<Root>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child2")

return new List<XElement> {

new XElement("Child5"),

new XElement("Child5")

};

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

This example is coded a little bit artificially – more typically we would want to replace a particular element with a number of elements that are contained in a collection that is returned by some query. Let’s say that we wanted to effectively remove a certain element from the tree, and replace that element with its children elements. We want to transform this:

<Root>

<Child>

</Child>

</Root>

To this:

<Root>

</Root>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "Child")

return element.Elements();

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<Root>

<Child>

</Child>

</Root>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

This is why it’s important that the Transform function returns object instead of XNode or XObject. This is of course an exact parallel to the XElement constructor – the XElement constructor takes object as an argument so that you can pass either an XObject (or objects of derived classes) or a collection of XObject. The Transform function returns object so that you can return either an XObject or a collection of XObject. (There are other reasons that the XElement constructor takes object as a parameter, but these are the relevant reasons for this discussion.)

More Sophisticated Selection of Nodes to Transform

In the above examples, the code selected elements based on the element name. In XSLT, of course, we often write involved XPath expressions to select nodes to transform. Sometimes we want to select nodes for transformation where the node has specific ancestors. Let’s say that we want to transform a paragraph only if the paragraph is in a content control:

<body>

abc

<sdt>

def

</sdt>

</body>

We can code it like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "p" && element.Parent.Name == "sdt")

return new XElement("p", "different content");

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<body>

abc

<sdt>

def

</sdt>

</body>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

This will be efficient – if the node isn’t a paragraph node, then the Boolean expression will be short-circuited, and no extra work will be done. If the node is a paragraph node, then the code consists of following a pointer in the ancestor linked list, and another efficient XName comparison. I've written even more involved node selection code than this - select a paragraph to transform based on contents of the previous paragraph, for example. (Note that if you are selecting nodes in reverse document order, you may want to use an approach that gives good performance.)

Selectively Transforming Descendant Nodes

Sometimes we want to transform just a specific set of the descendant nodes, ignoring everything else. In the following query, the highlighted line serves the same purpose as xsl:apply-templates in an XSLT style sheet. This is the code that sends each of the descendants through the transform function. If we include a call to the Where extension method in that query, we can cause only a subset of the descendants to be sent through the recursive function.

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

Let’s say that we want to perform a transform on the following XML document:

<Root>

<Child>

</Child>

</Root>

We want to transform all of the ’A’ elements to ‘Apple’, the ‘B’ elements to ‘Banana’ and the ‘C’ elements to ‘Carot’, but only for all of the elements where Id equals ‘1’. We want the resulting XML file to look like this:

<Root>

<Child>

</Child>

</Root>

We can code the transform like this:

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "A")

return new XElement("Apple",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "B")

return new XElement("Banana",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "C")

return new XElement("Carot",

element.Attributes(),

element.Nodes().Select(e => Transform(e)));

if (element.Name == "Child")

return new XElement("Child",

element.Attributes(),

element.Elements()

.Where(e => e.Attribute("Id").Value == "1")

.Select(e => Transform(e)));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

Only the XML elements that satisfy the predicate in the Where clause will be sent back through the transform function.

Notice that in this case, I used the Elements axis instead of the Nodes axis. Based on the needs of my transform, I can use any of a variety of axes to specify which nodes will be sent back through the recursive transform function.

Using Multiple Recursive Functions in the Same Transform

These types of transforms are always written in the pure style. In other words, no variables are ever mutated the while doing this transform. The transform always produces a completely new XML tree and never modifies the source tree. This gives us the freedom to call out to other recursive transforms that are also implemented in the pure style.

Let’s say that we’re transforming an Open XML document and we’ve written an interesting transform just for the paragraph element, we can write our transform for the main document part to call the paragraph transform function as appropriate. Because both transforms are written in a pure fashion, we can do this without worrying about introducing bugs associated with side effects in our code.

Following is the source document that we want to transform. This XML is ‘pseudo’ Open XML markup – namespaces are removed and the XML is simplified for the purposes of demonstration.

<body>

<r>

<t>This is a short sentence.</t>

</r>

<r>

<t>This is a different sentence.</t>

</r>

<sdt>

<r>

<t>This is a sentence in a content control.</t>

</r>

</sdt>

</body>

</document>

We want to produce the following document:

<body>

<r>

<t>THIS IS A SHORT SENTENCE.</t>

</r>

<r>

<t>THIS IS A DIFFERENT SENTENCE.</t>

</r>

<r>

<t>THIS IS A SENTENCE IN A CONTENT CONTROL.</t>

</r>

</body>

</document>

We could code the transformation like this:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Xml.Linq;

class Program

{

// This paragraph transform might do something very interesting - perhaps splitting

// runs in a new way or doing sophisticated modifications to styling. In this example,

// the code will transform the text node to upper case.

static object ParagraphTransform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

if (element.Name == "t")

return new XElement("t", ((string)element).ToUpper());

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => ParagraphTransform(n)));

}

return node;

}

// This transform might do something very interesting with content controls. In this

// case the transform will do something very simple - it removes the content control

// and promotes its children elements to the level of the content control.

static object Transform(XNode node)

{

XElement element = node as XElement;

if (element != null)

{

// call into the recursive transform for the paragraph

if (element.Name == "p")

return ParagraphTransform(element);

// remove the content control

if (element.Name == "sdt")

return element.Nodes().Select(n => Transform(n));

return new XElement(element.Name,

element.Attributes(),

element.Nodes().Select(n => Transform(n)));

}

return node;

}

static void Main(string[] args)

{

XElement root = XElement.Parse(

@"<document>

<body>

<r>

<t>This is a short sentence.</t>

</r>

<r>

<t>This is a different sentence.</t>

</r>

<sdt>

<r>

<t>This is a sentence in a content control.</t>

</r>

</sdt>

</body>

</document>");

XElement newRoot = (XElement)Transform(root);

Console.WriteLine(newRoot);

}

Because this transformation was coded in this style, artifacts of Open XML such as custom XML elements and nested content controls are handled automatically without us taking any special care to handle them.

Sometimes it’s useful to write small utility functions for some of the functionality supplied by each recursive transform function. It may be useful to put each transform function and its associated utility functions in a static class.

Normalization of Empty Elements

There is one point regarding empty elements that is worth mentioning – empty elements <Tag></Tag> are normalized to use the empty-element tag <Tag />. This post contains a discussion on empty elements.

I had an unfortunate bike accident last week and broke my right arm, and I’m right handed! I used the voice recognition features of Windows 7 to write this post. These features are great! I don’t think that the process of writing this post was appreciably slower than usual.

Eric White's Blog : C#

Recursive Approach to Pure Functional Transformations of XML

Identity Transform

Cloning vs. Attaching

Removing Nodes from the Newly Cloned Tree

Replace an Element with another Element

Replace an Element with Multiple Elements

More Sophisticated Selection of Nodes to Transform

Selectively Transforming Descendant Nodes

Using Multiple Recursive Functions in the Same Transform

Normalization of Empty Elements

Microsoft Promises No Patent Prosecution for the MONO Project