Mark Fussell's WebLog : XML

System.Xml v2 Articles posted by Alex Homer

mfussell — Wed, 22 Jun 2005 10:47:00 GMT

Alex has written two good overview articles on System.Xml in .NET v2.0 - part 1 on the XmlReader and part 2 on the XmlWriter.

Anders Hejlsberg on XML and Programming Languages

mfussell — Tue, 14 Jun 2005 10:36:00 GMT

One good reason to sign up for PDC 2005 is to hear Anders elaborate more on this work

.NET 2.0 XML Perf Comparison with Sun

mfussell — Tue, 14 Jun 2005 09:50:00 GMT

As presented in my article on System.Xml Beta 1 (which was effectively re-written into the Beta 2 version of my System.Xml book) the #1 feature in the .NET 2.0 release for System.Xml was performance. In the past Sun has come out bashing the XML performance support in .NET 1.1 which I wrote about here. Now all has been vindicated officially with an XML performance comparison report between .NET 2.0 Beta2, .NET 1.1, and Sun Java 1.5 Platforms. Simply by looking at the bar charts, in the majority of scenarios .NET 2.0 Beta 2 was twice as fast as the Sun implementation when parsing XML. Superb!

Furthermore here is a comparison article for Web Services performance between .NET 2.0 Beta 2, .NET 1.1, Sun JWSDP 1.5 and IBM WebSphere 6.0. Of course these numbers were helped by the improvements in System.Xml. Now that I work on the WSE team we are also applying the perf love on higher level services such as XML digtial signature and WS-Security. However for those people who ask whether WSE 3.0 will be based on the XmlReader and XmlWriter for the pipeline processing, I am afraid that this is not going to happen. WSE is fundamentally XmlDocument (XML DOM) based and it is too much to move this to a stream based API in the 3.0 release. However the memory working set in WSE 3.0 has been improved substantially and when using MTOM combined with the ASMX pipeline and HTTP, it is possible to stream binary data from a service onto the network stream effeciently.

Now we just need to complete the performance comparison work by having an up-to-date face-off with the latest XSTL processors using the XSLTMark benchmark and the .NET 2.0 XslCompiledTransform class

ADO.NET and System.Xml v. 2.0--The Beta Version (2nd Edition) now Published

mfussell — Thu, 24 Mar 2005 01:35:00 GMT

In time for the upcoming Visual Studio 2005 and .NET v2.0 Beta 2 release, the updated version of my ADO.NET and System.Xml book has now been published. See this book list for more information and where to download the code examples from. Alternatively just buy a copy so that I can feed my children and bribe my wife to convince her that the hours of writing were worthwhile.

If you bought a copy of the first version of this book, this version is a total re-write and twice as thick, reflecting that a year and a half changes a product in many ways. Given that this is on ADO.NET it also covers features from SQL Server 2005 and complements Bob Beauchemin's et al book A First Look at Microsoft SQL Server 2005 for Developers. I would not buy this book unless you have a sufficient understanding of either ADO.NET or System.Xml today, since this simply covers the delta from .NET v1.1 to .NET v2.0. For example there is a chapter on the XPathNavigator and XmlDocument class enhancements, but you have to understand the existing API to get the most benefit from this. The MSDN2 documentation here is a good place to start (if you want to know more about MSDN2 read this blog entry).

If you feel compelled to provide feedback, I would love to hear suggestions as to what to add or improve especially for the chapters on System.Xml.

Combining the XmlReader and XmlWriter classes for simple streaming transformations

mfussell — Sat, 12 Feb 2005 08:15:00 GMT

The XmlReader and XmlWriter classes can often be combined to provide simple streaming transformations rather than resorting to XSLT which requires a the document to be loaded into memory. This class combination is often faster and uses less memory, although it requires more code and is less flexible in the types of transformations possible. However for many scenarios it is ideal. Say for example you wanted to add a new element in several repeating places to an existing document. The XmlWriter.WriteNode method is useful in pulling from an XmlReader and pushing to an XmlWriter to achieve this, but it does have a limitation in that it writes the current node and all its children to the XmlWriter without providing more fine-grained control.

The code below shows a method called WriteShallowNode, which writes individual nodes to the XmlWriter from the XmlReader. In this way you can change individual nodes during the transformation process.

//This method is useful for streaming transformation with the

//XmlReader and the XmlWriter. It pushes through single nodes in the stream

static void WriteShallowNode( XmlReader reader, XmlWriter writer )

{

if ( reader == null )

{

throw new ArgumentNullException("reader");

}

if ( writer == null )

{

throw new ArgumentNullException("writer");

}

switch ( reader.NodeType )

{

case XmlNodeType.Element:

writer.WriteStartElement( reader.Prefix, reader.LocalName, reader.NamespaceURI );

writer.WriteAttributes( reader, true );

if ( reader.IsEmptyElement )

{

writer.WriteEndElement();

}

break;

case XmlNodeType.Text:

writer.WriteString( reader.Value );

break;

case XmlNodeType.Whitespace:

case XmlNodeType.SignificantWhitespace:

writer.WriteWhitespace(reader.Value);

break;

case XmlNodeType.CDATA:

writer.WriteCData( reader.Value );

break;

case XmlNodeType.EntityReference:

writer.WriteEntityRef(reader.Name);

break;

case XmlNodeType.XmlDeclaration:

case XmlNodeType.ProcessingInstruction:

writer.WriteProcessingInstruction( reader.Name, reader.Value );

break;

case XmlNodeType.DocumentType:

writer.WriteDocType( reader.Name, reader.GetAttribute( "PUBLIC" ), reader.GetAttribute( "SYSTEM" ), reader.Value );

break;

case XmlNodeType.Comment:

writer.WriteComment( reader.Value );

break;

case XmlNodeType.EndElement:

writer.WriteFullEndElement();

break;

}

Performing simple transformations with this method is very straightforward. The code below (written for .NET v2.0) reads the movies.xml document and for those dvd elements whose attribute genre is an action type, a new publisher element is written to the output.

Input document:

<?xml version="1.0"?>

<review>This film was <b>impressive</b> in its creativity.</review>

<stats>

</stats>

</dvd>

….

</dvdstore >

static void AddElementWithWriteShallowNode()

{

XmlWriterSettings settings = new XmlWriterSettings();

settings.Indent = true;

using (XmlReader reader = XmlReader.Create("movies.xml"))

{

using (XmlWriter writer = XmlWriter.Create(Console.Out, settings))

{

while (reader.Read())

{

if (reader.IsStartElement("dvd") &&

reader.GetAttribute("genre") == "action")

{

//Write the dvd element

WriteShallowNode(reader, writer);

//Now add a new publisher element to the output

writer.WriteElementString("publisher",

"metal.sword.com", "Samurai Films");

}

else

WriteShallowNode(reader, writer);

}

Output document:

<?xml version="1.0"?>

<p:publisher xmlns:p='metal.sword.com'>Samurai Films</p:publisher>

<review>This film was <b>impressive</b> in its creativity.</review>

<stats>

</stats>

</dvd>

….

</dvdstore >

Without this method you have to use the XmlWriter WriteXXX methods to re-write the XML document to the output if all you wanted to do was inject a single publisher element into the tree.

For an excellent and complete application example of using this approach see Dan Wahlin’s Generate Dynamic Maps and Flight Routes with XML and SVG

System.Xml V2 book - C'est Fini

mfussell — Fri, 04 Feb 2005 09:14:00 GMT

I finished reviewing the final versions of my chapters for the ADO.NET and System.Xml v 2.0 book which was a huge relief. Having read through it numerous times you start to get numb to your writing style. Needless to say the chapters where all re-written from scratch with completely new content throughout with my favorite being chapter 9, simply because I was closest to the XmlReader and XmlWriter. For instance it has an example of an XmlWriter.WriteShallowNode (XmlReader reader) method which enables you to do easier XmlReader to XmlWriter transformations by enabling single nodes to be written, rather than whole sub-trees. Here is the list of the chapters from the v2 book.

• Chapter 1: New Concepts in Data Access

ADO.NET Enhancements

• Chapter 2: ADO.NET Data Management Enhancements

• Chapter 3: Provider Factories, Schema Discovery, and Security

• Chapter 4: The DataSet and DataTable Classes

SQL Server 2005 Integration

• Chapter 5: ADO.NET and SQL Server 2005

• Chapter 6: SQL Server 2005 CLR Hosting

• Chapter 7: XML in SQL Server 2005

XML Enhancements

• Chapter 8: XML in the .NET Framework

• Chapter 9: Reading and Writing XML

• Chapter 10: XML Serialization Enhancements

• Chapter 11: XML Document Stores

• Chapter 12: Transforming XML Documents

Many XML features fell by the wayside since the first edition of the book, from XQuery to the XmlAdapter, but the essence of the v2.0 release, which was about improving the performance of the core XML APIs, providing type information and improved usability, has held together very well. Now that I work on WSE, which is effectively a messaging filter and processing pipeling built directly on top of the System.Xml, simply moving to the v2.0 release is going to give very measurable performarnce in throughput processing.

Binary XML: A Stillborn Spec.

mfussell — Fri, 14 Jan 2005 08:35:00 GMT

I read with continued dismay another article about binary XML and the bizarre Fast Infoset project. Tim Bray must be crying in bed at night over Sun's continued persistence that ASN.1 should have been the data interchange format for the world. It lost, XML won and XML works. Go and solve other more useful problems. You only have to look at the failure of XML 1.1, as it split the XML standard into incompatible versions to draw the same analogy with binary XML. CPUs continue to fall in price and if needs dictate, XML processing in hardware will take off to solve the issues of processing time. Not yet another proprietary data format.

Just as bad is the W3Cs continued mismanagement by creating working groups such as the Binary Characterization Working Group to trying and determine all the known uses cases that a binary format could apply to. Given that they "own" XML this is a bit like cutting off your nose to spite your face. It also is interesting to observe that very little of significance has emerged from the WC3 for around three years now. XQuery, although significant, is going to turn up very late to the party blaming traffic. XML 1.1 has caused everyone to run around like headless chickens wondering how to sort out the mess where, the smallest of updates can cause the maximum breaking change. And XML Schema is such a confused child no one quite knows how to approach it, for fear of having to spend years understanding its complex irrational behavior. The W3C needs to desperately find some inspirational individuals, nay dictators, to drive some independent innovation if it intends to "ship" anything significant in the next five years.

System.Xml V2 Decisions

mfussell — Thu, 16 Dec 2004 09:20:00 GMT

Arpan has posted on some of the reasons for XmlDocument being the main XML store in System.Xml and the difficulty of supporting standards that are not yet at a recommendation status. Ironically I now find myself in the alternate universe to this where WS-* specifications seem to pop in and out of existence on a daily basis, having to incorporate these into a fast changing project that is laying the path to Indigo. I have been working on WSE 3.0 planning this week and hope to post some more details of the release in the coming weeks. Currently these are the WSE 3.0 themes that I am using to drive the project, which sets the precedence for the features.

VS2005 (Whidbey) Integration
“WSE 3.0 and VS2005 together provide the platform to build advanced web service implementations.”
Indigo Interoperability and Strategic Direction
“Building solutions with WSE 3.0 and VS2005 engages you directly on the path to Indigo.”
Servicing and Customer Support
“WSE 3.0 continues to maintain focus on customer needs, WS-* standards support and performance enabling you to concentrate on your business needs and less on infrastructure.”

As always you have to pick from a huge pile of features and I wish that the team was three times as big. Fortunately other constraints such as being backwardly compatibile with previous releases are not necessary with WSE and so this makes it easier to incorporate new features and remove redundant ones, which creates a very dynamic project.

Beta Book Pain, Performance Love

mfussell — Fri, 12 Nov 2004 08:13:00 GMT

Writing books on beta products is probably the most painful thing that you can ever endure even if you are in the product team (or was as in my case). I finally finished up the last XML chapter in the beta version of ADO.NET and System.Xml V2. It is a vast brain dump of the delta between V1.1 and V2.0 for System.Xml coupled with the numerous changes since the PDC 2003 preview release. However I was not able to cover the #1 feature enough of System.Xml V2.0 - performance. This will have to wait for any potential future final version of the book depending on sales of this one of course. All I can say is that you will not be disappointed - the XML team has done an incredible job. Every component is significantly more performant and more performance improvements will happen between now and RTM. I just saw the new perf numbers for the XmlDocument (DOM) class scenarios (load, edit, delete, insert, save etc) and they have shown an impressive leap. Sun came out at the beginning of the year bashing the performance of the XML support in .NET. I cannot wait for them to run their numbers again on V2.0 and once it ships that they will "go dark" on perf for a long time. Simply by adopting the Beta 2 release any XML processing that you have will be screamingly fast.

Using the XmlSecureResolver to restrict access to web sites

mfussell — Fri, 15 Oct 2004 07:48:00 GMT

Having just written an article on XmlResolver and looked through the MSDN docs I realised that there was little in terms of explanation on how to use the XmlSecureResolver. As always code speaks volumes sIo put together this example which shows how the XmlSecureResolver can be used to restrict access to specific web sites. The XmlSecureResolver secures another implementation of an XmlResolver by wrapping the supplied XmlResolver and restricting the resources that it has access to. For instance, the XmlSecureResolver has the ability to prohibit access to particular internet sites or zones.

public static void UsingXmlSecureResolverToAccessURIs()

{

try

{

WebPermission wb = new WebPermission(PermissionState.None);

wb.AddPermission(NetworkAccess.Connect,

@"http://msdn.microsoft.com/xml/rss.xml");

wb.AddPermission(NetworkAccess.Connect,

@"http://www.w3.org/TR/REC-xml/");

PermissionSet ps = new PermissionSet(PermissionState.None);

ps.AddPermission( wb );

XmlTextReader reader = new XmlTextReader

(@"http://msdn.microsoft.com/xml/rss.xml");

reader.XmlResolver = new XmlSecureResolver(new XmlUrlResolver(), ps);

while (reader.Read()) ;

// This call fails as the site is not in the permission set

reader = new XmlTextReader(@"http://msdn.microsoft.com/sql/rss.xml");

reader.XmlResolver = new XmlSecureResolver(new XmlUrlResolver(), ps);

while (reader.Read()) ;

}

catch (Exception e)

{

Console.WriteLine("Unsuccessful Read Access");

Console.WriteLine(e);

}

When this code is run, only those URIs added to the WebPermission class are allowed to be accessed by the XmlTextReader. All others are denied and result in a security exception being thrown. This is extremely useful in restricting the sites that can be visited, especially when the list of sites can be constrained with a Regex expression.

System.Xml Beta 2 Changes

mfussell — Fri, 15 Oct 2004 07:02:00 GMT

Dare and I compiled the Upcoming Changes to System.Xml in .NET Framework 2.0 Beta 2 which he published on the MSDN XML developer center. Mostly my previous article still stands without the #6 and #3 gone. However as a result of implementing the editablable XPathNavigator over the XmlDocument class this has forced us to re-address performance issues with the DOM.Performance is still the top goal for the version 2.0 release and we continues to make improvements. Recently we found that by rewriting the date handling support in the XML Schema processor gave significant performance improvements in certain scenarios. The goal performance goal - to be as good as or better than MSXML 4.0 which has the leading performance benchmarks, particulary for XSLT, today.

Win-Dev 2004 talks on XQuery in SQL Server and System.Xml V2.0

mfussell — Tue, 05 Oct 2004 07:07:00 GMT

I am doing two talks at the Win-Dev 2004 Conference on the Web Services track and the Databases and Data Access so if you are going to the conference or live in the Boston area and want to meet up to have a chat, drop me a mail. The talks are;

WS6 - Whidbey System.Xml
Mark Fussell

Microsoft's System.Xml proved to be revolutionary for working with XML in a variety of different ways. This innovation has continued in the new Whidbey release with various enhancements and new features that simplify development and increase flexibility. This session will introduce you to the various improvements found in Whidbey's System.Xml.

D4 - XQuery in SQL Server 2005 and System.Xml
Mark Fussell

XML brings with it some powerful query languages for dealing with the hierarchical format that it typical of an XML document. The newest and richest query language is known as XQuery. This talk provides an overview of the XQuery language and explores XQuery functions and XQuery DML inside SQL Server 2005. It also details the XQuery support in the .NET Framework as part of System.Xml namespace that enables the querying of XML sources stored outside of the database such as locally stored files or XML received via webservices. By combining queries over SQL Server 2005 and other XML sources XQuery can be used as a data integration language.

Building Custom XmlResolvers Article on MSDN XML Developer Center

mfussell — Mon, 04 Oct 2004 06:25:00 GMT

In between re-writing and updating the chapters for the beta version of the my book A First Look at ADO.NET and System.Xml V2.0, I found some time to write an article on Building Custom XmlResolvers for MSDN. What's so great about this class is that it enables you to redirect the location of the XML that you want to load, as well as allowing you to define your own schemes, if necessary (e.g. db:// to load from a database). There are all sorts of great things that you can do with an XmlResolver and you can come up with some fairly intelligent caching solutions if your applications needs this. One aspect that I did not cover in this article is the use of the credentials property of the XmlResolver which there is a good KB article about here

As this went to press I was talking to a PM on the Frontpage team who complained that one problem they were having is that they did multiple round trips to a SQL Server data base in order to load <xsl:include> stylesheets i.e. one trip per include and this was obviously having a impact on stylesheet compile time in some scenarios. No problem I said, simply write your own XmlResolver that batches up all the <xsl:include> requests into a single batch call to the database and caches them in-memory. You will need to parse the <xsl:include>s in advance is order to know which stylesheets to load, but a quick read with an XmlTextReader on the XSLT stylesheet can tell you the list, which you can then give to the XmlResolver.

It would be good to hear of any uses that you have for custom XmlResolvers in your applications.

TechEd 2004 XML Talk Slides

mfussell — Tue, 01 Jun 2004 06:39:00 GMT

I have finally posted the slides and sample code for my TechEd 2004 talk on XML in the .NET framework. The samples require a copy of Visual Studio 2005 (i.e. the community drop distributed at TechEd) in order to compile and run.

Article on the XML Support in SQL Server 2005

mfussell — Mon, 24 May 2004 19:48:00 GMT

I worked with Shankar to write this article on the XML Support in Microsoft SQL Server 2005 which has just been published on the MSDN XML Developer Center. I would be very interested in comments on how you see an application need for storing XML content in a relational database along with querying and working with this data. E.g. The ability to store WordML documents created from Office 2003.