Marcelo's WebLog : Performance

MSXML XHTML DTDs - making the web better

marcelolr — Mon, 30 Nov 2009 22:00:00 GMT

The fix to requests for XHTML DTD files from the W3C Web server has been recently released. Windows Update should offer the fix automatically, but you can download and install the fix manually from the following links for various MSXML versions:

http://support.microsoft.com/?kbid=973688 [MSXML4 SP2... if you haven't tried SP3 yet see below]
http://support.microsoft.com/?kbid=973685 [MSXML4 SP3]
http://support.microsoft.com/?kbid=973686 [MSXML6 Out-Of-Band, for XP SP2 and Win2K3]
http://support.microsoft.com/?kbid=973687 [MSXML3 and MSXML6 for all OSes where these components are in-band]

What exactly is this about? Well, there are a number of cases where this can happen, but one common scenario that we've seen people run into when writing AJAX web pages is to request a page of information from the server, load it into an XML document and extract some information or merge it into the existing page.

The problem with this approach is that when you load the document into an MSXML document "by hand" (ie, not through responseXML on the XHR object), DTD processing is enabled, and the DOCTYPE declaration directs MSXML to go look up the XHTML DTD so you can use entities like  .

Multiply this by each web browser accessing a popular site, and you can imagine why no-one is happy with this situation: web sites break when the resource is not available, the W3C servers get overloaded, and users of web sites lose functionality with odd scripting errors.

The fix caches the XHTML DTDs in MSXML - these resources haven't changed for years and will likely end up in different URLs if something new comes along. So now clients save at least one round-trip (possibly more) and always get the DTD support they need.

Try running this bit of script in a .js file to see the awesomeness in action.

If you don't have the update, you might get an error such as the following.

You have error The server did not understand the request, or the request was invalid.
Error processing resource 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'.

If you do have the update, you'll see the parsed XML. Run this with Fiddler open and you'll "see" the absence of network activity.

Enjoy!

Performance Tools

marcelolr — Sat, 13 Jun 2009 00:00:00 GMT

I've written about performance tools in the past - it's a topic that interests me quite a bit, because I believe that perceived performance is a very fundamental part of the user experience and can wow you or frustrate you immensely. And I believe "real performance" (so to speak) is very important of course, especially when you don't have a person sitting in front of the computer, but I think every developer will appreciate the difference that a 2' checkin script has vs. a 20' checkin script or a 60' script. 'Nuff said.

In any case, this post about Visual Studio 2010 profiler tools shows some fabulously productive stuff. And all with Visual Studio usability, rather than what it feels like to use some of the more arcane tools (that are admittedly very well suited to automated scenarios, we'll always grant them that).

Anyway, read up and enjoy!

DTD resolution and network behavior

marcelolr — Tue, 21 Apr 2009 00:00:00 GMT

Following up from my last post, danieldsmith asked about a couple of additional details. I think I responded in the comments, but because (a) it's generally useful, and (b) I had some problems with my network connection (because I monkeying around with it), the reply may have been lost. So here we go again.

First, is caching implemented for the resolved DTDs? No, the default resolver doesn't do any caching - files will always be requested. So if you're loading documents with DTDs in a tight loop, you might get a pretty significant perf improvement here.

Second, what happens if there is no network connectivity? Well, the framework can't do validation without the DTD, and even if it could it would be unable to expand entities, so as you might guess, the problem isn't silently ignored - you'll get an exception instead.

Trying this on my machine at home, I got two slightly different exception - your mileage may vary. The first time I had Fiddler running, and I got a System.Net.WebException with a Message value "The remote server returned an error: (502) Bad Gateway.", and a Status value of ProtocolError. There was a Response assigned to it, which had a message of its own, "Fiddler - DNS Lookup Failed".

Then I shut down Fiddler and got another WebException, although this time the message was "The remote name could not be resolved: 'www.w3.org'", and the Response of the exception was null.

So now you know. Keep an eye on your network activity, and turn off features that you don't use to get perf / stability improvements without having to go and make changes to your application.

As a side note, I like to run Fiddler every now and then, just to see what kind of web traffic my machine is generating - it's awfully interesting to see the kinds of things that are going on sometimes.

Enjoy!

Faster XML - don't resolve DTDs if you don't need to

marcelolr — Sat, 18 Apr 2009 01:00:00 GMT

When loading an XML document through XmlDocument.Load or XDocument.Load, the default behavior when finding a DTD reference is to go resolve the URL, which typically means one or more web requests. Often, however, the DTD is there more as a marker of what the document contains than anything else, and you might not want to reference entities or do validation.

For example, let's say I have the following code. This will load an XML document from a string, then write it out to the console.

const string xmlText = 
@"<!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'>
<html lang='en' xml:lang='en' xmlns='http://www.w3.org/1999/xhtml'>
<head><title>Hello</title></head>
<body><h1>Hello from XHTML</h1></body>
</html>";

XmlDocument doc = new XmlDocument();
doc.LoadXml(xmlText);
Console.WriteLine(doc.InnerXml);
Console.ReadKey();

The problem here is that there is no reason for us to access the DTD for this particular application, and yet the following requests will go out.

http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd
http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent
http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent
http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent

The first solution, available on Silverlight, is to use the XmlPreloadedResolver. By default it will contain DTDs for XHTML and RSS, but you can add your own. In this case, you get the same level of functionality as if the real resources had been requests. For example, you can use entity references such as   to mean "non-breaking white space". If you assign an instance as your XmlResolver, you're good to go.

A second solution, which is also available on the full .NET Framework, is to simply set the XmlResolver property to null. In this case, no requests will be made at all, but you won't be able to use entity references nor have the document validated.

Note that this isn't just available on XmlDocument, but also through XmlReaderSettings, anytime you're creating an XmlReader (which can of course be used with both XmlDocument and XDocument, along with many other parts of the framework).

In the next release of the .NET Framework, there will be a few other ways in which you may save the resource requests, but these options are available to you today.

Enjoy!

Velocity at DevConnections

marcelolr — Sat, 21 Mar 2009 00:00:00 GMT

Next Monday I'll be at DevConnections in Orlando, talking about Velocity (Microsoft Project Code Named "Velocity", if you like formality).

It's going to be a bit of a challenge to sum up all that awesomeness into a single session, but I promise I'll do my best, and I'll be around for the rest of the event in case anyone wants to get in touch. Contact through email and blog will be slow, but should get back in rhythm after the event.

In case you find the online list a tad long to search through, the session is titled an unsurprising MDA02: Microsoft Project Codenamed Velocity: A Distributed In-memory Cache.

See you in Orlando!

XmlSchema and XmlSchemaSet thread safety

marcelolr — Tue, 17 Mar 2009 00:00:00 GMT

Here's a good word of warning: even if an object "feels" read-only because you're not calling code to modify it, if it's not documented as safe for use from multiple threads, then you shouldn't risk it.

In particular, I'd like to talk about XmlSchema and XmlSchemaSet today. Building these has a cost associated, and so it's nice to be able to build them once and then reuse them. But you have to be very careful in doing this. The docs say that all instance methods are not safe for multiple thread usage, but you don't really use them directly during validation, so it's hard to tell from the outside what's safe and what's not.

In a nutshell, the only thing you can do that is safe for concurrent usage is to use a validating reader. Here's the sample code to try this out (for some reason, this "breaks" more on 64-bit machines, but it's unsafe on all architectures).

First, a little helper to create an XmlSchema.

private XmlSchema CreateSchema()
{
  string schemaText = @"<?xml version='1.0'?>
<xs:schema id='play' targetNamespace='http://tempuri.org/play.xsd'
 elementFormDefault='qualified' xmlns='http://tempuri.org/play.xsd'
 xmlns:xs='http://www.w3.org/2001/XMLSchema'>
 <xs:element name='myShoeSize'>
  <xs:complexType>
   <xs:simpleContent>
    <xs:extension base='xs:decimal'>
     <xs:attribute name='sizing' type='xs:string' />
    </xs:extension>
   </xs:simpleContent>
  </xs:complexType>
  </xs:element>
</xs:schema>";

  using (StringReader reader = new StringReader(schemaText))
  {
    return XmlSchema.Read(reader, null);
  }
}

Next, a simple XmlSchemaSet.

private XmlSchemaSet CreateSchemaSet(XmlSchema schema)
{
  XmlSchemaSet set = new XmlSchemaSet();
  set.Add(schema);
  set.Compile();
  return set;
}

Finally, some validation:

private void ValidateDocument(XmlSchemaSet set)
{
  string doc = @"<myShoeSize xmlns='http://tempuri.org/play.xsd' sizing='123' />";
  XmlReaderSettings settings = new XmlReaderSettings();
  settings.Schemas = set;
  settings.ValidationEventHandler += new ValidationEventHandler(settings_ValidationEventHandler);
  using (StringReader reader = new StringReader(doc))
  using (XmlReader x = XmlReader.Create(reader, settings))
  {
    while (x.Read()) { }
  }
}

private int failCount;
void settings_ValidationEventHandler(object sender, ValidationEventArgs e)
{
  System.Threading.Interlocked.Increment(ref failCount);
}

Now, armed with these, I will show you some code that is thread-safe, but that a single line reorder would cause to break.

XmlSchema schema = CreateSchema();
Thread[] threads = new Thread[10];
XmlSchemaSet set = CreateSchemaSet(schema);
for (int i = 0; i < threads.Length; i++)
{
  threads[i] = new Thread((x) =>
    {
      for (int j = 0; j < 1000; j++)
      {
        // If the CreateSchemaSet were here
        // instead of outside this would break!
        //
        // Don't add the schema to the 
        // XmlSchemaSet from multiple threads!
        //
        // XmlSchemaSet set = CreateSchemaSet(schema);
        //
        ValidateDocument(set);
      }
    });
}

Array.ForEach(threads, (t) => t.Start());
Array.ForEach(threads, (t) => t.Join());
this.Text = "Failure count: " + failCount; 

The part before the thread creation is all thread-safe; the stuff inside the callback is happening on multiple threads at the same time. You can only use the set for validation here!

Enjoy!

Fast way to convert XmlDocument into XDocument

marcelolr — Sat, 14 Mar 2009 00:00:00 GMT

This question came up in the forums a little while ago, and I thought it might be interesting to do some comparisons.

I first came up with a few ways of turning an XmlDocument into an XDocument.

private static XDocument DocumentToXDocument(XmlDocument doc)
{
  return XDocument.Parse(doc.OuterXml);
}

private static XDocument DocumentToXDocumentNavigator(XmlDocument doc)
{
  return XDocument.Load(doc.CreateNavigator().ReadSubtree());
}

private static XDocument DocumentToXDocumentReader(XmlDocument doc)
{
  return XDocument.Load(new XmlNodeReader(doc));
}

Next I whipped up a function to time these with something quick and dirty. I make sure the past activity doesn't both much in terms of leaving garbage, and I warm up the action a bit (I also warm up the Stopwatch methods, just in case).

private static long Time(int count, Action action)
{
  GC.Collect();
  for (int i = 0; i < 3; i++)
  {
    action();
  }

  Stopwatch watch = new Stopwatch();
  watch.Start();
  watch.Stop();
  watch.Reset();
  watch.Start();

  for (int i = 0; i < count; i++)
  {
    action();
  }

  long result = watch.ElapsedMilliseconds;
  watch.Stop();
  return result;
}

And finally, all together:

StringBuilder sb = new StringBuilder();
sb.Append("<parent>");
for (int i = 0; i < 1000; i++)
{
  sb.Append(" <child>text</child>");
}
sb.Append("</parent>");

string text = sb.ToString();
XmlDocument doc = new XmlDocument();
doc.LoadXml(text);

long docToXDoc = Time(1000, () => DocumentToXDocument(doc));
long docToXDocNavigator = Time(1000, () => DocumentToXDocumentNavigator(doc));
long docToXDocReader = Time(1000, () => DocumentToXDocumentReader(doc)); 

Note that the actual numbers don't matter much, as this is my laptop running a bunch of things in the background, in the debugger and whatnot, but the relative values are interesting to see.

These are the values I got (they vary a bit each run, but not by much).

Using OuterXml: 1973 ms.
Using a navigator over the document: 1254 ms.
Using a reader over the document: 1154 ms.

Not surprisingly, avoiding the creation of a big string just to re-parse it is a big win - save the planet, use less CPU power!

Enjoy!

XmlDocument.PreserveWhitespace and memory usage

marcelolr — Sat, 07 Mar 2009 01:00:00 GMT

The PreserveWhitespace property is sometimes misunderstood. As this MSDN page describes, this affects non-significant whitespace nodes. Significant whitespace is always preserved.

So what is significant whitespace? The XmlSignificantWhitespace explains: it's the whitespace under the scope of an xml:space='preserve', or the whitespace between elements in a mixed content node (with both elements and text).

For many documents, non-significant whitespace is, well, not significant. It's just there so elements align and are more readable to humans, but processors aren't expected to do anything with it.

So if your program doesn't need it, there's no need for you to pay the price of creating objects to represent them and keep them in your document tree. If you do "typical" indentation, where every element is on a different line and you indent as elements get deeper, then you may actually save yourself some memory by getting rid of the whitespace. Thankfully, PreserveWhitespace defaults to false for XmlDocument, but I've seen cases where folks turn it to true because they aren't quite sure of what it does and it feels safer to not throw something away.

Here's a little snippet that shows the impact (your mileage may vary of course, depending on CPU architecture, version, compiler settings, etc).

bool[] preserves = new bool[] { true, false };
foreach (bool preserve in preserves)
{
  GC.Collect();

  // Build a document.
  StringBuilder s = new StringBuilder();
  s.AppendLine("<root>");
  for (int i = 0; i < 1000; i++)
  {
    s.AppendLine(" <element with='attributes' />");
  }
  s.AppendLine("</root>");

  string text = s.ToString();
  XmlDocument doc = new XmlDocument();
  doc.PreserveWhitespace = preserve;
  doc.LoadXml(text);

  Console.WriteLine(
    "PreserveWhitespace: " + preserve);
  Console.WriteLine(
    "Memory used before collection: {0}", 
    GC.GetTotalMemory(false));
  GC.Collect();
  Console.WriteLine(
    "Memory used after full collection: {0}",
    GC.GetTotalMemory(true));

  Console.WriteLine();
  GC.KeepAlive(doc);
}

And this is the output I get on my machine.

PreserveWhitespace: True
Memory used before collection:     456616
Memory used after full collection: 280208

PreserveWhitespace: False
Memory used before collection:     405480
Memory used after full collection: 232072

So (280208 - 232072) / 280208 ~= 17%, which isn't too bad for a line of code you don't have to write (because it's the default after all).

Enjoy!

Cloning an XmlDocument doesn't make the copy thread safe

marcelolr — Thu, 05 Mar 2009 01:00:00 GMT

This could be a gotcha, but cloning an XmlDocument doesn't quite make a completely independent copy.

The CloneNode method is intended in general to be a way of creating copies of XML nodes under the same XmlDocument scope / owner XmlDocument. Cloning the XmlDocument does create a copy, but it shares the same scope, which has pretty much shrank to the shared NameTable / implementation at that point.

So you can write a thread-safe implementation of NameTable and use that in the constructor of the original document, or you create a new document and then ImportNode, which will create a distinct node under the new document.

Enjoy!

StackOverflowException with XslCompiledTransform

marcelolr — Fri, 20 Feb 2009 01:00:00 GMT

Pawel recently looked into a customer report about XslCompiledTransform failing with a StackOverflowException after an upgrade to .NET Framework 3.5 SP1. Same stylesheet, same everything, but now the exception was showing up when it wasn't before.

There were a number of interesting gotchas associated with this.

The default stack size is smaller when running under IIS, which can lead to spending time figuring out whether you actually have the correct repro or not if you're running in a standalone app.
It turns out that the stylesheet had always been close to the stack limit, but a change in the JIT to improve speed increased stack usage some, and pushed the stack over the limit.
There are ways to rewrite the stylesheet to produce the same results but not consume as much stack internally (yay! - no configuration changes needed)

An interesting point that shouldn't be overlooked is that if you're not going to be changing the XSL at run-time, the XSLT compiler is a great choice.

For the real-deal details, I'll refer you to the MSDN Forum thread again.

Enjoy!

The one where lame excuses are given

marcelolr — Sat, 07 Feb 2009 01:00:00 GMT

Yes, yes, yes - I've been busy. I'm sure you saw that one coming. Also I've been working on things which are top-secret, yadda-yadda.

To reduce my lameness level, however, I'm back to writing for 2009 and I'll begin by linking to a very nice article which has certainly aged well: XML Reader with Bookmarks. This is great for those cases in which you mostly need forward-only processing of XML, but for a very few select cases you might need to rewind (also known as "peek ahead some amount"). A great example of "pay for play" - much better than having to choose between forward-only or all-in-memory.

And one of the benefits of having a long-running blog is that I can now look back and think - what I was I writing about in early 2005? Being effective with Outlook, it seems. I was a lot less prolific back then as well - clearly Avalon (WPF these days) took a lot of my time, that with the programmatic stuff, the new XAML markup and whatnot...

Any implied references to the continued relevance of XML over time and across technologies is, of course, mostly unintentional.

Depth vs. width in ADO.NET Data Services $expand

marcelolr — Tue, 02 Dec 2008 01:00:00 GMT

The $expand option in the query filter is used to request entities associated with the last segment of a request. This is often used to ensure that entities can be materialized with their relationships wired together and to reduce the number of roundtrips to the server.

For example, if http://www.example.com/service.svc/Customers returns all customers, then http://www.example.com/service.svc/Customers?$expand=Orders could be used to return all customers with all their Orders, assuming customers have an Orders collection.

You can use request multiple references and collections to be exapnded. One gotcha about the $expand syntax is that there is a different syntax for "drilling" through references: the forward slash is used to indicate drilling in, while a comma is used to separate the parallel expansion paths.

Here are some examples that will hopefully clarify this (omitting the path to the service).

/Customers?$expand=Orders. Returns customers and the orders of the customers.
/Customers?$expand=Orders,Preferences. Returns customers and the preferences of the customers.
/Customers?$expand=Orders,Orders/OrderDetails,Preferences. Returns customers, the orders of the customers, the order details of the orders of the customers, and the preferences of the customers.
/Customers?$expand=Orders/OrderDetails,Preferences. Same as the last one - when you go deep, all the parents are implied, so Orders/OrderDetailsOrders.
/Customers?$expand=Orders/OrderDetails,Orders/ShippingInfo. Returns customers, the orders of the customers, the order details of the orders of the customers, and the shipping informatio of the orders of the customers. Note that the ',' is used to separate paths starting from the last segment (Customers), so if you want to expand properties in parallel from a deeper entity, you need to specify the path again, as in this example.

Hope this was helpful - please post comments if you have further questions!

Limiting how many entities are returned from an ADO.NET Data Service

marcelolr — Thu, 20 Nov 2008 01:00:00 GMT

If you want to limit the number of entities returned from a service, you have a few alternatives...

If you're writing code for the client, the easiest way is to just use the '$top' clause to limit how many entities will be returned. Note that '$top' doesn't account for entities you expand using '$expand', so be careful with what you expand if you're using it.

That pretty much covers it for the options on the client - the server however is a more interesting thing.

First, you have the configuration options. MaxResultsPerCollection allows you to limit not only the number of entities returned at the top level of a request, but also at each collection within, so clients can't use '$expand' to overwhelm the server. Because a batch request can have multiple queries, you can combine that with MaxBatchCount.

The problem with using the configuration on the server is that it's meant more as a security feature than as a general paging mechanism, and as such the responses returned will (a) likely cause the client to error, because the response ends mid-stream, and (b) the client will have no good way to "continue" asking for data from where the last request left off.

A better approach might be to ask clients to be explicit about how many entities they're willing to handle at once using the '$top' option, and overriding the OnStartProcessingRequest method on the data service to check that the URL has indeed specified the '$top' option. Of course, you still want to set the configuration to safeguard your server!

Enjoy!

Windows Performance Analysis Developer Center

marcelolr — Wed, 05 Nov 2008 01:00:00 GMT

Congratulations to the folks working on the Windows Performance Toolkit and friends!

There's now a developer center you can use as a portal with pointers to resources, downloads, forums, FAQs, and guides to learning about the tools and getting started.

http://msdn.microsoft.com/en-us/performance/default.aspx

As you can see from the Powerful Features topic, some of the fun things you can do include viewing machine-wide activity, cross-referencing I/O with processes and stacks, and activity across different processes. Awesome tool when you need to go beyond a single user process and start looking at system-wide activity to investigate performance.

Enjoy!

Profiling WPF Tools - not just for WPF

marcelolr — Sat, 01 Nov 2008 00:00:00 GMT

Windows Presentation Foundation (WPF) has recently had a refresh to its performance tools. You can read more about it on the What's New for Performance Profiling Tools for WPF post.

The suite of tools includes a bunch of very interesting tools, one of which is a string allocator profiler. The good news is - you don't need to be working on a WPF application to be able to use it! You can just point it to any app you have, and you'll get a tree view of function calls and the count and size of strings allocated - very handy stuff to find where those allocation are coming from.

Enjoy!