Microsoft XML Team's WebLog : XSD

Understanding XML Schema Sets in the XSD Designer

XmlTeam — Thu, 04 Jun 2009 22:48:00 GMT

We have recently blogged about the new XML Schema Designer and the various views over schemas offered by it. We visit here the concept of schema sets, which are actually the central organizing concept around what is shown in the designer for a buffer in Visual Studio for which we have XML Schema information (be it an XSD file or an XML file or Visual Basic project with associated schemas). A schema set can be thought of as a collection of pairs, the first part of each pair being an XML namespace and the second part being the location of an XSD file that is associated to that namespace in the set. A namespace can have multiple files associated to it in a particular set and a file can be part of a multiple namespaces (due to the XML Schema concept of chameleons, whereby a file without a targetNamespace attributed defined for its schema element automatically assumes the targetNamespace of any and all files that include it). We explore below the ways in which schema sets are built and computed and how they are visualized in the XML Schema Designer.

A schema set is basically built by walking the tree of external references from a particular set of root XSD files. An external reference is either an include element, which brings in the schema at the specified schemaLocation attribute into the schema in the same namespace as the current schema, or an import element, which specifies a particular XML namespace to import as well as, optionally, a particular schema location where the schema processor can find an XSD file for that namespace (there is also a redefine element defined in XML Schema, but we can treat as essentially equivalent to an include for this discussion). To illustrate these concepts and how they get applied, let’s see them in use in a particular sample industry schema (brainml.xsd, an XML Schema for neurological modeling defined at http://brainml.org ).

When we first open brainml.xsd in Visual Studio, this becomes the root of the XML Schema Set to be constructed and displayed in the Schema Explorer tree that comes up to help visualize the hierarchy of a schema set. Analyzing the external references (which must always be declared at the top before any globals are defined), we see the following two:

<xs:import namespace="http://www.w3.org/XML/1998/namespace" />

<xs:import namespace="urn:bml/brainml.org:internal/BrainMetaL/1" schemaLocation="citation.xsd"/>

This gets interpreted as a request to bring in schemas for the two namespaces “http://www.w3.org/XML/1998/namespace” and “urn:bml/brainml.org:internal/BrainMetaL/1”. In the second case, we are also given a location hint where to find an xsd file for this namespace (more on how the first one is resolved a bit later), so we bring in the citation.xsd file into the set and, analyzing it, we see the following externals:

<xs:import namespace="http://www.w3.org/XML/1998/namespace" />

<xs:import namespace="http://www.w3.org/1999/xlink" schemaLocation="xlink.xsd"/>

<xs:include schemaLocation="brainmetal.xsd"/>

The first import is like the one we had previously seen for the XML namespace and will get resolved the same way. The second again brings in a new namespace into the set and a new file (xlink.xsd) for this namespace. The third includes another file (brainmetal.xsd) that has the same namespace (“urn:bml/brainml.org:internal/BrainMetaL/1”) as the targetNamespace of the current file. This process gets repeated for the rest of the unprocessed files in the set, though no new references are introduced in any of these, so finally we end up with the following tree view in our schema explorer of the namespaces and files in the set.

So how did the xml.xsd file get found and associated to the xml namespace in the set? The answer is that, aside from schema location, Visual Studio also has other places that it looks in for schemas to resolve namespaces when a specific schema location is not provided (remember, the schemaLocation is just a hint to the schema processor, which can apply its knowledge of the environment to figure out how to resolve a namespace). Visual Studio will look in the current project, solution and even other open schemas to resolve schema references, and also comes preconfigured with a set of well known schemas, such as the schema for the xml namespace that is being referenced here.

So how would we know what the well known namespaces and associations that are available in a particular context are? This can be seen through the schema dialog. For example, in the case above, the property window for the brainml.xsd code buffer shows a “Schemas” property and, clicking on it, brings up the following dialog.

This dialog shows a table view of namespaces for which Visual Studio has a known association and the locations of the files that are known to provide schemas for those namespaces. The left hand column, labeled “Use”, allows us to control when or if these known associations are used. The default option is “Automatic”, which means use the schema if needed to resolve an import (such as the current scenario of finding a schema for the well known xml namespace). An option of “Use” says the schema is to be used in the current set. Note that this option is pre-selected for the files computed to be in the set; selecting it for a new file would essentially introduce a new root into the set computation described above. Finally, there is also a do not use option to allow us to exclude a file from a set that would otherwise be included in one of the above scenarios.

There are also two special values (localized strings that are not valid XML identifiers) used where a namespace name would normally appear in the schema explorer to help visualize and distinguish two error conditions that could arise in building a schema set. One is the “Not Found or Invalid” name that is used when the path specified in a schemaLocation is either not found (i.e. an include of a non-existent or non-readable file) or if the file is present but cannot be parsed as an XML schema (i.e. it is either not valid XML or we do not find a schema root element). For example, if we edit brainmetal.xsd in the set above to have the root element read “schemaInvalid” our schema explorer view of the schema changes as in the following diagram.

There is another special name that can appear in place of the namespace, and that is the “Unauthorized Zone” name. The files that appear under this name are files that were attempted to be imported or included into the schema set by a schema file in a different security zone that does not have permissions to access the zone that file is in. This is similar to the Internet Explorer policy whereby a web page cannot redirect or read from a location on the user’s machine or intranet (i.e. a different security zone) unless the machine’s zone policy has been configured to allow this. The schema processor is essentially acting as a proxy for the remote site when requesting included or imported files on their behalf and thus enforces the zone security policies that are in place. This prevents any possible attacks whereby processing schema externals can be used by a malicious external site to either force opening a file or be used in combination with other exploits to potentially post back or gleam information about the user system.

For example, imagine there is a schema available in an external web site named “Remote_Import_Local.xsd” that attempts to import a local file from the path “d:\schemas\Security\Local.xsd”. Even if this path exists in your local machine and there is a valid schema file there, it will not be included in the schema set (and will in fact not even be opened as part of building the set), and you will instead get the following view in the schema explorer.

The inclusion of Local.xsd in the “Unauthorized Zone” and the warnings in the error pane about not being able to resolve the schema location are an indication to the end user that the schema they were visiting attempted to bring in a schema from a zone that it is not authorized to access.

-Fred Garcia

SDE, XML Tools Team

XmlSchemaSet Thread Safety

XmlTeam — Tue, 28 Apr 2009 00:59:00 GMT

Here's a good word of warning: even if an object "feels" read-only because you're not calling code to modify it, if it's not documented as safe for use from multiple threads, then you shouldn't risk it.

As an example, let’s look at XmlSchema and XmlSchemaSet. Initializing these has a cost associated with it, and so it's nice to be able to build them once and then reuse them. But you have to be very careful in doing this. The docs say that all instance methods are not safe for multiple thread usage, but you don't really use them directly during validation, so it's hard to tell from the outside what's safe and what's not.

In a nutshell, the only thing you can do that is safe for concurrent usage is to use a validating reader. Here's the sample code to try this out (for some reason, this "breaks" more on 64-bit machines, but it's unsafe on all architectures).

First, a little helper to create an XmlSchema.

private XmlSchema CreateSchema()
{
string schemaText = @"<?xml version='1.0'?>
<xs:schema id='play' targetNamespace='http://tempuri.org/play.xsd'
elementFormDefault='qualified' xmlns='http://tempuri.org/play.xsd'
xmlns:xs='http://www.w3.org/2001/XMLSchema'>
<xs:element name='myShoeSize'>
<xs:complexType>
   <xs:simpleContent>
    <xs:extension base='xs:decimal'>
     <xs:attribute name='sizing' type='xs:string' />
    </xs:extension>
   </xs:simpleContent>
</xs:complexType>
</xs:element>
</xs:schema>";

using (StringReader reader = new StringReader(schemaText))
{
    return XmlSchema.Read(reader, null);
}
}

Next, a simple XmlSchemaSet.

private XmlSchemaSet CreateSchemaSet(XmlSchema schema)
{
XmlSchemaSet set = new XmlSchemaSet();
set.Add(schema);
set.Compile();
return set;
}

Finally, some validation:

private void ValidateDocument(XmlSchemaSet set)
{
string doc = @"<myShoeSize xmlns='http://tempuri.org/play.xsd' sizing='123' />";
XmlReaderSettings settings = new XmlReaderSettings();
settings.Schemas = set;
settings.ValidationEventHandler += new ValidationEventHandler(settings_ValidationEventHandler);
using (StringReader reader = new StringReader(doc))
using (XmlReader x = XmlReader.Create(reader, settings))
{
while (x.Read()) { }
}
}

private int failCount;
void settings_ValidationEventHandler(object sender, ValidationEventArgs e)
{
System.Threading.Interlocked.Increment(ref failCount);
}

Now, armed with these, I will show you some code that is thread-safe, but that a single line reorder would cause to break.

XmlSchema schema = CreateSchema();
Thread[] threads = new Thread[10];
XmlSchemaSet set = CreateSchemaSet(schema);
for (int i = 0; i < threads.Length; i++)
{
threads[i] = new Thread((x) =>
    {
      for (int j = 0; j < 1000; j++)
      {
        // If the CreateSchemaSet were here
        // instead of outside this would break!
        //
        // Don't add the schema to the
        // XmlSchemaSet from multiple threads!
        //
        // XmlSchemaSet set = CreateSchemaSet(schema);
        //
        ValidateDocument(set);
      }
    });
}

Array.ForEach(threads, (t) => t.Start());
Array.ForEach(threads, (t) => t.Join());
this.Text = "Failure count: " + failCount;

The part before the thread creation runs on a single thread, and so there are no multi-thread concerns; the stuff inside the callback is happening on multiple threads at the same time. You can only use the set for validation here!

Enjoy!

Marcelo Lopez Ruiz

http://blogs.msdn.com/marcelolr/

LINQ to XSD Alpha 0.2

XmlTeam — Thu, 21 Feb 2008 19:37:00 GMT

I’m happy to announce that today we are re-releasing the LINQ to XSD Preview Alpha 0.2 for Visual Studio 2008. The previous preview release of LINQ to XSD targeted Beta 1 of Visual Studio 2008 but did not work on later builds. Many of the people who originally downloaded and tried the LINQ to XSD Preview Alpha requested an update for the final release of Visual Studio 2008 – this release is it. You can download it from here.

The LINQ to XSD preview illustrates our initial thinking on a strongly-typed programming experience over LINQ to XML. Instead of working with untyped XML trees, LINQ to XSD allows you to program in terms of strongly-typed classes, generated based on an XSD schema. The LINQ to XSD project is still incubation, but we are looking at ways to incorporate its functionality into future Visual Studio and .NET Framework releases. We are very much interested in gathering your feedback on the product, so please download it, give it a try, and send us some feedback.

I know it’s taken us a while to get this release out and some of you have been wondering what’s happening with this project. The team has only recently resurfaced from the Visual Studio 2008 release push and has begun looking at how we can take LINQ to XSD forward again. We’re looking forward to getting your feedback on this preview and on our future work on strongly-typed XML programming.

Please send us your comments, questions, and suggestions via the LINQ forum on MSDN: http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=123&SiteID=1

Shyam Pather

Dev Manager, Data Developer Experience Team

Chris Lovett Interview

XmlTeam — Fri, 16 Nov 2007 11:34:00 GMT

Chris Lovett was interviewed by book author Michael van Otegem recently and he asked some very interesting questions:

Please tell us who you are and what you do.
I’m an architect on the Data Programmability Tools team in SQL Server, and I work on XML tools that ship in Visual Studio. As an architect I do lots of different things including: product development on the XML editor in Visual Studio 2008; cross-group collaboration to make sure all our tools work together; playing with other concepts like XML Notepad; and thinking about and communicating our strategy and future directions for our technologies.

How long have you worked for Microsoft and what did you do before your current position?
I have worked on pretty much every XML core technology from Microsoft starting with MSXML in 1997, then to System.Xml in the .NET frameworks v1, and v2. and for the last few years I’ve been focusing on building XML tools in Visual Studio – for example, I was the primary developer on the XML editor in Visual Studio 2005. Before Microsoft I worked at IBM on OS/2 applications, then I joined the IBM Apple/IBM joint venture called Taligent, then I started my own company in Sillicon Valley with friends from Taligent during the height of the .com boom and that’s what led me to Microsoft.

A lot of the .NET Framework works with XML in some form or another. Why is XML such a key component and what were the challenges you faced because of that?
For one thing the .NET frameworks were designed during the peak of the XML hype curve, but more seriously, many folks at Microsoft were waking up to the fact that not everything had to be buried in code. Some things could be very neatly described at a higher “declarative” level and Tim Berners-Lee showed the world that “markup” is a great way to do that. So XML became the way to specify configuration information (.config files), and build information (MSBuild files), and setup information (WIX), object remoting with SOAP, and security permissions and so on. All of these domain specific uses of XML were then supported by our core System.Xml classes. It’s interesting to note that even those teams that didn’t adopt XML back in .NET v1.0 are fixing that. For example, we have the new AJAX work from the ASP.NET team and we have XAML in the Windows Presentation Framework (I’m a huge fan of WPF by the way. I’ve done a lot of UI development in the last 20 years and I have to say WPF totally rocks).   So XML is touching everything from database, management, communications, content publishing and now even into the user interface layer. I was amazed at the last PDC where just about every talk showed some snippet of XML somewhere during their talk.

I remember when we started XML at Microsoft most people thought we were crazy. The biggest challenge was convincing people the cost of parsing and storing verbose XML tags was worth it and our team has been working on performance, and scalability ever since. But the real reason XML become a key component of .NET, (and Windows, and SQL Server and Office) is because it achieved true cross-platform interoperability and because it was good enough for that job.

What do you (personally) like the most about XML and its associated standards?
Simplicity, cross-platform interoperability and huge adoption. The great thing about XML is that it is humble. It’s not trying to solve world hunger. Just invent your own tags, group them into structures that make sense for your domain and voilà. It is very simple and it is this same simplicity that helped HTML take the world by storm.   XML then improved on HTML by providing a clean separation between data and UI. It brought MVC to the masses so to speak.

The other day my 12 year old son was all excited and just had to show me what he discovered. He was editing Age of Empires XML files to tweak the behavior of the game using Notepad.    I asked him how he knew that he could do that and how he knew the XML syntax. He didn’t know what “syntax” meant, but he knew how to edit XML! Then the same week my doctor was all excited when he heard I worked on XML because he was involved in a software purchasing decision at our local hospital and it came down to their level of XML support. I couldn’t believe my ears.

The funny thing is that most programmers don’t really like XML. Probably because it doesn’t use curly brackets, so most programmers treat XML a bit like the ugly duckling. But the reality is that the whole world gets markup – to them markup makes our programming world more approachable. There are still way too many programmers that don’t get this.
What would be your #1 tip to people learning XML, XML Schema, XSLT and XQuery?
First of all I would say that XML 1.0 is the foundation. A must have. Can’t go wrong there, learn it, and learn how XML encoding works – that’s the number one issue people have with XML 1.0 – people don’t take the time to understand how UTF-8 encoding works which is a pretty important foundation to XML. Don’t worry too much about DTD, because we now have XML schema.

XML Schema (XSD) is a layer on top of XML 1.0. When you need a way to describe your XML structure in “standard contract” XSD is handy and most importantly – it’s there and it’s a standard. So don’t re-invent the wheel, but I’m not going to say that XSD is the be all and end all of data modeling, because it isn’t. A lot of things are missing, which is why people had to invent things like Schematron, and why Microsoft is working on EDM and SML and so on. Model driven development is now on the peak of the hype curve so I expect that modeling will be a battle ground for a long time to come. So take a pragmatic approach to XSD - use it if it fits your purpose. Some folks use other modeling approaches then have a tool that spits out the XSD – and that’s fine too.

Same goes for XQuery and XSLT – I think of these as being yet another layer above XSD. We did XSLT and XQuery because we figured that XML is data therefore people will want ways to query and transform that data. Makes sense, and I’ve done a lot of XSLT development, I still use it for specific tasks, but some things are a bit tedious. I find myself escaping to script a lot. XSLT 2.0 is a good improvement, but again, these things are not going to be the be all and end all of query and transformation languages.

For example, I’m a huge fan of the work they are doing in Visual Basic 9 with XML literals connected to our new Language Integrated Query (LINQ). It makes a lot of sense, because instead of having to “escape” to script, you just write the code you need right there in place – you have a complete general purpose programming language at your finger tips.   VB-XML integration is big leap forward for VB programmers and allows those developers, who may not be as familiar with the standard XML technologies like XSLT, to easily process XML data inside their programs. It’s a huge advantage to the VB programmer and we think it helps make VB an extremely compelling language for XML programming – it makes me want to write VB again, and I’ve heard many others say the same thing. However, it is VB-specific so development teams that need cross-platform interoperability at the query/transform layer are likely to stick to the standard technologies like XSLT and XQuery.

There’s a very interesting tension going on here where general purpose languages like VB and C# are moving up into declarative space with LINQ but not going all the way into declarative, versus SQL, XQuery and XSLT which are fully declarative with no side effects, which are therefore more optimizable, but sometimes rather incomplete as programming languages go and rather hard to author in some cases. I really don’t know how it’s going to end up. I think we should continue innovating on both approaches and see what happens. It should be very interesting.

As for all the myriad other XML standards out there, there’s a lot of hype that you have to sort though. To me it’s a funny thing to see programmers going to town making XML more complicated with layer upon layer of new concepts. I remember going to a conference and people were telling me “stop! – we can’t take any more”. There is genius in simplicity. I’m glad to see the renewed focus on simple REST-ful XML based services for this reason. If simple works, why complicate it. Conversely, if it isn’t simple, chances are people just won’t use it.

What can we expect from Microsoft in the future in the XML arena? Will support for XQuery 1.0 and XSLT 2.0 become part of Microsoft’s offering?
Microsoft is a pretty big place, so it’s hard for me to know all that is going on with XML across the company. But I do know about .NET, Visual Studio and SQL server. As I’ve mentioned before we are shipping the XML support in Visual Basic 9.0 with XML literals, XML axis properties and integration with LINQ to XML. LINQ to XML is our API which we are adding to the XML runtime in .NET 3.5, it is a new XML object model that is designed to work well with the Language Integrate Query capabilities of C# and VB.    We are also shipping some cool new features in the XML tools in Visual Studio 2008, including an incremental parser with extensibility API based on LINQ to XML that 3rd party XML designers can build on. We are also adding data breakpoints in the XSLT debugger and we have a new command line tool named “xsltc.exe” which takes XSLT and generates a .NET assembly which you can then deploy with your app instead of the XSLT source so you don’t have to compile XSLT on the server. Anton Lapounov has a great blog that talks about that.   There is not much else new in the System.Xml runtime because Visual Studio 2008 is essentially a service pack release of the .NET 2.0 runtime, so we’ve fixed some bugs there. We are also working on some XML features in Silverlight and we put up a preview of our LINQ to XSD work on MSDN. We are working on a new XSD designer and you will see more CTP's on as this takes shape.

As for XQuery, you probably know we have a subset of XQuery already supported inside SQL Server. We currently have no official plans that we can announce on a client side XQuery engine but we are definitely interested in expanding client side query processing. LINQ offers a path to this (for both relational as well as XML). ESQL provides another client-side investment. We are open to customer feedback on the relative importance of client side XQuery compared to all these other possibilities. Meanwhile we are doing some XQuery improvements in SQL Server 2008, adding LET, better datetime support, and lax validation.

As for XSLT 2.0 - we’ve heard from customers and understand the improvements in XSLT 2.0 over XSLT 1.0, but right now we’re in the middle of a big strategic investment in LINQ and EDM for the future of the data programming platform which we think will create major improvements in programming against all types of data. But we are always re-evaluating our technology investments so if your readers want to ramp up their volume on XSLT 2.0 please ask them to drop us a line with their comments.

Meanwhile I was rather surprised by the positive feedback to my little XML Notepad 2007 tool. It now has over 1 million downloads and is still going strong. Not bad for a few weeks work and no marketing. So something about this tool hit the sweet spot. The interesting thing there is it reaches out to the non-programmer community and I think that is the key and it has the right balance of simplicity and usability. A Swedish customer said it is “logam” – just enough. I think you should expect to see more from Microsoft in the future that helps to make XML something that everyone on the planet can deal with easily and in a way that integrates deeply with everything else Microsoft provides.

As the XML hype is wearing off, folks are realizing that not everything that made it through the standards process needs to be implemented.   So I think you will see Microsoft continue to innovate on new XML technologies and tools like LINQ to XML and VB 9.0 XML and you’ll see Microsoft taking a more pragmatic customer-demand-driven approach to standards.    Microsoft will probably never implement every standard that comes out but I’m confident you will see Microsoft continue to be committed to the really important XML standards, like XML 1.0 and XSD, and any other standard that is essential to achieving cross-platform interoperability, including Open XML. There is enormous power in the cross-platform reach of XML and the huge industry adoption it has and I’m happy to see that Microsoft is continuing to do some really innovative work with XML.

XML Tools Screencasts

XmlTeam — Mon, 04 Jun 2007 20:11:00 GMT

These videos provide some great information and a preview of the new XML Editor, XSLT Debugger, and EDM Wizard coming in Orcas, as well as a sneak preview of a new XSD Designer and EDM Designer that we can expect to see released after the upcoming Orcas release.

XML Schema Facets

XmlTeam — Fri, 01 Jun 2007 00:23:00 GMT

I have received a number of requests over a period of time asking how to get all the facets defined on simple types or complex types with simple content in an XML Schema. I recently wrote a code sample for the same and I thought i will post it for wider consumption. The code sample will handle restrictions on lists, unions as well as facets defined on base types.

public static void GetSchemaTypeFacets(XmlSchemaType schemaType) {

  if (schemaType == null || schemaType.Datatype == null) { //Complex type with complex content
   return;
  }

  XmlSchemaSimpleType st = schemaType as XmlSchemaSimpleType;
  if (st != null) {
   GetSimpleTypeFacets(st);
  }
  else { //Complex type simple content
   XmlSchemaComplexType ct = schemaType as XmlSchemaComplexType;
   Debug.Assert(ct != null);
   XmlSchemaSimpleContent simpleContent = ct.ContentModel as XmlSchemaSimpleContent;
   Debug.Assert(simpleContent != null);
   XmlSchemaSimpleContentRestriction simpleRest = simpleContent.Content as XmlSchemaSimpleContentRestriction;
   if (simpleRest != null) { //Can also be simple content extension
    PrintFacets(simpleRest.Facets);
   }
   GetSchemaTypeFacets(ct.BaseXmlSchemaType);

}
}

public static void GetSimpleTypeFacets(XmlSchemaSimpleType st) {
   if (st == null || st.QualifiedName.Namespace == XmlSchema.Namespace) { //Built-in type, no user-defined facets
     return;
   }
   switch(st.Datatype.Variety) {
    case XmlSchemaDatatypeVariety.List:
      XmlSchemaSimpleType itemType = (st.Content as XmlSchemaSimpleTypeList).BaseItemType;
      GetSimpleTypeFacets(itemType);
    break;

    case XmlSchemaDatatypeVariety.Union:
      XmlSchemaSimpleTypeUnion union = st.Content as XmlSchemaSimpleTypeUnion;
      foreach(XmlSchemaSimpleType stu in union.BaseMemberTypes) {
        GetSimpleTypeFacets(stu);
      }
    break;

    case XmlSchemaDatatypeVariety.Atomic:
      XmlSchemaSimpleTypeRestriction rest = st.Content as XmlSchemaSimpleTypeRestriction;
      PrintFacets(rest.Facets);
    break;

default:
break;
}

if (st.BaseXmlSchemaType != null) { //Chain to the base type
GetSimpleTypeFacets(st.BaseXmlSchemaType as XmlSchemaSimpleType); //BaseType of a simple type is always a simple type
}

}

private static void PrintFacets(XmlSchemaObjectCollection facets) {
  foreach(XmlSchemaFacet facet in facets) {
   Console.WriteLine(facet.GetType().ToString() + " " + facet.Value);
  }
}

Let me know if you have any questions.

-Priya Lakshminarayanan

To Trust, or Not to Trust?

XmlTeam — Tue, 03 Apr 2007 02:58:00 GMT

Validation of an XML document against an Xml Schema guarantees that the structure and content of the xml conforms to the types defined in the schema. Does this mean that we automatically elevate the trust level of a document that has passed schema validation? Can we use schema validation as a security layer to our application?

The general recommendation is that validation of an xml document should not preclude the need for secure coding practices in the application that consumes the validated data. That being the case, we know of applications where length facets are used to ensure that a input parameter is not longer than the specified length, pattern facets are used to verify that the input does not pose the risk of SQL/Command injection etc.

W3C Xml Schema is a complicated specification open to a lot of interpretation and we have not reached a stage yet where all the schema processors are 100% compatible. Consider the case where the regular expression implementation in a particular schema processor is different from that specified in the XSD specification. Suddenly, the pattern facet that is supposed to protect the application from injection attacks is no longer safe.

If you are one among the people who answered yes to the questions at the beginning of the article, read on for ways to tighten the security of a validation episode using the XmlSchemaValidationFlags in the System.Xml.Schema namespace in the .NET Framework 2.0

XmlSchemaValidationFlags Explained

XmlSchemaValidationFlags was introduced in .NET Framework 2.0 in order to mitigate security threats and improve interoperability while performing schema validation using the validating XmlReader or XmlSchemaValidator.

The enumeration has the following values:

Enum Value	Description	XmlReaderSettings Default
None	Identity constraints, Schema Location hints, Inline schemas and validation warnings will all be ignored
ProcessIdentityConstraints	Perform validation for xs:ID, xs:IDREF, xs:key, xs:keyref, xs:unique	Yes
ProcessInlineSchema	Load any inline schemas in the xml instance being validated and add the schema for validation of subsequent xml nodes	No
ProcessSchemaLocation	Load schemas by following the location hints specified in xsi:schemaLocation and xsi:noNamespaceSchemaLocation attributes and use the schemas for validation of subsequent xml nodes	No
ReportValidationWarnings	Report any warnings encountered during the validation of the xml instance	No
AllowXmlAttributes	Allow xml:* attributes even if they are not defined in the schema. The attributes will be validated based on their data type	Yes

Security Implication of XmlSchemaValidationFlags

· DO TURN ON the ReportValidationWarnings flag

By default this flag is not turned on while creating a validating XmlReader using the XmlReaderSettings. (This was done so that users can perform partial validation of an xml instance without having to deal with a large number of warnings for the portions that don’t have a schema. For eg: Validating a WordML document with user content against the user’s schema. Another reason was to improve performance as every warning entails creation of an exception object)

Consider the following example of an order schema and an instance order.xml

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://tempuri/Orders.org" xmlns="http://tempuri/Orders.org" elementFormDefault="qualified">

<xsd:element name="order">

<xsd:complexType >

<xsd:sequence>

<xsd:element name="orderid" type="xsd:int"/>

<xsd:element name="item" type="xsd:string" maxOccurs="unbounded"/>

<xsd:element name="address" type="xsd:string"/>

</xsd:sequence>

</xsd:complexType>

</xsd:element>

</xsd:schema>

<item>umbrella</item>

<address>1234 wallaby way</address>

</order>

The <orderid> element is invalid according to the schema (A100 is not a valid xsd:int) but validation using a validating XmlReader with the default settings returns without any errors. This happens because the namespace in the xml and the namespace in the schema do not match (http://tempuri/Order.org Vs http://tempuri/Orders.org) and strict schema validation occurs only after finding the schema definition for an element whose name AND namespace match a definition in the schema (Schema-Validity Assessment (Element))

In this case, it is only a warning that schema information could not be found due to the namespace mismatch and since warnings are turned OFF by default, the user sees no evidence that the validation did not happen. If the flag is turned ON, the user should see the following warning:

Could not find schema information for the element 'http://tempuri/Order.org:order'.

An Error Occurred at: file:///E:/bugrepro/order.xml, (1,2)

Note: Warnings will be reported when this flag is turned ON AND a validation event handler is hooked up (to XmlReaderSettings, XmlDocument or XmlSchemaValidator)

· DO NOT TURN ON the ProcessSchemaLocation flag

If this flag is turned ON, Schema Location hints in the xml document (xsi:schemaLocation and xsi:noNamespaceSchemaLocation attributes) are followed by the validation engine using the default XmlUrlResolver (unless the XmlResolver property is specifically set to NULL or a secure resolver in which case it takes precedence)

The default resolver does not protect against cross-zone re-direction and adding schemas at validation time by the instance document might change validation outcome by adding new types, redefining existing types etc.

· DO NOT TURN ON the ProcessInlineSchema flag

In addition to allowing new types and redefining existing types by way of the schema allowed inline in the xml document, this will also pose a threat to users who are dependent on strict validation to map their XML into objects since any element can now contain a whole schema as its child node and might cause unexpected errors in the X-O mapping.

· MAY TURN ON the AllowXmlAttributes flag

If allowing attributes from the xml namespace (on any or all elements in the instance even though not specifically allowed by the schema) will not pose a risk to your application.

· MAY TURN ON the ProcessIdentityConstraints flag

If processing of xsd:key. xsd:keyref, xs:ID, xs:IDREF is important to your application and you have determined that the scope of the key/keyref is not such that it might cause a Denial of Service attack.

All of the above are merely guidelines for a secure validation episode using the System.Xml.Schema namespace. The flags you choose are greatly dependent on your application needs but keep in mind the security implications of validated data the next time you are tempted to completely trust such data.

Hope you enjoyed reading this and I greatly appreciate your feedback.

-Priya Lakshminarayanan