Microsoft XML Team's WebLog : Schema Validation

XmlSchemaSet Thread Safety

XmlTeam — Tue, 28 Apr 2009 00:59:00 GMT

Here's a good word of warning: even if an object "feels" read-only because you're not calling code to modify it, if it's not documented as safe for use from multiple threads, then you shouldn't risk it.

As an example, let’s look at XmlSchema and XmlSchemaSet. Initializing these has a cost associated with it, and so it's nice to be able to build them once and then reuse them. But you have to be very careful in doing this. The docs say that all instance methods are not safe for multiple thread usage, but you don't really use them directly during validation, so it's hard to tell from the outside what's safe and what's not.

In a nutshell, the only thing you can do that is safe for concurrent usage is to use a validating reader. Here's the sample code to try this out (for some reason, this "breaks" more on 64-bit machines, but it's unsafe on all architectures).

First, a little helper to create an XmlSchema.

private XmlSchema CreateSchema()
{
string schemaText = @"<?xml version='1.0'?>
<xs:schema id='play' targetNamespace='http://tempuri.org/play.xsd'
elementFormDefault='qualified' xmlns='http://tempuri.org/play.xsd'
xmlns:xs='http://www.w3.org/2001/XMLSchema'>
<xs:element name='myShoeSize'>
<xs:complexType>
   <xs:simpleContent>
    <xs:extension base='xs:decimal'>
     <xs:attribute name='sizing' type='xs:string' />
    </xs:extension>
   </xs:simpleContent>
</xs:complexType>
</xs:element>
</xs:schema>";

using (StringReader reader = new StringReader(schemaText))
{
    return XmlSchema.Read(reader, null);
}
}

Next, a simple XmlSchemaSet.

private XmlSchemaSet CreateSchemaSet(XmlSchema schema)
{
XmlSchemaSet set = new XmlSchemaSet();
set.Add(schema);
set.Compile();
return set;
}

Finally, some validation:

private void ValidateDocument(XmlSchemaSet set)
{
string doc = @"<myShoeSize xmlns='http://tempuri.org/play.xsd' sizing='123' />";
XmlReaderSettings settings = new XmlReaderSettings();
settings.Schemas = set;
settings.ValidationEventHandler += new ValidationEventHandler(settings_ValidationEventHandler);
using (StringReader reader = new StringReader(doc))
using (XmlReader x = XmlReader.Create(reader, settings))
{
while (x.Read()) { }
}
}

private int failCount;
void settings_ValidationEventHandler(object sender, ValidationEventArgs e)
{
System.Threading.Interlocked.Increment(ref failCount);
}

Now, armed with these, I will show you some code that is thread-safe, but that a single line reorder would cause to break.

XmlSchema schema = CreateSchema();
Thread[] threads = new Thread[10];
XmlSchemaSet set = CreateSchemaSet(schema);
for (int i = 0; i < threads.Length; i++)
{
threads[i] = new Thread((x) =>
    {
      for (int j = 0; j < 1000; j++)
      {
        // If the CreateSchemaSet were here
        // instead of outside this would break!
        //
        // Don't add the schema to the
        // XmlSchemaSet from multiple threads!
        //
        // XmlSchemaSet set = CreateSchemaSet(schema);
        //
        ValidateDocument(set);
      }
    });
}

Array.ForEach(threads, (t) => t.Start());
Array.ForEach(threads, (t) => t.Join());
this.Text = "Failure count: " + failCount;

The part before the thread creation runs on a single thread, and so there are no multi-thread concerns; the stuff inside the callback is happening on multiple threads at the same time. You can only use the set for validation here!

Enjoy!

Marcelo Lopez Ruiz

http://blogs.msdn.com/marcelolr/

XML Schema Facets

XmlTeam — Fri, 01 Jun 2007 00:23:00 GMT

I have received a number of requests over a period of time asking how to get all the facets defined on simple types or complex types with simple content in an XML Schema. I recently wrote a code sample for the same and I thought i will post it for wider consumption. The code sample will handle restrictions on lists, unions as well as facets defined on base types.

public static void GetSchemaTypeFacets(XmlSchemaType schemaType) {

  if (schemaType == null || schemaType.Datatype == null) { //Complex type with complex content
   return;
  }

  XmlSchemaSimpleType st = schemaType as XmlSchemaSimpleType;
  if (st != null) {
   GetSimpleTypeFacets(st);
  }
  else { //Complex type simple content
   XmlSchemaComplexType ct = schemaType as XmlSchemaComplexType;
   Debug.Assert(ct != null);
   XmlSchemaSimpleContent simpleContent = ct.ContentModel as XmlSchemaSimpleContent;
   Debug.Assert(simpleContent != null);
   XmlSchemaSimpleContentRestriction simpleRest = simpleContent.Content as XmlSchemaSimpleContentRestriction;
   if (simpleRest != null) { //Can also be simple content extension
    PrintFacets(simpleRest.Facets);
   }
   GetSchemaTypeFacets(ct.BaseXmlSchemaType);

}
}

public static void GetSimpleTypeFacets(XmlSchemaSimpleType st) {
   if (st == null || st.QualifiedName.Namespace == XmlSchema.Namespace) { //Built-in type, no user-defined facets
     return;
   }
   switch(st.Datatype.Variety) {
    case XmlSchemaDatatypeVariety.List:
      XmlSchemaSimpleType itemType = (st.Content as XmlSchemaSimpleTypeList).BaseItemType;
      GetSimpleTypeFacets(itemType);
    break;

    case XmlSchemaDatatypeVariety.Union:
      XmlSchemaSimpleTypeUnion union = st.Content as XmlSchemaSimpleTypeUnion;
      foreach(XmlSchemaSimpleType stu in union.BaseMemberTypes) {
        GetSimpleTypeFacets(stu);
      }
    break;

    case XmlSchemaDatatypeVariety.Atomic:
      XmlSchemaSimpleTypeRestriction rest = st.Content as XmlSchemaSimpleTypeRestriction;
      PrintFacets(rest.Facets);
    break;

default:
break;
}

if (st.BaseXmlSchemaType != null) { //Chain to the base type
GetSimpleTypeFacets(st.BaseXmlSchemaType as XmlSchemaSimpleType); //BaseType of a simple type is always a simple type
}

}

private static void PrintFacets(XmlSchemaObjectCollection facets) {
  foreach(XmlSchemaFacet facet in facets) {
   Console.WriteLine(facet.GetType().ToString() + " " + facet.Value);
  }
}

Let me know if you have any questions.

-Priya Lakshminarayanan

To Trust, or Not to Trust?

XmlTeam — Tue, 03 Apr 2007 02:58:00 GMT

Validation of an XML document against an Xml Schema guarantees that the structure and content of the xml conforms to the types defined in the schema. Does this mean that we automatically elevate the trust level of a document that has passed schema validation? Can we use schema validation as a security layer to our application?

The general recommendation is that validation of an xml document should not preclude the need for secure coding practices in the application that consumes the validated data. That being the case, we know of applications where length facets are used to ensure that a input parameter is not longer than the specified length, pattern facets are used to verify that the input does not pose the risk of SQL/Command injection etc.

W3C Xml Schema is a complicated specification open to a lot of interpretation and we have not reached a stage yet where all the schema processors are 100% compatible. Consider the case where the regular expression implementation in a particular schema processor is different from that specified in the XSD specification. Suddenly, the pattern facet that is supposed to protect the application from injection attacks is no longer safe.

If you are one among the people who answered yes to the questions at the beginning of the article, read on for ways to tighten the security of a validation episode using the XmlSchemaValidationFlags in the System.Xml.Schema namespace in the .NET Framework 2.0

XmlSchemaValidationFlags Explained

XmlSchemaValidationFlags was introduced in .NET Framework 2.0 in order to mitigate security threats and improve interoperability while performing schema validation using the validating XmlReader or XmlSchemaValidator.

The enumeration has the following values:

Enum Value	Description	XmlReaderSettings Default
None	Identity constraints, Schema Location hints, Inline schemas and validation warnings will all be ignored
ProcessIdentityConstraints	Perform validation for xs:ID, xs:IDREF, xs:key, xs:keyref, xs:unique	Yes
ProcessInlineSchema	Load any inline schemas in the xml instance being validated and add the schema for validation of subsequent xml nodes	No
ProcessSchemaLocation	Load schemas by following the location hints specified in xsi:schemaLocation and xsi:noNamespaceSchemaLocation attributes and use the schemas for validation of subsequent xml nodes	No
ReportValidationWarnings	Report any warnings encountered during the validation of the xml instance	No
AllowXmlAttributes	Allow xml:* attributes even if they are not defined in the schema. The attributes will be validated based on their data type	Yes

Security Implication of XmlSchemaValidationFlags

· DO TURN ON the ReportValidationWarnings flag

By default this flag is not turned on while creating a validating XmlReader using the XmlReaderSettings. (This was done so that users can perform partial validation of an xml instance without having to deal with a large number of warnings for the portions that don’t have a schema. For eg: Validating a WordML document with user content against the user’s schema. Another reason was to improve performance as every warning entails creation of an exception object)

Consider the following example of an order schema and an instance order.xml

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://tempuri/Orders.org" xmlns="http://tempuri/Orders.org" elementFormDefault="qualified">

<xsd:element name="order">

<xsd:complexType >

<xsd:sequence>

<xsd:element name="orderid" type="xsd:int"/>

<xsd:element name="item" type="xsd:string" maxOccurs="unbounded"/>

<xsd:element name="address" type="xsd:string"/>

</xsd:sequence>

</xsd:complexType>

</xsd:element>

</xsd:schema>

<item>umbrella</item>

<address>1234 wallaby way</address>

</order>

The <orderid> element is invalid according to the schema (A100 is not a valid xsd:int) but validation using a validating XmlReader with the default settings returns without any errors. This happens because the namespace in the xml and the namespace in the schema do not match (http://tempuri/Order.org Vs http://tempuri/Orders.org) and strict schema validation occurs only after finding the schema definition for an element whose name AND namespace match a definition in the schema (Schema-Validity Assessment (Element))

In this case, it is only a warning that schema information could not be found due to the namespace mismatch and since warnings are turned OFF by default, the user sees no evidence that the validation did not happen. If the flag is turned ON, the user should see the following warning:

Could not find schema information for the element 'http://tempuri/Order.org:order'.

An Error Occurred at: file:///E:/bugrepro/order.xml, (1,2)

Note: Warnings will be reported when this flag is turned ON AND a validation event handler is hooked up (to XmlReaderSettings, XmlDocument or XmlSchemaValidator)

· DO NOT TURN ON the ProcessSchemaLocation flag

If this flag is turned ON, Schema Location hints in the xml document (xsi:schemaLocation and xsi:noNamespaceSchemaLocation attributes) are followed by the validation engine using the default XmlUrlResolver (unless the XmlResolver property is specifically set to NULL or a secure resolver in which case it takes precedence)

The default resolver does not protect against cross-zone re-direction and adding schemas at validation time by the instance document might change validation outcome by adding new types, redefining existing types etc.

· DO NOT TURN ON the ProcessInlineSchema flag

In addition to allowing new types and redefining existing types by way of the schema allowed inline in the xml document, this will also pose a threat to users who are dependent on strict validation to map their XML into objects since any element can now contain a whole schema as its child node and might cause unexpected errors in the X-O mapping.

· MAY TURN ON the AllowXmlAttributes flag

If allowing attributes from the xml namespace (on any or all elements in the instance even though not specifically allowed by the schema) will not pose a risk to your application.

· MAY TURN ON the ProcessIdentityConstraints flag

If processing of xsd:key. xsd:keyref, xs:ID, xs:IDREF is important to your application and you have determined that the scope of the key/keyref is not such that it might cause a Denial of Service attack.

All of the above are merely guidelines for a secure validation episode using the System.Xml.Schema namespace. The flags you choose are greatly dependent on your application needs but keep in mind the security implications of validated data the next time you are tempted to completely trust such data.

Hope you enjoyed reading this and I greatly appreciate your feedback.

-Priya Lakshminarayanan