Typed XML in SQL Server 2005

Moving on

2007-10-26T07:52:00Z

I'm over 2 months late in writing this post. please forgive me for the delay, especially those of you who were nice enough to leave comments and questions and expected a response.

This is my last posting on my "Typed XML" blog. After almost 9 years at Microsoft I felt the need to do something else and on August 13th I joined the ranks of a well known online retailer. You'll still find some of my contributions in SQL Server 2008 when it is released. The entire team did a great job addressing some of the issues most often mentioned by customers. I don't want to steal anybody's thunder (or attract the attention of any lawyers :-) ) so I won't say much more but keep an eye out for that next version. I's going to rock!

Those 8 and 3/4 years at Microsoft were quite a ride. Thanks to everybody who helped make that time special.

Silent XQuery failures

2007-04-20T03:10:00Z

A newsgroup post I read earlier today reminded me that there are cases where failure to get the expected result when using XQuery can sometimes be difficult to diagnose.

Here’s what the poster was doing

declare @xml xml

set @xml = '<root><test></test></root>'

set @xml.modify('replace value of (/root/test/text())[1] with "test

new value"')

select @xml

This user expected to get something like <root><test>test new value</test></root> but instead, the resulting instance looked like <root><test /></root>.

The server never gave any error or warning, yet the XML instance appeared to be untouched.

The problem, as you might have guessed already, is that XPath expression (/root/test/text())[1] returns the empty sequence. In the original instance, ‘test’ is an empty element. It has no children at all. Therefore the expression (/root/test/text())[1] doesn’t point to any existing node. The solution to this problem is to insert a new test node inside the ‘test’ element, like this

set @xml.modify('insert text{"test new value"} as first into (/root/test)[1] ')

In general, when you use the modify() method and the target XPath expression returns the empty sequence, nothing happens. The only exception to this rule is when the server can statically determine that the result will be empty. That’s when XML schemas come in handy.

For example, let’s look at the following example

DECLARE @x XML

SET @x = '<root><x/><y/><z/></root>'

SET @x.modify('delete /root[1]/a[1]')

Let’s suppose I made a typo and instead of typing ‘z’ in my XPath expression I typed ‘a’. The query doesn’t do anything because with this particular XML instance the XPath expression /root[1]/a[1] returns the empty sequence (it doesn’t point to any existing node). There is no way to know that before executing the query. Statically, the type of this XPath expression is element(a, xdt:untyped)?

Now, let’s try something similar when a schema collection is associated with the XML variable.

CREATE XML SCHEMA COLLECTION SC AS '

</sequence>

</complexType>

</element>

</schema>'

DECLARE @x XML(SC)

SET @x = '<root><x/><y/><z/></root>'

SET @x.modify(' delete /root[1]/a[1]')

This time the query fails with the following error message:

XQuery [modify()]: There is no element named 'a' in the type 'element(root,#anonymous) ?'.

The server was able to use the schema to determine that my XPath expression will never point to an existing node and sent me a helpful error message. Even though typing your data doesn’t mean you’ll catch all those mistakes, it still helps when debugging XML queries.

-
Disclaimer:
This posting is provided “AS IS” with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.

Stardust memories

2007-03-14T04:04:00Z

The famous Stardust hotel and casino was torn down today. I was fond of the place since this was the hotel I stayed at when I first visited Vegas in the mid nineties. During our most recent trip to Nevada last summer my wife and I made sure we stopped by to bid farewell to the place. Its history was deeply linked to that of the town. Illustrious artists performed there. If you’ve seen the movie Casino, the story was in large part inspired by events involving the ownership of the Stardust. Yet, despite its past, the Stardust still had to go to make room for a new 4+ billion resort. By comparison, when the famous Parisian concert hall L’Olympia was threatened by developers, there was a public outcry to save it. Oh well … nothing is forever and I’ll always have those wonderful memories.

Mixed content (part 3)

2007-03-14T02:22:00Z

It’s been a few weeks since my last post about mixed content so let’s pick up right where we left off.

This time we will look at DML operations on mixed content. We’ll keep using the XML schema as well as the table I created in the first post of this series.

Let’s insert an instance to work with.

INSERT INTO T VALUES (3, CONVERT(XML(SC_Letter), '<l:letter xmlns:l="urn:letter" xmlns:t="urn:letter:tags">

<l:date>2006-10-15-08:00</l:date>

<l:sender>

<l:FirstName>James</l:FirstName>

<l:LastName>Kirk</l:LastName>

<l:Line1>1234 Elm Street</l:Line1>

<l:City>Redmond</l:City>

<l:ZIPCode>98052</l:ZIPCode>

<l:State>WA</l:State>

</l:sender>

<l:recipient>

<l:FirstName>Jean-Luc</l:FirstName>

<l:LastName>Picard</l:LastName>

<l:Line1>13 rue des Lilas</l:Line1>

<l:City>Ronchin</l:City>

<l:ZIPCode>59555</l:ZIPCode>

<l:Country>France</l:Country>

</l:recipient>

Dear Sir,

I received your letter dated <t:date>2006-10-10+00:00</t:date> this morning. It would be my pleasure to attend the annual symposium <t:title>XML database systems in class-B Federation Spacecrafts</t:title>.

Please contact my assistant <t:person>Mr. Spock</t:person> at <t:email>spock@space-federation.org</t:email> to make all necessary arrangements.

Sincerely,

James T. Kirk

</l:letter>', 1))

DML operations on elements

There are no behavior differences between element-only content and mixed content. You can insert, delete, and replace the value of elements provided the resulting instance validates against the schema.

For example, let’s update the name of the assistant mentioned in the letter.

UPDATE T SET xmlCol.modify(' declare namespace l = "urn:letter";

declare namespace t = "urn:letter:tags";

replace value of (/l:letter//t:person[data(.) = "Mr. Spock"])[1] with "Capt. Spock"')

WHERE iCol = 3

You can query the instance back and see for yourself that the value of the element has been updated.

We could also decide not to mention the assistant by name and delete the element.

UPDATE T SET xmlCol.modify(' declare namespace l = "urn:letter";

declare namespace t = "urn:letter:tags";

delete (/l:letter//t:person[data(.) = "Capt. Spock"])[1]')

WHERE iCol = 3

If you query the instance back you can see that the letter now says

Please contact my assistant at <t:email>spock@space-federation.org</t:email> to make all necessary arrangements.

Something very interesting happened here that is not immediately visible. If you remember, in our previous posts, we used the following query to examine the nodes in our instance

SELECT x.query('.'), x.query('if (. instance of text()) then "text" else

if (. instance of element()) then "element" else ""')

FROM T

CROSS APPLY xmlCol.nodes('/*:letter[1]/node()') AS Tref(x)

WHERE iCol = 3

Had we run this query prior to deleting the <person> element, we would’ve seen this as part of our results (partial results printed to save space)

. Please contact my assistant	text
<p1:person xmlns:p1="urn:letter:tags">Capt. Spock</p1:person>	element
at	text
<p1:email xmlns:p1="urn:letter:tags">spock@space-federation.org</p1:email>	element
to make all necessary arrangements. Sincerely, James T. Kirk	text

Now that we’ve deleted the element, the results look like this (partial results again)

. Please contact my assistant at	text
<p1:email xmlns:p1="urn:letter:tags">spock@space-federation.org</p1:email>	element
to make all necessary arrangements. Sincerely, James T. Kirk	text

Since we delete an element that was between two text nodes, those nodes merged to form only one.

DML operations on text nodes

Text nodes can be updated, inserted or deleted.

First, let’s replace the value of a text node.

UPDATE T SET xmlCol.modify(' declare namespace l = "urn:letter";

declare namespace t = "urn:letter:tags";

replace value of /l:letter[1]/text()[6] with ". I suggest that you talk to my asisstant, who can be contacted at "')

WHERE iCol = 3

You can query the instance to see that the text of the letter was modified appropriately.

Now let’s insert a text node after an existing one

UPDATE T SET xmlCol.modify(' declare namespace l = "urn:letter";

declare namespace t = "urn:letter:tags";

insert text{"		PS: Live long and prosper."} as last into /l:letter[1]')

WHERE iCol = 3

If you look at the nodes, you’ll see that the new text node we inserted merged with the one preceding it to form one text node so that all text between two elements tags counts as only one text node (partial results printed below)

. I suggest that you talk to my asisstant, who can be contacted at

text

<p1:email xmlns:p1="urn:letter:tags">spock@space-federation.org</p1:email>

element

to make all necessary arrangements.

Sincerely,

James T. Kirk

PS: Live long and prosper.

text

A new SQL CLR blog

2007-03-09T01:24:00Z

One of my colleagues, Brad Richards just started blogging about SQL CLR performance. If you use SQL CLR I strongly suggest that you take a look at this blog. Brad is very knowledgeable about performance and you're likely to learn some very valuable information.

Mixed content (part 2)

2007-01-17T02:32:00Z

Following last week's introduction to the concept of mixed content, let's look in detail at a validated XML instance with mixed content.

Taking a look at the children nodes of the <letter> element

Let’s reuse the instance we stored in our table last time. We can query it and look at all the nodes under the “letter” root element. With the nodes() method we can output a table. This table will contain a row for each node under <letter>. The first column contains the node itself, the second column contains the value “text” if it is a text node or “element” if it is an element node.

SELECT x.query('.'), x.query('if (. instance of text()) then "text" else

if (. instance of element()) then "element" else ""')

FROM T

CROSS APPLY xmlCol.nodes('/*:letter[1]/node()') AS Tref(x)

WHERE iCol = 1

The results look something like this (I formatted the cells containing element nodes so that the whole XML fragment is visible. If you run the query yourself you’ll notice no such formatting.)

<p1:date xmlns:p1="urn:letter">2006-10-15-08:00</p1:date>	element
<p1:sender xmlns:p1="urn:letter"><p1:FirstName>James</p1:FirstName><p1:LastName>Kirk</p1:LastName><p1:Line1>1234 Elm Street</p1:Line1><p1:City>Redmond</p1:City><p1:ZIPCode>98052</p1:ZIPCode><p1:State>WA</p1:State></p1:sender>	element
<p1:recipient xmlns:p1="urn:letter"><p1:FirstName>Jean-Luc</p1:FirstName><p1:LastName>Picard</p1:LastName><p1:Line1>13 rue des Lilas</p1:Line1><p1:City>Ronchin</p1:City><p1:ZIPCode>59555</p1:ZIPCode><p1:Country>France</p1:Country></p1:recipient>	element
Dear Sir, I received your letter dated	text
<p1:date xmlns:p1="urn:letter:tags">2006-10-10Z</p1:date>	element
this morning. It would be my pleasure to attend the annual symposium	text
<p1:title xmlns:p1="urn:letter:tags">XML database systems in class-B Federation Spacecrafts</p1:title>	element
. Please contact my assistant	text
<p1:person xmlns:p1="urn:letter:tags">Mr. Spock</p1:person>	element
at	text
<p1:email xmlns:p1="urn:letter:tags">spock@space-federation.org</p1:email>	element
to make all necessary arrangements. Sincerely, James T. Kirk	text

Let’s now reprint the entire instance, as we submitted it to the server, with the text nodes colored in green.

<l:letter xmlns:l="urn:letter" xmlns:t="urn:letter:tags">

<l:date>2006-10-15-08:00</l:date>

<l:sender>

<l:FirstName>James</l:FirstName>

<l:LastName>Kirk</l:LastName>

<l:Line1>1234 Elm Street</l:Line1>

<l:City>Redmond</l:City>

<l:ZIPCode>98052</l:ZIPCode>

<l:State>WA</l:State>

</l:sender>

<l:recipient>

<l:FirstName>Jean-Luc</l:FirstName>

<l:LastName>Picard</l:LastName>

<l:Line1>13 rue des Lilas</l:Line1>

<l:City>Ronchin</l:City>

<l:ZIPCode>59555</l:ZIPCode>

<l:Country>France</l:Country>

</l:recipient>

Dear Sir,

Please contact my assistant <t:person>Mr. Spock</t:person> at <t:email>spock@space-federation.org</t:email> to make all necessary arrangements.

Sincerely,

James T. Kirk

</l:letter>

You’ll notice that all text between two elements tags counts as one text node. Our instance contains five of them. You can verify the accuracy of that count with the following query, which returns 5 also

SELECT xmlCol.value('count(/*:letter/text())','int') FROM T WHERE iCol = 1

Whitespace-only text nodes

You might also have noticed that our instance contained portions of text made only of whitespace characters (between the “date” and the “sender” element for example). Those weren’t preserved as text nodes. Whitespace-only text nodes are actually a very interesting topic that isn’t limited to mixed content and we’ll probably talk about it some more in the future but for now, let’s look at what one should do when they wish to preserve them. There is an optional parameter in the T-SQL convert() function, that when set to 1, forces the server to preserve whitespace-only text nodes when converting from a string type to XML. In our case, we should do this.

INSERT INTO T VALUES (2, CONVERT(XML(SC_Letter), '<l:letter xmlns:l="urn:letter" xmlns:t="urn:letter:tags">

<l:date>2006-10-15-08:00</l:date>

<l:sender>

<l:FirstName>James</l:FirstName>

<l:LastName>Kirk</l:LastName>

<l:Line1>1234 Elm Street</l:Line1>

<l:City>Redmond</l:City>

<l:ZIPCode>98052</l:ZIPCode>

<l:State>WA</l:State>

</l:sender>

<l:recipient>

<l:FirstName>Jean-Luc</l:FirstName>

<l:LastName>Picard</l:LastName>

<l:Line1>13 rue des Lilas</l:Line1>

<l:City>Ronchin</l:City>

<l:ZIPCode>59555</l:ZIPCode>

<l:Country>France</l:Country>

</l:recipient>

Dear Sir,

Please contact my assistant <t:person>Mr. Spock</t:person> at <t:email>spock@space-federation.org</t:email> to make all necessary arrangements.

Sincerely,

James T. Kirk

</l:letter>', 1))

Let’s rerun the same query as last time for this new instance

SELECT x.query('.'), x.query('if (. instance of text()) then "text" else

if (. instance of element()) then "element" else ""')

FROM T

CROSS APPLY xmlCol.nodes('/*:letter[1]/node()') AS Tref(x)

WHERE iCol = 2

This time the results look like this (once again I altered the way results are displayed for your reading comfort)

	text
<p1:date xmlns:p1="urn:letter">2006-10-15-08:00</p1:date>	element
	text
<p1:sender xmlns:p1="urn:letter"><p1:FirstName>James</p1:FirstName> <p1:LastName>Kirk</p1:LastName><p1:Line1>1234 Elm Street</p1:Line1><p1:City>Redmond</p1:City> <p1:ZIPCode>98052</p1:ZIPCode><p1:State>WA</p1:State></p1:sender>	element
	text
<p1:recipient xmlns:p1="urn:letter"><p1:FirstName>Jean-Luc</p1:FirstName><p1:LastName>Picard</p1:LastName><p1:Line1>13 rue des Lilas</p1:Line1><p1:City>Ronchin</p1:City> <p1:ZIPCode>59555</p1:ZIPCode><p1:Country>France</p1:Country></p1:recipient>	element
Dear Sir, I received your letter dated	text
<p1:date xmlns:p1="urn:letter:tags">2006-10-10Z</p1:date>	element
this morning. It would be my pleasure to attend the annual symposium	text
<p1:title xmlns:p1="urn:letter:tags">XML database systems in class-B Federation Spacecrafts</p1:title>	element
. Please contact my assistant	text
<p1:person xmlns:p1="urn:letter:tags">Mr. Spock</p1:person>	element
at	text
<p1:email xmlns:p1="urn:letter:tags">spock@space-federation.org</p1:email>	element
to make all necessary arrangements. Sincerely, James T. Kirk	text

Now there are 8 text nodes under the “letter” element as evidenced by the following query:

SELECT xmlCol.value('count(/*:letter/text())','int') FROM T WHERE iCol = 2

This time following whitespace-only text nodes have been preserved

All characters between the opening “letter” tag and the opening “date” tag
All characters between the closing “date” tag and the opening “sender” tag”
All characters between the closing “sender” tag and the opening “recipient” tag.

Next time we'll look at DML operations and the effect they can have on mixed content.

Mixed content (part 1 of many)

2007-01-12T03:57:00Z

I started working on this two days ago. I quickly realized that when I’m done I’ll most likely have material for multiple posts. My original plan was to finish the article in the next few days, and then publish it as several posts over time. However I eventually decided to start posting now, while the work is still in progress, and let any feedback or questions you might have guide me.

Description

Mixed content is one of the great but often overlooked features of xml schema. A mixed complex type makes it legal for text nodes to appear between the elements that are normally expected by the validator.

In order for mixed content to be allowed the <xsd:complexType> node of the type's declaration must contain an attribute named "mixed" with a value of "true".

For a more complete explanation you can start with the Section of the W3C's xsd primer that deals with mixed content at http://www.w3.org/TR/xmlschema-0/#mixedContent.

In this post we will examine SQL Server 2005's capabilities and behavior when dealing with mixed content.

Example

As an example, we’ll deal with letters. Letters are ideal candidates for mixed content because they can contain both structured and unstructured information. The sender's and recipient(s) addresses for example constitute structured data for which we can design content models. The body of a letter itself is free-flow text but within it we can identify certain valuable pieces of information such as dates or email addresses using XML elements.

Here is a sample schema for letters.

CREATE XML SCHEMA COLLECTION SC_Letter AS '

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"

targetNamespace="urn:letter"

elementFormDefault="qualified"

xmlns="urn:letter">

<xsd:element name="letter">

<xsd:complexType mixed="true">

<xsd:sequence>

<xsd:element name="date" type="xsd:date"/>

<xsd:element name="sender" type="addressType"/>

<xsd:sequence minOccurs="1" maxOccurs="unbounded">

<xsd:element name="recipient" type="addressType"/>

</xsd:sequence>

<xsd:any processContents="strict" namespace="urn:letter:tags" minOccurs="0" maxOccurs="unbounded"/>

</xsd:sequence>

</xsd:complexType>

</xsd:element>

<xsd:complexType name="addressType">

<xsd:sequence>

<xsd:element name="Title" type="xsd:string" minOccurs="0"/>

<xsd:element name="FirstName" type="xsd:string" minOccurs="0"/>

<xsd:element name="LastName" type="xsd:string"/>

<xsd:element name="Line1" type="xsd:string"/>

<xsd:element name="Line2" type="xsd:string" minOccurs="0"/>

<xsd:element name="City" type="xsd:string"/>

<xsd:element name="ZIPCode" type="xsd:string" minOccurs="0"/>

<xsd:choice>

<xsd:element name="State" type="xsd:string"/>

<xsd:element name="Country" type="xsd:string"/>

</xsd:choice>

</xsd:sequence>

</xsd:complexType>

</xsd:schema>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"

targetNamespace="urn:letter:tags"

elementFormDefault="qualified" xmlns="urn:letter:tags">

<xsd:element name="person" type="xsd:string"/>

<xsd:element name="date" type="xsd:date"/>

<xsd:element name="title" type="xsd:string"/>

<xsd:element name="URI" type="xsd:anyURI"/>

<xsd:element name="email" type="emailType"/>

<xsd:simpleType name="emailType">

<xsd:restriction base="xsd:string">

<xsd:pattern value="[\w\-\.]+@([\w\-]+\.)+[\w\-]{2,4}"/>

</xsd:restriction>

</xsd:simpleType>

</xsd:schema>'

Now let’s create a table with a typed XML column

CREATE TABLE T (iCol int primary key, xmlCol XML(SC_Letter))

And finally, let's insert a sample XML instance

INSERT INTO T VALUES (1, '

<l:letter xmlns:l="urn:letter" xmlns:t="urn:letter:tags">

<l:date>2006-10-15-08:00</l:date>

<l:sender>

<l:FirstName>James</l:FirstName>

<l:LastName>Kirk</l:LastName>

<l:Line1>1234 Elm Street</l:Line1>

<l:City>Redmond</l:City>

<l:ZIPCode>98052</l:ZIPCode>

<l:State>WA</l:State>

</l:sender>

<l:recipient>

<l:FirstName>Jean-Luc</l:FirstName>

<l:LastName>Picard</l:LastName>

<l:Line1>13 rue des Lilas</l:Line1>

<l:City>Ronchin</l:City>

<l:ZIPCode>59555</l:ZIPCode>

<l:Country>France</l:Country>

</l:recipient>

Dear Sir,

Please contact my assistant <t:person>Mr. Spock</t:person> at <t:email>spock@space-federation.org</t:email> to make all necessary arrangements.

Sincerely,

James T. Kirk

</l:letter>')

As you can see the 'letter" element contains interleaved element and text children. This is legal because the type was declared with mixed=''true".

It is important to understand that this property is not recursive. In our example it would be illegal to add text nodes under the "recipient" element since its type is not mixed. You can for example try the following INSERT query.

INSERT INTO T VALUES (0, '

<l:letter xmlns:l="urn:letter" xmlns:t="urn:letter:tags">

<l:date>2006-10-15-08:00</l:date>

<l:sender>

<l:FirstName>James</l:FirstName>

This is a text node

<l:LastName>Kirk</l:LastName>

<l:Line1>1234 Elm Street</l:Line1>

<l:City>Redmond</l:City>

<l:ZIPCode>98052</l:ZIPCode>

<l:State>WA</l:State>

</l:sender>

</l:letter>')

There is a text node under the “sender” element. Since “sender” is of type “addressType” and that type wasn’t defined as mixed, its presence violates the schema and the query predictably fails with the following error message:

XML Validation: Text node is not allowed at this location, the type was defined with element only content or with simple content. Location: /*:letter[1]/*:sender[1]

Next time we’ll look at text nodes, how the validator handles them, and how we can query them…

xsi:nil magic (part 2/2)

2006-12-05T03:56:00Z

Last time we looked at what happens to the xsi:nil attributes when replacing the value of a simply typed element. In this post, we’re going to look at complex types.

First let’s create a schema collection with complex types and elements.

CREATE XML SCHEMA COLLECTION SCnil_cplx AS '

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="urn:nil" xmlns="urn:nil">

<xs:complexType name="CT_emptiable">

<xs:choice minOccurs="0" maxOccurs="unbounded">

<xs:element name="a" type="xs:string"/>

<xs:element name="b" type="xs:byte"/>

</xs:choice>

</xs:complexType>

<xs:complexType name="CT_nonEmptiable">

<xs:choice minOccurs="1" maxOccurs="unbounded">

<xs:element name="a" type="xs:string"/>

<xs:element name="b" type="xs:byte"/>

</xs:choice>

</xs:complexType>

<xs:element name="N1" type="CT_emptiable" nillable="true"/>

<xs:element name="E1" type="CT_emptiable" nillable="false"/>

<xs:element name="N2" type="CT_nonEmptiable" nillable="true"/>

<xs:element name="E2" type="CT_nonEmptiable" nillable="false"/>

</xs:schema>'

CREATE TABLE T (idCol int primary key, xmlCol XML(SCnil_cplx))

Let’s insert 4 rows, containing instances of the 4 top level elements defined in the schema collection.

INSERT INTO Tcplx VALUES (1,'<ns:N1 xmlns:ns="urn:nil"><a>Data</a>2</ns:N1>')

INSERT INTO Tcplx VALUES (2,'<ns:N2 xmlns:ns="urn:nil"><a>Data</a>2</ns:N2>')

INSERT INTO Tcplx VALUES (3,'<ns:E1 xmlns:ns="urn:nil"><a>Data</a>2</ns:E1>')

INSERT INTO Tcplx VALUES (4,'<ns:E2 xmlns:ns="urn:nil"><a>Data</a>2</ns:E2>')

Now let’s see what happens when we delete all the children elements under each of those top level elements.

Let’s start with element ‘E1’ which is not nillable, and whose content model is emptiable (which means an empty element is valid with respect to the schema definition).

UPDATE Tcplx

SET xmlCol.modify('declare namespace ns="urn:nil"; delete /ns:E1[1]/*')

WHERE idCol = 3

SELECT xmlCol FROM Tcplx WHERE idCol = 3

The DML operation is successful and following its completion the instance looks like this.

<ns:E1 xmlns:ns="urn:nil" />

Let’s do the same thing with element ‘E2’ which is not nillable and whose content model is not emptiable (which means an empty element is invalid with respect to the schema)

UPDATE Tcplx

SET xmlCol.modify('declare namespace ns="urn:nil"; delete /ns:E2[1]/*')

WHERE idCol = 4

As could be expected this query fails with the following message:

XML Validation: Invalid content. Expected element(s): a,b. Location: /*:E2[1]

Now that we’ve established the behavior for non nillable elements let’s look at ‘N1’, which is nillable and whose content model is emptiable.

UPDATE Tcplx

SET xmlCol.modify('declare namespace ns="urn:nil"; delete /ns:N1[1]/*')

WHERE idCol = 1

SELECT xmlCol FROM Tcplx WHERE idCol = 1

The DML operation is successful and the instance now looks like this

<ns:N1 xmlns:ns="urn:nil" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true" />

Notice that even though, an empty instance of N1 would have been valid with respect to the schema, when all the children element were deleted the server nilled the instance by adding the attribute xsi:nil with a value of “true”.

Running a similar query on an instance of ‘N2’, which is nillable and non emptiable also yields interesting results.

UPDATE Tcplx

SET xmlCol.modify('declare namespace ns="urn:nil"; delete /ns:N2[1]/*')

WHERE idCol = 2

SELECT xmlCol FROM Tcplx WHERE idCol = 2

Even though the element isn’t emptiable, the query succeeds, and the resulting instance looks like this:

<ns:N2 xmlns:ns="urn:nil" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true" />

Once again, when all the children elements are deleted, the element is nilled by adding xsi:nil=”true”.

In conclusion, the behavior for elements with complex types is simple. If an element is nillable, removing its children elements will nill it through the addition of xsi:nil=”true”.

xsi:nil magic (part 1/2)

2006-12-01T05:19:00Z

A few months ago I wrote a post about the interesting behavior of the xsi:type attribute. Today we’re going to look at his no so distant relative xsi:nil.

But first let’s do a quick recap. Any element can be made nillable by adding the attribute nillable=”true” to its declaration. Practically it means that this element can take a special value we call “nil”. An instance of this element will be “nilled” if it has no content and contains the boolean attribute nil from namespace http://www.w3.org/2001/XMLSchema-instance (usually associated to the prefix “xsi” but this is not a requirement) with a value of “true”.

As an example, let’s look at the following schema collection.

CREATE XML SCHEMA COLLECTION SCnil AS '

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="urn:nil" xmlns="urn:nil">

<xs:complexType name="CT_emptiable">

<xs:choice minOccurs="0" maxOccurs="unbounded">

<xs:element name="a" type="xs:string"/>

<xs:element name="b" type="xs:byte"/>

</xs:choice>

</xs:complexType>

<xs:simpleType name="ST">

<xs:restriction base="xs:decimal">

<xs:minInclusive value="0"/>

</xs:restriction>

</xs:simpleType>

<xs:element name="E_ST" type="ST" nillable="true"/>

<xs:element name="E_CT_e" type="CT_emptiable" nillable="true"/>

</xs:schema>

Now if we create a table with a typed XML column we can easily create and store nilled instances of elements E_ST and E_CT_e

CREATE TABLE T (iCol int primary key, xmlCol XML(SCnil))

INSERT INTO T VALUES (1,

'<E_ST xmlns="urn:nil" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>')

INSERT INTO T VALUES (2,

'<E_CT_e xmlns="urn:nil" xmlns:z="http://www.w3.org/2001/XMLSchema-instance" z:nil="true"/>')

You may notice that in the second example I used prefix “z” for the http://www.w3.org/2001/XMLSchema-instance namespace. As I said earlier “xsi” is the commonly used prefix but it is by no means the required one.

Now let’s look what being “nilled” means for a simply typed element.

If you query the data of a nilled simply typed element, the result will be the empty sequence. You can verify this by running the following query

SELECT xmlCol.query('declare namespace ns="urn:nil"; data(/ns:E_ST[1])instance of empty()')

FROM T WHERE iCol = 1

The result is “true”.

Now how do we “un-nil” such an element? We could try to delete the xsi”nil attribute but this is not allowed. A query such as

UPDATE T SET xmlCol.modify('declare namespace xsi="http://www.w3.org/2001/XMLSchema-instance"; delete /*[1]/@xsi:nil')

WHERE iCol = 1

will return the following error message

XQuery [T.xmlCol.modify()]: The XQuery syntax '@{http://www.w3.org/2001/XMLSchema-instance}:nil' is not supported.

The solution is to update the value of the element, like this

UPDATE T SET xmlCol.modify('declare namespace ns="urn:nil"; replace value of /ns:E_ST[1] with 1.0 cast as ns:ST?')

WHERE iCol = 1

SELECT xmlCol FROM T WHERE iCol = 1

The XML instance now looks like this

<E_ST xmlns="urn:nil" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">1</E_ST>

Notice that the xsi:nil attribute is no longer present.

The reverse operation consists in updating the element’s value with the empty sequence, like this

UPDATE T SET xmlCol.modify('declare namespace ns="urn:nil"; replace value of /ns:E_ST[1] with ()')

WHERE iCol = 1

SELECT xmlCol FROM T WHERE iCol = 1

The XML instance now looks like this

<E_ST xmlns="urn:nil" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true" />

The element’s content is gone and attribute xsi:nil has been added with the Boolean value “true”.

I’m going to stop here for today. In the next installment we’ll look at what happens with complex content elements.

About my article in SQL Server Standard Magazine

2006-11-16T04:26:00Z

In my previous post I mentioned the article I wrote for the September issue of SQL Server Standard Magazine. I heard of one person who was having trouble running the code samples from the article. When I received my copy of the magazine I realized the font the article was printed in made it hard to distinguish between ‘(‘ or ‘)' and ‘[‘ or ‘]'. I scanned an examplefrom the magazine and fed the image to an OCR program and it recognized all of the square brackets as parentheses.

An experienced XQuery user would certainly know when to use the brackets vs. the parentheses, but if you're a new to this technology and are using the article as a starting point, you should be aware that there is a difference. In any case, if you have any questions don't hesitate to use the contact form to send me an email.

The latest issue of SQL Server Standard magazine is all about XML.

2006-09-01T20:42:00Z

The September issue of SQL Server Standard magazine is devoted to XML. I contributed an article where I look at ways to use XML to solve relational problems. Check it out!

Complementing XSD with CHECK constraints

2006-08-23T02:43:00Z

I was recently asked to look at a customer’s question. This person wanted to write a schema that would validate instances like the following one.

<a>

<an_element att1="data 1" att2="data 2"/>

</a>

</TopElement>

The customer had two requirements for the children of element ‘a’.

1. They should be allowed to have any valid unqualified name.

2. They are required to contain both attributes att1 and att2.

The first requirement can easily be taken care of using element wildcards but I couldn’t come up with a way to define the second constraint in an XML schema.

There are constraints like this one that are not expressible (or not easily expressible) using XSD but that are easy to check using a simple XQuery. The usual workaround is to use a T-SQL CHECK constraint.

First, let’s create a schema collection

CREATE XML SCHEMA COLLECTION SC AS '

</sequence>

</complexType>

</element>

</sequence>

</complexType>

</element>

</schema>

If we type an XML column with this collection, we will accept all unqualified elements as children of ‘a’, whether they contain the required attributes or not. In order to implement this second requirement, we are going to use a CHECK constraint.

Since CHECK constraints don’t allow calls to XML data type methods, we need to create a user-defined function.

CREATE FUNCTION dbo.checkAttrs(@x XML(SC))

RETURNS bit

BEGIN

RETURN ~(@x.exist('/TopElement/a/*[not( ./@att1 and ./@att2 )]'))

END

The function defined above checks for the presence of a child element of ‘a’ that doesn’t contain both attributes att1 and att2. If such an element exists, the function returns 0. If none is found, the function returns 1.

With this function, we can now create a table with an XML column that behaves according to the user’s specifications.

CREATE TABLE T(xmlCol XML(SC)

CHECK (1 = dbo.checkAttrs(xmlCol)))

When an XML instance is inserted into or modified inside table T it is first validated against the schemas present in the ‘SC’ schema collection. Then the server verifies that CHECK constraint holds true. If the instance doesn’t validate against the schema, or if it violates the check constraint, it is rejected.

In conclusion this short example shows how you can harness the power of XQuery through the T-SQL CHECK syntax in order to implement constraints that cannot be defined with XSD alone.

xsi:type magic

2006-08-11T20:12:00Z

As I said in my previous post, the past few weeks have been demanding and I had little time to spend on blog post. However I recently rediscovered an interesting behavior in typed XML DML and I felt I should make some time for a quick post.

Today’s post deals with xsi:type. If you don’t know about this subcasting feature of XML Schema, here is a quick recap.

An element doesn’t always have to be validated according to the type it is associated with in its schema declaration. It is possible to validate it according to any subtype of the schema type. For example, if I have a declaration for an element ‘e’ of type ‘xsd:decimal’, I can decide to validate any instance of ‘e’ with any subtype of ‘xsd:decimal’, such as ‘xsd:byte’. In order to do this, my instance of ‘e’ must contain an attribute named ‘type’ qualified with namespace ‘http://www.w3.org/2001/XMLSchema’. The value of this attribute must be a QName that references the type to use for validation. In our example, my instance would look like this:

If you are not familiar with this feature and need to find out more, you can read about it in the W3C’s primer document at http://www.w3.org/TR/xmlschema-0/#UseDerivInInstDocs

Before we go any further, let’s create a schema collection containing our definition of element ‘e’.

CREATE XML SCHEMA COLLECTION SC AS '

</schema>'

Additionally, let’s create a table ‘T’ with a typed XML column, and let’s insert two instances of ‘e’. The first one is just a regular instance. The second one makes use of the subcasting mechanism through the presence of the xsi:type attribute.

CREATE TABLE T (iCol int primary key, xmlCol XML(SC))

INSERT INTO T VALUES (1, '<e>1</e>')

INSERT INTO T VALUES (2, '<e xsi:type="xs:long" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema">1</e>')

Attribute xsi:type isn’t treated like other user-declared attributes. It cannot be queried or deleted, and its value cannot be replaced. For example, let’s run the following query

UPDATE T

SET xmlCol.modify('delete /e[1]/@xsi:type')

WHERE iCol = 2

We get the following error message

XQuery [T.xmlCol.modify()]: The XQuery syntax '@{http://www.w3.org/2001/XMLSchema-instance}:type' is not supported.

So is there a way to manipulate the xsi:type attribute? The short answer is “yes”, but there are limitations.

Let’s run the following XML DML query

UPDATE T

SET xmlCol.modify('replace value of /e[1] with xs:byte(1)')

Now let’s look at what happened to the two instances in the table. The first one now looks like this

And the second one looks like this

Notice that the first instance remains unchanged, but in the second instance element ‘e’ is now subcasted to type xsd:byte. We can run an “instance of” query to show this.

select xmlCol.query('data(/e[1]) instance of xs:byte?') FROM T

returns

false

true

The rules for replace value of are the following

· If the destination element isn’t subcasted (i.e. doesn’t contain an xsi:type attribute) then the destination value is promoted to the schema type of the element

· If the destination element is subcasted (i.e. contains an xsi:type attribute) then it is recasted to the destination value’s type

If you’re inserting a non-subcasted element but know that you might want to change its types later, all you need to do is add an xsi:type attribute with a value equal to the QName of the element’s type as declared in the schema. In our case we’d do this

INSERT INTO T VALUES (3,

'<e xsi:type="xs:decimal" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema">1</e>')

Technically this instance is equivalent to <e>1</e> (since element ‘e’ is declared of type ‘xsd:decimal’ in the schema) but because it contains the xsi:type attribute, whenever its value is updated it will also take the type of the new value.

Explaining the reasons behind some of XML datatype's limitations

2006-06-29T07:24:00Z

If you use XML in SQL Server 2005, and especially if you use it in association with schemas, I suggest you read Brandon's recent blog post about the limitations that were placed on the XML datatype. Brandon is one of our developers and he's got first-hand information about the design decisions that lie behind those limitations.

On a different note, I know my contributions to this blog have been few and far between recently. There's no shortage of topics I'd like to address, just a shortage of time.

If you leave me now…

2006-06-21T21:46:00Z

A man named Vincent Ferrari is making headlines today. He called AOL’s costumer service to cancel his account and ran into a less than cooperative representative. The twist is, Vincent was taping the call and posted the recording on his blog.

This article from MSNBC.com quotes AOL’s response to the story. As expected the representative is blamed for “[violating] customer service guidelines and practices” and he is “no longer with the Company”. Puh-lease!

Given that AOL is literally hemorrhaging customers these days, is it hard to believe that the company would have an aggressive customer retention policy? Since Vincent posted his story a lot of people have come forward to relate similar experiences. Are we to believe that they all ran into a few bad apples?

My wife used to be a long time AOL customer. Back 4 or 5 years ago, when we decided to cancel her account, the customer service reps also tried very hard to retain our business. I’ve heard similar stories from friends or coworkers dating back years. If AOL was aggressively trying to hold on to customers back when it was doing pretty well, is it hard to believe it would be even more aggressive today?

As always the little guy takes the blame though. I’m not saying the rep didn’t violate some policy or guideline, but his behavior didn’t come out of the blue. Aggressive policies from management breed aggressive behavior. You see it all the time (look no further than your local car dealerships). Maybe AOL needs to come to terms with the fact that dial-up is a dying business. I don’t believe it will disappear overnight, but it is a market which is bound to get a lot smaller than it currently is. AOL should be hard at work looking for replacement income streams rather than pressuring departing customers to stay with them for a little while longer.