Irwin Dolobowsky's Blog

SqlXml / MDAC Clarification

irwando — Mon, 07 Feb 2005 21:26:00 GMT

Apparently there's been a bit of confusion out there lately about the list of deprecated technologies in the Data Access Road Map. Both Ken and Bryant blogged about it, and I just wanted to clarify a few things:

- This article specifically referrs to SqlXml in MDAC. That's the SqlXmlx.dll file that provides nifty things like Templates and Updategams through SqlOleDB directly. This is actually a fairly old (1.0+) version of our code.

- SqlXml will be removed from future versions of MDAC, as in MDAC > 2.X. SqlXml still provies its own OleDb provider that can be used with MDAC or SQLNCLI (in Sql Server 2005). We will remain in the 2.X line.

- Notification Services has a dependency on SqlXml, but on the stand-alone version that contains Bulkload. The MDAC plans should not affect this at all.

So, SqlXml will be hanging around for a while, have no fears. Hope that clarifies a few things.

Xml Team is back!

irwando — Wed, 05 Jan 2005 07:28:00 GMT

Well, we're all back from the holidays here, spinning up again, looking forward to kicking out Yukon Beta 3 and Whidbey Beta 2. As I promised, changes are coming after the new year. The Xml Team blog should replace this one in the next day or two. In the mean time, go check it out here. I'm also going to add all of our indiviual blogs as links to the XmlTeam blog for a one-stop shop for finding all of your friendly neighborhood XML team members.

Site Change - New Year, New Blog

irwando — Thu, 16 Dec 2004 01:16:00 GMT

Tomorrow, hopefully, you'll all notice some changes to the blog here on the MSDN XML Dev Center. No, I'm not leaving, I promise. However, the blog here is being changed to move to a new blog - the XmlTeam blog. This blog will have contributions from various members of the XML team. My hope is that with more folks contributing you'll see more frequent posts and a bigger variety of posts.

You can see the new blog now up at http://blogs.msdn.com/XmlTeam .

He's Back Folks!

irwando — Sat, 11 Dec 2004 00:41:00 GMT

I know everyone missed me the last two weeks, but rest assured, I'm back and on top of things. Thanks to all those who were patient in waiting for a response from me. I was bouncing around the East for two weeks in Bermuda and New Jersey, loving every minute of it. Of course, now I'm back in bright, sunny Seattle. Well, I'm back in Seattle at least.

Coming Up

I know articles have been a bit slow lately, mostly from folks out on vacation. End of November to the end of December is always a bit of a slow down around here as everyone heads out for the holidays. Fear not though, we've got a few in the pipe:

Venkat Prasad writes a piece on Exchanging XML data with SQL Server 2000 and Reporting Services via Web Services.
Helena Kupkova will show us how to create bookmarks in XML Streams with the ResetableXmlReader.
Radhakrishnan Srikanth will present a series of articles on different components within System.Xml giving advice on usage patterns and scenarios. Should be great intro reference material for anyone developing XML apps.

We'll also be mixing in some Sql Server XML related articles now and then, and hopefully some other app-building stuff as well. We're actually doing some internal app-building here to test our Whidbey components, and I'm hoping to turn a lot of the development work there into articles up here. I've also set a goal for myself of the new year for updating the home page layout. We're going to do a bit of re-organizing which should result in a more dynamic, relevant front page.

From You!

Keep the questions and feedback coming, I appreciate it all. Over the last couple of weeks I have seen a few general problem areas:

XML in Compact Framework - Unfortunately, this is not something our team owns. The Mobile team owns this, so if you have questions about XML components in the Compact Framework head over there.
XML Downloads - The page is a bit old, that should be updated next week. Just to clarify the latest bits are:

As I continue to see more trends in questions I'll try and point them out up here.

Of course, it wouldn't be a blog post if I didn't ask for some feedback. Starting soon (right after the new year) we'll being thinking about Tech Ed '05. The XML group should be there with talks and a booth again, but is there something you specifically want to see? A particular topic discussed in a talk or other forum? No promises, of course, but send me your ideas and I'll see what I can do.

Happy Holidays!

Praise XML!

irwando — Sat, 20 Nov 2004 00:07:00 GMT

Read through my RSS items, I found a reference to this post about "The Church of XML". Seems like someone's been burned by this stuff a bit, huh? While the article is a obviously dripping with sarcasm, I actually thought it was a pretty funny read that, at the center, may actually have a few grains of truth. Here's what I seemed to get out of it (putting a little more positive spin on things):

XML is here to stay and you're probably (eh, definitely) going to have to deal with it
Just because XML is here doesn't mean you have to use it; there are times, many of them, when it isn't needed

The first point is something we all need to accept now and move on. Our group has recently begun our roadmap work for the next version of our products and I have been writing the section on SqlXml. One of the sections involves messaging to customers as to what benefits the proposed technologies will offer. Right off the bat I start with, "Plain and simple, if a customer is producing an application which transmits or receives data or other information from another system, they will more likely than not have to deal with XML." Anyone want to argue with me on that? Feel free to drop me a line, but I'm going to stand by that. XML has become the defacto standard for communication between systems. Unless you control the whole system being built and will never need to share with someone out of your control, I'm better that XML will be involved somehow, somewhere.

This brings me to the second part. If you do control everything, if you have a small app that doesn't communicate over a network or does so with a restricted set of clients, do you need XML? Maybe, maybe not. I have, in the past, recommended folks not use it. Building a small app to act as a front end to Sql Server that only a couple of folks around the office need access to? I'm pretty sure Dataset and ADO.NET will satisfy you quite well. Now am I going to stop folks from using XML if they want to? Nah, that's just not good for my job security. But I'll gladly recommend an alternate technology if need be. I think that's why I love being in the SqlXml group here. Really, it's all about how to get XML to and from other formats so you don't have to deal with it when it isn't appropriate. Hopefully going forward we will expand that beyond just to and from the database.

News from the Front

First, congrats to Jean Paoli on his being awarded the IDEAlliance XML Cup Award.

As for the rest of us around here, most of our work now has been focused on Sql Server 2000 Service Pack 4, Sql Server 2005, and Visual Studio 2005 Beta 2. We're also beginning roadmap planning for the next version of our products after that. Always an interesting process where we get to review what we've done, where we want to go, and what is going to get us there. Its still early in the process, but I'll try and keep everyone up to date on thought as I can.

Ooooh, good segue here...if you want to learn a lot more about this and want to help us out, we're hiring! Click here to check the career center for jobs available in the group. There's a place up there for you to submit your resume if you're interested. I'll even make a special deal for my loyal readers of one week, if you contact me with a link to your resume on-line, I'll forward it along for you. Please indicate preference for a position.

That's it for this week folks. Thanks to everyone who's already sent in questions and comments on the site, I've enjoyed speaking to you all. There will probably not be a post next week due to Thanksgiving, but I should be back the week after that. Happy Turkey Day to all!

First Post (redux)

irwando — Wed, 10 Nov 2004 11:46:00 GMT

Redux? Yes, that seems to be appropriate. I think I've done 2 or 3 of these "first post" entries now, but a new endeavor calls for a new First Post for what I'm sure will become the millions of my fans out there. So here we go...

Welcome to my blog! My name is Irwin Dolobowsky and I've been a Program Manager at Microsoft for 3.5 years now, all within the XML group. After interning at Microsoft in 2000, I came back to work on SqlXml, which I've owned since about month two of my employment. Since then I've also had the opportunity to work on various technologies here, including System.Xml and the SOAP Toolkit. My latest gig is this place - I've now taken ownership of the MSDN XML Developer Center from Dare. Whether this is a good or bad thing remains to be seen.

What can you expect out of my blog? Well, I must admit my knowledge of all things XML is not nearly as extensive as my predecessor, but I think I can hold my own. Expect to hear about what the latest out of the XML team is here at Microsoft as well as comments and pointers on whatever pops up on my XML Radar (which is actually RSS Bandit and a Search Folder searching titles, but it works quite well). I've also have some ideas on how to update this Developer Center to make it more dynamic and a one-stop shop for all XML related info at Microsoft. If you're curious, the following are a list of items I'd like to change here:

Bring out the MVP's - I love these folks. I always enjoy meeting up with them at conferences or chatting with them over MSN Messenger. They're out there championing our technologies every day. I want to let everyone know who they are and what they do! Other Dev Centers have the MVP of the month, links to blogs, latest posts from them, etc. Well, so too shall the XML Developer Center
Bring out the Team - My goal is to walk around the office here and poke some folks with a stick to get them to start up (or post to existing) blogs and newsgroups. We've got some brilliant folks here, and while we many not always have the most intellectual conversations, a lot of the time we do. A number of people here are already blogging, but you have to find them! We'll make that easier.
The Latest and Greatest - Releases, Newsgroup Postings, KB articles. These are dynamic things and to keep track of all of them requires visits to several sites and applications. Instead, come here to see what the latest happenings are and decide what you want to drill down into.

You'll see some other changes as well, perhaps a few tweaks to site layout and organization. I've gotten a few comments already, but if you have more feedback for me on the above or anything about the Dev Center, please drop me a line. I look forward to hearing from you! And now, off to play a little Halo 2...

SqlXml 3.0 SP3 Now Available!

irwando — Tue, 26 Oct 2004 18:21:00 GMT

SqlXml 3.0 SP3 has just been released and is now availalbe for download from http://www.microsoft.com/downloads/details.aspx?FamilyId=51D4A154-8E23-47D2-A033-764259CFB53B&displaylang=en.

This release includes bug fixes and QFE rollups, as well as the removal of SqlXml's dependency on the SOAP Toolkit. Users no longer have to install the SOAP Toolkit to enable SqlXml Web Services functionallity. This also simplifies the installion process on Windows Server 2003, on which the SOAP Toolkit was not supported. This removes barriers our customers have toward upgrading their servers to Windows Server 2003.

Thanks, and spread the word!

Irwin

Still Here, New Beginnings

irwando — Mon, 25 Oct 2004 07:30:00 GMT

Yes, for my faithful readers, I am still here. It has been quite some time since I last posted, but I hope to make these much more frequently again. Curious what I've been working on? Well...

SqlXml 3.0 SP3 - We're fixing issues with SqlXml running on Windows Server 2003, including removing our dependency on the SOAP Toolkit. This should be out this week.
SqlXml in Sql Server 2005 - Still working on that. As anyone who grabbed Beta 2 saw, we're there! Now to finish nailing that down.
SqlXml Long Term - We had planned on a new mapping stack in Whidbey, but that was cut due to varying circumstances. However, there are those of us who still believe greatly in a mapping technology as well as the need for tools and components which shred XML into ANY other form. I'm currently investigating the possibilites there.
.Net Framework 1.0 SP3 & 1.1 SP1 - I was the System.Xml contact point for the Framework's recent service packs, so I spent lots of time running around with shipping resposibilities.

As you can see, I've managed to keep myself pretty busy lately.

New Responsibilities

As Dare mentinoed, he is leaving our group for greener pastures and is leaving me the Xml Dev Center on MSDN to run. I'm very excited about this opportunity and have already formed some plans for updating it. Current thinking has the Dev Center becoming more dynamic with links to latest KB articles, downloads, conference information, maybe standards information, etc. I hope to make it a one-stop shop for any XML related information you need. Of course, the biggest thing is to make everyone out there happy with it and make it useful for you. So if you have suggestions for me, please leave a comment and let me know what you'd like to see!

Chat with us now!

irwando — Thu, 08 Jul 2004 20:08:00 GMT

Sorry, got this rather late, but for anyone interested, the System.Xml team be on for the next hour or so:

Title: C# and XML

Use XML? Use C#? Have some questions, comments, or just want listen to people talk about your favorite technologies? Join members of the C# team and Dare Obasanjo <http://blogs.msdn.com/DareObasanjo/> of the XML/Data team for a discussion about C# and XML.

Chat room: http://communities2.microsoft.com/home/chatroom.aspx?siteid=34000014

The Daily WTF

irwando — Wed, 07 Jul 2004 01:10:00 GMT

I'm sure many of you have seen it, but the Daily WTF is among the sites I check everyday (works well with the “daily” part). If not, check it out, it provides a new, amusing bit of computing wizardry each day. Today's hit particularly close to home.

Two New Articles - Bulkload in .Net and FOR XML in SQL Server 2005

irwando — Mon, 28 Jun 2004 20:45:00 GMT

There are a few new SqlXml-related articles up on MSDN.

First we have Amar Nalla's XML to SQL: Using SQLXML Bulkload in the .NET Framework which gives great examples on constructing annotated schemas and using Bulkload within .Net. Amar goes into great detail on how to feed Bulkload with a stream, a process that takes some work when going from .Net through managed interop to Bulkload. If you're using Bulkload within .Net check it out, its a great article and well worth your time to read.

A little bit older is Michael Rys's article What's New in FOR XML in Microsoft SQL Server 2005 which discusses enhancements to the FOR XML extensions to T-SQL offered by SQL Server. I've played with the new extensions a bit, not nearly as much as I should have, but am none the less very enthused about them. I think the new PATH syntax will go a long way toward helping people construct complex shapes from there relational table quicker and easier than they could be for with EXPLICIT.

Of course, this brings up the question, if we have this great new PATH mode for making complex XML shapes, why do we need mapping? Its a question I've been asked on more than a few occasions, so here are a few reasons why I still consider mapping a valuable investment:

Composability - Let's say you want to produce results that conform to the same XML schema from several queries. If you're using FOR XML, each time you have a new query, you have to make sure the output matches your target schema. However, with mapping, you define your output once, and can then query it in many ways, each time only re-writing the query. You can also then allow others to give you queries to execute and you're still guaranteed the same output format - not possible with FOR XML.

Updates - FOR XML is wonderful for producing XML from your relational data. But what about if you want to update that same data? Now you have to have a stored procedure to do the update and a FOR XML to retrieve the data, and you have to figure out how the data produced by the FOR XML statement maps back to the database. With mapping, we provide both query and update technologies, so you only have to think about one thing - the XML.

Multiple Databases - This actually relates back to Composability. You have a few different SQL databases, some 2000, some 2005, maybe with the same relational schemas, maybe not. With mapping, you can establish a mapping to each database and then query them using the same XPath queries each time, and let us handle generating the query for that particular database. If you're using FOR XML, you have to write a query for each of those databases and make sure they all resolve to the same schema.

This isn't to say I don't think FOR XML is useless or anything (though I'm obviously a bit biased toward mapping), quite the contrary. You will get much more control of the queries you're doing by using it. There is just simply more you can do by constructing the query yourself, things SQLXML does not support. You can store those queries inside stored procedures within the database, making your DBA much happier I'm sure. You can leverage your existing SQL knowledge without having to know about mapping and XPath as well.

So what am I saying here? Which one you use depends on what you consider most valuable. Do you need every last bit of performance and do you need to fine tune every query? Go with FOR XML. Can you give up some control of your queries in exchange for more composability or the ability to update? Mapping is the way to go. You can even mix the two together, don't be afraid! Query all you like with FOR XML, but when you get that 30 meg XML document, look to Bulkload to get it loaded into your database. Its ok, all of the pieces will play very nicely together, I guarantee.

Bulkload Sample Code

irwando — Sat, 29 May 2004 06:08:00 GMT

As promised, the sample code for my presentation. The Object Demo was from Andy Conrad's article “Death, Taxes, and Relational Databases, Part 2”. The other demo was mine, and here's the code. Again, the point of this demo was to show how two parties can exchange data from their databases using SqlXml. Each database has a different schema and XML is used to transfer the data. Both parties agree on a common XML schema for the data to to be transfered, and then each create a mapping to that XML schema for their database. An XPath is issued against mapping one to retrieve data and bulkload is then used with the data and mapping two to load the second database.

Mapping against Northwind:

<xs:schema elementFormDefault='qualified' xmlns:xs='http://www.w3.org/2001/XMLSchema' xmlns:sql='urn:schemas-microsoft-com:mapping-schema'> <xs:element name='Employee' sql:relation='Employees' sql:key-fields='EmployeeID'> <xs:complexType> <xs:sequence> <xs:element name='AddressList' sql:is-constant='1'> <xs:complexType> <xs:sequence> <xs:element name='Address' sql:is-constant='1'> <xs:complexType> <xs:sequence> <xs:element name='Street' type='xs:string' sql:field='Address' /> <xs:element name='City' type='xs:string' sql:field='City' /> <xs:element name='State' type='xs:string' sql:field='Region' /> <xs:element name='Zip' type='xs:string' sql:field='PostalCode' /> <xs:element name='Type' type='xs:string' sql:mapped='false' /> </xs:sequence> </xs:complexType> </xs:element> </xs:sequence> </xs:complexType> </xs:element> <xs:element name='FirstName' type='xs:string' sql:field='FirstName' /> <xs:element name='LastName' type='xs:string' sql:field='LastName' /> <xs:element name='EmployeeID' type='xs:int' sql:field='EmployeeId' /> <xs:element name='Title' type='xs:string' sql:field='Title' /> </xs:sequence> </xs:complexType> </xs:element>

<xs:annotation> <xs:appinfo> <sql:relationship name='EmployeeAddress' parent='Employee' parent-key='EmployeeId' child='Address' child-key='EmployeeId' /> </xs:appinfo> </xs:annotation> </xs:schema>

Code to Query:

   try
   {
    SqlXmlCommand mycmd = new SqlXmlCommand("Provider=SQLOLEDB;Server=localhost;database=Northwind;uid=test;password=test;");
    mycmd.CommandType = SqlXmlCommandType.XPath;
    mycmd.SchemaPath = "NWEMP.xml";
    mycmd.CommandText = "/Employee";
    mycmd.RootTag = "root";

    FileStream fs = new FileStream("EmpList.xml", System.IO.FileMode.Create);
    mycmd.ExecuteToStream(fs);
    fs.Close();
   }
   catch(Exception ex)
   {
    Console.WriteLine(ex.ToString());
   }

Mapping against Sample DB (Same database as in Andy's article, making the ID field an identiy column):

<xs:schema elementFormDefault='qualified' 
xmlns:xs='http://www.w3.org/2001/XMLSchema' 
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'  
xmlns:sql='urn:schemas-microsoft-com:mapping-schema'
id='EmployeeMappingSchema'> 

<xs:element name='Employee' sql:relation='Employee' sql:overflow-field='Overflow' sql:key-fields='EmployeeID'> 
  <xs:complexType> 
    <xs:sequence> 
      <xs:element name='AddressList' sql:is-constant='1'>
        <xs:complexType> 
          <xs:sequence> 
            <xs:element name='Address' sql:relation='Address' sql:relationship='EmployeeAddress' sql:key-fields='AddressType'>
              <xs:complexType> 
                <xs:sequence> 
                  <xs:element name='Street' type='xs:string' sql:field='Street' /> 
                  <xs:element name='City' type='xs:string' sql:field='City' /> 
                  <xs:element name='State' type='xs:string' sql:field='State' /> 
                  <xs:element name='Zip' type='xs:string' sql:field='Zip' /> 
                  <xs:element name='Type' type='xs:string' sql:field='AddressType' default="Work"/> 
                </xs:sequence> 
              </xs:complexType> 
            </xs:element>
          </xs:sequence> 
        </xs:complexType> 
      </xs:element>
      <xs:element name='FirstName' type='xs:string' sql:field='FirstName' /> 
      <xs:element name='LastName' type='xs:string' sql:field='LastName' /> 
      <xs:element name='EmployeeID' type='xs:int' sql:field='EmployeeID' /> 
      <xs:element name='Title' type='xs:string' sql:field='Title' /> 
    </xs:sequence> 
  </xs:complexType> 
</xs:element>

<xs:annotation> <xs:appinfo> <sql:relationship name='EmployeeAddress' parent='Employee' parent-key='EmployeeID' child='Address' child-key='EmployeeID' /> </xs:appinfo> </xs:annotation> </xs:schema>

Code to Bulkload (using COM Interop):

   try
   {
    SQLXMLBULKLOADLib.SQLXMLBulkLoad3Class objBL = new SQLXMLBULKLOADLib.SQLXMLBulkLoad3Class();
    objBL.ConnectionString = "Provider=SQLOLEDB;Server=localhost;database=SampleEmployees2;uid=test;password=test;";
    objBL.ErrorLogFile = "error.xml";
    objBL.KeepIdentity = false;
    objBL.Execute ("SEDB.xml","EmpList.xml");
   }
   catch(Exception ex)
   {
    Console.WriteLine(ex.ToString());
   }

Tech Ed Wrap-Up

irwando — Fri, 28 May 2004 08:53:00 GMT

So I was hoping to have updated the blog a bit more during the week, but the internet access in the Cabana areas was iffy, and sitting back in the room here I usually just fell asleep. This was my first Tech Ed US (I went to Barcelona last year) and I can say that it was very tiring but also very enjoyable. I just got back from the big party at Sea World and some drinks with friends, and thought that I'd throw down my thoughts before enjoying a well-deserved nights rest. You can see some stuff from the trip on my personal blog here.

Overall, I thought the conference was great. Every attendee I've talked to seems to agree and has told me that they've found it to be very well run and very valuable. Always makes you good to hear that MS is doing well, so thanks to any and all of you who came out to see us.

Booth duty was probably the most tiring part, standing up for 3 hours at a clip. Data Access seems to be a a bit of a nitch area - either you love it a lot and have a deep, technical issue or you don't care about it at all. So, our visitors tended to be fewer, but always great quality when they stopped by. It was a great chance to hear how folks are using our stuff and answer some questions.

The Cabanas were a great idea too - a bit more inimate area in which to sit and chat and answer questions. It was a great place to hang out though the internet access was kinda iffy for me. It was great that customers could come find us whenever they needed a SQL question answered.

My talk went pretty well I think. Probably had 80-100 folks in the room, not bad for the last session on Thursday right before the big party. Thanks to Kent Tegels for the praise and great write up here. I did purposefully skip details on the next SqlXml release, mostly because we're still defining it. As a result of pushing back our mapping plans to post-Whidbey, SqlXml has gained a renewed interest as the client platform for XML and Yukon. So, we're looking at what we can do to improve it. Details to follow as I get them.

Here are some other folks who've been blogging on Tech Ed if you're interested (in no particular order):

And with that, I'm off to collapse into bed. As promised, I'll get my sample code posted up ASAP, hopefully in the morning.

Thanks again to all who attended. Please feel free to leave any comments/questions for me here.

New Article - Xml Support in Microsoft Sql Server 2005

irwando — Fri, 21 May 2004 22:30:00 GMT

There's a great new article up on MSDN giving an overview of the XML Support in Sql Server 2005:

XML Support in Microsoft SQL Server 2005

How do I know its great? Well, because I contributed of course! Just a little, I did the section on SqlXml. Shankar Pal and Mark Fussell did most of it. Check it out for a great overview though.

Off to San Deigo tomorrow to get in a little golf before Tech-Ed. Demos and slides are mostly done, hopefully everyone will enjoy my talk. I've ended up with 23 slides, but that also includes a nice bulkload demo and an objects out of the database demo using SqlXml. Hopefully that will be “Extreme” enough for everyone.

Tech Ed Prep

irwando — Fri, 07 May 2004 21:08:00 GMT

The past couple of weeks I've been working on my slide deck and demos for Tech Ed (DAT 405 if you're coming). My biggest concern is to make sure that my talk isn't stale. I want to ensure that anyone who comes, whether they're brand new to SqlXml or they've used it for years, gets something out of the talk. Therefore, I'm approaching it a bit differently.

Usually we give the standard history of SqlXml, go through each feature, explain what it does, one or two small demos and an “application” demo which usually consists of a small winforms app and a few lines of code that fill a text box with XML. This year though I've (we've, to those who've helped put it together) tried to focus more on real-world application scenarios.

For example, for one of my demos I've taken Dare's RSSBandit news aggrigator and SqlXml-ified it. In this demo alone I get to cover Annotated XSD, XPath, Managed Classes, Updategrams, and Bulkload. Making it even better is that you can see how each of these fit into an actual application rather than just a form with some text and buttons.

Oh, and I think the other demo will be Andy Conrad's excellent work on using SqlXml to create Decoupled Domain Models (apparently this is the new buzz word for the old buzz word of “business objects“.

I'm hoping that this will be a much more enjoyable way to present the capabilities of SqlXml, both for me so I don't have to talk about slides for 45 minutes and for attendees so they (YOU!) can see SqlXml in action rather than just listening to these abstract thoughts.

So, come by and meet me, pound me with all those buring questions I know you have, yell at me for not having feature X, or just come out for drinks. Webdata as a while will also have a booth there, I encourage you to come check it out.

Developer.com on Managed SqlXml Classes

irwando — Fri, 07 May 2004 20:59:00 GMT

Someone recently forwarded me this article on Developer.com about the SqlXml Managed Classes. It was a nice little article giving a simple introduction on the usage of our managed classes along with a very simple mapping. Certainly enough to get people interested in looking for more info.

The more interesting part of the aricle for me though was this:

“By now you probably know that SQL Server 2005 (formerly code-named "Yukon") will include tight integration between SQL Server and the .NET Framework, as well as new XML features. But did you know that Microsoft regularly releases a library of .NET and XML add-ons that works with SQL Server 2000? Called SQLXML, this library can help you with quite a few XML-related tasks.”

It reminded me of something that has been a topic of discussion around the group here lately - the delta between Sql Server 2000 and Sql Server 2005 XML features is actually quite large, larger than you might think if you consider all of the SqlXml web releases. All of the feature we have worked on and release over the last 5 years will be included in SS2k5, making it a long list of new XML technologies for Sql Server. Consider all we've done since 2000:

Web Release 1:

Updategram
Bulkload

SqlXml 2.0

Annotated XSD (instead of XDR)
Client-Side FOR XML
SqlXml Managed Classes
Diffgram Support

SqlXml 3.0

Web Services Support

Current release - SqlXml 3.0 SP2

ID Propagation Support for Bulkload
NULL Support for Web Services

That's not a bad list. Hopefully we'll be able to add a few more things to it to really make Sql Server 2005 special. All leading toward my future plan - “SqlXml - Now includes Sql Server!”

Viewer's Choice, New Dev Center, Community

irwando — Tue, 30 Mar 2004 00:37:00 GMT

Viewer's Choice

So I've notice lately that several TV shows, most notably “Friends”, have gone to a viewer's choice system where folks vote on their favorite episodes to see. This seems to happen to either a) lengthen the season or b) raise ratings during slow times. Seems like a good idea to me, so here's what I'm going to do. Given my obvious lack of attention to the blog here, I want to know what folks would like to hear about. Are there any SqlXml topics you'd like opinions or thoughts on? Any guidance on issues you're having? Leave a comment, let me know, and I'll use them to decide what to write about next.

New Dev Center

Thanks in no small part to the efforts of Dare, the new XML Developer Center on MSDN was just launched. Hopefully it will be a one stop shop for all XML related topics here at MS. Included in that is of course SqlXml, though the pickings are somewhat slim at the moment. Have no fear however, as we're working on more content. Maybe Dare will even add this thing to the list on the site if I manage to post a bit more often.

Community

Besides responding to your interests here, I'm making a new effort to get back into the newsgroups much more. It's kind of hard sometimes, remembering that in everything else going on during the day that I should stop for a bit and browse through. Thankfully, there are a lot of other great folks on there answering questions as well. I have made some observations though:

Bulkload seems to be by far the most popular technology in SqlXml. However, folks don't tend to like having to annotate a schema to use it. From what I've read, a lot of people are counting on the default mapping and working around that. I'd still recommend giving us an annotated schema, especially when you get into nested shapes. Trust me, take the time to learn it, it isn't that terrible.
Lots of people are doing FOR XML EXPLICIT queries rather than Annotated XSD. To me, when doing complex queries, Annotated XSD is much, much easier. However, I do understand the drawbacks to it. You can't do it inside a stored proc. You can't do it on the server. You lose these optimizations that the database has built in. I'd be interested to know what SqlXml users would think about AXSD in a hosted CLR function in Yukon rather than FOR XML EXPLICIT. Or is it simply a matter of people still prefer SQL over mapping? Personally I love AXSD + XPath, but my demo apps don't really compare to massive database systems.
Another feature vs. feature - OpenXml vs. Bulkload. Again, server vs. client technology. Again, I choose the client one, but then again, I do own the thing. For a database guy, I seem to run a way from it an awful lot, huh?

I think those are the big items. Hopefully a few of them will turn into extended blog posts or articles. I think my first article will be on type conversions in XPath over Annotated XSD. A bunch of folks hit this and its quite a confusing story. I'm working on that now and will of course blog about it when done.

XML Templates and Sql Junkies

irwando — Thu, 19 Feb 2004 00:15:00 GMT

On the front page of SqlJunkies today is an article about XML Templates in SqlXml. In “An Introduction to XML Templates with Sql Server 2000”, Denise White gives a brief overview of what XML Templates are and how to access them over HTTP using our ISAPI extension. There are some good examples in there, mostly using embedded FOR XML queries, and all are download-able from the site.

For anyone who doesn't know, or doesn't want to read the article quite yet, XML Templates are a great little technology for creating dynamic XML documents that have sections populated with data from Sql Server 2000. Basically, you create a valid XML document however you like, and anywhere you want to have data dynamically filled in, you add a special element within our namespace. This could be a “sql:query” element which supports the execution of FOR XML queries and stored procedures, or a “sql:xpath-query” element which indicates the contents are a XPath statement to be executed over a Annotated XSD (XML <-> Relational mapping) file.

There are a variety of ways to execute templates. The article covers executing them through our ISAPI component. However, they can also be executed within applications through ADO / SqlOleDb. Support for templates is built into MDAC 2.7 and 2.8 within the provider. We also ship a set of managed wrappers (SqlXmlCommand) for our native components so the functionality can be accessed within a .Net application. Finally, in the 3.0 release of SqlXml, we added support for accessing these things over web services via our ISAPI component.

So why is this so useful? Well, think about it a bit. Normally if you wanted to do something like this, you'd have to first create the static XML document, walk to the first place to insert dynamic data, execute a query using a different component, fill in the pieces (possibly having to convert the data to XML), and then repeat for any other areas to be filled in. Instead, we take that burden off of you, allowing you to create these templates that we fill in. As mentioned in the article, templates can be parameterized so that the same template can be used repeatedly.

One area Denise did not mention was updates with Updategrams. Updategrams are our technology for updating the database via a diff of XML fragments. Our processor interprets the diff as Inserts / Updates / Deletes against the database depending on elements appearing or not appearing in the before and after images. Updategrams are in fact a part of templates and another tag (”updg:sync”) that can be included within a template. So, in a single file, you could make an update to a Sql Server and then select data back out.

For example. Consider the following template in which I take in several parameters, update the database, and then present the updated data by issuing a XPath against an Annotate schema:

<?xml-stylesheet type="text/xsl" href="updateContact.xsl"?>

<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"
xmlns:updg="urn:schemas-microsoft-com:xml-updategram">
<sql:header>
    <sql:param name="contactID"></sql:param>
    <sql:param name="name"></sql:param>
    <sql:param name="title"></sql:param>
    <sql:param name="email"></sql:param>
    <sql:param name="phone"></sql:param>
    <sql:param name="fax"></sql:param>

    <sql:param name="bname"></sql:param>
    <sql:param name="btitle"></sql:param>
    <sql:param name="bemail"></sql:param>
    <sql:param name="bphone"></sql:param>
    <sql:param name="bfax"></sql:param>
  </sql:header>

<updg:sync mapping-schema='Contact.xsd'>
    <updg:before>
      <Contact contactID="$contactID" name="$bname" title="$btitle" email="$bemail" phone="$bphone" fax="$bfax"/>
    </updg:before>
    <updg:after>
    <Contact contactID="$contactID" name="$name" title="$title" email="$email" phone="$phone" fax="$fax"/>
    </updg:after>
</updg:sync>

<UpdatedRecord>
    <sql:xpath-query mapping-schema='Contact.xsd'>
      /Contact[@contactID=$contactID]
    </sql:xpath-query>
</UpdatedRecord>
</ROOT>

Here is the annotated XSD file I'm using with it:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:sql="urn:schemas-microsoft-com:mapping-schema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">

<xs:annotation>
    <xs:appinfo>
       <sql:relationship name="Contacts_Customer"
                         parent="Contacts"
                         parent-key="customerID"
                         child="Customers"
                         child-key="customerID"/>
    </xs:appinfo>
</xs:annotation>

<xs:element name="Contact" sql:relation="Contacts">
  <xs:complexType>
    <xs:attribute name="contactID" type="xs:int" use="required" sql:field="contactID"/>
    <xs:attribute name="customerID" type="xs:int" use="optional" sql:field="customerID"/>
    <xs:attribute name="companyName" type="xs:string" use="optional" sql:relationship="Contacts_Customer" sql:relation="Customers" sql:field="companyName"/>
    <xs:attribute name="name" type="xs:string" use="required" sql:field="name"/>
    <xs:attribute name="title" type="xs:string" use="optional" sql:field="title"/>
    <xs:attribute name="email" type="xs:string" use="optional" sql:field="email"/>
    <xs:attribute name="phone" type="xs:string" use="optional" sql:field="phone"/>
    <xs:attribute name="fax" type="xs:string" use="optional" sql:field="fax"/>
  </xs:complexType>
  </xs:element>
</xs:schema>

Sorry for any bad formatting, you may want to copy these into notepad to look at.

Anyway, in the template I've defined a bunch of parameters within the “sql:header“ block. Depending on how this template is being executed, the parameters are passed in at run time (In Denise's article, over http as part of the address. They could, for example, also be passed in via form post.). The “updg:sync“ tags indicate an updategram. In this case, because the element will appear in the before and after block with the same key field (contactID), this is an update. Were I to change that so there was a different contact ID in the before and after, it would appear as two different elements, prompting a delete of the element in the before block and an insert of the one in the after.

After performing the update, I do a simple query against the database to get out the contact I just updated. Within the XML declaration, I also specified an XSL file to execute. This will be done after the query results are returned. Using this, I can build a dynamic website with update capabilities with no coding what-so-ever.

There are limitations here of course. Given that this is not a programming environment, it does not have the richness of ASP.Net. However, if you're still interested in templates, we do provide a set of managed classes (wrappers around our native code) that could be used within ASP.Net pages to execute templates. This would allow you to use templates for retrieving you XML data from Sql Server while still giving you the full power of ASP.Net and the .Net Framework.

Obligatory Intro Post

irwando — Fri, 13 Feb 2004 09:34:00 GMT

Well, I suppose that my first post should be somewhat of an intro post so you have some idea of who I am. Hopefully, this will reassure you that I am not a complete nut, but rather do know a bit what I'm talking about. Late night does seem the time to do this, as evidenced by my browsing the latest posts on http://weblogs.asp.net. Seems lots of folks use the night to do their first posts. I suppose that prevents the real posts during the day from getting too cluttered with junk. Ah well, on with it then.

I began working at Microsoft about 2.5 years ago, having interned here the summer before that. Both my internship and my full time employment have found me in the Webdata XML team, specifically the SqlXml team. The team was born around the time Sql Server 2000 was being worked on to address a rising issue in data access - XML was the new kid on the block gaining popularity, a lot of data was in relational databases, and we wanted them to play together. Thus was born SqlXml.

Our first version, which I often refer to as version 0, went out as part of Sql Server 2000. It included support for Xml Templates, querying over annotated XDR, the FOR XML syntax, and an ISAPI to access all that functionality via http. Recognizing that XML would be advancing way too fast to keep up with it on Sql Server's release cycle, the SqlXml team decided to start issuing new versions via the web.

Xml For Sql Server Web Release 1 contained 2 important advancements - Updategrams, which provide for update operations via an Xml diff format, and XmlBulkload, to quickly and easily shred XML data into your Sql Server. After that, the name of our product was officially dubbed “SqlXml”. SqlXml 2.0 brought us support for Annotated XSD as well as a set of Managed Wrappers for our native components so they be used in .Net Applications. Finally, SqlXml 3.0 added support for Web Services. We then released 2 service packs for SqlXml 3.0, the first addressing mostly security issues, the second adding ID Propagation support for Bulkload.

I joined up on the team just at the end of the 2.0 release cycle. The 3.0 release was given to me completely and I've owned all of the SqlXml releases since then (so when you have issues, I'm the guy at fault). I continue to work on that while also addressing SqlXml in the future. We're working on integrating the idea of mapping XML to relational data into our Whidbey offerings, making it much more of a mainstream product.

Now, why this Blog? Why not? Everyone's doing it! More seriously though, I'm hoping I can provide some cool notes and good answers to lingering SqlXml issues, as well as offer some looks at the future. Hopefully folks will find this thing enjoyable and useful reading. And if not, there's always complaining about this week's episode of Friends.

So, please check back now and then, see how I'm doing. Feel free to submit questions and I'll try and address them here. Oh, and check out all the folks I've linked too, they're smart and usually a good read.