All About Interop : XML

REST Q & A - enough already

DotNetInterop — Mon, 18 Aug 2008 18:04:00 GMT

Tim Bray at Sun has some questions and comments on REST.

First, let me say that the religious wars over WS-* and REST seem to be generated by a very small number of people who have a very large amount of spare time. Don't these people have things to do?

Can it possibly be enlightening or helpful for anyone at this time to read Mr Tim Bray's opinion, once again, that XSD and WSDL are "lousy and malformed"? Really? Wait, let's read four other "Thinkers" who chime in with their own artfully crafted prose stating their position on how much they dislike XSD.

This is all utter baloney. While one class of people will debate endlessly about the number of angels that can fit on the head of a pin, there are other people who have real jobs. They need to get stuff done, they need to connect systems together in their own enterprise, or connect with partners. Or they need to build frameworks and tools to help other people connect systems together. Yes, XSD isn't as simple as we'd like. Yes, it's hard to implement the spec. Yes, there are strange corners in the spec. Ok, already. We got it. We've heard it all. And governments are wasteful and the tax system is unfair, too. We mortals who have stuff to do - We have already accepted the imperfect nature of our circumstances. We get that it isn't all as it should be, if we had designed everything with the foreknowledge that comes from experience. That's the paradox, isn't it? You only go around once, but you learn as you go. .... Ok then, being practical we-have-stuff-to-do people, we don't sit around and dwell on what might have been, but we accept the reality and move on. Ferpeetesake! GET ON WITH IT, Bray.

Bray also repeated the complaint that WS-* was produced by "A Microsoft/IBM-driven process that was cripplingly product-linked and political." Such a naive opinion to be repeating. The way Bray wrote it, he could easily duck and say "it wasn't ME who said it, I was only repeating what someone else said. . ." Uh-huh.

Let me say this about that: While it is true that the representatives of large software vendors will inevitably represent the interest of the vendors in any spec standardization and rationalization process, to conclude that the vendors are unimportant or somehow harmful to the process is naive beyond belief. Suppose that the vendors did not participate in the WS-* process. Then what? Then WS-* is unsupported and you have no tools to use. Then the vendors go and build something else and you have fragmentation in the marketspace and no agreement on what to use, and interop is still a pipe dream. Sure the vendors change the process, but isn't that what the horse-trading process is about? Without the vendors, what would you have gotten?

And don't even tell me REST. REST is only possible having followed the WS-* effort. REST benefits from all the foundation work on XML (whether you like XSD or not).

Ok, so large vendors were hard to work with. How about some cheese with that whine? Once again, isn't this just the way the world works? It's like buying a house and later complaining that the seller wouldn't give you the price you wanted. It's a negotiation, Bray. You can't get everything you want.

But that's the strange part. Recently, Bray himself cited a book about donkeys, and how they find tranquility in a chaotic world. Donkey sense. But this is exactly what Bray is NOT doing.

-----

Bray mentioned a dearth of tools around REST. He cited Restlet and Jersey as exceptions. I don't know Jersey, but I do know Restlet, a lightweight REST framework for Java. If he mentions Restlet, why would Bray not also mention Windows Communication Foundation (WCF), I wonder? is it lack of knowledge? just jingoism? (edit: I suspect the latter, having read Bray's smug self-satisfied tone in his previous comments on the REST support in WCF. It's obvious he himself knows about WCF; I conclude he just doesn't want anyone else to know.) What I tried to post to Bray's comment section on his post of today (didn't work for some reason, the "pre-parse anlayzer could not be found" or some such drivel) is this: WCF since v3.5 provides a nice framework for building servers and clients that use message-based communication models, whether SOAP and WS-* based or REST based.

People can object and say that "it's not purely REST" but the beauty of WCF is that it is MESSAGE BASED, which Mr Bray apparently loves - and you the developer can defer your protocol choice (though not your arch model) until deployment.

Unlike Restlet, WCF is mainstream, in wide use. It is well supported by mainstream development tools, and there is plenty of good information on the web in the form of tutorials, sample apps, and so on. WCF is well tested and performance-tuned. It is of course very interoperable: regardless whether you use WCF on the client-side or on the server-side, you're gonna be able to connect with arbitrary REST endpoints, regardless of the language or platform those other endpoints use.

I would say more, but I gotta go; I have stuff to do.

Connecting to Exchange using JAX-WS, part 1

DotNetInterop — Tue, 29 Jul 2008 21:52:00 GMT

Yes, it is possible to Connect to Exchange using JAX-WS

The fear and trepidation I had when considering the effort to connect a Java application to Exchange Server 2007 was well founded. It wasn't easy. I'm going to spoil the ending and tell you, yes, it is possible to connect Java apps to Exchange Server via Web services, and you can do it securely and with good performance and reliability. But it took some starts and stops to make it happen.

Warning: Old Guy Reminiscing Ahead

Back a few years, I was a technical marketing guy for the Microsoft platform, and my main role Was talking to customers. One of the things I talked with customers about often, was interop. Pretty much every customer I talked to already had a signficant investment in information technology, they had a strategy that was in place and working, they had systems in place, working. They had off-the-shelf apps, they had some custom-built apps, and they had skillz. They were looking to make adjustments, to evolve their systems, to improve. It reminded me of the principles of Agile Development.

Do you remember XP? Do you remember how trendy that was, like 3 or 4 years ago. Everyone was talking about XP, everyone was looking to apply XP principles into their approaches. That seems a long time ago. XP is so 2004.

The approach that companies took with their information systems architecture was something like what developers did in a different domain with XP. Lots of little iterations, constant improvement, and steadily you get closer to your goal.

I remember talking to customers about web services technology, about using standard XML-based protocols to enable better connections between disparate information systems. Yes, Virginia, this was actually a theme of my conversations back only a few years ago. Now it seems so obvious. Now that everything speaks XML and interconnection is fairly easy, the wisdom of using standard protocols to gain efficiencies in IT is apparent. But only a short time ago this issue was not so clearly understood.

To make my case that web services and XML based protocols were the future, I'd show customers the investments that Microsoft was making into them. I'd tell them about all the standards was Microsoft working on. But the thing that customers wanted was hard evidence - not standards or papers. They wanted proof of XML usage in Microsoft products. So I'd talk about all the XML-based protocols that were already in the existing Microsoft systems they had: the web services libraries in the .NET Framework, the tooling in Visual Studio (but these were fairly obvious). The XML shredding capability that was introduced in SQL 2000 and improved in SQL 2005. Actually with the FOR XML clause you can do some pretty nifty things with the data that is already in your database. The MSXML component - now sort of just accepted as part of the landscape, a key, if almost forgotten enabler of AJAX - but 6 or 7 years ago it was undergoing rapid development.

These are all pretty good examples of Microsoft's investment in XML and WS-*. But one of the things that really surprised people was the web services capabilities we were introducing into Office (with WordML and ExcelML and so on), and the Exchange Server.

Exchange Server 2007 and Web services

Exchange Server 2003 had some limited web services capabilities. With Exchange Server 2007, Microsoft delivered an email and collab server that exposed all of its features through standard web services interfaces. Today if you have an Exchange Server, you can point your browser to http://my.exchange.server.com/ews/Exchange.asmx and get a WSDL for the server. Wow. That is pretty cool.

Standards are standards, and complying to a standard doesn't guarantee interop. If you are designing a web services interface, you can get better interop by keeping your WSDL fairly simple, and constraining your interface definitions to avoid some of the more esoteric nooks and crannies of the XML Schema spec. This is advice we've all been following for years. It works.

The problem with that advice is that it does not work if the information systems at the end of the wire are not simple, or if the requirements are not simple. For example, if you want WS-Addressing or delegated security, things can get complicated. And, similarly, Exchange Server is not simple. It stores lots of different types of things (email messages, contacts, tasks, notes, calendar entries, requests, resent items, recurrences, mime content type, cancellations, suggestions for new times, and so on). And then it compounds that complexity by adding in an unstructured aspect - document attachments and other unstructured data. Then there is stuff like impersonation, delegation (your admin assistant can update your calendar), out-of-office messages, and so on. It's pretty rich.

With all that capability, necessarily, the interface to Exchange is not going to be simple. Look at the user interface in Outlook - it is pretty rich. And the same is true with the network interface on the wire, described in the WSDL.

As a result, the WSDL for Exchange is complicated. It's big. It's got lots of nested structures. There are unions and restrictions and headers and bodies. It breaks all the rules for designing schema with interop in mind.

Anyway, who needs web services, right? We can use WebDAV!

Interop and the Exchange Server web services interface

First decision: which web services library to use? My first thought was to use the tried-and-true Apache AXIS 1.4 library that I had been depending on since 2006. It wasn't THAT old, and it worked for all the other interop tasks I had tried it on. It connected to .NET systems using ASMX or WCF. It was simple. I knew the tools.

My first step was to grab the Exchange.wsdl, the types.xsd and the message.xsd. I knew that I needed to add a service definition to the end of the Exchange.wsdl file, because most WSDL compilers want it:

   <wsdl:service name="ExchangeServices">
     <wsdl:port name="ExchangeServicePort" binding="tns:ExchangeServiceBinding">
       <soap:address location="https://myexchange.company.com/EWS/Exchange.asmx"/>
     </wsdl:port>
   </wsdl:service>

</wsdl:definitions>

But very early the work with AXIS 1.4 was hitting roadblocks. That library generated source code that did not compile. It choked on quite a few things. Other people have walked this path but custom modifications to the generated code did not sound like fun to me. I quickly decided that I'd need to move to something else.

My next idea was to move to AXIS2, which, coincidentally, is also at version 1.4. Now why they chose to call it "AXIS2 1.4" instead of "AXIS 2.4" is beyond me. There is a significant breakage in back-compatability with the move to AXIS to AXIS2, but still the version numbers are very confusing.

AXIS2 introduces a number of changes over AXIS (1). The AXIS guys improved the XML schema support, for one. Also, they made the XML serialization engine pluggable, so you can use the built-in simple library (called ADB), or you can switch to use JiBX or XMLBeans.

As with AXIS1, I ran the Exchange WSDL through the AXIS2 tools to generate the client-side proxies. This time, the code actually compiled. I was pleased to see that there was just one giant Java file generated for the entire Exchange interface. In AXIS 1.4, there was a whole series of files, 447 of em. Too many, thought I. But AXIS2 v1.4 generated just one file. Cool!

And it was a big one. Opening the file in emacs - emacs asked me, "are you sure you want to open this file? it's really big!" Seriously. And compiling that single file with javac, I actually got a heap overflow. Wow! That's a big file. I just used the -X heap magic incantations on that file and I was able to compile it.

I wrote up a simple proof of concept Java class that used the generated AXIS stub classes to connect to Exchange Server. When I ran the code, I got a failure: "required attribute localTraversal is null" I had no idea where that was leading me. I searched in the WSDL and schema for Exchange but couldn't find any elements or attributes by that name. I considered ripping open the AXIS2 source code to try to troubleshoot it. Wasn't real excited about that prospect. I had no idea where that was leading me.

My next step was to try the XMLbeans serialization option in AXIS 2. With this, I was back to generating ~ 400 Java files. But I didn't care, as long as the interop worked. And I had to update my client application code, because switching from ADB to XMLBeans necessitates a change in the programming model. I didn't care, though. Interop! (The things I do for interop!)

The AXIS2 + XMLbeans combo was looking promising, but also failed, this time with

java.lang.ClassCastException: org.apache.xmlbeans.impl.values.XmlComplexContentImpl cannot be cast to com.microsoft.schemas.exchange.services._2006.types.Folder
IdType

This is apparently a common error, if there is a duplication in classes. I don't think I had that. I keep a clean workspace. Some of the XMLBeans people suggested on forums to upgrade to the later version of XMLBeans. There was no better suggestion I found. The AXIS2 library bundles XMLBeans 2.3.0, but XMLBeans standalone was at v2.4.0. So I downloaded 2.4.0, and tried to manually compile the schema with the scomp tool.

c:\apache\xmlbeans-2.3.0\bin\scomp -out GeneratedTypes.jar contract\ExchangeServices.wsdl  -compiler c:\sunjdk\bin\javac.exe

But, then, another runtime exception:

Exception in thread "main" java.lang.NoClassDefFoundError: com/microsoft/schemas/exchange/services/_2006/types/FolderIdType
        at ExchangeClient.Run(ExchangeClient.java:161)
        at ExchangeClient.main(ExchangeClient.java:217)
Caused by: java.lang.ClassNotFoundException: com.microsoft.schemas.exchange.services._2006.types.FolderIdType
        at java.net.URLClassLoader$1.run(Unknown Source)

It was one problem after another. Also, I had found the documentation and examples to be completely lacking for AXIS2. I couldn't find anyplace that gave me any hints. Why was it so hard to find information on this thing? And why did I have to use Class.Factory.Create(); instead of just calling new Class(). How dumb is this? And finally, there were 59 JAR files I needed to reference for AXIS2. That's what the doc told me. Commons-logging, xmlbeans, axis-api, axis-rt, corba, neethi, axiom, anogen, and on and on. I don't even know what those things are. The whole thing felt very... messy. I've already experienced JAR Hell and I don't want to go back there. I'm sure each one of those jars is required, but AXIS2 *could* be a little more seamless. As it is, it feels like a science project.

Ok, time to switch horses. The AXIS Stuff was not giving me joy. My next attempt would be to use JAXWS.

With JAXWS, there is one runtime JAR. One. The code-gen produces... a JAR. How refreshing! I don't need 400 class files. I can just reference a JAR. (Yes, I know there are 400 classes inside the jar. I don't care. It's a single jar). Hmm, someone ought to tell the AXIS project about this upstart JAXWS. The stub classes get generated and compile nicely. (small victory, cross fingers and hope for more). I am starting to like JAXWS.

The programming model exposed by JAXWS-generated client-side classes is a little different than what I was used to. Rather than using return values from a stub method for the response, JAXWS has this idea of using the javax.xml.ws.Holder class for return values. Kinda odd, but... whatever. Maybe there is a way to change the client-side programming model, but I wasn't going to worry about that right then.

There is some stuff in the Exchange interface (WSDL) having to do with impersonation and SerializedSecurityContext in each request on the EWS Service port, which is not useful. I'm using neither of those things, yet they are included in every Web services stub method. Passing null for these parameters does not satisfy, I get an error saying that I cannot set that to xsi:nil unless the schema has nillable='true'. I tried fiddling with the WSDL.. to set it to nillable in the right spots... That didn't work, so I just removed that stuff from the WSDL completely, from every wsdl:message in the WSDL file:

        <wsdl:message name="GetFolderSoapIn">
                <wsdl:part name="request" element="tns:GetFolder" />
<!-- dinoch - change for JAXWS
                <wsdl:part name="Impersonation" element="t:ExchangeImpersonation"/>
                <wsdl:part name="S2SAuth" element="t:SerializedSecurityContext"/>
                <wsdl:part name="MailboxCulture" element="t:MailboxCulture"/>
-->
                <wsdl:part name="RequestVersion" element="t:RequestServerVersion"/>
        </wsdl:message>

and also from every soap:operation:

                <wsdl:operation name="GetItem">
                        <soap:operation soapAction="http://schemas.microsoft.com/exchange/services/2006/messages/GetItem" />
                        <wsdl:input>
<!--
                                <soap:header message="tns:GetItemSoapIn" part="Impersonation" use="literal"/>
                                <soap:header message="tns:GetItemSoapIn" part="S2SAuth" use="literal"/>
                                <soap:header message="tns:GetItemSoapIn" part="MailboxCulture" use="literal"/>
-->
                                <soap:header message="tns:GetItemSoapIn" part="RequestVersion" use="literal"/>
                                <soap:body parts="request" use="literal" />
                        </wsdl:input>
                        <wsdl:output>
                                <soap:body parts="GetItemResult" use="literal" />
                                <soap:header message="tns:GetItemSoapOut" part="ServerVersion" use="literal"/>
                        </wsdl:output>
                </wsdl:operation>
                <wsdl:operation name="CreateItem">

Ok, I was getting somewhere. With this change I was able to send a FindItem message over the wire to Exchange, and get message IDs back. Very cool. The approach I was taking was to use findItem to get the item IDs, and then use GetItem on the Ids to get the full content. The FindItem() was working flawlessly, but the GetItem() was not. I was not getting the results I expected. No exceptions, but my data was not coming back.

To troubleshoot this, I used Fiddler2 to see the HTTP messages back and forth between Java and Exchange. Because the communication was using SSL for encryption, Fiddler2 does not work. Some people have used Charles, a similar web debugging proxy. But I already had Fiddler, so... With the fiddler tool I could see exactly why the GetItem message was not giving me joy. (By the way, if you use Fiddler to decrypt SSL traffic, it's necessary to specify to the Java app that it should accept any SSL traffic - basically disable cert validation in the application. Only do this for testing purposes!!)

Correct flow from a .NET app to Exchange:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetItem xmlns="http://schemas.microsoft.com/exchange/services/2006/messages">
      <ItemShape>
        <BaseShape xmlns="http://schemas.microsoft.com/exchange/services/2006/types">Default</BaseShape>
        <AdditionalProperties xmlns="http://schemas.microsoft.com/exchange/services/2006/types">
          <FieldURI FieldURI="item:Subject" />
          <FieldURI FieldURI="item:Body" />
          <FieldURI FieldURI="item:Categories" />
          <ExtendedFieldURI PropertySetId="0006200E-0000-0000-C000-000000000046" PropertyId="35584" PropertyType="Integer" />
          <ExtendedFieldURI PropertyTag="0x10F3" PropertyType="String" />
          <ExtendedFieldURI PropertyTag="0x3008" PropertyType="SystemTime" />
        </AdditionalProperties>
      </ItemShape>
      <ItemIds>
        <ItemId Id="AAAZAERpbm8uQ2hpZXNhQG1pY3Jvc29mdC5jb20ARgAAAAAALBHixAbt0hGt6gCAX5/eGgcAd75VstDi0RGt3QCAX5/eGgAAAnopngAAEKx2OMNuVUuHEMvofVOxpwANFT4iewAA" ChangeKey="AAAAABQAAAAdMskaJAAzTpK1s6mN4vCnAAhyCw==" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />

Broken flow from a JAXWS app to Exchange:

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
  <S:Header>
    <ns2:RequestServerVersion xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types" Version="Exchange2007_SP1" />
  </S:Header>
  <S:Body>
    <GetItem xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types">
      <ItemShape>
        <ns2:BaseShape>Default</ns2:BaseShape>
        <ns2:AdditionalProperties>
          <ns2:Path xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:PathToUnindexedFieldType" FieldURI="item:Subject" />
          <ns2:Path xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:PathToUnindexedFieldType" FieldURI="item:Body" />
          <ns2:Path xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:PathToUnindexedFieldType" FieldURI="item:Categories" />
        </ns2:AdditionalProperties>
      </ItemShape>
      <ItemIds>
        <ns2:ItemId ChangeKey="AAAAABQAAAAdMskaJAAzTpK1s6mN4vCnAAhyCw==" Id="AAMkAGUwNmJhYWVjLTI1ZTgtMTFkMy05MDc5LTAwODA1ZjMxZjgyNgBGAAAAAAAsEeLEBu3SEa3qAIBfn94aBwB3vlWy0OLREa3dAIBfn94aAAACeimeAAAQrHY4w25VS4cQy+h9U7GnAA0VPiJ7AAA=" />

Exchange wasn't sending me the data I expected, because the JAXWS request message was not what Exchange expected it to be. With this understanding, I looked into the contract. For the element in question, the schema used a ref="type" structure, and this was the only use of the ref in the entire contract. Clearly causing a problem; JAXWS was formatting it strangely. I modified it to eliminate the ref, just as a test.

  <!-- Arrays of paths -->
  <xs:complexType name="NonEmptyArrayOfPathsToElementType">
    <!-- using choice here so that proxy generator will not flatten the array and thus lose the
                item element names -->
    <xs:choice maxOccurs ="unbounded">
<!-- dinoch - change for jaxws -->
  <xs:element name="Path" type="t:BasePathToElementType"/>
  <xs:element name="FieldURI" type="t:PathToUnindexedFieldType" />
  <xs:element name="IndexedFieldURI" type="t:PathToIndexedFieldType" />
  <xs:element name="ExtendedFieldURI" type="t:PathToExtendedFieldType"/>

<!--
      <xs:element ref="t:Path"/>
-->

    </xs:choice>
  </xs:complexType>

After re-generating the client-side stubs, I was able to get the GetItem message to work nicely. Here's the message from JAXWS after the change in schema:

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
  <S:Header>
    <ns2:RequestServerVersion xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types" Version="Exchange2007_SP1" />
  </S:Header>
  <S:Body>
    <GetItem xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types">
      <ItemShape>
        <ns2:BaseShape>Default</ns2:BaseShape>
        <ns2:AdditionalProperties>
          <ns2:FieldURI FieldURI="item:Subject" />
          <ns2:FieldURI FieldURI="item:Body" />
          <ns2:FieldURI FieldURI="item:Categories" />
        </ns2:AdditionalProperties>
      </ItemShape>
      <ItemIds>

After that, it was a matter of writing up the code to use the jaxws-generated stubs. It all works nicely now. It's pretty darn fast, too!

Let me summarize what I did to get Java to talk to Exchange:

Modified the exchange.wsdl to include a service definition, and to remove all occurrences of impersonation, S2SAuth, and MailboxCulture from each wsdl:message.
Modified the types.xsd to eliminate the ref="t:Path", and replace it with actual elements
Use Fiddler2 to trace and debug the SOAP messages.

The bottom line is that web services interop is possible even with complicated schema and interfaces, if you are willing to get your hands dirty on the schema, and if you choose the right tools.

In part 2 of this story, I'll show you some of the application code that uses the JAXWS-generated client classes, and I'll pack up the source code for everything too.

Great resources:

Cheers,
-Dino

Facebook Thrift, Google ProtoBufs, and Interop

DotNetInterop — Thu, 10 Jul 2008 21:49:00 GMT

Dare had an en-pointe analysis of the emergence of various new non-standarcd communications protocols, such as Facebook Thrift and Google Protocol buffers, and how they compare to the standards-based comms protocols like RSS, ATOM or even WS-*. Dare correctly points out that these tools can be useful if you tightly control the endpoints involved in the communication, and, if you're certain the set of communicating endpoints will never expand to include anything you don't tightly control (acquisition? partner?).

[I updated this blog entry a couple days after it was first published. The text added is enclosed in square brackets.]

After one is accustomed to fire and police protection, and very smooth roads, and storm sewers that keep the roads clear when it rains, it can be easy to forget, and to chafe under the income tax burden that keeps all those government systems functioning.

In the same way, paying the angle-bracket-tax can get annoying after a while, too. Starting in 1999 or so, we, as an industry, moved away from binary [remote-procedure call] protocols such as those dragged in by DCE RPC, CORBA IIOP, Microsoft DCOM, or Java RMI. Instead we have invested heavily in the development of tools and frameworks that use an XML-based message-passing paradigm to improve interop. We achieved critical mass on [message-based]communication standards, and everyone was happy. Except now some of us are not, and we apparently want RPC again.

We want binary serialization because it is so darn efficient. We want opaque data formats because those angle brackets are just so tiresome.

Well, we embrace those efficiencies at our peril. We have been here before! I am not saying that there is no place for Protocol Bugg^H^Hffers. What I am saying is that we as an industry shifted to XML-based message passing for a good reason, and that reason has not disappeared. We're used to the smooth roads and working sewers. Let's not have a revolution and toss all that aside. Use optimized communications where it makes sense. [And don't drag in RPC when all we want is binary serialization.] Dare shows the way.

Last thing - just dreaming a bit here... wouldn't it be cool for developers if there was a single programming framework that could do angle brackets as well as optimized binary transfers? [And everything was a message? Not a remote procedure call in sight?] Wouldn't it be cool if you could code it once, and then select the serialization mechanism at configuration time, in metadata. Wait, wouldn't it be even coooler if .. this is gonna sound crazy, but.... wouldn't it be cool if we could have multiple communication endpoints running at the same time, each dedicated to a different wire protocol, but using the same backing business logic. One endpoint could speak XML, another could speak binary, and they could run at the same time. That would be cool, wouldn't it?

Open Source and Interoperability

DotNetInterop — Wed, 18 Jun 2008 17:42:00 GMT

Open. Source. Is. Not. Interoperability.

Ted Neward is an entertaining and talented writer, not to mention a stand-up guy, and a first-rate technogeek.

But that doesn't mean he is always right. I was just reading and old post of his which I missed during my unscheduled sabbatical, in which he writes:

They [Microsoft] need to have an interoperability story that developers can believe in, which means some kind of open-source-friendly play,

I still, still totally do not get why Interop and Open Source are used so interchangeably, by so many people. Often by people whose viewpoints I respect, such as the esteemed Mr Neward. Or if not used interchangeably, why they are viewed as so closely related by so many people. I do not see these two things as mutually dependent. They are completely independent factors.

Interop is not open source. Open source does not guarantee interop. Period.

I am not making quality judgments on either open source or interop. I am not saying that one of them is good and one is bad. I am saying they are two different things. They are "silicon based computers" and "big endian bit ordering". They are related, they are neighbors, they are acquaintances, but they are not interchangeable. Nor does one imply the other.

Repeat after me:

Interop is not open source.
Interop does not require open source implementations
Open source does not guarantee Interop

People will disagree with this; I expect to hear some feedback saying "With open source I can see the code and therefore it is easier to build something that interconnects with it." I'm not buying that. That is lazy thinking or bad engineering or both. I can imagine that open source would imply interop, if you limit the definition of interoperability to mean "connect with previously written code in the same executable image." In other words, merging distinct code bases. This is a case where just linking up to a shared library can work, but sometimes seeing the code allows some better opportunity.

But this limited definition of interoperability is definitely not the mainstream one that companies and organizations are dealing with today. They want to connect large lego blocks together: Connect System A and System B together in a business process flow that makes sense, and which may evolve over time. Large-grained interop. This is not merging two distinct code bases together. Oftentimes the "code base" is not available - it is a pre-packed app. Maybe it is Oracle, or maybe it is a Salesforce.com app. Maybe the other end is a Sharepoint Portal, or a Medical Records management system from Lawson. These might be called system-to-system interop or app-to-app interop if you like, where network protocols are the thing, not compile-time-constants. Data interfaces, not code classes or interfaces.

In that situation, which I am arguing is THE mainstream challenge that architects and devs confront when they use the word "interop", looking at source code is not helpful, and I won't hesitate to argue, I think it would be counterproductive. Yeah, you read me right - it actually is harmful to look at the code if you want to connect two big apps together.

What is necessary to enable interop in these cases is PROTOCOLS, people. Standard protocols would be nice to have, but don't misunderstand - standardized protocols are not a requirement for interop. The requirement is for PUBLISHED protocols, not necessarily standard ones. PUBLISHED NETWORK PROTOCOLS ALLOW INTEROP. This is why a Java or .NET app can connect to an IBM transaction processing system, even though the on-the-wire protocols are completely closed and proprietary to IBM. The protocols are documented. They are closed yet published. And because IBM's DTP protocols are published (not publicly per se, but published to those who license the protocols), anyone can implement the client-side of the exchange.

The same goes true for, say, Microsoft SQL Server. There is an on-the-wire protocol known as TDS, also known as tabular data stream. Microsoft publishes TDS under license to partners, but I believe some have reverse-engineered it. The fact that the protocol is constant and published means that, for example, DataDirect can build an OLEDB driver or a type-4 JDBC driver for SQL Server.

It's the Protocols, Silly!

Standardized protocols are nice to have because the standard, if it is based on freely-available IP (or at the very least, IP available under RAND terms), encourages wider adoption and thus the virtuous cycle inherent in Metcalfe's Law. Standardized protocols are essential if you want broad interoperability, which of course is really important, and is what most of us are after anyway. Above I said that standards are not required, and I stand by that statement. But practically speaking, standards are almost a sine-qua-non of meaningful interop.

Web Services and XML are just common protocols, and their widespread adoption (owing in large part to the fact that they are standards) is the true source of their value to companies and organizations (again see Metcalfe's Law). It is not required that web service endpoints, either client or server, be implemented with an open-source web services stack, in order to get good interoperability. Instead it is essential that the endpoints conform to the standard protocol definitions. And the corollary is, the protocol definitions must be sufficiently clear, complete, relatively simple to implement, and relatively simple to test, such that faithful implementations of the protocols can eb validated easily and will interconnect transparently.

This is why .NET WCF clients can easily interconnect with Web Services enpoints running under IBM WebSphere App Server, though neither the .NET Framework nor the WebSphere web services libraries are open source.

Open Source Is Not Interoperability.

Java/XML Binding Options? WebSphere prefers JAXB 2.0...

DotNetInterop — Mon, 24 Mar 2008 17:54:00 GMT

I'm looking to put together more Java and .NET interop samples. If any of you have any particular requests, let me know. One of the key areas for interop I'm looking at is XML serialization.

Today I was scanning the web looking for insight into the leading Java-to-XML binding frameworks. For those of you who are .NET literate but not Java literate, "XML data binding" is the term Java people (and maybe programmers for other platforms too) use for what we .NET-heads call "XML Serialization". According to my unofficial count, there are, let's see... ah...roughly..... a zillion different options. XStream, XMLBeans, JAXB, JiBX, Castor, Zeus, Quick, etc. Then there are the vendor options, like Electric XML (from The Mind Electric, which was bought by Webmethods, which was bought by SoftwareAG, which ... ?? who knows what they are doing with Glue now? when I visited the webmethods.com website, it was down?). At one point, Oracle had its own option, IBM had its own option, BEA's was what became XMLBeans (via the "Beehive" gambit), and there were many others, I'm sure.

Let a thousand flowers bloom indeed. The problem with this "1000 flowers" approach is it presents a challenge to any developer who wants to just solve a problem. Where's the critical mass? Every developer has to go and evaluate this decision, and who knows which is the right choice for any given set of circumstances? Choice is great but too many choices can lead to higher decisions costs, and too much redundancy can stifle progress. One set of devs learns techniques and tools for JiBX and the next set learns the same for XMLBeans and ne'er the twain shall meet.

As I was looking, I learned that IBM's WebSphere Feature Pack for Web Services, released in June 2007, provides JAXB 2.0 support as the preferred Java-XML binding mechanism.

Seeing that JAXB is also the JCP-blessed option, it seems like the question "Which Java-XML binding framework is the mainstream one?" has been answered. Maybe this discussion has been over for a long time and I just missed the memo?

I am sure there will be numerous little frameworks that continue to explore the boundaries of XML-to-Java data binding, but JAXB seems like the mainstream winner approach for now. It is the Honda Accord or Toyota Camry of Java-to-XML binding - a safe, reliable, well-supported, well-known choice. Other options may have nifty features or cooler styling, but parts, service and maintenance on the Accord is going to be cheap and easy in the long term. You can't go wrong wiith that.

I've produced numerous examples of how to interoperate between Java and .NET using XML, and I chose other frameworks for the Java side, including XMLBeans and most recently, XStream, but at this point I think JAXB is going to be my mainstream preference for an Java XML data binding framework going forward.

If anyone has any guidance that disagrees with this, please advise!

How to Build a REST app in .NET (with WCF)

DotNetInterop — Thu, 20 Mar 2008 21:46:00 GMT

My prior post talked about how NOT to write a REST app in .NET. I mentioned WCF as the preferred option. In this post, I'll describe the steps for how you should do it.

Some background

First up, you should use WCF to build your REST app. WCF is the Windows Communication Foundation. (Microsoft has such a strong penchant for creating acronyms, that there are web sites dedicated solely to cataloging them. ) WCF is part of the .NET Framework, first added in .NET 3.0. WCF is used by applications developers when building apps that communicate, whether using REST, SOAP, or something else. These days, most applications need to communicate.

The basic metaphor in WCF is that services receive and respond to incoming communication, and clients initiate those communications. The communication is modeled as an interface in code, and the service is a class or type that implements that interface. Services run within a host on any Windows computer - a desktop, laptop or a big server machine. You can also run services on Windows Mobile devices like smart phones!

Therefore for any WCF app, you need a service interface, a service implementation, and a host.

We'll be building a REST service - an application that receives and understands HTTP GET Requests according to the REST pattern.

No Visual Studio required

Contrary to what you may believe, you do not need to have Visual Studio in order to build .NET applications, including WCF applications. But all of the posts I have seen on the topic assume Visual Studio. VS2008 is the commercial application developer's tool that Microsoft sells; the 2008 version includes some nice WCF design and development tools. It is by far the easiest way to build a WCF app; using VS2008 to build WCF apps also allows you to take advantage of other Visual Studio goodies like unit testing, Intellisense, debugging, profiling, and so on. I highly recommend it. If, however, you just want to get something done quickly and you don't have and don't want to pay for the commercial tool, you can also build WCF apps using a text editor (like emacs), and the command line tools included in the .NET SDK (a free download). In this post, I will show you how.

Hosting a WCF Service

WCF provides various options when it comes to hosting a service. These options are:

"Self hosted" - within a Console or Windows Forms application.
Hosted within IIS
As a Windows Service
Hosted within Windows Activation Service. This option was added with IIS7, available in Windows Vista or Windows Server 2008.

You can read more about these options, and how to choose between them, here.

My Recommendation: Do your WCF Prototype work in a self-hosted app.

If you are developing an application to prototype some ideas, then you probably want the simplest environment possible. This, for me, is often the self-hosted route - it gives me the most control and the lightweight approach I need. Hosting WCF services within IIS can also be a lightweight approach. For this post, though, I'm going to show you how to write a WCF app that is self-hosted in a console EXE. This way you can use the console output as a diagnostic tool while tweaking and tuning the app. You can start it and stop it manually. It's easy and accessible.

There is a nifty WCF Service Host that is included as part of Visual Studio 2008, called WcfSvcHost.exe. This is a Windows Forms app that hosts any WCF service you create. It collapses into the system tray, it's got some nice usability features. On the other hand, (1) I'm presuming no Visual Studio, so you don't have WcfSvcHost.exe, dear reader. And (2), this hosting app doesn't give me the control I want. [In particular, it does not give me control over how to handle errors in the service layer, which is something I really want.]

Rolling your own host for a WCF service is a pretty simple affair. Here's my code to do it.

    1 namespace ionic.samples.WcfRest
    2 {
    3 
    4     // console-based app to Host the WCF / REST service 
    5     public class ConsoleServiceHost
    6     {
    7         public static void Main()
    8         {
    9             string addressRoot = "http://localhost:8080/";
   10 
   11             // get the type we are hosting 
   12             System.Type t1 = typeof(ionic.samples.WcfRest.NorthwindDataService);
   13 
   14             // Create a new ServiceHost to start and stop the service 
   15             // (The URI specified here is used for MEX inquiries)
   16             System.ServiceModel.ServiceHost serviceHost1 =
   17                 new System.ServiceModel.ServiceHost(t1, new System.Uri(addressRoot)); 
   18 
   19             // Create the REST binding
   20             System.ServiceModel.WebHttpBinding restBinding = new System.ServiceModel.WebHttpBinding();
   21 
   22             // Create the REST endpoint.  We need to do this explicitly in order to add a custom behavior to it. 
   23             var restEndPoint =
   24                 new System.ServiceModel.Description.ServiceEndpoint
   25                     (System.ServiceModel.Description.ContractDescription.GetContract(typeof(ionic.samples.WcfRest.IDataService)),
   26                     restBinding,
   27                     new System.ServiceModel.EndpointAddress(addressRoot + "rest"));
   28 
   29             // Add a customm behavior to the REST endpoint
   30             restEndPoint.Behaviors.Add(new ionic.samples.WcfRest.FaultingWebHttpBehavior());
   31 
   32             // Add the endpoint to the service
   33             serviceHost1.Description.Endpoints.Add(restEndPoint);
   34 
   35             // Open the ServiceHost to create listeners and start listening for messages
   36             serviceHost1.Open();
   37             System.Console.WriteLine("Service implementation: " + t1.ToString());
   38             System.Console.WriteLine("The service is ready.");
   39             System.Console.WriteLine("Service Address: " + addressRoot + "rest");
   40             System.Console.WriteLine();
   41             System.Console.WriteLine("Press <ENTER> to terminate the service.");
   42             System.Console.WriteLine();
   43             System.Console.ReadLine();
   44 
   45             // Close to shutdown the service
   46             serviceHost1.Close();
   47         }
   48     }
   49 }

I will call your attention to some highlights there: First, you will see that things like base addresses, service interfaces, and service implementation classes are all specified in code (line 12, 17, 25, etc). This stuff could be specified in config files, and commonly is. But in this case, I am specifying all of it in code. There is no config file required here.

Second, is the use of WebHttpBinding, on line 20. This is the binding in WCF, new for .NET 3.5, that supports REST.

Finally, a subtle bit - notice the custom behavior I added to the endpoint. FaultingWebHttpBehavior on line 30. That behavior gives me the error handling control I mentioned earlier. More about that later.

Ok, that's the host. Now I need the service interface and the service implementation.

The Interface

The database I am using is the Northwind sample database which has been shipped with Microsoft database products for years. My service will simply return information for orders in the database - order date, customer, order contents, shipping address, and so on.

This is a simplistic interface, with just one method. It takes a single parameter as input - the order ID - and returns order information in an XML document. Here's the final product:

    1     [ServiceContract(Namespace = "urn:ionic.samples.2008.03.wcfrest")]
    2     public interface IDataService
    3     {
    4         [OperationContract]
    5         [WebGet(
    6             BodyStyle = WebMessageBodyStyle.Bare,
    7           RequestFormat = WebMessageFormat.Xml,
    8           ResponseFormat = WebMessageFormat.Xml,
    9           UriTemplate = "Northwind/Order/{orderId}")]
   10         OrderInfoMsg GetOrderInfo(string orderId);
   11     }

Things to notice:

Standard WCF-isms like OperationContract and ServiceContract.
The WebGet attribute is new for .NET 3.5, and is the one you want to use for a method that handles REST requests.
The UriTemplate specifies the form of the URI requests this method will be tickled for. Also it does automagic URI-to-method-parameter mapping.

The interface for a service in WCF is exhibited at multiple levels - in code (C# in this case), and on the wire (XML in this case). When doign web services design, a best practice for interoperability is to define the WSDL first, and then implement the services and clients from that WSDL. But REST has no WSDL, and the development style is more rapid iteration. In keeping with that tradition, in my case, I defined the interface in C#, and then iterated on it until I got XML in a shape I liked.

The OrderInfoMsg type, which is the response type I finally arrived at here, looks like this:

    1     [DataContract(Namespace = "urn:ionic.samples.2008.03.Rest")]
    2     public class OrderInfoMsg
    3     {
    4         [DataMember]
    5         public OrderT Order;
    6         [DataMember]
    7         public ApplicationFaultT Fault;
    8 
    9         [DataMember]
   10         public MessageInfoT MessageInfo;
   11 
   12 
   13         public OrderInfoMsg()
   14         {
   15             MessageInfo = new MessageInfoT();
   16         }
   17     }

There are 3 parts to the class - one part contains information about the order, one part to contain information about any fault that occurred on the server side, and a final part to contain generic message information. The fault information is key, because REST doesn't formally specify a way to handle faults, or what they look like. So your REST application has to consider that itself. My approach is to include a way to pass fault information back to the caller explicitly. The MessageInfo part gets filled implicitly, with a timestamp and other information. Think of that as a basic message "dog tags".

I won't show all the code for the other classes. They are just basic Data Transfer Objects.

The Service Implementation

The service implementation is obviously going to be pretty basic. All we need to do is query SQL Server for the order information, and then form an XML document in reply.

Here's where there is some fun involved. We can use LINQ-to-SQL to do the queries and fill the Data Transfer Object. Check the code:

    1   public OrderInfoMsg GetOrderInfo(string orderId)
    2   {
    3       int n = 0;
    4       try
    5       {
    6           n = System.Int32.Parse(orderId);
    7       }
    8       catch
    9       {
   10           return new OrderInfoMsg
   11           {
   12               Fault = new ApplicationFaultT
   13               {
   14                   message = "Bad input, please pass a valid order number",
   15                   code = 11
   16               }
   17           };
   18       }
   19 
   20       OrderInfoMsg oim = null;
   21       try
   22       {
   23           oim =
   24           (from o in db.Orders
   25             where o.OrderID == n
   26             select new OrderInfoMsg
   27               {
   28                   Order = new OrderT
   29                   {
   30                       OrderID = o.OrderID,
   31                       Customer = (from c in db.Customers
   32                                   where o.CustomerID == c.CustomerID
   33                                   select new CustomerT
   34                                   {
   35                                       CompanyName = c.CompanyName,
   36                                       ContactName = c.ContactName,
   37                                       ContactTitle = c.ContactTitle,
   38                                       Phone = c.Phone
   39                                   }).Single(),
   40                       OrderDate = o.OrderDate,
   41                       ShipInfo = new ShipInfoT
   42                       {
   43                           ShipName = o.ShipName,
   44                           ShipAddress = o.ShipAddress,
   45                           ShipCity = o.ShipCity,
   46                           ShipPostalCode = o.ShipPostalCode,
   47                           ShipCountry = o.ShipCountry,
   48                       },
   49                       Items =
   50                       (from od in db.Order_Details
   51                         where od.OrderID == o.OrderID
   52                         select new ItemT { 
   53                             ProductName = od.Product.ProductName, 
   54                             Quantity = od.Quantity, 
   55                             UnitPrice = od.UnitPrice }).ToArray()
   56                   }
   57               }).Single();
   58 
   59       }
   60       catch (System.Exception exc1)
   61       {
   62           oim = new OrderInfoMsg
   63           {
   64               Fault = new ApplicationFaultT
   65               {
   66                   message = "Could not retrieve Order Information. Maybe you passed bad input?",
   67                   innerMessage = "Exception: " + exc1.Message,
   68                   code = 12
   69               }
   70           };
   71       }
   72 
   73       return oim;
   74 
   75   }

Lines 4 through 18 are just validation of the input. The LINQ stuff starts on line 21. Line 26 is where I create the new OrderInfoMsg and the following lines fill that data structure based on information retrieved from the SQL database.

At first that LINQ stuff may look novel and confusing, but it is really a simple way to do database queries. Keep in mind though, that the REST stuff in WCF is not dependent on LINQ in any way. You could create and fill the OrderInfoMsg any old way, including using an old-fashioned DataReader if you like.

But I find LINQ a nice upgrade. At line 49, you can see I fill an array of line items for the order. That's nice, concise, readable code.

In line 62 you can see some error handling, and the use of the object initializers that are new for C# 3.0. Again, nice, clean code.

Where does this LINQ stuff come from?

The LINQ mappings for the Northwind database are defined in a generated C# file, which is not shown here, and which, I confess, I produced using Visual Studio. I know, I know. I promised that VS wasn't required. But I figured that digression was allowed, seeing as how the main point of this post is not LINQ but how to use WCF to build a REST app. You don't need LINQ to retrieve data. It's just a little extra BAM! for you in this sample.

The FaultingWebHttpBehavior

I found that when there was an error in the service, it would return an HTML page to the caller. This has been noticed by others. A fellow by the name of Andre posted code for an endpoint behavior that would prevent the generation of HTML messages. I liked it and used it here. This is the custom behavior I added to the endpoint, in the service host code, shown above.

    1 namespace ionic.samples.WcfRest
    2 {
    3     /// <summary>
    4     /// A <see cref="WebHttpBehavior"/> that returns error messages to the caller 
    5     /// when an exception occurs on the server/service. 
    6     /// </summary>
    7 
    8     public class FaultingWebHttpBehavior : System.ServiceModel.Description.WebHttpBehavior
    9     {
   10 
   11 
   12         protected override void AddServerErrorHandlers(System.ServiceModel.Description.ServiceEndpoint endpoint,
   13                                System.ServiceModel.Dispatcher.EndpointDispatcher endpointDispatcher)
   14         {
   15             endpointDispatcher.ChannelDispatcher.ErrorHandlers.Clear();
   16             endpointDispatcher.ChannelDispatcher.ErrorHandlers.Add(new ErrorHandler());
   17         }
   18 
   19         public class ErrorHandler : System.ServiceModel.Dispatcher.IErrorHandler
   20         {
   21             #region IErrorHandler Members
   22 
   23             public bool HandleError(System.Exception error)
   24             {
   25                 return true;
   26             }
   27 
   28             public void ProvideFault(System.Exception error,
   29                         System.ServiceModel.Channels.MessageVersion version,
   30                         ref System.ServiceModel.Channels.Message fault)
   31             {
   32                 System.ServiceModel.FaultCode faultCode =
   33                   System.ServiceModel.FaultCode.CreateSenderFaultCode(error.GetType().Name,
   34                                               "urn:ionic.samples.serviceexception");
   35                 fault = System.ServiceModel.Channels.Message.CreateMessage(version, faultCode, error.Message, null);
   36             }
   37 
   38             #endregion
   39         }
   40     }
   41 }

The REST messages

The request looks something like this:

    1 GET /rest/Northwind/Order/11049 HTTP/1.1
    2 Accept: */*
    3 Accept-Language: en-us
    4 UA-CPU: x86
    5 Accept-Encoding: gzip, deflate
    6 User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0;)
    7 Host: dinoch-2:8080
    8 Proxy-Connection: Keep-Alive

Obviously, any application on any computer can generate that sort of request. I used an IE browser, but it could be anything, and in a real REST app it won't be just a browser.

And the response looks like this:

    1 HTTP/1.1 200 OK
    2 Content-Length: 1189
    3 Content-Type: application/xml; charset=utf-8
    4 Server: Microsoft-HTTPAPI/2.0
    5 Date: Thu, 20 Mar 2008 20:34:59 GMT
    6 
    7 <OrderInfoMsg ...>

The prettified XML looks like this:

    1 <OrderInfoMsg xmlns="urn:ionic.samples.2008.03.Rest" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
    2   <Fault i:nil="true"/>
    3   <MessageInfo>
    4     <ApplicationId>319a80e4-3e64-4328-a5d1-468b354c370a</ApplicationId>
    5     <ApplicationName>WcfRestService, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null</ApplicationName>
    6     <OkToStore>true</OkToStore>
    7     <generated>2008-03-20T13:34:59.3459363-07:00</generated>
    8     <originatingHost>dinoch-2</originatingHost>
    9   </MessageInfo>
   10   <Order>
   11     <Customer>
   12       <CompanyName>Gourmet Lanchonetes</CompanyName>
   13       <ContactName>André Fonseca</ContactName>
   14       <ContactTitle>Sales Associate</ContactTitle>
   15       <Phone>(11) 555-9482</Phone>
   16     </Customer>
   17     <Items>
   18       <Item>
   19         <ProductName>Chang</ProductName>
   20         <Quantity>10</Quantity>
   21         <UnitPrice>19.0000</UnitPrice>
   22       </Item>
   23       <Item>
   24         <ProductName>Queso Manchego La Pastora</ProductName>
   25         <Quantity>4</Quantity>
   26         <UnitPrice>38.0000</UnitPrice>
   27       </Item>
   28     </Items>
   29     <OrderDate>1996-05-24T00:00:00</OrderDate>
   30     <OrderID>11049</OrderID>
   31     <ShipInfo>
   32       <ShipAddress>Av. Brasil, 442</ShipAddress>
   33       <ShipCity>Campinas</ShipCity>
   34       <ShipCountry>Brazil</ShipCountry>
   35       <ShipName>Gourmet Lanchonetes</ShipName>
   36       <ShipPostalCode>04876-786</ShipPostalCode>
   37       <ShipRegion i:nil="true"/>
   38     </ShipInfo>
   39   </Order>
   40 </OrderInfoMsg>

Clean, simple, easy.

But wait! There's more! What if you don't want XML but want that data in JSON? A simple flip of an attribute in the Service definition gives us this as a response:

    1 {
    2   "Fault":null,
    3   "MessageInfo":{
    4     "ApplicationId":"319a80e4-3e64-4328-a5d1-468b354c370a",
    5     "ApplicationName":"WcfRestService, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
    6     "OkToStore":true,
    7     "generated":"\/Date(1206047182582-0700)\/",
    8     "originatingHost":"dinoch-2"
    9     },
   10   "Order":{
   11     "Customer":{
   12       "CompanyName":"La maison d'Asie",
   13       "ContactName":"Annette Roulet",
   14       "ContactTitle":"Sales Manager",
   15       "Phone":"61.77.61.10"
   16       },
   17     "Items":[
   18       {"ProductName":"Guaraná Fantástica","Quantity":10,"UnitPrice":4.5000}
   19     ],
   20     "OrderDate":"\/Date(833180400000-0700)\/",
   21     "OrderID":11051,
   22     "ShipInfo":{
   23       "ShipAddress":"1 rue Alsace-Lorraine",
   24       "ShipCity":"Toulouse",
   25       "ShipCountry":"France",
   26       "ShipName":"La maison d'Asie",
   27       "ShipPostalCode":"31000",
   28       "ShipRegion":null
   29       }
   30     }
   31 }

Pretty nifty, eh?

That's how to write a REST app in .NET.

Where are we?

Well, I hope I've shown you a couple things here. First, it's easy to use WCF to build REST apps, using the .NET Framework 3.5. Second, you can build WCF apps with free command line tools. Finally, LINQ is cool, and is a nice complement to REST approaches.

Grab the source code attached here and try it yourself.

Date and Time Values and Java Interop: a concrete example using XStream

DotNetInterop — Wed, 10 Oct 2007 17:51:00 GMT

Last month I posted on Date and Time values and interop. One of the key points I made is that you can lose information (specifically timezone information) when you transition from .NET to something else. If you stay within .NET, then you don't lose that information, even if you serialize to XML and then de-serialize. If you transition outside of .NET, then you may lose information.

What do I mean by "serialize"?

It is possible to produce an XML instance document that contains the state stored within a .NET type. Imagine a Person object that contains a name, a birthdate, and a coolness rating. These three data items might appear as three elements within the XML instance document that is produced via "serialization". This is exactly what happens within a web services call.

Digression: Within a traditional request/response communication transaction, there are four transition points, points at which data enters or leaves an application. Outbound transitions occur when data is emitted from the app. We call this serialization. Inbound transitions occur when data enters the app from outside; this happens via de-serialization.

The first outbound transition occurs when transmitting the request on the caller. The request is converted from an object graph stored in the memory of the .NET application, into an XML instance document, via serialization. The serialized request is then transmitted via some mechanism to the server. On the receiving (server) side, the request is then converted into an in-memory object graph; this is an inbound transition, which occurs via de-serialization. The server then performs some action and sends the response, which is a mirror image of the request, from the point of view of serialization and de-serialization. Both outbound and inbound transitions are of the kind I described in that prior post, a transition you need to be careful about.

With each transition, there are two types of pitfalls, In my prior post I spoke of the loss of information. There is also the converse pitfall - one side of the transmission could provide more information than can be handled by the other. Let's look at this case now.

An Example

Here's a concrete example. I'm going to take some data and serialize it to XML using .NET, and then de-serialize it from XML into an instance of a Java object. For now I am not going to concern myself with the transport - the details of how the XML instance document gets from the .NET app to the Java application. It could be any one of a number of ways: REST-style invocation, shared filesystem, shared database, message queue, smtp, ftp, tcp transfer, and so on. It could be wrapped in syntactic sugar so you can call it a "web services call". It doesn't really matter. I'm concerned here only with the serialization and de-serialization.

For the Java side, I selected the XStream Java XML Serialization package (version 1.2.2). The thing about Java is, there are about a thousand and one XML Serialization libraries out there, and you know, every one of them is a little different. Apparently many people each thought they could build a better mousetrap, and so there are now a zillion choices for Java-based XML serialization libraries. As to which is "best" ? I will leave that to someone with more time than I have. In any case, XStream is pretty easy to use, and seems to be reasonably common, so I chose that. (I have previously used JAXB as well as the Apache XMLBeans library. This time, I chose a different library.)

Ok, now in this scenario, the classes on the .NET side and on the Java side are pretty simple. The .NET class looks like this:

    1   [XmlRoot("person")]
    2   public class Person 
    3   {
    4     public Person() {}
    5     public Person(string name, DateTime birthdate, int rating) 
    6     {
    7       _birthdate= birthdate;
    8       _rating= rating;
    9       _name= name;
   10     }
   11 
   12     [XmlElement("name")]
   13     public string _name;
   14 
   15     [XmlElement("rating")]
   16     public int _rating;
   17 
   18     [XmlElement("birthdate", DataType="dateTime")]
   19     public System.DateTime _birthdate;
   20   }

You can see that one of the data members in the .NET type is a DateTime value. There are a number of options for serializing a DateTime value into XML. One option is to serialize it as a number, for example a time_t quantity which from my Un*x days is a 32-bit quantity representing the number of seconds that have elapsed since January 1st, 1970. This is an interesting option but not one I explored for this example. Another set of options is to serialize it as one of the xsd built-in types, like xsd:dateTime, xsd:date, or xsd:time, as defined in the XML Schema standard. I chose the xsd dateTime type, as you can see from the attribute on line 18 in the code above.

There is a corresponding Java class; It looks like this:

    1   public class MyData
    2   {
    3     // fields to be serialized
    4     public String name;
    5     public int count; 
    6     public java.util.Date date;
    7   }

The Java class lacks annotations because the XStream library I chose allows me to specify the Java-to-XML binding in code, programmatically.

Serializing from .NET

I use this kind of code to serialize from an instance of the object in .NET to an XML document:

    1       XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
    2       ns.Add( "", "" );
    3 
    4       System.Console.WriteLine("\n\nThe serialized Person:\n");
    5       s1.Serialize(new XTWFND(System.Console.Out), d1, ns);
    6       System.Console.WriteLine("\n");

The XmlSerializerNamespaces thing just tells the serializer to omit the default namespaces. I do this only to tidy up the resulting XML. Also, that XTWFND is defined like so:

    1   /// XmlTextWriterFormattedNoDeclaration
    2   /// helper class : eliminates the XML Documentation at the
    3   /// start of a XML doc. 
    4   /// XTWFND = XmlTextWriterFormattedNoDeclaration
    5   public class XTWFND  : System.Xml.XmlTextWriter {
    6     public XTWFND (System.IO.TextWriter w) : base(w) { Formatting= System.Xml.Formatting.Indented;}
    7     public override void WriteStartDocument () { }
    8   }

It is a helper class, an XmlTextWriter that indents and does not emit an XML Declaration line; again, this is for cleaner XML but is not strictly necessary for interop.

The XML resulting from this code looks like this:

<person>
  <name>Yiifusxxwic</name>
  <rating>859</rating>
  <birthdate>2007-10-08T22:56:45.8697911-07:00</birthdate>
</person>

Serializing from Java

In Java with XStream, we can do something similar. The code to serialize looks like this:

    Person p1= getInstance();
    XStream xstream= new XStream();
    xstream.alias("person", Person.class);
 
    System.out.println(xstream.toXML(p1));

And then XStream infers the element names from the names of the members in the class. The resulting XML document looks like this:

<person>
  <name>Mvsnyzilgen</name>
  <rating>426</rating>
  <birthdate>2007-10-08 23:17:11.679 PDT</birthdate>
</person>

Looks pretty similar to the XML generated from .NET, right? Yes, it is Similar, but not the same. And because the XML is not the same format on either end of the wire, interop isn't happening.

When .NET serializes the DateTime value, it uses the xsd:dateTime format, which includes a timezone specified in "offset from UTC". The default Date serialization in XStream does not. But, there is a nice ISO8601DateConverter in XStream. This gives you application code like so:

    1     Person p1= getInstance();
    2     XStream xstream= new XStream();
    3     xstream.alias("person", Person.class);
    4     xstream.registerConverter(new ISO8601DateConverter());
    5 
    6     System.out.println(xstream.toXML(p1));

This modified usage gives you XML out of the Java application like this:

<person>
  <name>Kgdtqgcdyxg</name>
  <rating>211</rating>
  <birthdate>2007-10-08T23:21:44.931-07:00</birthdate>
</person>

...which is much closer to what we want.

Deserializing

De-serializing from within .NET is pretty simple. We can do it with code like this:

            string xmlFromXstream =
                "<person>\n" +
                "  <name>Jjqbrkqcnuk</name>\n" +
                "  <rating>934</rating>\n" +
                "  <birthdate>2007-10-08T23:21:44.931-07:00</birthdate>\n" +
                "</person>\n";
 
            Person d2 = (Person)s1.Deserialize(new System.Xml.XmlTextReader(new System.IO.StringReader(xmlFromXstream)));

And slurping in the XML as generated from XStream, just works.

De-serializing from within Java with XStream is similar. The code looks like this:

        String xmlFromDotNet = "<person>" +
        "  <name>Sxgiyg</name>" +
        "  <rating>859</rating>" +
        "  <birthdate>2007-10-08T22:06:28.8700931-07:00</birthdate>" +
          "</person>";
 
        Person p2 = (Person)xstream.fromXML(xmlFromDotNet);

But this does not work. This is because .NET serializes time values with a higher precision than Java can deal with. Look closely at the birthdate values generated from .NET and compare them to the birthdate values generated from Java. If you look carefully you will see that the dates generated from .NET have 7 decimal digits of precision, while Java has just 3. And Java is not able to de-serialize from the more precise representation generated by .NET! Ack!

What to do? The easiest thing to do, if you don't mind the loss of precision, is to modify the .NET class to serialize the dateTime value into something XStream can handle. We can do this by modifying the type to look like this:

    1   [XmlRoot("person")]
    2   public class Person
    3   {
    4     private static string formatString= "yyyy-MM-ddTHH:mm:ss.fffzzz";
    5 
    6     public Person() { }
    7     public Person(string name, DateTime birthdate, int rating)
    8     {
    9       _birthdate = birthdate;
   10       _rating = rating;
   11       _name = name;
   12     }
   13 
   14     [XmlElement("name")]
   15     public string _name;
   16 
   17     [XmlElement("rating")]
   18     public int _rating;
   19 
   20     [XmlElement("birthdate")]
   21     public string _bdayString
   22     {
   23       get { return _birthdate.ToString(formatString); }
   24       set 
   25       { 
   26       try {
   27         _birthdate = 
   28           System.DateTime.ParseExact(value,
   29                                new string[] {formatString}, 
   30                    new System.Globalization.CultureInfo("en-US", true), 
   31                                System.Globalization. DateTimeStyles.AllowWhiteSpaces);
   32       }
   33       catch {}
   34       }
   35     }
   36 
   37     [XmlIgnore]
   38     public System.DateTime _birthdate;
   39   }

This "hides" the actual DateTime value from the XML serializer with the XmlIgnore attribute, but then serializes in its place, a string, formatted just the way Java wants it. The getter and setter on the string property allows the serialization and de-serialization to work reflexively, using the same format. The XML generated from that Person type works nicely with the Java code I showed previously.

Where are We?

This example shows that .NET and Java can use XML to interop, but it shows some of the pitfalls associated with DateTime values and loss of information on serialization. In particular it shows that you need to serialize an common dateTime format (ISO8601 mostly works) and you need to be careful of precision on either side. We did not cover Nillable Date values - that's another pitfall I will leave for another day.

Check the attachments for the full source code.

Open XML a superb standard - says Mono dude

DotNetInterop — Wed, 12 Sep 2007 05:25:00 GMT

Slashdot reports that Miguel de Icaza, founder of GNOME and Mono, recently remarked in a Google groups discussion that Open XML is a “superb standard” and that the spec has been unfairly criticized. “OOXML is a superb standard and yet, it has been FUDed so badly by its competitors that serious people believe that there is something fundamentally wrong with it. This is at a time when OOXML as a spec is in much better shape than any other spec on that space. Besides, it is always better to have two implementations and then standardize than trying to standardize a single implementation.”

The slashdot post:
http://linux.slashdot.org/linux/07/09/10/2343256.shtml

The google groups discussion thread:
http://groups.google.com/group/tiraniaorg-blog-comments/browse_thread/thread/2a07b8b50038d8c8/d582162af2d63d57

Fact Check – IBM’s Steve Mills claims on MS and SOA

DotNetInterop — Fri, 10 Aug 2007 03:13:00 GMT

We saw an article on CNET that included quotes from Steve Mills of IBM directly related to Interop, SOA and Microsoft. Since Interop is one of our passions, we thought it would be worthwhile to respond. Given that we are approaching an election here in the US, we are employing a CNN technique. Let's check the facts.

Claim: Steve Mills is Senior VP of IBM Software.

Microsoft Fact Check: TRUE. Mr. Mills runs IBM Software, which is about a USD $18B business for IBM. By the way, IBM Software Group contributes about 40% of IBM's pre-tax profit, according to a recent ComputerWorld article.

Claim: Microsoft's approach to SOA [is] stymied by its emphasis on linking Microsoft-compatible processes. "We're doing all platforms; all applications," IBM Software Group executive Steven Mills told ZDNet.co.uk. "We're integrating everything. Microsoft is trying to provide connectivity capabilities for those that are running on Windows platforms. That's a profound difference."

Microsoft Fact Check: Mr. Mills seems to be commenting on Microsoft SOA infrastructure software, including the .NET Framework (and WCF, and WF, and all the constituent technologies), BizTalk Server, Host Integration technologies, and so on. If the claim is that this Microsoft software runs only on Windows, then this is TRUE. This is not a particularly novel, interesting or relevant observation. Incidentally, it applies to Mr. Mills' portfolio as well: CICS and IMS for example. Or consider DB2: it runs on many systems, but the capabilities in the mainframe version differ from those versions for other platforms.

Mr. Mills seems to be saying something different: he seems to be implying, because BizTalk Server (as one example) runs only on Windows, then it can connect only to other Windows-based systems or applications. That is FALSE. The facts are: the vast majority of the over 7,000 BizTalk Server customers use the technology to connect with assets on UNIX, Linux and mainframe based systems: 92% perform heterogeneous platform integration. In fact, Microsoft has long offered a technology that does nothing but integrate with IBM's iSeries and zSeries technologies. (it accomplishes this via IBM's proprietary protocols, which Microsoft must license).

Claim: "Their perspective is how to make Windows environments connect, as long as you're using Microsoft technology. Our view is: how do you make every environment connect whether you are using Microsoft or anyone else's technology," Mills said.

Microsoft Fact Check: This is the continuation of Mr. Mills' previous thought. In virtually every enterprise above a certain size or age, the rule is heterogeneity in IT. Whether you're an international bank, a specialty manufacturer, or a chain of sushi restaurants, there's a myriad of information systems and applications you need to deal with, and better connections among those systems allows you to run your business better. That's what SOA is about, and that's what Microsoft's SOA platform infrastructure is designed to help with. Thus, FALSE.

Claim: Mills claimed there is a big difference between IBM and Microsoft's approaches, saying that, in contrast to Microsoft, IBM uses open standards for XML and web services.

Microsoft Fact Check: FALSE. Implying that Microsoft does not support standards like XML and WS-* is odd. This denies the reality of the past 7 years, during which Microsoft has repeatedly been recognized by independent analysts as leading the industry drive toward defining and implementing open protocol standards such as XML, and WS-*. It is also an attempt to revise history as Microsoft and IBM partnered on Web Services Standards in 1999 which led to the original specifications. Mr. Mills himself stood up with Bill Gates to talk about the two companies' collaboration on these standards. Moreover, Microsoft and IBM have continuously defined additional Web service specifications and publicly tested interoperability. Examples include WSDL, WS-Security, WS-ReliableMessaging, etc.

But let's talk specifics. Take one example: XML Serialization. XML Serialization has been part of the .NET Framework since v1.0. XML Serialization allows a developer to map between instances of objects in a program, and instances of XML documents, very simply. This is a key interop-enabling capability; it means, in part, that a .NET application running on Windows can very simply emit an XML instance document that can then be consumed by any other XML-capable application, running on any other platform. Or vice versa. And it's a foundation for other, higher-level interop capabilities: start with XML Serialization and add in message transport and tools, and you then can get Web services.

This XML support has continued through the versions of .NET across the years, and has been expanded significantly. WCF, shipped in .NET 3.0 in December 2006, added some new capabilities and more mature models – dealing with XML in .NET apps is now both easier and more flexible. The list of open standard protocols supported in WCF includes: HTTP1.1, XML, SOAP1.1, SOAP1.2, WS-Addressing, XOP, MTOM, WS-Security (including x.509, Kerberos, and SAML 1.1 token profiles), WS-Policy, WS-Trust, WS-Coordination, WS-AtomicTransaction, WS-SecureConversation, WSDL 1.1, WS-MetadataExchange, WS-Transfer. Looking forward, in the .NET Framework v3.5 (currently at beta 2), we've included support for REST-style web services, as well as internet syndication protocols like ATOM and RSS, and other protocols like JSON.

Microsoft does support optimized protocols and formats in WCF for Windows-to-Windows communication. IBM provides similar optimizations for their products, as do all SOA platforms.

Claim: Microsoft and IBM have tussled over XML standards.

Microsoft Fact Check: TRUE. But, like the observation that Windows is the OS that underlies Microsoft SOA infrastructure, this is not an interesting part of the story. The picture that is painted by the article would lead one to suspect that a tussle over standards is exclusionary or at least stalling productivity. Coexistence of multiple standards and encouraging market participation in standards evolution is paramount. The OOXML / ODF point in the article is like saying you won't be able to send digital pictures to your mom because there isn't a single standard for graphic presentation. Clearly there are multiple standards for digital images, gif, jpeg, png and so on - which has arguably helped drive utilization. The far more important point is implementation of multiple standards which Microsoft technology does.

OOXML and ODF are about documents. Discussion of documents needs to include all standards, many of which Microsoft supports and has lead. Examples include HTML, CSS, etc.

Claim: "The [Microsoft] MSDN mechanism is a lightweight messaging infrastructure in a message-based environment, whereas IBM delivers a fully functioning infrastructure," [Mr. Mills] said.

Microsoft Fact Check: FALSE, judging from the wide variety of customer case studies that describe how customers utilize .NET and the rest of the Microsoft platform to broadly exploit the benefits of SOA. Microsoft customers use the Microsoft SOA infrastructure to run the core information systems that power their businesses. IBM's Web sites discusses the "entry points" to SOA. Microsoft has equivalent products in Microsoft Office Sharepoint, IIS/ASP.NET, BizTalk Server, WCF, Visual Studio and adaptors for existing applications.

As a side note, MSDN is a developer-outreach program, including a website, a software subscription and licensing offering, a magazine, and more. It's not directly related to SOA infrastructure. IBM has developerWorks, which is similar in intent.

In Closing

Do you want to know just how serious we are about Interop and Performance? Take a look at our .NET Stock Trader application which we modeled after IBM's Trade 6.1 performance application. We've demonstrated our ability to interoperate with applications that run on other platforms including IBM WebSphere applications running on Windows, Linux, or Unix™ . Don't miss this part: the .NET application duplicates the function of the Websphere app, but with far better performance.

[This article was drafted by Dino Chiesa and Steven Martin. It is being posted on both of our respective blogs.]

Interop between ASMX and WCF Services

DotNetInterop — Fri, 20 Jul 2007 04:08:00 GMT

My esteemed colleague, Kevin, is working on a project where the customer has a bunch of ASP.NET (ASMX) web services, and they are looking at migrating to WCF. The customer's goal is simplicity: in stage 1, they'd like to keep their ASMX files, keep the IIS hosting, but add a .svc file to also host the service also in WCF. In stage 2, they'd like to turn off the ASMX endpoint and use only the WCF (.svc) endpoint. All this while keeping the existing client code unchanged: no new code, no re-compile, nothing. A .NET app with a web service stub generated from wsdl.exe should be able to talk to the new service hosted in WCF.

This ought to be pretty simple, right? Kevin found an MSDN article on this topic, How to: Migrate a ASP.NET Web Service Code to the Windows Communication Foundation. This article seems to work, though it doesn't address the specific requirements for Kevin's customer, which included simplicity. Following this article, which by the way is the "right" way to migrate from ASP.NET to WCF, you essentially re-implement everything in WCF. Isn't there a simpler way?

What Kevin did was pretty straightforward and sensible, and he thought it should work. He had a simple .ASMX file, which looked like this:

<%@ WebService Language="C#" CodeBehind="~/App_Code/CommunicationService.cs" Class="CommunicationService" %>

The code-behind file contained the implementation of the service. It looked like this:

[WebService(Namespace = "urn:namespace1")] 
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)] 
public class CommunicationService : System.Web.Services.WebService 
{ 

  public CommunicationService() { } 

  [WebMethod] 
  public ComplexType SendCommunication(CommunicationEntity Request) {...}

  ... 

Kevin created a new .svc file for the WCF stuff:

<%@ ServiceHost Language="C#" CodeBehind="~/App_Code/CommunicationService.cs" Service="CommunicationService" %>

And then he decorated the existing web service implementation class with the appropriate WCF attributes, something like this:

[WebService(Namespace = "urn:namespace1")] 
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)] 
[ServiceContract(Namespace= "urn:namespace1")] 
[XmlSerializerFormat] 
public class CommunicationService : System.Web.Services.WebService 
{

  public CommunicationService() { } 

  [WebMethod] 
  [OperationContract(Action= "SoapAction1Here")] 
  public ComplexType SendCommunication(CommunicationEntity Request) {...} 

  ...

The OperationContract attribute decorates the service method, like WebMethod does for ASP.NET. The XmlSerializerFormat attribute on the service class tells WCF to not use the default DataContract serialization behavior, but instead to use the XmlSerializer. This, ostensibly, to maintain consistency and compatibility with data types that have been decorated with the XmlSerializer attributes, like XmlElement, XmlIgnore, XmlType, and so on.

This all sounds pretty easy right? The problem is that the actual XML messages expected by WCF for this service, were different than the XML messages expected and accepted by ASP.NET. Now it's XML, so, we don't need an exact string match. We need equivalence of the Xml Infoset. That means, element names must be the same, XML namespaces must be the same, but prefixes attached to the namespaces can be different. But we did not have infoset equivalence! The first puzzle was, WHY?

As in many scenarios involving interop between different web services libraries, it's the XML namespaces that cause lots of insidious problems. In this case, the ASP.NET-hosted web service was serializing and de-serializing the CommunicationEntity, using an XML namespace derived from the XmlTypeAttribute attached to that type definition. On the other hand, WCF was serializing the same object using the XML namespace associated to the ServiceContract. If those two xml namespaces are the same, no problem. But in general, they are different, and in this specific case, they were different. The result was that ASMX clients (and by this I mean .NET apps compiled with webservice stubs generated from wsdl.exe in .NET 2.0) would send a serialized XML stream to the WCF service, which was ever so close to the format WCF wanted. But close doesn't cut it, and so WCF would not de-serialize the request, and the app received a null instance for CommunicationEntity.

This stinks.

You would think it would be automatic, but it sure isn't.

The next puzzle was, how to fix this? If you are familiar with the ASP.NET and XmlSerializer model, there are lots of attributes you can apply to types to specify the namespaces to use, the element names to use, and so on. It's very flexible, so flexible that some people find it unfathomable. The WCF serializer is much simpler, which is nice, but on the downside, there is no special attribute that WCF uses to specify the namespace here.

After a little spelunking, I found what I think is a useful hack to work around this problem. It is a custom ServiceHost, which, in the OnOpening() event, changes the MessageParts where the part namespace disagrees with the namespace specified in the XmlTypeAttribute associated to the CLR type for that message part. Huh?

What this means is, now the WCF endpoint and the ASP.NET endpoint serialize and de-serialize the same way, for complex message types. Even better, when you query the WCF service for metadata, you get the "correct" WSDL – with the updated xml namespace.

The ServiceHost uses reflection to inspect the message parts and decide on whether to change the XML namespace or not. Reflection is expensive at runtime, but this happens only once, when the service is opened. So it should have essentially no impact, after startup of the service.

Here's the code:

using System; 
using System.ServiceModel; 
using System.ServiceModel.Description; 

// if your original .svc file was: 
// <%@ ServiceHost Language="C#" Debug="true" Service="CommunicationService" CodeBehind="~/App_Code/CommunicationService.cs" %> 
// 
// then use this, to get the custom service host: 
// <%@ ServiceHost Language="C#" Debug="true" Factory="dinoch.wcf.fixup.EnforceXmlTypeServiceHostFactory" 
//     Service="CommunicationService" CodeBehind="~/App_Code/CommunicationService.cs" %> 
// 
namespace dinoch.wcf.fixup 
{ 
  /// <summary> 
  /// This custom ServiceHost modifies the ServiceDescription to use the "preferred" 
  /// namespaces on message parts. The ServiceDescription is used by the WCF service 
  /// in generating metadata (eg WSDL) and also in driving the serialization behavior. 
  /// 
  /// We do this because the default behavior in WCF when using the XmlSerializer format 
  /// (via the XmlSerializerFormat attribute) is to NOT use the namespace specified in the 
  /// XmlTypeAttribute, if such an attribute is attached to the data type. This means that 
  /// WCF services will not accept the same input messages as ASMX Services, and will not 
  /// generate the same response messages as ASMX services - they will differ by XML namespace. 
  /// 
  /// The way this works - on opening the service, which happens before WSDL is generated and 
  /// before any messages and dispatched to methods, this ServiceHost modifies the ServiceDescription 
  /// of the service. It examines message parts, looks for XmlTypeAttributes attached to the 
  /// types associated to those message parts, and then compares the xml namespace used in the message 
  /// part to the xml namespace specified in the XmlTypeAttribute. If they namespaces differ, then 
  /// this ServiceHost overrides the default namespace, and uses the namespace from the XmlTypeAttribute 
  /// in its stead. 
  /// 
  /// Because this ServiceHost modifies the ServiceDescription before starting up the service, at runtime, 
  /// this technique has the same effect as applying an (imaginary) attribute 
  /// to the service implementation code; the metadata (WSDL) generated from this service 
  /// will indicate the desired namespace for the message parts. 
  /// 
  /// This ServiceHost has no effect on service operations that do not use the XmlSerializer. 
  /// 
  /// There is one catch: running svcutil.exe over the assembly 
  /// (as opposed to the service itself, hosted and running within IIS) will not 
  /// detect the proper namespaces. 
  /// 
  /// To use this service host, use something like this in your .svc file: 
  /// <%@ ServiceHost 
  ///     Language="C#" 
  ///     Debug="true" 
  ///     Factory="dinoch.wcf.fixup.EnforceXmlTypeServiceHostFactory" 
  ///     Service="CommunicationService" 
  ///     CodeBehind="~/App_Code/CommunicationService.cs" 
  /// %> 
  /// 
  /// For some additional background, see http://msdn2.microsoft.com/en-us/library/Aa395224.aspx 
  /// 
  /// </summary> 
  public class EnforceXmlTypeServiceHost : ServiceHost 
  { 
    public EnforceXmlTypeServiceHost(Type t, params Uri[] baseAddresses) : base(t, baseAddresses) { } 

    public EnforceXmlTypeServiceHost(object singletonInstance, params Uri[] baseAddresses) 
      : base(singletonInstance, baseAddresses) { } 

    public EnforceXmlTypeServiceHost() : base() { } 
    protected override void OnOpening() 
    { 
      ServiceEndpointCollection sec = this.Description.Endpoints; 
      foreach (ServiceEndpoint se in sec) 
      { 
        if (se.Address.ToString().EndsWith(".svc")) // eliminate mex endpoints 
        { 
          foreach (OperationDescription opDesc in se.Contract.Operations) 
          { 
            // Are we using XmlSerializer? 
            if (opDesc.Behaviors.Find<XmlSerializerOperationBehavior>() != null) 
            { 
              foreach (MessageDescription messDesc in opDesc.Messages) 
              { 
                for (int i = 0; i < messDesc.Body.Parts.Count; i++) 
                { 
                  MessagePartDescription part = messDesc.Body.Parts[i]; 
                  System.Reflection.MemberInfo info = (System.Reflection.MemberInfo)part.Type; 
                  System.Xml.Serialization.XmlTypeAttribute[] attributes = 
                    (System.Xml.Serialization.XmlTypeAttribute[])
                    info.GetCustomAttributes(typeof(System.Xml.Serialization.XmlTypeAttribute), true); 
                  if ((attributes!= null) && (attributes.Length > 0)) 
                  // check to see if the xml namespace specified to the XmlSerializer via 
                  // the XmlTypeAttribute attribute, disagrees with the xml namespace used 
                  // in the WCF message description. 
                  if (attributes[0].Namespace != part.Namespace) 
                  { 
                    // the namespaces do not agree. we need to swap them. 
                    // this won't work, the property is readonly 
                    // messDesc.Body.Parts[i].Namespace = attributes[0].Namespace; 
                    // Duplicate the old part description, except for the XML namespace 
                    MessagePartDescription newPart = new MessagePartDescription(part.Name, attributes[0].Namespace); 
                    newPart.Index = part.Index; 
                    newPart.MemberInfo = part.MemberInfo; 
                    newPart.Multiple = part.Multiple; 
                    newPart.ProtectionLevel = part.ProtectionLevel; 
                    newPart.Type = part.Type; 
                    // Replace the old part with the new one 
                    messDesc.Body.Parts[i] = newPart; 
                  } 
                } 
              } 
            } 
          } 
        } 
      } 

      base.OnOpening(); 
    } 
  } 

  public class EnforceXmlTypeServiceHostFactory : System.ServiceModel.Activation.ServiceHostFactory 
  { 
    protected override ServiceHost CreateServiceHost(Type serviceType, Uri[] baseAddresses) 
    { 
      return new EnforceXmlTypeServiceHost(serviceType, baseAddresses); 
    } 
  } 
}

And here's how you specify the custom ServiceHost in the .svc file:

<%@ ServiceHost
    Language="C#"
    Factory="dinoch.wcf.fixup.EnforceXmlTypeServiceHostFactory"
    CodeBehind="~/App_Code/CommunicationService.cs"
    Service="CommunicationService" %>

I hope this is useful! The same principle would apply, of course, when migrating from ASMX to WCF on the server side, if you are using a Java-based client, or a web service client on any other platform. So this definitely falls in the arena of interop. If any of you have other requests on interop topics relating to WCF, let me hear 'em!

Cheers,

-Dino

Using Java to program openXML documents

DotNetInterop — Sat, 24 Mar 2007 00:34:00 GMT

I've blogged about this before. Now I See there is a sourceforge project supporting the use of Java to access openXML docs. Brian Jones has the scoop:

http://blogs.msdn.com/brian_jones/archive/2007/03/20/programming-against-the-openxml-formats-with-java.aspx

Open XML Workshop in Redmond, March 13-15

DotNetInterop — Fri, 16 Feb 2007 21:13:00 GMT

Want to learn about Open XML from the source? There's an opportunity coming up March 13-15 on the Microsoft campus in Redmond, Washington. It's called the Open XML Deep Dive developer workshop . If you’re adopting Open XML, and need deep technical training on the details of the format, including the actual markup, architecture, and developer topics, this is your chance to get up to speed fast. There will be other workshops for IT pros and other non-developers, coming soon.

Targeted toward Microsoft's training partners and ISVs, the workshop will cover Open XML architecture, the Open Packaging Convention, the three main schemas (WordprocessingML, SpreadsheetML, and PresentationML), DrawingML, custom XML support, and XSLT applications for Open XML. We’ll have presentations of 60-90 minutes alternating with hands-on lab time, and all attendees will have a workstation with the labs, Office 2007, Windows Vista, and Visual Studio 2005 installed.

There will be C++, Java and C# samples used in the hands-on labs.

Basic understanding of XML and related topics is a prerequisite.

To register for the workshop, contact Wendy Fotre (a-wendyf@microsoft.com). Registration is on a first-come/first-served basis; get your registration in early. There is no cost for the workshop, but attendees are responsible for their own travel arrangements and lodging.

Lessons learned using RSS with .NET 2.0: Simplicity and Speed

DotNetInterop — Tue, 09 Jan 2007 01:52:00 GMT

RSS, .NET 2.0, and Interop

For many people, when they think of Interop with respect to .NET development, they think, first and foremost, Web Services. This is good as far as it goes but...Web services is neither necessary nor sufficient for interop in many cases. RSS is a great example of an XML document protocol that is very useful for interop - in practice, interop between RSS Readers, RSS producers, aggregators, and so on. The document format is RSS, the protocol over which that document is exchanged is often HTTP - the web protocol - but of course it could be *anything*. Any protocol. How does .NET 2.0 support RSS for Interop?

A Digression - IE7 and RSS

I installed IE7 a while back, and I am pretty happy about the RSS support built-in to it. Navigate to an RSS feed and you get a reasonable user-friendly display of the feed. It's not a feed reader or rss manager, but it's nice to not see the angle brackets in the browser, when I just want to browse an RSS doc.

Separately, I wanted an app that could dynamically create a digest of a website - this is a sort of tactical app I needed for something at work. Obviously, it's gotta be RSS or ATOM. I wanted build the app in .NET, and it will run as a web app. This means ASP.NET.

How can I create an RSS doc from within .NET? I know about the RSS.NET project on sourceforge, and I have actually used that DLL in some other projects - RSS.NET was integrated into the Club Web Starter Kit that was created as a usable example of what you could do with ASP.NET 2.0. It was good for what it did, but I had a few problems (this was a while ago) with particular variants of RSS. I made some changes but never bothered to raise a bug on the sourceforce project, basically it was more effort than I was signed up for. In any case it worked, but I was a bit shy of the license, and so on.

Now I Was doing some more RSS stuff, and with my bad developer habits , I ignored what already existed and set off to build my own RSS library. After much effort, many errors, and... oh, wait. Hold on a second. It wasn't much effort after all, and there were no errors and backtracking. Basically it took me 10 minutes to build an RSS library in .NET, and it was very straightforward. Here's the resulting class diagram from Visual Studio 2005:

The process was pretty simple and the same approach should work with any XML dialect. How did I do it?

The tools in the .NET SDK make this simple

The first thing I did was grab a sample RSS document. Then I ran it through the xsd.exe tool, which is part of the .NET 2.0 SDK. The result of that was an XSD document, describing the schema for RSS. Then I ran that through xsd.exe , and the output of that was a .cs source file, modelling the RSS document.

Aside: Lots of developers have this idea that the output of a tool, any tool, is sacrosanct, and cannot be changed. As if the tool knows better than you, what you need. I don't subscribe to that philosophy. The output of one tool is just the input to the next tool, in my opinion.

I took the .cs source file that was kindly generated for me by xsd.exe, and doctored it up, added a few helper methods to the generated classes, tweaked the names of the classes and so on, and I had myself a simple RSS library, perfectly suited to my purposes. (Note: the diagram above is from Visual Studio. It essentially is a graphcial model of the generated class. I didn't actually use any premium feature in Visual Studio to generate the class. I used only the .NET SDK Tools. VS just generated the pretty picture.)

Honestly**, the entire effort on this was about 10 minutes. As I prototyped the app, I saw what I needed and what was missing from the RSS object model, and I tweaked it a little as I went along. A few iterations and the project was done. Boom. I didn't need to examine the license for a third-party tool, I didn't need to understand the model for the third party code. It was small and simple (Really Simple, you might say!). If you want to see the source, see the link below.

[**Honestly, don't you hate when people say, "Honestly,..." As if most of the time they're not being honest, but this time they really are....]

That's what I love about .NET development: Things are surprisingly easy to accomplish. This approach should work with any XML dialect. I am sure there are billions and billions out there, and most of them, if they are simple enough, should be workable with this approach.

What I did here was generate a library that allows any .NET application to produce an RSS document from any content type it can access. I could also have extended this so that it could consume RSS streams, but that was out of scope for my project, so I didn't do it. You should be able to do something similar for any XML Schema, including internal-only schema that you use only for exchanging documents inside the walls of your corporation or organization.

Happy angle brackets to you!

Use Office 2003 XML Schema, no worries

DotNetInterop — Tue, 31 Oct 2006 17:49:00 GMT

In the past I've posted some articles [1, 2] about generating Office 2003 documents from a server-side Java application. I also posted some example code. Of note, the Office 2003 XML Reference Schemas are now also available under the Open Specification Promise. Basically, with the OSP, Microsoft irrevocably promises not to assert any Microsoft Necessary Claims against you for making, using, selling, offering for sale, importing or distributing any implementation to the extent it conforms to a Covered Specification. In other words, Microsoft promises not to bother you if you use the specs covered under OSP. This promise is subject to some conditions, check the OSP website for full details.

-Dino

Peter Galli of eWeek on Microsoft and its work on Interop

DotNetInterop — Mon, 30 Oct 2006 21:33:00 GMT

Editorial excerpt of the Week. From an article mostly about Sender ID, a specification for e-mail authentication, Peter Galli commented on Microsoft's recent work on interop:

“Over the past four months Microsoft has announced a number of key interoperability activities focused on business and technical activities, including the establishment of an Interoperability Customer Executive Council, the Open XML Translator Project, and the strategic relationship with XenSource for the development of technology to provide interoperability between Xen-enabled Linux and Windows Server virtualization.”

(Peter Galli, reporter from eWEEK. “ Microsoft Opens Access to Its Sender ID Spec ”)

I don't know that the past four months have seen a significant increase in interop activity from Microsoft. Maybe an increase in marketing of the activity, but it seems to me that Microsoft has been doing lots of basic interop work for a long time. WS-* efforts, interop plugfests, the original SOAP specification in 2000, even XML standards in 1997.

On a side note, I am looking into a Java/JSP server-side implementation of the Cardspace protocol. This would allow a Windows Vista client browser using Cardspace to authenticate to any Java-based website. Any interest in that? Drop me a line. For more on Cardspace, see Kim Cameron's identity blog.