All About Interop : Java

Geneva supports OASIS WS-Trust, SAML

DotNetInterop — Fri, 07 Nov 2008 19:09:00 GMT

Last week at the PDC 2008, Microsoft released the public beta of “Geneva”.

“Geneva” is three things:

Geneva Server. This is a security token service (STS), as defined in the OASIS WS-Trust specification. This thing issues and transforms claims, manages user access, and enables automated federation.

Geneva Framework. This is a managed (.NET) Framework that helps developers build claims-aware applications and services, that connect to the STS. You can use it to process claims on either side of an authorization transaction (requestor or responder).

Windows CardSpace Geneva. This is just an extension of the CardSpace thing in Windows you know and love today. chances are, you've seen it, but you don't use it. In a nutshell - CardSpace is a set of Windows features and user-interface that lets users navigate access decisions and control how personal information is used. Everyone has multiple claims as part of their identity: you are a student at UW, you are an employee of BigCorp, you are a member in good standing of a particular club, you have received a particular security clearance, You have a bank account with number 4444-444-44 at BigBank, etc. CardSpace lets you decide which of the many claims you can make about your identity, to disclose to a particular service or server. Rather than disclosing "everything" about you to every server or service, you disclose only what you need to disclose for the particular transaction. That is one aspect of the identity model, and CardSpace is the thing in Windows that makes that possible.

CardSpace technology, and actually, the Identity Metasystem concept, is pretty cool. If you haven't looked at it, you should. The problem with CardSpace and more generally, using claims-based access control (CBAC) in an application, has always been that it was impractical. Microsoft delivered a client (CardSpace), but we didn't deliver an STS! And we didn't deliver an easy way for the server to evaluate and verify claims. Therefore developers didn't have an easy way to employ CBAC in their apps. Geneva will change this.

Now, some of you are thinking, "ok, this sounds interesting but I don't know what you are talking about with this WS-Trust thing and 'claims based access control'. And I can understand that. Here's the thing - instead of hand-crafting access-control logic into your application, instead of managing your own user list and access control list, you can "outsource" this job to an STS. The Authorization Manager (sometimes called AzMan) is similar in philosophy, so if you understand the utility of AzMan, then you will get the idea of CBAC. But AzMan is not standards based, not federated, not truly claims-based (it is role based, which is less general than claims-based), and not usable in Web services transactions. Imagine generalizing AzMan and federating it and using only standard protocols, and that's where we're headed with Geneva. You're gonna want to use this.

I said "Geneva is standards based." The key to standards is support by the vendors, in tools, frameworks, servers, and so on. Currently, WCF from Microsoft supports WS-Trust, as does the WSIT from Sun. WebSphere App Server v7.0 announced support for WS-Trust just last month, but I haven't had the time to test it with Geneva. Not sure of other frameworks. The key is, with Geneva, the server is now here, and people can and will start building on this. I think with the release of the Geneva CTP, we'll start to see broader adoption of WS-Trust and standards-based CBAC among frameworks.

One last thing to point out: In the cloud, the Microsoft Services Connector and the .NET Access Control Service, both announced at PDC as well, are built on “Geneva” technology and share the same claims architecture.

More Information:
Kim Cameron’s blog
Vittorio Bertocci’s blog
Get the Geneva beta
Single stop resource on Geneva

A list of “Geneva” sessions recordings at PDC:

code	Title	Presenter(s)	Link
BB11	Identity Roadmap for Software + Services	Kim Cameron and Vittorio Bertocci	http://channel9.msdn.com/pdc2008/BB11/
BB42	Identity: "Geneva" Server and Framework Overview	Stuart Kwan and Caleb Baker	http://channel9.msdn.com/pdc2008/BB42/
BB43	Identity: "Geneva" Deep Dive	Jan Alexander	http://channel9.msdn.com/pdc2008/BB43/
BB44	Identity: Windows CardSpace "Geneva" Under the Hood	Rich Randall	http://channel9.msdn.com/pdc2008/BB44/

Connect WebSphere and Java applications to SQL Server

DotNetInterop — Thu, 06 Nov 2008 00:55:00 GMT

FYI: The MS JDBC Driver is completely tested by IBM for use with WebSphere App Server v6.1 and v7.0.

From the IBM Announcement for WebSphere Application Server v7.0:

WebSphere Application Server V7.0 does not include the WebSphere Connect JDBC driver that had been provided in WebSphere Application Server V6.1. To replace the functionality provided by the WebSphere Connect JDBC driver, WebSphere Application Server V7.0 is tested with the DataDirect Connect for JDBC driver and the Microsoft SQL Server JDBC Driver, which may be obtained from DataDirect Technologies or Microsoft.

Connect your WebSphere/Java apps to SQL Server with this driver. Actually, you can connect *any* Java application to SQL this way - it doesn't have to be running in WebSphere. If you develop in Jetty or Tomcat or JBoss or Geronimo, the MS JDBC Driver will work fine for you.

If you have .NET applications, or Office apps, of course you can connect them to SQL Server also. In this way, SQL Server can serve as a nice interop point between Java and .NET applications, or between Java and MS Office (Let's say... an Excel spreadsheet).

Some notes:

the MS JDBC Driver is a free download.
It is a full type-4 (100% Java) JDBC 3.0 driver, fully tested and compliant with Sun's verification suite.
It connects to SQL Server 2000, 2005 or 2008
It works from any JVM on any OS platform, including Linux
It works with the Express version of SQL Server (which is also a free download) and it works with the Standard and Enterprise versions, too.
There is forum support available for the driver
There's a tech preview of a JDBC 4.0 compliant driver, here.

If this is the sort of thing that interests you, be sure to check the team blog for the JDBC driver.

It just got easier to build REST apps in .NET

DotNetInterop — Thu, 30 Oct 2008 00:48:00 GMT

The REST starter kit that we launched at PDC is now live! :: http://www.msdn.com/wcf/rest .

The WCF REST Starter Kit is a set of features, Visual Studio templates, samples and guidance that help people build REST apps using WCF. WCF, part of the .NET Framework, got some enhancements for REST support in the .NET 3.5 release. This starter kit is a bunch of blueprints and supporting material to make it easier to use what is in the Framework for REST.

Like, blueprints and guidelines for how to do caching, security, error handling, help page support, conditional PUT, push style streaming, type based dispatch and semi-structured XML support. Whoa!

The WCF REST Starter Kit also provides Visual Studio templates for creating REST style services such as an Atom feed service, a REST-RPC hybrid service, Resource singleton and collection services and an Atom Publishing Protocol service. In addition, there are samples, documentation and other guidance that help to use these new capabilities.

The goal is to provide a toolset that simplifies building RESTful services today, and to take feedback from the developer community on the features provided in the WCF REST Starter Kit that will shape future REST capabilities in WCF 4.0.

Eclipse tools for Silverlight and .NET

DotNetInterop — Mon, 13 Oct 2008 23:00:00 GMT

There is now an Eclipse project, Eclipse4SL, to produce tools that work with Microsoft Silverlight. See Neil Hutson's blog post for some details.

For those of you who have day jobs and find it hard to keep up with all the code-names and product names coming out of Microsoft, Silverlight is the name a cross-browser, cross-platform, and cross-device plug-in for rich interactive applications for the Web. Huh? Break it down:

Silverlight is a browser plug-in: Runs in IE, Firefix, Opera, and more.

As a plug-in, Silverlight provides a runtime, which is at it's core, .NET based. It is a lightweight version (subset) of the .NET Framework. When you write declarative code, you'll be using a .NET langauge

Silverlight is cross-platform: Microsoft produces versions to run on Windows and Mac and the Mono project is doing Moonlight for Linux.

Silverlight is for Rich Interactive apps. This means high-def video, audio, hypertext, dynamic data.

Bottom line, Silverlight is like Adobe's Flash, but built on a broad platform (.NET and the CLR) with mainstream language support (C# and VB) as opposed to a relatively narrow platform and runtime requiring ActionScript.

Because Silverlight uses a lightweight NET runtime, if the Eclipse4SL project succeeds, developers will be able to use Eclipse tools to create .NET applications. How about that! You will be able to edit XAML and C# in Eclipse!

Milestone 1 for Eclipse4SL is set for this week: October 14th there'll be a CTP of the tools. Feature complete in December. Nice! I love to see this. Love it, love it, love it.

From the Eclipse4SL website, Interop is a key criterion for success: Increased Interoperability: Eclipse will contain functionality that will help Java Developers build Silverlight applications that work better with Java Web Services using REST, SOAP, JSON and other standards. There's a company called Soyatec, a longtime Eclipse Foundation member, doing the analysis and development work.

For my part, I'm so glad we're getting to the point where we all can work together collaboratively to produce interesting stuff for customers. Next milestone: when a proejct like this doesn't warrant blog posts entitled "interesting times indeed."

Weblogic JMS with .NET

DotNetInterop — Thu, 09 Oct 2008 18:36:00 GMT

This came out last month, I just learned about it from an email from Jesus today.

WebLogic Server 10gR3 now has an officially-supported .NET client for its JMS provider. Very cool. Some of you might be wondering, just what does that mean? It means there is a queue mechanism in the WebLogic Server, for a long time this has been called "Weblogic JMS". And you can connect to that queue subsystem via .NET applications. James Bayer has a blog entry on it, which you should check out.

pet peeve alert! Calling a queue "JMS" is strange and confusing. It would be like calling the Oracle database "Oracle JDBC" or "Oracle ADO.NET". Oracle Database is not the same as JDBC. JDBC is an API. ADO.NET is the name for an API. The database is not the API. The database (whether Oracle, Microsoft SQL Server, IBM DB2, or something else) exposes mutliple API families - some for Java, some for .NET languages, some for C and C++. The programmatic interfaces are distinct from the implementation.

With JMS, Sun and the Java guys have confused the issue. Maybe even on purpose: remember back when JMS was conceived, Sun had aspirations to make Java itself a platform - everything contained within Java as other things like the operating system and the database itself, and to the point, a messaging subsystem, recede into the background. Well, that didn't really happen, despite the huge popularity of Java as a programming language. Operating systems still matter, databases still matter, message queues did not get subsumed into the programming language. Java is a language and a VM and a base class library, but not a platform. Regardless, the side effect of Sun's unfulfilled platform aspirations for Java is that today, people call message and queue subsystems "JMS." Developers speak of JMS topics, JMS queues, JMS messages. But it is clear: just as JDBC is a Sun/Java API for databases, JMS is a Java API for Messaging subsystems.

IBM has always been very clear about this with its MQ product, later called MQSeries and then changed to "WebSphere MQ".   (For this post I'll just call it MQ.)   MQ is the implementation, and MQ exposes a bunch of different APIs - for C, for .NET, for Java, and I'm sure, for other languages. In what looks like a bit of schizophrenia, IBM also has message service within WebSphere Application Server, which is called WebSphere JMS if I am not mistaken, and which... you guessed it, exposes more than just a JMS programming interface.

For some reason, the queue infrastructure for some products has been called "JMS" as if that is the only API that has ever mattered. Seems like a result of shortsighted and wrongheaded product planning to me. And it's darned confusing, to boot.

A separate issue is that JMS is not merely an API! JMS actually injects some message protocol behavior, specifically the RFH2 header formatting stuff, which makes for tricky interop with other, non-Java applications through queue subsystems. Designing JMS this way presented a big, unnecessary obstacle to interop, and I've blogged on it previously.   IBM has had to introduce new APIs for MQ and its other queue subsystems, specifically to work around this interop barrier. I don't know a better way to solve it, than the way IBM did. But make a note to yourself - if you ever design a queue system or queue API, keep the API separate from the application protocol and message formatting.

Check it out!

Using the WSDL-First design approach with WCF services

DotNetInterop — Fri, 05 Sep 2008 17:07:00 GMT

Couple of helpful posts from the archives for you.

If you are care about interop, then you probably want to use a contract-first or WSDL-first design for your web services. In other words, you will not simply decorate server-side code annotations (in Java) or attributes (in .NET) and generate WSDL dynamically from that running code. That approach works in demonstrations, but it does not lend itself to good interoperability in the long run.

So you start with WSDL. Since you are WCF-savvy, you are confining yourself to the subset of XML Schema that is friendly to WCF. The restriction isn't too onerous, though, especially as it improves interoperability significantly, and also simplifies your life since you don't have to read and understand all of the XML Schema spec! Ok, so following those rules, you write the WSDL. And then you generate from that WSDL the server-side stub code, and the client-side proxy code, using the web services toolkits of your choice. On the .NET side, that toolkit is obviously WCF (and the tool itself is svcutil.exe); on the Java side, maybe the JAXWS reference implementation, or AXIS2, and so on. But the trick is, the generated code on the .NET side is muy bueno if you write your WSDL following a few additional simple rules, which are not mentioned in the above linked doc.

For example, if you want to pass a collection in your interface, and you will be using WCF on one or both ends of the wire, then you want to use an explicit complexType for the array in the WSDL/XSD. This is not required by WSI Basic Profile, nor is it mentioned anywhere in the official WCF doc as far as I know. Nonetheless, follow this rule for best results. Do not define your arrays with an element and a @maxOccurs="unbounded" attribute in your schema; if you do, you will get the XmlSerializer with WCF, rather than the DataContract serializer, and that is probably not what you want. More on this tip here.

Also, use only one part in each wsdl:message (to comply with WS-I Basic Profile) and set the message part attribute @name = "parameters" to make the messages document/literal wrapped, versus bare. If you're not sure, you want wrapped.

Finally, another tip: when using collections in service interfaces (or more specifically, data contracts), you can use the svcutil.exe tool to specify that collections be represented in the generated .NET code as one of the Generic collection types, rather than a vanilla array. This is done with the /collectionType option on the svcutil.exe command line. See this post for more detail on that idea.

I think there might be a similar codegen hint on other web services toolkits but I could not find the right switch on AXIS2.

Web services interop with WCF - Relevant Forum Questions

DotNetInterop — Thu, 04 Sep 2008 17:45:00 GMT

I was just browsing the WCF forum on MSDN. Here's a summary of the recent interesting threads dealing with interop:

Check it out.

HOW TO: Configure the jakarta Tomcat connector for IIS (serve JSP from IIS)

DotNetInterop — Tue, 12 Aug 2008 17:57:00 GMT

Steve Sfartz offers a tip for configuring the Tomcat connector on IIS, to allow IIS and Tomcat to run from the same domain. This will allow numerous interop scenarios, one of which is to allow IIS to serve Silverlight apps while the Tomcat engine can serve JSP and Servlets.

Changing of the Guard: AXIS out, JAXWS in

DotNetInterop — Wed, 30 Jul 2008 19:03:00 GMT

I have come to a conclusion: I no longer want to deal with the hassle that has become Apache AXIS.

Back in the day, when interconnecting Java to .NET was still novel, I started using Apache SOAP to show how interop between .NET and JAva was possible. I was looking for something simple and easy, something that would allow interconnection without a ton of pain. Apache SOAP was that thing. But Apache SOAP was brittle. It had scads of problems with xsi:type and SOAP Section 4 encoding. I needed something more flexible, something easier.

Then came Glue from The Mind Electric. Glue was the easier, more flexible web services stack. It was simple. It worked. It was actively maintained. Graham Glass came regularly to Redmond to make sure the interop with .NET was good.

Then TME got bought by WebMethods, and Glue disappeared. I needed to pick a new Java stack. When AXIS came out, I switched, looking for better tools, and something that would be reliable, maintained longer term.

At that time, I looked at Sun's web services stuff, but it was always bundled with a Web server or Open Office or a Sun Ray or something else from Sun I didn't want. It was like Scott McNealy's proverbial furball. I just wanted a working web services stack, I didn't want to buy a Solaris Server. I didn't want a furball. So I went with AXIS. I could use it on Tomcat, on JBoss, or on Jetty. I could use it client or server side. I wrote a few pages on the AXIS wiki, specifically around how to get it to work with .NET. I sat on the AXIS users mail list, responded to interop questions.

AXIS (Version 1) worked great for a long time, still does in fact. It has connected to many different endpoints for me - Excel Services, WCF endpoints, .NET, SQL Server, and many other pieces. It was neat and clean; the tool options made sense. The code it generated was pretty clean.

But AXIS1 is no longer being maintained. Any new standards (like MTOM - heh, new since 2005) will not be supported in AXIS1. Also, it didn't work so well when it came to connecting to Exchange. AXIS2 likewise. And that allowed me to step back and re-evaluate. AXIS2 is current, but has become very large, the doc is poor, the support is invisible, the generated code smells, and the seams are everywhere. Fifty nine jar files? Really? Do I need this hassle? With JAX-WS, do I need AXIS any longer? I think not.

I'm switching to JAX-WS.

This is kinda backwards, because my goal is to show customers that they can in fact connect disparate systems together. Given that goal, my own interests should rank near the bottom in the list of criteria for selecting a web services library. It should be - what are customers using? What do customers want to see? But I honestly cannot believe customers will continue to put up with the furball that AXIS2 has become.

And if I am in a position to make a recommendation, I will recommend JAX-WS.

It works. There is a single runtime jar file. The tools are better. The interop is good. It's fast. I'm happy.

I'm changing my decision but I haven't changed my criteria. I still want something that just works, simply. Basically I want the closest philosophoical equivalent of .NET, but in Java. AXIS used to be that. Now, JAX-WS fits that bill.

I still love .NET and WCF. But if I need to connect to .NET from Java, I will use JAX-WS.

Connecting to Exchange using JAX-WS, part 1

DotNetInterop — Tue, 29 Jul 2008 21:52:00 GMT

Yes, it is possible to Connect to Exchange using JAX-WS

The fear and trepidation I had when considering the effort to connect a Java application to Exchange Server 2007 was well founded. It wasn't easy. I'm going to spoil the ending and tell you, yes, it is possible to connect Java apps to Exchange Server via Web services, and you can do it securely and with good performance and reliability. But it took some starts and stops to make it happen.

Warning: Old Guy Reminiscing Ahead

Back a few years, I was a technical marketing guy for the Microsoft platform, and my main role Was talking to customers. One of the things I talked with customers about often, was interop. Pretty much every customer I talked to already had a signficant investment in information technology, they had a strategy that was in place and working, they had systems in place, working. They had off-the-shelf apps, they had some custom-built apps, and they had skillz. They were looking to make adjustments, to evolve their systems, to improve. It reminded me of the principles of Agile Development.

Do you remember XP? Do you remember how trendy that was, like 3 or 4 years ago. Everyone was talking about XP, everyone was looking to apply XP principles into their approaches. That seems a long time ago. XP is so 2004.

The approach that companies took with their information systems architecture was something like what developers did in a different domain with XP. Lots of little iterations, constant improvement, and steadily you get closer to your goal.

I remember talking to customers about web services technology, about using standard XML-based protocols to enable better connections between disparate information systems. Yes, Virginia, this was actually a theme of my conversations back only a few years ago. Now it seems so obvious. Now that everything speaks XML and interconnection is fairly easy, the wisdom of using standard protocols to gain efficiencies in IT is apparent. But only a short time ago this issue was not so clearly understood.

To make my case that web services and XML based protocols were the future, I'd show customers the investments that Microsoft was making into them. I'd tell them about all the standards was Microsoft working on. But the thing that customers wanted was hard evidence - not standards or papers. They wanted proof of XML usage in Microsoft products. So I'd talk about all the XML-based protocols that were already in the existing Microsoft systems they had: the web services libraries in the .NET Framework, the tooling in Visual Studio (but these were fairly obvious). The XML shredding capability that was introduced in SQL 2000 and improved in SQL 2005. Actually with the FOR XML clause you can do some pretty nifty things with the data that is already in your database. The MSXML component - now sort of just accepted as part of the landscape, a key, if almost forgotten enabler of AJAX - but 6 or 7 years ago it was undergoing rapid development.

These are all pretty good examples of Microsoft's investment in XML and WS-*. But one of the things that really surprised people was the web services capabilities we were introducing into Office (with WordML and ExcelML and so on), and the Exchange Server.

Exchange Server 2007 and Web services

Exchange Server 2003 had some limited web services capabilities. With Exchange Server 2007, Microsoft delivered an email and collab server that exposed all of its features through standard web services interfaces. Today if you have an Exchange Server, you can point your browser to http://my.exchange.server.com/ews/Exchange.asmx and get a WSDL for the server. Wow. That is pretty cool.

Standards are standards, and complying to a standard doesn't guarantee interop. If you are designing a web services interface, you can get better interop by keeping your WSDL fairly simple, and constraining your interface definitions to avoid some of the more esoteric nooks and crannies of the XML Schema spec. This is advice we've all been following for years. It works.

The problem with that advice is that it does not work if the information systems at the end of the wire are not simple, or if the requirements are not simple. For example, if you want WS-Addressing or delegated security, things can get complicated. And, similarly, Exchange Server is not simple. It stores lots of different types of things (email messages, contacts, tasks, notes, calendar entries, requests, resent items, recurrences, mime content type, cancellations, suggestions for new times, and so on). And then it compounds that complexity by adding in an unstructured aspect - document attachments and other unstructured data. Then there is stuff like impersonation, delegation (your admin assistant can update your calendar), out-of-office messages, and so on. It's pretty rich.

With all that capability, necessarily, the interface to Exchange is not going to be simple. Look at the user interface in Outlook - it is pretty rich. And the same is true with the network interface on the wire, described in the WSDL.

As a result, the WSDL for Exchange is complicated. It's big. It's got lots of nested structures. There are unions and restrictions and headers and bodies. It breaks all the rules for designing schema with interop in mind.

Anyway, who needs web services, right? We can use WebDAV!

Interop and the Exchange Server web services interface

First decision: which web services library to use? My first thought was to use the tried-and-true Apache AXIS 1.4 library that I had been depending on since 2006. It wasn't THAT old, and it worked for all the other interop tasks I had tried it on. It connected to .NET systems using ASMX or WCF. It was simple. I knew the tools.

My first step was to grab the Exchange.wsdl, the types.xsd and the message.xsd. I knew that I needed to add a service definition to the end of the Exchange.wsdl file, because most WSDL compilers want it:

   <wsdl:service name="ExchangeServices">
     <wsdl:port name="ExchangeServicePort" binding="tns:ExchangeServiceBinding">
       <soap:address location="https://myexchange.company.com/EWS/Exchange.asmx"/>
     </wsdl:port>
   </wsdl:service>

</wsdl:definitions>

But very early the work with AXIS 1.4 was hitting roadblocks. That library generated source code that did not compile. It choked on quite a few things. Other people have walked this path but custom modifications to the generated code did not sound like fun to me. I quickly decided that I'd need to move to something else.

My next idea was to move to AXIS2, which, coincidentally, is also at version 1.4. Now why they chose to call it "AXIS2 1.4" instead of "AXIS 2.4" is beyond me. There is a significant breakage in back-compatability with the move to AXIS to AXIS2, but still the version numbers are very confusing.

AXIS2 introduces a number of changes over AXIS (1). The AXIS guys improved the XML schema support, for one. Also, they made the XML serialization engine pluggable, so you can use the built-in simple library (called ADB), or you can switch to use JiBX or XMLBeans.

As with AXIS1, I ran the Exchange WSDL through the AXIS2 tools to generate the client-side proxies. This time, the code actually compiled. I was pleased to see that there was just one giant Java file generated for the entire Exchange interface. In AXIS 1.4, there was a whole series of files, 447 of em. Too many, thought I. But AXIS2 v1.4 generated just one file. Cool!

And it was a big one. Opening the file in emacs - emacs asked me, "are you sure you want to open this file? it's really big!" Seriously. And compiling that single file with javac, I actually got a heap overflow. Wow! That's a big file. I just used the -X heap magic incantations on that file and I was able to compile it.

I wrote up a simple proof of concept Java class that used the generated AXIS stub classes to connect to Exchange Server. When I ran the code, I got a failure: "required attribute localTraversal is null" I had no idea where that was leading me. I searched in the WSDL and schema for Exchange but couldn't find any elements or attributes by that name. I considered ripping open the AXIS2 source code to try to troubleshoot it. Wasn't real excited about that prospect. I had no idea where that was leading me.

My next step was to try the XMLbeans serialization option in AXIS 2. With this, I was back to generating ~ 400 Java files. But I didn't care, as long as the interop worked. And I had to update my client application code, because switching from ADB to XMLBeans necessitates a change in the programming model. I didn't care, though. Interop! (The things I do for interop!)

The AXIS2 + XMLbeans combo was looking promising, but also failed, this time with

java.lang.ClassCastException: org.apache.xmlbeans.impl.values.XmlComplexContentImpl cannot be cast to com.microsoft.schemas.exchange.services._2006.types.Folder
IdType

This is apparently a common error, if there is a duplication in classes. I don't think I had that. I keep a clean workspace. Some of the XMLBeans people suggested on forums to upgrade to the later version of XMLBeans. There was no better suggestion I found. The AXIS2 library bundles XMLBeans 2.3.0, but XMLBeans standalone was at v2.4.0. So I downloaded 2.4.0, and tried to manually compile the schema with the scomp tool.

c:\apache\xmlbeans-2.3.0\bin\scomp -out GeneratedTypes.jar contract\ExchangeServices.wsdl  -compiler c:\sunjdk\bin\javac.exe

But, then, another runtime exception:

Exception in thread "main" java.lang.NoClassDefFoundError: com/microsoft/schemas/exchange/services/_2006/types/FolderIdType
        at ExchangeClient.Run(ExchangeClient.java:161)
        at ExchangeClient.main(ExchangeClient.java:217)
Caused by: java.lang.ClassNotFoundException: com.microsoft.schemas.exchange.services._2006.types.FolderIdType
        at java.net.URLClassLoader$1.run(Unknown Source)

It was one problem after another. Also, I had found the documentation and examples to be completely lacking for AXIS2. I couldn't find anyplace that gave me any hints. Why was it so hard to find information on this thing? And why did I have to use Class.Factory.Create(); instead of just calling new Class(). How dumb is this? And finally, there were 59 JAR files I needed to reference for AXIS2. That's what the doc told me. Commons-logging, xmlbeans, axis-api, axis-rt, corba, neethi, axiom, anogen, and on and on. I don't even know what those things are. The whole thing felt very... messy. I've already experienced JAR Hell and I don't want to go back there. I'm sure each one of those jars is required, but AXIS2 *could* be a little more seamless. As it is, it feels like a science project.

Ok, time to switch horses. The AXIS Stuff was not giving me joy. My next attempt would be to use JAXWS.

With JAXWS, there is one runtime JAR. One. The code-gen produces... a JAR. How refreshing! I don't need 400 class files. I can just reference a JAR. (Yes, I know there are 400 classes inside the jar. I don't care. It's a single jar). Hmm, someone ought to tell the AXIS project about this upstart JAXWS. The stub classes get generated and compile nicely. (small victory, cross fingers and hope for more). I am starting to like JAXWS.

The programming model exposed by JAXWS-generated client-side classes is a little different than what I was used to. Rather than using return values from a stub method for the response, JAXWS has this idea of using the javax.xml.ws.Holder class for return values. Kinda odd, but... whatever. Maybe there is a way to change the client-side programming model, but I wasn't going to worry about that right then.

There is some stuff in the Exchange interface (WSDL) having to do with impersonation and SerializedSecurityContext in each request on the EWS Service port, which is not useful. I'm using neither of those things, yet they are included in every Web services stub method. Passing null for these parameters does not satisfy, I get an error saying that I cannot set that to xsi:nil unless the schema has nillable='true'. I tried fiddling with the WSDL.. to set it to nillable in the right spots... That didn't work, so I just removed that stuff from the WSDL completely, from every wsdl:message in the WSDL file:

        <wsdl:message name="GetFolderSoapIn">
                <wsdl:part name="request" element="tns:GetFolder" />
<!-- dinoch - change for JAXWS
                <wsdl:part name="Impersonation" element="t:ExchangeImpersonation"/>
                <wsdl:part name="S2SAuth" element="t:SerializedSecurityContext"/>
                <wsdl:part name="MailboxCulture" element="t:MailboxCulture"/>
-->
                <wsdl:part name="RequestVersion" element="t:RequestServerVersion"/>
        </wsdl:message>

and also from every soap:operation:

                <wsdl:operation name="GetItem">
                        <soap:operation soapAction="http://schemas.microsoft.com/exchange/services/2006/messages/GetItem" />
                        <wsdl:input>
<!--
                                <soap:header message="tns:GetItemSoapIn" part="Impersonation" use="literal"/>
                                <soap:header message="tns:GetItemSoapIn" part="S2SAuth" use="literal"/>
                                <soap:header message="tns:GetItemSoapIn" part="MailboxCulture" use="literal"/>
-->
                                <soap:header message="tns:GetItemSoapIn" part="RequestVersion" use="literal"/>
                                <soap:body parts="request" use="literal" />
                        </wsdl:input>
                        <wsdl:output>
                                <soap:body parts="GetItemResult" use="literal" />
                                <soap:header message="tns:GetItemSoapOut" part="ServerVersion" use="literal"/>
                        </wsdl:output>
                </wsdl:operation>
                <wsdl:operation name="CreateItem">

Ok, I was getting somewhere. With this change I was able to send a FindItem message over the wire to Exchange, and get message IDs back. Very cool. The approach I was taking was to use findItem to get the item IDs, and then use GetItem on the Ids to get the full content. The FindItem() was working flawlessly, but the GetItem() was not. I was not getting the results I expected. No exceptions, but my data was not coming back.

To troubleshoot this, I used Fiddler2 to see the HTTP messages back and forth between Java and Exchange. Because the communication was using SSL for encryption, Fiddler2 does not work. Some people have used Charles, a similar web debugging proxy. But I already had Fiddler, so... With the fiddler tool I could see exactly why the GetItem message was not giving me joy. (By the way, if you use Fiddler to decrypt SSL traffic, it's necessary to specify to the Java app that it should accept any SSL traffic - basically disable cert validation in the application. Only do this for testing purposes!!)

Correct flow from a .NET app to Exchange:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetItem xmlns="http://schemas.microsoft.com/exchange/services/2006/messages">
      <ItemShape>
        <BaseShape xmlns="http://schemas.microsoft.com/exchange/services/2006/types">Default</BaseShape>
        <AdditionalProperties xmlns="http://schemas.microsoft.com/exchange/services/2006/types">
          <FieldURI FieldURI="item:Subject" />
          <FieldURI FieldURI="item:Body" />
          <FieldURI FieldURI="item:Categories" />
          <ExtendedFieldURI PropertySetId="0006200E-0000-0000-C000-000000000046" PropertyId="35584" PropertyType="Integer" />
          <ExtendedFieldURI PropertyTag="0x10F3" PropertyType="String" />
          <ExtendedFieldURI PropertyTag="0x3008" PropertyType="SystemTime" />
        </AdditionalProperties>
      </ItemShape>
      <ItemIds>
        <ItemId Id="AAAZAERpbm8uQ2hpZXNhQG1pY3Jvc29mdC5jb20ARgAAAAAALBHixAbt0hGt6gCAX5/eGgcAd75VstDi0RGt3QCAX5/eGgAAAnopngAAEKx2OMNuVUuHEMvofVOxpwANFT4iewAA" ChangeKey="AAAAABQAAAAdMskaJAAzTpK1s6mN4vCnAAhyCw==" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />

Broken flow from a JAXWS app to Exchange:

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
  <S:Header>
    <ns2:RequestServerVersion xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types" Version="Exchange2007_SP1" />
  </S:Header>
  <S:Body>
    <GetItem xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types">
      <ItemShape>
        <ns2:BaseShape>Default</ns2:BaseShape>
        <ns2:AdditionalProperties>
          <ns2:Path xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:PathToUnindexedFieldType" FieldURI="item:Subject" />
          <ns2:Path xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:PathToUnindexedFieldType" FieldURI="item:Body" />
          <ns2:Path xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns2:PathToUnindexedFieldType" FieldURI="item:Categories" />
        </ns2:AdditionalProperties>
      </ItemShape>
      <ItemIds>
        <ns2:ItemId ChangeKey="AAAAABQAAAAdMskaJAAzTpK1s6mN4vCnAAhyCw==" Id="AAMkAGUwNmJhYWVjLTI1ZTgtMTFkMy05MDc5LTAwODA1ZjMxZjgyNgBGAAAAAAAsEeLEBu3SEa3qAIBfn94aBwB3vlWy0OLREa3dAIBfn94aAAACeimeAAAQrHY4w25VS4cQy+h9U7GnAA0VPiJ7AAA=" />

Exchange wasn't sending me the data I expected, because the JAXWS request message was not what Exchange expected it to be. With this understanding, I looked into the contract. For the element in question, the schema used a ref="type" structure, and this was the only use of the ref in the entire contract. Clearly causing a problem; JAXWS was formatting it strangely. I modified it to eliminate the ref, just as a test.

  <!-- Arrays of paths -->
  <xs:complexType name="NonEmptyArrayOfPathsToElementType">
    <!-- using choice here so that proxy generator will not flatten the array and thus lose the
                item element names -->
    <xs:choice maxOccurs ="unbounded">
<!-- dinoch - change for jaxws -->
  <xs:element name="Path" type="t:BasePathToElementType"/>
  <xs:element name="FieldURI" type="t:PathToUnindexedFieldType" />
  <xs:element name="IndexedFieldURI" type="t:PathToIndexedFieldType" />
  <xs:element name="ExtendedFieldURI" type="t:PathToExtendedFieldType"/>

<!--
      <xs:element ref="t:Path"/>
-->

    </xs:choice>
  </xs:complexType>

After re-generating the client-side stubs, I was able to get the GetItem message to work nicely. Here's the message from JAXWS after the change in schema:

<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
  <S:Header>
    <ns2:RequestServerVersion xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types" Version="Exchange2007_SP1" />
  </S:Header>
  <S:Body>
    <GetItem xmlns="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:ns2="http://schemas.microsoft.com/exchange/services/2006/types">
      <ItemShape>
        <ns2:BaseShape>Default</ns2:BaseShape>
        <ns2:AdditionalProperties>
          <ns2:FieldURI FieldURI="item:Subject" />
          <ns2:FieldURI FieldURI="item:Body" />
          <ns2:FieldURI FieldURI="item:Categories" />
        </ns2:AdditionalProperties>
      </ItemShape>
      <ItemIds>

After that, it was a matter of writing up the code to use the jaxws-generated stubs. It all works nicely now. It's pretty darn fast, too!

Let me summarize what I did to get Java to talk to Exchange:

Modified the exchange.wsdl to include a service definition, and to remove all occurrences of impersonation, S2SAuth, and MailboxCulture from each wsdl:message.
Modified the types.xsd to eliminate the ref="t:Path", and replace it with actual elements
Use Fiddler2 to trace and debug the SOAP messages.

The bottom line is that web services interop is possible even with complicated schema and interfaces, if you are willing to get your hands dirty on the schema, and if you choose the right tools.

In part 2 of this story, I'll show you some of the application code that uses the JAXWS-generated client classes, and I'll pack up the source code for everything too.

Great resources:

Cheers,
-Dino

Connecting Java to Exchange over WebDAV, with Apache HttpClient

DotNetInterop — Mon, 21 Jul 2008 19:47:00 GMT

Earlier I mentioned a Java Exchange Connector I had seen, and in that post I also said that I had some code that connected from Java to Exchange Server via WebDAV. Some people have asked for the code, so here it is.

I use a set of Apache libraries to make this possible: Apache Commons HttpClient for the Java SSL client capability, and Apache Jakarta Slide for the WebDAV piece. As you may have heard, Slide was discontinued as a project a while ago. So maybe you are thinking, Slide? What good is that code if it depends on an abandoned Apache library?. A reasonable point. Honestly though, the Slide stuff is really not that critical for this scenario. Basically, WebDAV is a standard for formatting queries that get sent over HTTP. But in my case, I am just cons-ing up strings that contain the queries. It would be pretty easy to factor out the WebDAV/Slide stuff, I think. I didn't bother to do it, only because I wrote the code a long time ago, well before Slide was discontinued, and I didn't really feel like investing the time to re-factor it now.

The httpclient and slide jars also drag in the commons-logging and codec jars from Apache.

I have the full working proof-of-concept code attached to this post as a zipfile. It is a JSP-based app that runs in Tomcat or Jetty or your favorite servlet container. Requires JDK 1.5 to build it. You can have a look and try it out yourself. Here's the code I use to send out a query to Exchange.

  public org.w3c.dom.Document search(String urlString, String request, int depth, String range) 
    throws Exception
  {
    Document doc= null;

    SearchMethod method = new SearchMethod(urlString, request); 
    try {
      //method.setRequestHeader("Content-type", "text/xml");
      method.setRequestHeader("depth", ""+depth) ;
      method.setRequestHeader("Translate", "f");

      // must set Content-Length explicitly.  Why doesn't the SearchMethod do this?  Who knows. . .
      method.setRequestHeader("Content-Length", String.valueOf(request.length()));

      if ((range!=null) && (range !=""))
          method.setRequestHeader("Range", range);

      int rc = httpclient.executeMethod(method);

      doc = method.getResponseDocument();
    }

    finally {
      // release any connection resources used by the method
      method.releaseConnection();
    }            

    return doc;
  }

Ok, that's just some boilerplate WebDAV stuff. Get a request, set the HTTP headers, send out the request. The magic really is in the request itself. This is what a WebDAV request looks like on the wire:

SEARCH /exchange/dinoch/Inbox HTTP/1.1 
depth: 1 
Translate: f 
Content-Length: 370
Range: rows =0-3
Content-Type: text/xml; charset=utf-8
User-Agent: Jakarta Commons-HttpClient/3.1
Host: mail.microsoft.com
Cookie: $Version=0; sessionid=40e73022-93d3-4739-99ff-bf60fd60fcee; $Path =/
Cookie: $Version=0; cadata=1 gGfmglWDAmcLGmstqx3kSpuecb1BVBmjFwiD0IIem2hm6IsfCLr7DSo63ncgdM7xNi3E5A==; $Path= /

<searchrequest xmlns='DAV:' >
  <sql> 
    SELECT "DAV:id", "DAV:href" , "urn:schemas:httpmail:subject", "urn:schemas: httpmail:from", "urn:schemas:httpmail:datereceived" 
    FROM SCOPE('shallow traversal of "https://mail.microsoft.com/exchange/dinoch/Inbox"') 
    WHERE "DAV:ishidden"=False AND "DAV:isfolder"=False 
  </sql>
</searchrequest>

In that request, I'm searching on my Exchange Inbox. I can also search on any folder: Calendar, Contacts, Notes, Tasks, any mail folder, and so on. The schema are different for different item types, so ya gotta be careful there. Anyway, in the above request, I search on the first 4 rows, and I ask for the fields: subject, id, href, from, and datereceived.

In the proof of concept, I create these queries using a file-based template, one for each type of query. I have a template for the inbox query, another template for the query of the tasks folder, and so on. This is the template for the Tasks query, for example:

<searchrequest  xmlns='DAV:'>
  <sql>
    SELECT 
       "DAV:href", 
       "DAV:displayname", 
       "DAV:getlastmodified",
       "urn:schemas:httpmail:subject",
       "urn:schemas:httpmail:textdescription",
       "http://schemas.microsoft.com/mapi/id/{00062008-0000-0000-C000-000000000046}/0x00008517" as DueDate2,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8101" AS Status,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8102" AS PercentComplete,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8104" AS StartDate,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8105" AS DueDate,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x810f" AS DateCompleted,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x811c" AS IsComplete,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8113" AS State,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8110" AS ActualEffort,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8111" AS EstimatedEffort,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x8518" AS Mode,
       "http://schemas.microsoft.com/mapi/id/{00062003-0000-0000-C000-000000000046}/0x811f" AS Owner

    FROM SCOPE('shallow traversal of  "##FOLDERPATH##"')
    WHERE "DAV:ishidden"=False AND "DAV:isfolder"=False  
    ORDER BY IsComplete, DueDate ASC 

 </sql>
</searchrequest>

One of the trickiest areas for me was just figuring out the MAPI schema for all the various fields that might be present in a given document type (like task, message, meeting request, contact, note, and so on) that can be stored by Exchange. The schema are not intuitive, nor did I find the documentation to be easily accessible. As you can see, there are some magic incantations above for the task document type.

Just so everyone is clear: The schema used is the same, regardless of the type of client. I am writing a Java app here. But you could use these same queries from a .NET app, a VBScript app, or a PHP app, or Ruby or whatever. The Exchange Server responds to the on-the-wire protocol, and doesn't care about the client you use.

Another key part is logging into the Exchange server. This is the method I use to login.

  public void login (String destUrl, String username, String password) 
    throws Exception
  {
    Username= username;

    String[] urlParts=destUrl.split("/");
    String server= urlParts[2]; 
    String authDllPath= "/exchweb/bin/auth/owaauth.dll";
    String AuthUrl= Protocol + "://" + server + authDllPath;

    //System.setProperty("javax.net.debug", "all");  // too much information
    //System.setProperty("javax.net.debug", "ssl");

    // the class ionic.ssl.SSLSocketFactoryImpl must be available for the classloader,
    // eg, on \lib\ext
    java.security.Security.setProperty("ssl.SocketFactory.provider",
                                       "ionic.ssl.SSLSocketFactoryImpl");

    if (!server.equals(Servername)) 
      throw new Exception ("Cross-site redirection attempt.");

    PostMethod method = new PostMethod(AuthUrl);

    try {

      //System.out.println("DavConnection: Logging in with u/p: " + username + " | " + password); 

      method.setFollowRedirects(false); // false == default

      NameValuePair[] data = {
        new NameValuePair("destination", destUrl),
        new NameValuePair("username", username),
        new NameValuePair("password", password)
      };
      method.setRequestBody(data);

      int rc = httpclient.executeMethod(method);
      int stat= method.getStatusLine().getStatusCode(); 
        
      if (stat== 302) {  // 302 = Moved Temporarily (this is success)

        // The response form Exch2003 says "Moved Temporarily" but 
        // if we are only logging in, we don't need to follow this link. 
        // The apache commons httpclient runtime will log an INFO
        // saying "302 received but followRedirects==false.  This is OK. 

        // String redirectLocation= null; 
        // Header locationHeader = method.getResponseHeader("location");
        // if (locationHeader != null) 
        // redirectLocation = locationHeader.getValue();
        // System.out.println("Redirect to: " + redirectLocation);

      }
      else 
        throw new Exception("Unexpected HTTP status code during login (" + stat +")");
    } 
    finally {
      // release any connection resources used by the method
      method.releaseConnection();
    }            
  }

This login method is exposed on a DavConnection object, the class/type that wraps all the interaction with Exchange server. After login, of course, I set the DavConnection into the http session. It gets returned as a cookie to the browser. Thereafter when the browser connects and presents its cookie, we have te active Exchange connection. We only login once.

At this point I want to talk about cookies. My favorite kind are oatmeal cookies, which I love to make. When I make 'em, I tend to eat them, all of them. So I don't make 'em that often. My second-favorite kind of cookies are HTTP Cookies. There are two sets of HTTP cookies in this application scenario. One set of cookies links the browser to the JSP app. Another set of cookies links the JSP app to Exchange Server.

The second set, the cookies that Exchange sends back to the client that is authenticating (in this case our JSP app) must be retained and presented back to the server on subsequent queries. Nicely for us, the Apache HttpClient library does that automagically for us. The DavConnection class embeds an HttpClient instance as a member; this HttpClient instance retains the cookies that keep the conversation with Exchange alive. mmm-kay?

Let's see what else? I guess you will want to have a look at a sample response from an Exchange WebDAV query. This is one response I got.

<a:multistatus xmlns:b="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/" xmlns:d="urn:schemas:httpmail:" xmlns:c="xml:" xmlns:a="DAV:">
  <a:contentrange>0-9</a:contentrange>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/RE:%20Developing%20and%20debugging%20without%20admin%20rights.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47Tx0AAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/RE:%20Developing%20and%20debugging%20without%20admin%20rights.EML</a:href>
        <d:subject>RE: Developing and debugging without admin rights</d:subject>
        <d:from>"Alun Jones" &lt;alunj@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T21:26:24.409Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/MarieHu%20team%20meeting:%20FY06%20plans%20%26%20budget.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxGAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/MarieHu%20team%20meeting:%20FY06%20plans%20%26%20budget.EML</a:href>
        <d:subject>MarieHu team meeting: FY06 plans &amp; budget</d:subject>
        <d:from>"Marie Huwe" &lt;mariehu@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T21:23:44.107Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Repeater%20control%20%3CItemTemplate%3E%20question.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxMAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Repeater%20control%20%3CItemTemplate%3E%20question.EML</a:href>
        <d:subject>Repeater control &lt;ItemTemplate&gt; question</d:subject>
        <d:from>"Mike Burdick" &lt;mikebu@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T21:23:06.000Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Work%20on%20the%20update%20to%20the%20Business%20Section%20of%20the%20RTB%20Slide%20Deck.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxZAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Work%20on%20the%20update%20to%20the%20Business%20Section%20of%20the%20RTB%20Slide%20Deck.EML</a:href>
        <d:subject>Work on the update to the Business Section of the RTB Slide Deck</d:subject>
        <d:from>"Paul Barcoe-Walsh" &lt;paulbwa@exchange.microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T20:52:48.000Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/RE:%20demo%20script%20updates%20-%2060min:%20403%20on%20notify%20ws.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxSAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/RE:%20demo%20script%20updates%20-%2060min:%20403%20on%20notify%20ws.EML</a:href>
        <d:subject>RE: demo script updates - 60min: 403 on notify ws</d:subject>
        <d:from>"Jonathan Moons" &lt;jmoons@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T20:46:05.000Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/ACTION:%20New%20launch%20BOM%20in%20effect.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxQAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/ACTION:%20New%20launch%20BOM%20in%20effect.EML</a:href>
        <d:subject>ACTION: New launch BOM in effect</d:subject>
        <d:from>"Bill Dunlap" &lt;bdunlap@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T20:43:09.000Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/You%20available%20to%20talk%20today_x003F_.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxRAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/You%20available%20to%20talk%20today_x003F_.EML</a:href>
        <d:subject>You available to talk today?</d:subject>
        <d:from>"Terry Leeper" &lt;tleeper@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T20:07:57.694Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/MarieHu%20team%20meeting:%20Overview%20of%20the%20Connected%20Systems%20Generico%20application.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxYAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/MarieHu%20team%20meeting:%20Overview%20of%20the%20Connected%20Systems%20Generico%20application.EML</a:href>
        <d:subject>MarieHu team meeting: Overview of the Connected Systems Generico application</d:subject>
        <d:from>"Marie Huwe" &lt;mariehu@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T20:02:45.693Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Blowfish%20implementations%20for%20.NET.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxKAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Blowfish%20implementations%20for%20.NET.EML</a:href>
        <d:subject>Blowfish implementations for .NET</d:subject>
        <d:from>"Kevin Hammond" &lt;kevinha@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T19:48:00.000Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
  <a:response>
    <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Out%20of%20Office%20AutoReply:%20SF%20and%20Interop.EML</a:href>
    <a:propstat>
      <a:status>HTTP/1.1 200 OK</a:status>
      <a:prop>
        <a:id>ARkAAAACUCbqAQAAQL47TxBAAAAA</a:id>
        <a:href>https://mail.microsoft.com/Exchange/dinoch/Inbox/Out%20of%20Office%20AutoReply:%20SF%20and%20Interop.EML</a:href>
        <d:subject>Out of Office AutoReply: SF and Interop</d:subject>
        <d:from>"Mike Wons" &lt;mikewons@microsoft.com&gt;</d:from>
        <d:datereceived b:dt="dateTime.tz">2005-04-11T19:35:23.673Z</d:datereceived>
      </a:prop>
    </a:propstat>
  </a:response>
</a:multistatus>

Just standard XML. I use an XSL sheet to transform that into some readable HTML. In the interest of completeness, here it is:

<xsl:stylesheet 
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    version="1.0"
    xmlns:a="DAV:"
    xmlns:m="urn:schemas:httpmail:" 
    xmlns:c="urn:schemas:contacts:" 
    xmlns:java="java" 
  >

<!-- Contacts.xsl                                                         -->
<!--                                                                      -->
<!-- This sheet is used to Transform XML obtained from an Exchange        -->
<!-- webdav query into html.                                              -->
<!--                                                                      -->
<!-- Tue, 12 Apr 2005  15:11                                              -->


<xsl:param name="PageNumber"/>
<xsl:param name="UserName"/>
<xsl:param name="FolderName"/>
<xsl:param name="PageName"/>
<xsl:param name="CurrentTimeFormatted"/>


<xsl:output 
    method="html" 
    indent="yes" />

  <xsl:template match="a:multistatus">
    <xsl:variable name="FolderDecoded"
              select="java:net.URLDecoder.decode($FolderName, 'UTF-8')" >
    </xsl:variable>
    <xsl:choose>
      <xsl:when test="number($PageNumber) > 1">
      <a>
          <xsl:attribute name="href"><xsl:value-of select="$PageName"/>?action=itemizeFolder&amp;ref=<xsl:value-of select="$FolderName"/>&amp;page=<xsl:value-of select="number($PageNumber)-1" /></xsl:attribute>
            &lt;&lt;prev</a>
      </xsl:when>
      <xsl:otherwise>
      &#160; &#160; &#160; &#160; &#160;
      </xsl:otherwise>
    </xsl:choose>

    &#160;
    <a>
          <xsl:attribute name="href"><xsl:value-of select="$PageName"/>?action=itemizeFolder&amp;ref=<xsl:value-of select="$FolderName"/>&amp;page=<xsl:value-of select="number($PageNumber)+1" /></xsl:attribute>
           next&gt;&gt;
    </a>
    
    <h2>Folder:<xsl:value-of select="$FolderDecoded"/></h2>
    <h3>User: <xsl:value-of select="$UserName"/><br/>
    as of: <xsl:value-of select="$CurrentTimeFormatted"/></h3>
    <table border='1'> 
    <xsl:apply-templates select="*" />
    </table> 
  </xsl:template>

  <xsl:template match="a:contentrange" />

  <xsl:template match="a:response">

    <tr>
     <xsl:choose>

      <!-- ============================================================ -->
      <xsl:when test="$FolderName = 'Inbox' or $FolderName = 'Sent Items' or $FolderName = 'Deleted Items'  or $FolderName = 'Junk E-mail'  " >
      <td>
        <a>
            <xsl:attribute name="href"><xsl:value-of select="$PageName"/>?action=get&amp;item=<xsl:value-of select="a:href" /></xsl:attribute>
              <xsl:value-of select="a:propstat/a:prop/m:subject" />
        </a> 
        </td>
        <td style="font-size:8pt;">
              <xsl:value-of select="translate(a:propstat/a:prop/m:from, '\\', '')" />
        </td>
        <td>
           <xsl:value-of select="substring(a:propstat/a:prop/m:datereceived,0,11)" />&#160;<xsl:value-of select="substring(a:propstat/a:prop/m:datereceived,12,8)" />
        </td>

      </xsl:when>

      <!-- ============================================================ -->
      <xsl:when test="$FolderName = 'Contacts'">
        <td>
        <a>
            <xsl:attribute name="href"><xsl:value-of select="$PageName"/>?action=get&amp;item=<xsl:value-of select="a:href" /></xsl:attribute>
             <b> <xsl:value-of select="a:propstat/a:prop/c:fileas" /></b>
        </a> 
        </td>
        <td style="font-size:8pt;">
              <xsl:value-of select="a:propstat/a:prop/c:telephoneNumber" />
        </td>
      </xsl:when>

      <!-- ============================================================ -->
      <xsl:otherwise>
      <!-- xsl:when test="$FolderName = 'Notes'"  -->
        <xsl:variable name="ItemName"> 
        <xsl:choose>
          <xsl:when test="substring(a:propstat/a:prop/a:displayname,string-length(a:propstat/a:prop/a:displayname)-3) = '.EML'">
            <xsl:value-of select="substring-before(a:propstat/a:prop/a:displayname,'.EML')" />
          </xsl:when>
          <xsl:otherwise>
            <xsl:value-of select="a:propstat/a:prop/a:displayname" />
          </xsl:otherwise>
        </xsl:choose>
        </xsl:variable>

        <td>
        <a>
            <xsl:attribute name="href"><xsl:value-of select="$PageName"/>?action=get&amp;item=<xsl:value-of select="a:href" /></xsl:attribute>
             <b> <xsl:value-of select="$ItemName" /></b>
        </a> 
        </td>
        <td style="font-size:8pt;">
              <xsl:value-of select="a:propstat/a:prop/a:getlastmodified" />
        </td>
      </xsl:otherwise>
     </xsl:choose>
    </tr>
  </xsl:template>
</xsl:stylesheet>

Of course, in the more general case, you're not going to just be displaying the result you got from querying exchange. Instead you will be extracting data and mashing it up with something else. In that case you will want to do XPath or DOM walky stuff on the XML doc. And you know how to do that, right?

All the code is available attached here. I hope you all find it useful!

Just a quick disclaimer before I go: I Really DO NOT recomend that people constructing new apps use WebDAV to connect to Exchange Server. Exchange Server 2007 supports a standards-compliant Web services interface, which is much easier to use than the WebDAV interface. If you have a choice, please consider using the web services option.

Cheers!
-Dino

Open Source and Interoperability

DotNetInterop — Wed, 18 Jun 2008 17:42:00 GMT

Open. Source. Is. Not. Interoperability.

Ted Neward is an entertaining and talented writer, not to mention a stand-up guy, and a first-rate technogeek.

But that doesn't mean he is always right. I was just reading and old post of his which I missed during my unscheduled sabbatical, in which he writes:

They [Microsoft] need to have an interoperability story that developers can believe in, which means some kind of open-source-friendly play,

I still, still totally do not get why Interop and Open Source are used so interchangeably, by so many people. Often by people whose viewpoints I respect, such as the esteemed Mr Neward. Or if not used interchangeably, why they are viewed as so closely related by so many people. I do not see these two things as mutually dependent. They are completely independent factors.

Interop is not open source. Open source does not guarantee interop. Period.

I am not making quality judgments on either open source or interop. I am not saying that one of them is good and one is bad. I am saying they are two different things. They are "silicon based computers" and "big endian bit ordering". They are related, they are neighbors, they are acquaintances, but they are not interchangeable. Nor does one imply the other.

Repeat after me:

Interop is not open source.
Interop does not require open source implementations
Open source does not guarantee Interop

People will disagree with this; I expect to hear some feedback saying "With open source I can see the code and therefore it is easier to build something that interconnects with it." I'm not buying that. That is lazy thinking or bad engineering or both. I can imagine that open source would imply interop, if you limit the definition of interoperability to mean "connect with previously written code in the same executable image." In other words, merging distinct code bases. This is a case where just linking up to a shared library can work, but sometimes seeing the code allows some better opportunity.

But this limited definition of interoperability is definitely not the mainstream one that companies and organizations are dealing with today. They want to connect large lego blocks together: Connect System A and System B together in a business process flow that makes sense, and which may evolve over time. Large-grained interop. This is not merging two distinct code bases together. Oftentimes the "code base" is not available - it is a pre-packed app. Maybe it is Oracle, or maybe it is a Salesforce.com app. Maybe the other end is a Sharepoint Portal, or a Medical Records management system from Lawson. These might be called system-to-system interop or app-to-app interop if you like, where network protocols are the thing, not compile-time-constants. Data interfaces, not code classes or interfaces.

In that situation, which I am arguing is THE mainstream challenge that architects and devs confront when they use the word "interop", looking at source code is not helpful, and I won't hesitate to argue, I think it would be counterproductive. Yeah, you read me right - it actually is harmful to look at the code if you want to connect two big apps together.

What is necessary to enable interop in these cases is PROTOCOLS, people. Standard protocols would be nice to have, but don't misunderstand - standardized protocols are not a requirement for interop. The requirement is for PUBLISHED protocols, not necessarily standard ones. PUBLISHED NETWORK PROTOCOLS ALLOW INTEROP. This is why a Java or .NET app can connect to an IBM transaction processing system, even though the on-the-wire protocols are completely closed and proprietary to IBM. The protocols are documented. They are closed yet published. And because IBM's DTP protocols are published (not publicly per se, but published to those who license the protocols), anyone can implement the client-side of the exchange.

The same goes true for, say, Microsoft SQL Server. There is an on-the-wire protocol known as TDS, also known as tabular data stream. Microsoft publishes TDS under license to partners, but I believe some have reverse-engineered it. The fact that the protocol is constant and published means that, for example, DataDirect can build an OLEDB driver or a type-4 JDBC driver for SQL Server.

It's the Protocols, Silly!

Standardized protocols are nice to have because the standard, if it is based on freely-available IP (or at the very least, IP available under RAND terms), encourages wider adoption and thus the virtuous cycle inherent in Metcalfe's Law. Standardized protocols are essential if you want broad interoperability, which of course is really important, and is what most of us are after anyway. Above I said that standards are not required, and I stand by that statement. But practically speaking, standards are almost a sine-qua-non of meaningful interop.

Web Services and XML are just common protocols, and their widespread adoption (owing in large part to the fact that they are standards) is the true source of their value to companies and organizations (again see Metcalfe's Law). It is not required that web service endpoints, either client or server, be implemented with an open-source web services stack, in order to get good interoperability. Instead it is essential that the endpoints conform to the standard protocol definitions. And the corollary is, the protocol definitions must be sufficiently clear, complete, relatively simple to implement, and relatively simple to test, such that faithful implementations of the protocols can eb validated easily and will interconnect transparently.

This is why .NET WCF clients can easily interconnect with Web Services enpoints running under IBM WebSphere App Server, though neither the .NET Framework nor the WebSphere web services libraries are open source.

Open Source Is Not Interoperability.

Java Exchange Connector and EWSJ

DotNetInterop — Tue, 17 Jun 2008 17:19:00 GMT

Just saw this on TheServerSide.com. The Exchange Web Services for Java (EWSJ - what happened to the 4?). It's a Java class library that ~~uses the Apache AXIS web services stack to connect~~ connects to Microsoft Exchange Server 2007, via the published and supported Exchange Web Services interfaces.

What you can do with this is connect to Exchange 2007 from any Java app. A good example is when building an HR or group scheduling app, in Java. (While I love .NET, not everyone has seen the light! and some poor developers are relegated to using Java even for new applications.) In the scenario where you write in Java and the enterprise messaging standard is Exchange, the EWSJ library may come in handy.

There's a beta of the Exchange Web Services for Java out now; it looks like the company updates the beta fairly regularly. They mention a free academic license, so I guess they will be licensing EWSJ for a fee for other purposes.

The company also has something called the JEC, which I guess is the Java Exchange Connector. I don't understand the difference between EWSJ and JEC. Based on reading, I'm guessing they do the same thing - provide access for Java apps into the Exchange store - but I think the JEC is based on the older WebDAV interface into Exchange, and EWSJ uses the web services interfaces. ~~Just a guess... I don't know. Maybe someone can enlighten me.~~ Confirmed: JEC uses WebDav and EWSJ uses EWS.

This EWSJ works with the Web services interfaces supported by Exchange 2007. By the way, those are the same web services interfaces I use from a Windows Forms app running in .NET, and the same interfaces I use from a Windows Mobile app running on my smartphone.

It brings up an interesting question - why pay for EWSJ - or any third-party library that wraps web services interfaces - when the webservices interfaces are public and free-of-charge? I mean, if you have the WSDL and XML Schema, and you have access to good Web services tooling (Apache AXIS for Java, WCF in .NET), then you could just build (generate) the client-side proxies and classes and get connected, right? But, there's an opportunity to add in value to the interface - like caching of content, caching of credentials, simplifying the interface, integrating with other frameworks, and so on. The approach I took with my .NET work was to use the tools and published interfaces. But I can see value in depending on a third party library that adds some additional capability. I don't know much about EWSJ, so I don't know what's been added, but I see the opportunity for the company to deliver something interesting there.

By the way, I have also previously demonstrated the use of WebDAV interfaces to get to Exchange Server from Java, prior to version 2007, as the JEC does it. You can get from a JSP to Exchange 2003 for example, using WebDAV. The way I did it was, like the JEC approach, to depend on the Apache libraries. In my case, just as with the JEC, I used the now-retired Jakarta Slide library and the httpclient from the commons package. I can publish this source code if you like. (But if you are running Exchange 2003, you really should upgrade. E2007 is much much better, and the web services interface is just one aspect.)

I was surprised to see that Jakarta Slide has been retired! Apparently it didn't attract enough developers. But it looks like there is a new WebDAV implementation in the Apache Jackrabbit project. I guess this is a challenge facing all open-source projects. With so much attention focused on AJAX and pretty web interfaces, internetworking protocols like WebDAV take a back seat. Good to see WebDAV is not totally dead at Apache.

EWSJ looks like one more interesting interop opportunity.

DotNetZip, open source library for .ZIP files, revs to v1.4

DotNetInterop — Fri, 30 May 2008 18:18:00 GMT

I have updated DotNetZip, the open source Zip Library for .NET, to v1.4.

Find it at http://www.codeplex.com/DotNetZip .

New features!! password support is here! Whoohoo!! This is PKzip (weak) encryption. Another new feature: you can now add entries into zips using content from streams. There are clearer exceptions generated in the case of errors. Beyond new features, there are a bunch of other improvements and fixes to the library, too. For example, the internal handling of zip files now uses streams for more optimal memory usage. As another example, there is now a large set of unit tests that ships with the library. As a result, reliability is significantly higher than in v1.3. Finally, the Zip utilities have been updated for better command-line parsing, more options and flexibility.

That's the v1.4 library, which is released and final.

For v1.5, I'll be focusing on Update capability - the ability to update an entry in an existing zip archive with new content, as well as the ability to Remove entries in existing zip archives. There's a preview of the v1.5 library already available on http://www.codeplex.com/DotNetZip .

If you want some background on the Zip Library, check out my prior posts on this topic. Just for your information, DotNetZip is licensed under the Microsoft Public License, which basically means you can use it free of charge, for any purpose, but there's no warranty. Check the full terms of the license on the CodePlex site.

As always if you have questions on usage, check the CodePlex forums.

Cheers!

Whatever happened to WSRP? (psst: I'll give you a hint, it rhymes with BEST, and you can do it on a mattress)

DotNetInterop — Thu, 03 Apr 2008 21:53:00 GMT

Remember when the Java vendors used to advocate WSRP as a way to accomplish Interop?

I never thought much of WSRP as a tool for implementing interop, and I went on the record with that.

Web services (without the RP add-on), and REST are what people are using for interop, and that really makes sense to me. I love it when I'm right. :)

PS: the WSRP FAQ on the OASIS website was last updated in 2004. *Yawn*.