Ulrich's SOAP Box

Working with Whitehorse, Pt2…

2005-07-14T03:04:00Z

I said in my last post that the benefit of using Whitehorse has been marginal so far. To my surprise, I never got a single email refuting this. I still stand by that comment, although I am being a little hard on Whitehorse.

At this stage in the design process, I've painted a picture of my application design, and a picture of my logical datacenter design. I use the word picture explicitly, because that is what it is. It is a visual representation, without any additional information (metadata) attached. Kind of a flat representation of a 3D world. Today, I'm going to describe how I started to add that additional information, initially to the application design.

If you refer back to the application design in my previous post, you'll see I started by selecting one area of functionality we would be delivering (point-of-sale customer and sales order), and then built up an application design. To the basic POS system, I added some additional features to support further development, like an event publisher to allow new systems to be attached to the application, and a library of common services (InfrastructureLibrary). Remember also that my application utilises published SAP Web Services, which in my initial design I emulates using an ASP.NET Web Service (SAPWS).

The first thing I did today was to try to access the real SAP Web Services. I consulted the SAP API documentation and found the address of the WSDL files that described the Web Services I planned to utilise. Then, within the Application Diagram, I deleted the dummy ASP.NET Web Service (SAPWS), and added an External Web Service instead. When I did this, I got a dialog, which allowed me to specify the path to the WSDL file for the SAP API I wanted to use. I entered the URL, and this created an external Web Service in the Application Designer.

At this point I bumped into a couple of issues. Firstly, SAP exposes each API call as a separate web service, instead of a single web service with multiple web methods. This meant that I had to create a total of 8 separate external web services in the application designer, one for each SAP API I planned to call. This could make for a pretty cluttered Application Design if you planned to access a large number of SAP APIs.

The second issue came up when I looked at the signature of the SAP web services. These were RPC/encoded web services, rather than document/literal. I wanted to stick to document/literal web services, following the guidelines of the WS-I Basic Profile. I believe that later versions of SAP provide better web service interfaces. However, it became apparent that putting a web service façade across the front of the provided SAP web services was going to be a good idea.

You can see my Application Design above. Note that I cropped it a bit, to focus on the area of interest. At this stage, I had linked to each of the SAP web services, and I had defined the web services (and web methods) for my SAPFacade (a less granular set of business web services, implemented in terms of native SAP web services). I still need to define the rest of my applications, but at this point I thought I'd have a little fun….

I checked everything into Visual Sourcesafe (I'm working stand-alone off my laptop, so no Team Server L), labelled the version, and then checked it all out again. Then I right-clicked on SAPFacade, and selected "Implement Application". VSTS created the ASP.NET Web Service project (in C#), automatically added web references to all of the SAP web services I had linked to in my Application Design, and created stubs for the methods of the two web services I had defined. Selecting one of the web methods, I was then immediately able to write the following code, with completion etc.

At this point, I really began to see some benefits from Whitehorse. Usually when I'm working with corporate customers, I don't get the luxury of completely "green fields" development. What I mean by that is that I invariably have to link my applications to existing applications/systems, and that I need to deploy those applications onto existing (sometimes shared) infrastructure (more about that next week). With a "static" design tool like Visio I typically can't pull in definitions from existing systems, and integrate them with my evolving design (although to be fair, I can pull in web services definitions in some other design tools). My Application Design was now starting to capture significant meta-data behind the visual elements, and more importantly, this meta-data is automatically transferred to the application developer (in the form of class definitions etc). I was beginning to get hooked!

One last really COOL feature! I talked above about the fact that I wasn't that happy with the exposed SAP web services, and I implemented a façade to those services which provided a web service interface that better suited me. Now of course as I continue to develop this application, I will build up a pretty comprehensive SAPFacade (a number of web services, grouped around specific business entities, such as SalesOrders, and Customers). Of course within my customer, other designers will be working on related applications, and these designers will benefit from the work I have done providing business level wrappers around the SAP web services. SO, what I can now do, is right-click on the SAPFacade application within the Application Designer, and select "Add To Toolbox". Now this component will be available on my toolbox, for re-use in other application designs (and I can export it for others to use as well). Of course I can use this to add other enterprise wide common components, like InfrastructureWS. Now that ROCKS!

More next week, as I start to "flesh out" my logical Datacenter Design.

An apology

2005-07-13T06:51:00Z

My post yesterday didn't work very well because I posted the pictures of the designer onto a web site that had download limitations. The result was that most people who tried to view the post couldn't see the pictures. Now fixed, thanks to Nigel Parker, who loaned me some webspace. Thanks Nigel

Tomorrow, I'll post the second article in this series.

Working with Whitehorse…

2005-07-12T05:37:00Z

I've done my fair share of technology presentations, and recently a lot of those have centered around the designers in Visual Studio Team System (aka Whitehorse). Now when you do a presentation, it's hard to really exercise the technology. You tend to stick to specific scripts, and avoid concepts that are complex, and take too long to explain (unless you want your audience to fall asleep).

As a result, I was both excited, and a little bit worried, when one of my corporate customers asked me to help them architect and design a new series of applications they were going to build using VSTS. Excited, because I was finally going to get a chance to really exercise this cool technology, and a bit worried, because I wondered just how much value it would actually add to the design process, and whether it would really work as promised (it still a beta after all).

This blog is the first in a series where I'm going to try to explain how I went about using Whitehorse, and how effective I found it.

The customer had already developed an enterprise architecture which covered off the major applications they needed to build in the coming months, and also provided a whole lot of guidelines around how those applications should be architected (leading towards a SOA). The customer also already had some applications deployed using VS 2003, in an existing datacenter environment.

We started by identifying a candidate application. This was a retail POS application which was going to be browser based, and would provide retail customer management, and order creation/update. At the back-end, this functionality was all provided by SAP. We planned to integrate with SAP via SAP Web Services.

We started by laying out our initial application design. You can see the initial design below. The various applications are laid out in tiers, with a presentation tier containing the POS web site (CustomerSalesOrder), an application tier containing process level services (StoreOpsPS), another application tier containing the more atomic web services (SAPFacade, StoreOpEvents, and InfrastructureServices), and a data tier.

The key applications for the retail POS system are the CustomerSalesOrder web site, and the StoreOpsPS higher level services. As we started looking at those more closely, we realised that there were going to be some common services which would be shared by various applications. We decided to group these as InfrastructureServices, and these included things like auditing, authentication etc. We also wanted to be able to add integration with other systems easily, so we decided to implement an event publisher (StoreOpsEventPublisher), which applications could subscribe to. This was not required for the initial application, but we felt it would come in handy later on.

At this stage, we did not have access to the SAP system, so we mimicked that with a ASP.NET Web Service called SAPWS. We put a couple of Web Services into this proxy based on the SAP documentation. Later we'll replace this with the actual SAP web services.

At this stage, we talked to the infrastructure team, and got them to help us lay out the existing infrastructure using the Logical Datacenter Designer. That diagram is shown here

As you can see, they have two logical tiers, with presentation and application in the WebZone, and then several "data" tiers (SAPZone, DatabaseZone, and MainframeZone). From my perspective, although SAP and the mainframe have application logic, I only access them as data servers.

One thing that was not immediately apparent to me was that I needed to actually connect up endpoints even within a zone. So, when I first built this it would not validate against my Application Design, because I hadn't specified the LocalWS, or the EventQueue (OK, so I should RTFM). I found that I could connect up various Web Services, database connections, and generic connections, and then set constraints around them. For example I was able to specify that the protocol for the EventQueue had to be MSMQ. Cool.

At this point, I created a deployment diagram, and then spent some time putting all the applications into the appropriate servers, and linking up these within and across the zones. This took an hour or so, mainly due to my incomplete understanding of how it all worked. I also laid out the operations on each of the Web Services, ready for these to be created.

So, after all this, did Whitehorse help me design this system? At this stage in the process, I'd have to admit the benefit was still marginal. It did help, in that it forced the various interested parties to cooperate, and design not only how the application was going to work, but also how it was going to be deployed, but at this stage, we could have done all this with Visio.

HOWEVER, stay tuned for my next blog, when I'll describe how we continued to work with these design views, and make them much richer than they are here...

A chance to win some cool prizes

2005-06-12T06:17:00Z

As I said in my previous post, I've been trying out a lot of beta software recently. One of the things I've been working with a lot is Visual Studio Team System. I reckon its pretty cool (actually, I'd have trouble going back to the previous version now), and really wish I'd had something like it in some of the projects I've done in the past.

To encourage you all to try it as well, we (the Microsoft New Zealand Developer and Platform team) have decided to run a competition to encourage you to try it out. The competition comes in two flavours, to appeal to both individual developers, and to development teams, and the prizes are pretty cool. There's about $100K in prizes up for grabs, for New Zealand developers only, so you have a great chance to win. Plus you get the chance to become world famous in New Zealand, because the winners get drawn at TechEd, which will give you some great exposure to the New Zealand development community.

To find out more, go to http://www.microsoft.co.nz/msdn/competition. Oh, and if you haven't already, get the VSTS beta here

Best of luck...

The life of a beta tester...

2005-06-07T05:22:00Z

Ha, this blogging seemed like a great idea at first, but its really hard to maintain. I've got a lot of respect for people like Scott who are pretty busy, and yet still manage to blog so frequently. I've been pretty busy presenting on various topics, and as part of that, I've been playing (don't tell my boss, but its all good fun) with a whole range of beta technologies. Currently, on my desktop I have VSTS Beta 2, SQL Server CTP, WSE 3.0 CTP, and BizTalk Server 2006. Now you'd think with all that beta software, almost nothing would work, but actually, things are going pretty well.

Scott Woodgate talks about Rolling Thunder, when he speaks about various cool features in the new BizTalk 2006, so I thought I'd share some of my own Rolling Thunder for the various technologies I've been checking out over the last 6 weeks, starting with BizTalk 2006.

Rolling Thunder #1 - The BizTalk Configuration Utility

One of the biggest complaints I've had from customers is that while installing BizTalk is pretty simple, the configuration wizard was just too complex. If you made any mistakes, and forgot to install a feature, or wanted to make changes, you often ended up unconfiguring (hopefully you remembered to do that), and then re-running the configuration. Well, the first thing that hit me in the eye was the new configuration wizard. Simplicity itself to run, and you could run and re-run it to install features, and then uninstall features as needed. I was pretty pleased about that, because I tried to install BizTalk onto SQL 2005, so I ended up uninstalling the BAM Events, and generally changing things around several times.

Rolling Thunder #2 - The BizTalk Flatfile Schema Wizard

Scott's already blogged about this, but its such a cool feature, I had to mention it as well. I wish I'd had something like this back in BizTalk 2000 days....

Rolling Thunder #3 - WSE 3 MTOM support

I was giving a talk about WSE and Indigo to our local .NET User Group. I was also pretty intensely using Visual Studio 2005 Team Suite for my development, so I wanted to show both WSE2 and WSE3. Rebecca Dias kindly gave me access to an early version of the WSE3 CTP (which you can get here), so I put together a bunch of samples showing security in WSE2 with Visual Studio 2003 (on a VPC), and then showed MTOM support in WSE3 directly off my laptop. A lot of people have been struggling with sending binary data across web services, with some using WS-Attachments, and some using MIME. Finally, with an implementation of MTOM available, we can all move to a standard way to do this. And it's so simple too.

I've also looked at the security features in WSE3, but more on that in another post. WSE3 is very cool though, and I recommend you check out the CTP.

And before anyone says, I know I should probably put all this beta software into a VPC (and I do for some builds and demos), but it really does work very well together.

Calling on Kiwi Ingenuity

2005-04-27T11:07:00Z

Scott Woodgate just announced the Connected Systems Developer Competition. If you build the best sample application that makes use of VS.NET 2005, SQL Server 2005, and BizTalk, you get to win $50,000 (US!). I'd really love this prize to go to someone from New Zealand. If I didn't already work for Microsoft, I'd give it a go myself.

Check out...http://blogs.msdn.com/scottwoo/archive/2005/04/27/412640.aspx

A long absence

2005-04-19T06:09:00Z

Well, its been a while since I last posted. What with the changes to our blogging infrastructure, plus my workload suddenly increasing dramatically, I just never seemed to get around to writing another blog.

So what have I been doing. Well, I got asked to do a couple of talks about Visual Studio Team System (VSTS), at local developer conferences, and then we had Connect 05 in Christchurch, Wellington and Auckland. Didn’t seem that bad, but I really like to make sure I am prepared for talks, so it all ended up taking a lot longer than I anticipated (but was very interesting).

VSTS is truly huge. I’ve spent the last 7 years building increasingly complex server systems on Microsoft technologies, everything from the Rugby World Cup web site, to message exchanges built on BizTalk Server, so I really relate to the designers that ship with VSTS. For the talk, the demo script involved a fairly simple 3 tier web application, which you hook together, and then deploy to a logical datacenter. Impressive in itself, but I decided to have a bit of fun, and ended up plugging in the application and datacenter designs of several of my past projects. This took a lot of time, but was pretty cool.

I’ll be doing a lot more with this stuff in the next few weeks, so stay tuned.

WSE 2.0 SP3 went live today on MSDN.

2005-03-01T10:00:00Z

Get it here.

Links from my Business Process Management session this morning...

2005-02-21T15:59:00Z

I promised a few links from my Business Process Management session this morning…

The BizTalk Server Security paper - http://www.microsoft.com/biztalk/techinfo/whitepapers/2004/security.asp

Using Microsoft Tools for Business Process Management - http://www.microsoft.com/biztalk/techinfo/whitepapers/2004/business_process.asp

BizTalk Server 2004: Understanding BPM Servers - http://www.microsoft.com/biztalk/techinfo/whitepapers/2004/bpmservers.asp

BizTalk Technical Guides, including deployment, performance, security - http://msdn.microsoft.com/BPI/TechGuides/default.aspx

Thanks to those of you who attended. I got some great questions, and feedback.

Martin Fowler on patterns

2005-02-21T10:30:00Z

Well, here I am at the Microsoft Regional Architect Forum on the Gold Coast. I just caught Martin Fowler’s keynote on Enterprise Application Architecture Patterns, and patterns generally. Martin is an engaging speaker, and I was pretty keen to catch his ideas on patterns after I worked with his colleague, Gregor Hohpe on the Integration Patterns book. Martin also gave out a URL to a survey he’s done on “good” (Martin was quick to stress that this is his idea of “good”) sources of pattern information. You can get that here

Which TCP/IP ports does BizTalk use?

2005-02-17T12:06:00Z

Check it out in this handy article

A nice analogy

2005-02-17T11:21:00Z

I’m a sucker for a nice analogy! I find they are a great way to help you to understand complex ideas and systems, by referencing stuff you understand well. Of course there is the danger that you can carry one too far, but I still find I understand and remember stuff better when someone provides a nice vision for me. For example, I’ve always liked Pat Helland’s analogies about IT, first the Emissaries and Fiefdoms, and then the Growth of American cities aligned to the development of IT.

Yesterday I presented at the Service Oriented Architecture conference here in Auckland, and I also sat in on a session by Mark Colan from IBM. Mark made a nice analogy between the way systems were originally developed in a monolithic fashion, then object-oriented, and then SOA, likening it to walls (bear with me here).

Mark started the analogy by showing us an ancient wall, composed of irregular bricks, with large mortar chunks in between, and even some pieces that had been added on years later, and cemented into place. This was analogous to our monolithic applications, held together by code, which largely could not be changed, although some retro-fit did happen, tacked into place by more custom code. The next picture was a more modern wall, composed of bricks that were all the same size, still obviously held together by mortar. This was like o-o, because the components fitted together well, but you still needed code to hook them together, and once hooked together, rearranging them is impractical. The last picture Mark showed was of a Lego “sculpture” of Harry Potter (you can see it if you ever fly into or out of Auckland Airport). This was SOA. Bricks still the same, but they could easily be composed in a lot of interesting ways, and even after being built, they could be rearranged into new compositions. Kind of licke the cement was now re-useable, as well as the bricks.

The other point Mark made was that the guy who built the Lego sculpture had several attempts at it before he got it right, which parallels the potential rapid development style possible with SOA.

Nice one Mark!

BizTalk MSMQ Adapter released

2005-02-17T10:57:00Z

Cool. The MSMQ Adapter (as different from the MSMQT Adapter) has now been released for BizTalk. You can download here.

I’ve been waiting for this, because I do a lot of development on my machine, and want to have MSMQ and BizTalk on the same machine. If I installed (registered) MSMQT, I wouldn’t be able to have MSMQ on that machine. If I had MSMQ, I couldn’t read the messages from the queue (you get the idea).

Now onto that Publish Subscribe pattern example…

Patterns Repository

2005-02-17T09:03:00Z

If you haven’t already seen it, check out http://patternshare.org. It’s a table of architecture and design patterns, split into Application, Integration, Operational etc patterns. It pulls in and references patterns from GoF, our Integration Patterns book, The Enterprise Application Patterns book, and other sources, and links you to the pattern synopsis if its available on the net. A cool resource for architects.

A new blog...

2005-02-08T14:06:00Z

A friend of mine, Alex James, has just started blogging. Alex owns his own small company, building some pretty slick software which allows applications to share data or datalayers. You can find his blog here, and download his software from his web site here.

Looking at the number of blogs though, it seems Alex has been spending too much time blogging. Get back to work man!