Josh Lee's Financial Services Blog

Financial Services technology issues

A Whole Batch of .....

I'm finally done with the seasons of standards meetings.  I just wrapped up the ACORD Subcommittee meeting in Ft. Lauderdale.  I have to say that I'm glad that I'm done with all that.  One thing that strikes me about standards and application design is how much people forget about the flexibility in programming, message design and enterprise architecture when they try to preserve legacy infrastructure or ways of thinking. 

One of these areas is the ever present function we now lovingly refer to as "batch".  The funny thing is that I've been hearing for years now, that "batch is hard" or that "you can't do batch with foo Microsoft product".  I even heard it this week regarding the ACORD XML message specification.  I heard someone say that they couldn't cobble together a batch message load using XML.  I thought that wa the most absurd thing that I had ever heard.  Perhaps one of the issues is that the size of a larger XML document does not lend itself to being strung together into multiple messages for a single batch message.

As industry standards are developed, one thing that they must keep in mind is that a more granular and composable approach allows batch messages to be more concatenated into a batch message without overloading the wire with data.  Second, a properly architected service layer would allow consumers to request just the data that they need at the time that they need it thus reducing the need for large message based batches with largely unnecessary data.  Smart client architectures can assist with that.  I just don't understand why batch processing kicks everyone's butt.  So let me set the record straight.  SQL Server 2000 can do batch, DTS allows a number of ways to load extract, transform and load data...and that's only one feature.  BizTalk Server 2004 can support batch...through workflow, message creation, and adapter infrastructure.  .NET code can also build a batch environment...so stop sweating this.  If you really have to do batch, just program it.

Now, I thought I'd give some more random questions and bullets that I thought of at the ACORD meeting.  These may not make much sense to those that didn't attend or don't engage with ACORD, but I'll try to give some context.

  • In the version 2.0 creation process, there seems to be an outstanding issue as far as how finely or coarsely you factor a service namespace.  They seem to be using a CRUD method grouping all business units under CRUD macro levels.  Would perhaps a business level designation be better or just an infrastructure that supports both?
  • The concept of one way messaging is something that still needs lots of work.  Sure there is a need like eventing and notifications, but perhaps some of the WS-I work is best suited there, and not a custom "framework".
  • When are we going to kill AL3....PLEASE !!!!!
  • ACORD has a concept of "Service Provider Extensions"...could these be better served using a WS-Policy infrastructure and not a custom schema redefinition scheme.  I'll have to look into that.
  • When will we finally have a convergence of the specification data elements so that when ACORD says "FirstName" (for example) it means one thing and not three?
  • ...and finally
  • There does appear to be very little mainstream support for the SOAP w/Attachments direction.  Many of the historically active contributors to the process are not supportive at all and in fact willing to wait for the right specification to compose with a web services architecture...um, like MTOM  :-)

And that's it for tonight.  Now...back to Halo 2.

- Josh Lee

Published Wednesday, November 10, 2004 11:27 PM by FinServGuy
Anonymous comments are disabled

This Blog

Syndication

Tags

No tags have been created or used yet.

News

Welcome to Josh Lee's Financial Services Blog!!!

This blog is intended for the Financial Services audience in Banking, Insurance and Capital Markets. It is the source for code, samples, architectures, patterns and discussions related to Microsoft technology in Financial Services.

Josh Lee is the Program Manager for Financial Services Architecture and past Strategy Director for Microsoft's technology in Financial Services.

Feedback and constructive discussion is welcome. ENJOY!!

© 2009 Microsoft Corporation. All rights reserved. Terms of Use  |  Trademarks  |  Privacy Statement
Microsoft
Page view tracker