Where do I stand today on ESB and the mythical "successful big project"?
I received an email from a long-time follower of my blog that I thought might serve to spark a little interesting debate and also serve as a marker in time for my opinions on the subject of ESB and the ever elusive "successfully delivered big project":
Hi Rich-
Followed your blog from loosely coupled. I am reading these thoughts about the ESB from your post and the loosely coupled post, right around the same time, I had been pushing the same thoughts at various Government of <country> project that I was involved with....somehow "figure stuff out first, build some stuff, and then worry about all this big infrastructure stuff" just does not appeal to people. Hence I have seen two rather large ESB projects crumble under the weight of the huge initial investment and the lack of interest from would-be users.
So I am interested in finding out....2 years after your "debate" with Dave Chapell [part1, part 2], where do you think the ESB situation is? Are the WS-* standards that mature yet? I am once again on a similar path, preaching the same message...
Would love to hear where you are with the ESB stuff now?
--cheers
<name>
<name>@<domain>.com
SOA Architect
To be honest, I don't see any change in my perception on this matter because my beliefs back then were based on painful experience. I haven't yet seen any revolutionary improvement in the way projects are executed that leads me to believe that someone has thought up a significantly "better way". This is largely because projects are lead by wetware ... wetware with egos, grudges, issues and which can be easily distracted, diverted, disillusioned, etc.
And so, as you've pointed out, we continue to see repeated implosions where projects to boil the ocean with a shiny new candle collapse under their own gravity. I don't know when people will start to wake up and smell the White Chocolate Latte (yum!:)) and realize that taking smaller, more achievable, more manageable steps which allow you to course-correct while you proceed, whilst also having to accept that you're going to learn stuff along the way which may lead you backwards on occasion ... is the only way we're going to proceed.
I said it before and I'll say it again - mistakes, learning and progress are not mutually exclusive - they're joined at the hip. There is absolutely no way you an architect something today that takes more than 10 months to build and expect that you will have designed and built this thing to withstand every scenario you want to throw at it - by the time you're done, the world will have changed considerably.
This, to me, if the crux of the issue - Stuff changes ... deal with it! Tools change, platforms change, ideas change, markets change ... and they all change faster now than ever before. Assume the velocity of change is only going to increase and you'll be better prepared to handle real life! :) If you don't and continue in the mistaken belief that the world will be the same 12 months from now, you're asking for trouble.
To the question on whether WS-* is mature, I'd definitely say that the WS-* protocols are significantly more mature than they were 2 years. Many are now onto their 2nd or 3rd revision after being used in production systems and products for 3-4 years. Several more have been through the wringer over the last few years and are now being submitted to standards organizations now that we have a pretty good feeling that they've reached an appreciable level of maturity.
Remember also that many of the WS-* protocols were born from a synthesis of what experience has taught us were already effective ways to get certain things done, so it wasn't like they were just created from an "ah-hah"! While all the usual discussions are still to be found on the forums and on the blogs. E.G. "Is WS-* as efficient on the wire compared to binary protocols", the usual answers still apply - the size of the data on the wire should rarely be a true inhibitor to adopting these protocols. This sounds like an interesting corollary to my suggestion above that things change - and are changing faster: but recognize that some things (like human behavior ... the kind of behavior that commissions and executes projects) change at a glacial pace where any increase in the rate of change takes an eon to see.
So to close (and because I'm taking my wife out to the movies tonight and there's a Starbucks on the way with a White Chocolate Latte with my name on it), I guess I'd say that little has really changed in the guidance relative to this space in the last two-three years. Products and platforms have improved enormously, tools are more plentiful and more productive, techniques are honed, but only marginally so.
But people will still believe (or be convinced by others) that they have to go boil the ocean with a candle ... just be sure not to get wet - you might catch a nasty chill! :)
