This phrase was used quite a bit this week at the Gartner AADI conference and while I am pretty skeptical when it comes to new acronyms and technology names, i do rather like this one for two reasons.  First it allows us to cut through some of the ambiguity of the SaaS/PaaS layers by getting to the heart of a very important cross-section of capabilities.  Second, this term starts moving us from the current understanding of applications as silo'd sets of user experiences based on antiquated boundaries into a set of discrete capabilities that should be location transparent and composable.  While there is a danger in overusing the "aaS" acronym to turn each possible capability (security, integration, data, search, mapping, etc) into a new acronym, this name does provide the opportunity to have constructive conversations on how capabilities can be both enabled and leveraged through APaaS to enable the next generation of distributed applications.

I'm attending Gartner's AADI conference this week.  They might as well call this the Application Platform as a Service (APaaS) conference as its all about the Cloud, SOA, BPM, and distributed application development.  It's a very nice to hear Gartner recognizing and promoting the fact that "BPM needs SOA to be successful" and "you can't skip SOA to get to the Cloud".  After all the "SOA is dead" messaging I've been seeing, its nice to see such an influential advisor recognize that dismissing SOA is an attempt to dismiss fundamental issues that are core to any distributed systems moving forward.

This week we published a technical whitepaper on Microsoft downloads that discusses in some detail how we enable service virtualization on the .NET platform through the Managed Services Engine (MSE).  Aaron Skonnard of Pluralsight authored this document and does a great job in not only going through specific activities such as versioning a service and defining new policies, but also the application of these activities in virtualizing services through the .NET Service Bus.  This is a very thorough document on the MSE, so if you are looking for a detailed understanding of our approach to service virtualization I strongly suggest you look it over.

As a US holiday, today has long been designated as "clean my office" day.  Since I am often on the road, my office is my home study and over time, a number of papers, notes, mail, and computer parts seems to somehow accumulate (I swear sometimes these things must be breeding) to the point that I am left with nothing more than a patch of surface area on my desk to work with.  I probably have disposaphobia because I carefully scrutinize every scrap of paper or item before throwing it away - and of course when in doubt it stays clear of the trash bag.  Inevitably, I will find some long lost note or article I was looking for weeks prior. 

One of the things I came across this time is quite pertinent - thoughts on how to define SLE's and SLA's in the MSE to accommodate business capabilities and processes.  Between vacation and year end activities over the last several weeks, I have been working out just how to represent capabilities in our service model.  In this context, capabilities can be thought of as business-level services that could be standalone or steps of a process.  Of course a process itself could be represented as a capability, which starts to demonstrate some of the complexity that needs to be represented by the service model.  Examples of capabilities would be "Get Customer" or "Process Order".  I'm currently viewing these as logical representations of one or more operations in the MSE.  The "or more" perspective comes from seeing that you could have multiple interfaces that represent functions that are very similar.  For example, you could get a customer by name or by an account number.  The response could also vary from simply a name to a complete set of addresses and order history.  These would be different operations and yet the same capability from the perspective of a business owner. 

There is much that has to be learned about how this is best modeled and surfaced to users via the UI.  For now, we are going to extend the MSE UI to this "3rd layer" of the model to allow business analysts to define capabilities that an architect would then map to the operation items.  That will at least give us a palette with which to play with and learn from.  Should be fun!  Certainly moreso than cleaning my office, which I should now get back to.

I'm in the Southeastern U.S. this week and next for the Microsoft Connections Spring Symposium.  I'm speaking on The Need for SOA in the New Economy, touching on the keys to success based on our experience with customers across the world.  There are four tracks with with some great sessions spanning Information Management, Application Development, and IT Strategy.  Below is the schedule and registration link if you can attend.

· April 14 - Ft Lauderdale

· April 16 - Tampa

· April 21 - Atlanta

· April 23 - Charlotte

Over the past year and a half, there have been many articles, books, and blogs covering SOA Governance.  While I initially thought this was a good shift by the industry to focus on the process and people side of SOA, I now realize that most of these efforts are actually attempts to expand the agenda of some specific technology or product as a silver bullet for SOA.  These "SOA Governance" discussions are actually speaking to Service Governance - tools and capabilities to help organizations manage and monitor their Web services.  While service management is an important capability in an organization, the narrow focus on service governance is really missing the mark for an IT organization.  What enterprises really need to focus on is IT Governance, not SOA Governance.  The best reference I have found so far on IT Governance is the book by Peter Weill and Jeanne Ross.  It does a great job of explaining how IT needs clear decision boundaries and an accountability framework to maximize the value and efficiency of IT.  If you have a lot of "services", then you will need SOA principles and service management to account for those elements of your environment, but it has to be part of the entire ecosystem requiring visibility and control.  Governing services provides little value if you don't have the appropriate governance over the systems and processes surrounding and supporting your services.  This is akin to a city defining laws for just one neighborhood or section of town.  While the laws may differ between zones, the entire area needs a well defined governance model so that everyone can coexist, whether they be services, processes, n-tier applications, objects, or hardware.  Services by definition cannot be treated as an island and any organization that just chooses to focus on SOA Governance is unfortunately doing just that.

In the services world, there has been a lot of attention on discovery: the need to discover the existence of services through a directory or tool.  UDDI attempted to solve this problem with limited success, but has gained "checkbox status" among organizations looking for SOA products and solutions.  I have seen organizations have more success with custom portals and wiki's because they are easily stood up and supported.  However, the real value from these approaches is the inclusion of additional content such as descriptions, samples, tips, and faq's.  I think this points to a bigger need for service consumers - aiding them in their decision on whether to use the service.  I think the whole idea of service discovery has really gotten off track because it over-simplifies the discovery process and obscures the significant milestones in the lifecycle of service consumption.  This is one of the biggest reasons organizations aren't seeing the level of service adoption or reuse that they anticipated.

Discovering that something exists requires much more than visibility.  It involves discernment, evaluation, and implementation.   Most approaches today actually do a very poor job of providing visibility and then jump to the implementation stage of the cycle.  Again, I go back to UDDI which allows you to search for a service and then provides a WSDL so you can implement it.  Most of the custom solutions take a similar approach, but any additional content often improves the implementation experience.  I believe both approaches miss something very fundamental by skipping the discernment and evaluation stages of service discovery.  They actually get off on the wrong foot by providing visibility at the wrong layer.  Most consumers aren't looking for a Web service, they are looking for a discrete operation.  The Customer service may or may not be useful to somebody looking for getCustomerOrder.  Perhaps the Order service would be more appropriate.  The bottom line is that service containers are arbitrary and discovering one is about as useful as coming across a bucket.  You know there is something in it, but you aren't sure what until you take a closer look (usually via the WSDL).  Well, if you had a 100 buckets, you'd be pretty frustrated by the time you got around to the 50th bucket and had yet to find what you were looking for.  The same principle applies to services.  We need to allow service consumers to discover operations so they can discern the service they will need for accessing that operation. 

Then we get to the evaluation stage.  Just because something is a functional match doesn't mean that it is appropriate to use.  What does it cost to use the service?  Can it handle the load, meet the appropriate response requirements, or support the necessary security model?  Does the service have a track record of successful usage or recurring outages?  These are just some of the factors that go into a well-formed decision on whether to use a given service and none of them are supported by the current approaches to service discovery.  Publishing a policy can support some of these requirements, but how many examples have you seen where policy was utilized for this purpose?  I think we still have a lot to learn concerning the content and makeup of this information, but we won't learn that until we start providing any information.

Once you realize you need all this information to support the Service Discovery Process, you'll have to figure out where this will be stored and how it will be maintained.  That is where a service repository becomes paramount.  This is not the same thing as a service directory because the repository is for a provider to store all the information relevant to their services and a directory is just the data that provider wants to publish to potential consumers.  A directory should use a standard, consistent interface for use across a disparate consumer base whereas a repository needs to be extensible and flexible to store any relevant information for a fairly small user base.  Anyone who has gone through the painful exercise of trying to get a service directory to function as a repository knows you simply can't get there from here.  That is why we started from scratch in building a service repository for the Managed Services Engine (MSE) that can store all service metadata.  It is build on a relational data model that can be extended to support new requirements and scenarios.  We then use a pub/sub model to share published information and updates to a UDDI or other service directory of choice.  There is also an API on the MSE service repository that can be used to search for specific operations through any user interface of choice.  With these capabilities, we have found that organizations can have a very rich service discovery process that transcends container visibility to help them quickly determine which service to use and capitalize on the benefits of SOA.  

It recently dawned on me with all the discussion around "SOA is dead" and the desire of analysts to discover the next thing, that IT mindshare is really very similar to a reality show.  I'm not just talking about the life imitating art aspect, I mean that the IT industry behaves and evolves very similar to American Idol, Dancing with the Stars, or any other reality show with contestants, judges, and a voting public.  If you think about it, analysts are just like the judges in these shows:  Gartner could be Simon Cowell; Forrester could be Len Goodman (guy from DWTS - I had to look it up :)); the Burton Group might be Randy Jackson; I won't share with you who I think is Paula Abdul, but I'm sure you'll have your own opinions! 

These judges are evaluating the individual contestants which are the technologies and solutions that are out there being "discovered".  These can be independent ideas or solutions, products, or technologies spawned from one or more vendors.  Regardless of their origin, the judges use their credibility to influence the public to vote for specific contestants.  The enthralled public are the IT organizations eager to take advantage of a new technology or approach to make them more successful.  They vote with their budget priorities and investments.  While they will often fall in line with the judges' general consensus, a good effort can cause the public to override them.  I like that part as it at least gives one hope that a good contestant can persevere regardless of any authoritative bias.

To be fair, I'm not a big fan of these reality shows, probably for the same reason that I don't get too hung up on the names and acronyms du jour of IT.  SOA was a one time winner of "IT Idol of the Stars", but now it is a new season and the judges are eager to find the next new talent.  Before we do so, we need to recognize that while there may be a "younger version" of technology out there waiting to be discovered, at the end of the day, SOA isn't going away because it is much more than a contestant - it is a paradigm.  Just like the music won't be leaving American Idol, distributed systems built on services won't be ripped out of IT strategies.  It might change its name and approach to evolve with the times (think Starship, not Chris Gaines), but just like old blue-eyes or Johnny Cash, I have no doubt that SOA will continue to get better and more appreciated with time. 

Last night we released the February 09 CTP of the Managed Services Engine (MSE) on CodePlex.  With this release, we are crossing over into the modeling paradigm with a new WPF-based UI that allows you to not only define your service components, but visualize the interdependencies and associate configurations and items through drag and drop functionality. 

Something else we are providing with this release are a series of short videos that designed to help you quickly get started with the updated service model and the new UI.  We've been working on this release for over six months and are excited about its potential to make service virtualization more approachable than ever, but we are looking for users and organizations to provide the ultimate validation.

So its been a while since my last post...  okay, about 3 months if you check the archive.  I've been heads down for just about that long to help get the next release of the MSE ready (which will be available later this month).  We have been working on some really exciting scenarios and capabilities that will really help take SOA to the next level.  Of course, by SOA, I really mean massively distributed systems.  In this next release, we are taking service virtualization one step further by representing every single component of the service, from the server all the way back to the source system.  This not only allows you to maximize the reuse of those items as building blocks, but also provides support for some very problematic ownership boundaries.  For example, an SAP system owner can define a policy for all services accessing his system, regardless of where they are exposed and who is using them.  We are essentially dismissing the constraints that are imposed by the limited approach of one service -> one owner -> one policy. 

Stay tuned for an announcement once the next release is available on CodePlex.  In the meantime, you can catch up on the service virtualization concept by reviewing the recent white paper written by Aaron Skonnard.  This is a great executive summary that helps to outline the key concepts and value proposition for this innovative approach to SOA.

I'm over in Barcelona this week for TechEd EMEA Dev Week.  Many thanks to all of you that attended my session on "SOA Governance and Service Virtualization" on Wednesday and asked some great questions.  It is great to get some international perspective to our solutions and approach to SOA, modeling, and IT governance. 

One of the key themes at this year's conference is the value of REST.  Jon Flanders is a self-declared Restafarian who went so far as to say that REST was more interoperable that SOAP and is in fact an architecture, not just a pattern and/or protocol.  That certainly got my attention since I certainly didn't share that view walking into the session and initially got me wondering what kind of world he is living in these days.  I wasn't able to catch up with him afterwards, but had dinner with Aaron Skonnard last night where he also shared a similar perspective.

Much of the motivation behind REST is the idea that SOAP is typically an RPC-based approach and has gotten way to complex.  Aaron had a rather insightful comment that massive vendor-support is now actually working against SOAP since a lot of capabilities have been added, for better or for worse.  I certainly agree that SOAP has been pushed way beyond its original intent - Wolf Gilbert once called this process as trying to turn SOAP into a better DCOM (non-Microsoft techies can substitute your proprietary remoting protocol of choice here).  I was initially a bit resistant to SOAP myself because I thought XML over HTTP was quite acceptable for most services - for evidence, please refer to my book "Architecting Web Services" from 2001... if you can find it!  :)  However, I also feel that making a complete cut over to REST from SOAP is also akin to throwing out the baby with the bath water.

Many of the benefits of REST (structured URI's, stateless services, well-defined entities, simple transports) can easily be adopted as good practices by developers of SOAP services.  Some benefits, like caching, are a bit more problematic, but can certainly be done with a little bit of work (this is probably the only real hands-down benefit to REST today, but is still based on certain pre-conditions).  However, some of the negatives (no standard/supported method for metadata exchange, serious security models, or other policies) are absolute show stoppers for many organizations/scenarios.  REST certainly has a place in the industry as evidenced by the widespread adoption in the Web 2.0 world and certain closed systems, but it certainly doesn't work for everyone.  My initial issue with Jon's claim that REST is far more interoperable was that it came with loads of unspoken caveats, as evidenced by his demo that had him grabbing message payloads from Fiddler and indiscriminately choosing which elements in the payload to ignore and embrace in code.  One could make the case that a pipe delimited flat file is more interoperable when you are the consumer AND the provider.  I also found it humorous when he listed all the SOAP-based security bindings and chuckled at how many options there were and blew them all off.  I'd like to see him do that in front of a recent customer of ours where they had no less than three different distributed security models that needed to be supported between various systems.  

My main issue with the Restafarian attitude is that they are addressing some of the development challenges of services, while btw introducing others, and still not really addressing the most pressing deficiencies for other roles in the ecosystem.   We will start making real progress in this area once we transcend this level of debate and move on to the more significant challenges to having ubiquitous, flexible, supportable services.  We need to empower the business to get more involved in the capability creation process and they couldn't understand, much less care about, the distinction between REST and SOAP.  That is an implementation detail and you can easily have it both ways thanks to WCF.  So regardless of which side of the fence you are on, if you think it is the answer, you need to reassess your question.  In my next post I'll share with you the question my team is trying to answer.

I'll be in Barcelona for TechEd Dev Week to present on SOA Governance & Service Virtualization (SOA01-IS) on November 12th at 9am.  My good friend and colleague, Wolf Gilbert (VP with SOA Software), will also be presenting on SOA Governance on Nov 6th at 1pm, during the IT Pro Week.  Please drop by if you want to see some of the latest on what Microsoft and SOA Software are doing in this space.

Last week, SOA Software announced their certification of the Managed Services Engine (MSE) as a governable solution, along with their continued support for .NET and BizTalk.  This release highlights some of the collaborative work taking place between Microsoft and SOA Software to provide a richer end-to-end solution for customers to leverage the strengths of WCF's integration and extensibility with SOA Software's rich management tools and policies. 

http://www.soa.com/index.php/section/company_press_detail/soa_software_announces_expanded_soa_governance_for_microsoft_net_framework/

Steve Martin today posted a preview of some of the announcements that will be made during PDC later this month:  http://blogs.msdn.com/stevemar/archive/2008/10/01/the-road-to-pdc-net-framework-4-0-and-dublin.aspx  I know that Microsoft has taken a couple of shots from people that thought the splitting of Oslo and Dublin signaled a lack of focus or strategy, but I actually think it reflects a natural maturation process around some of the scenarios that CSD is trying to tackle.  Enterprises are getting more complex and sophisticated and any attempt to streamline productivity or increase visibility and manageability of these environments is going to require a lot of harmonization.  The days of easy answers to problems are over and it is going to take models, servers, and tools to effectively orchestrate that level of coordination.  I am very much encouraged to see us tackling these scenarios because I see us not just delivering some new capabilities that are kind of cool, but I can see the big picture for how it can all come together to solve the bigger challenges of massively distributed systems (aka SOA).  I hope that attendees of the PDC will get at lease a glimpse of the concerts to come while getting their hands on a couple of the new instruments.  :)

I have been very critical of the SOA articles that have been written over the last several... well, ever since people started writing them.  They have typically come across as very self serving (simply promoting a product, vendor, or organization) and providing very little insight to others starting to embark on the SOA journey.  I belief this is a significant factor in why very, very, very few organizations are really making much progress with SOA.  We talk about the potential reuse that comes from SOA, but very few organizations have been able to reuse the lessons learned by other organizations that are either experiencing success or making mistakes.  That is pretty obvious when every article is rehashing the same topics, focusing on a single application and/or the basic value proposition of Web services.   I can't tell you how many articles I've been sucked into with an enticing title that simply talked about how somebody benefited from the interoperability of Web services.  While these articles can provide some value (especially if they get past the promotion of a particular product or vendor), they have contributed to the industry confusion between Web services and SOA and have at best neutralized their benefit.  Part of the problem is that organizations aren't opening up very much in this space - probably because a lot of early SOA efforts are mistakes and people don't like talking about those.  The other part of the problem is that SOA transcends technology by involving people and processes and technical writers aren't inclined to branch out into those dimensions, either because they don't recognize them, they don't want to write about them, or they don't think their readers are interested.

However, I have found the relevance and quality of articles improving quite a bit over the last few months.  I have found, of all places, CIO Magazine (http://www.cio.com/topic/1498/SOA) to be one of the best examples of this shift.  They have a dedicated section to SOA and have published some real gems like the recent article on how Qualcomm broke down their IT Silos through SOA and blogs on organizational challenges, skill sets, and testing.  I would have never even given CIO Magazine a thought if not for somebody specifically calling my attention to it a couple months ago (thanks Michael!)  This probably shouldn't have surprised me too much because CIO Magazine probably has the type of writer that is comfortable with the blend of organizational and technical challenges and their various dimensions.  Part of this improvement can also be attributed to organizations that are opening up a bit more to these writers.  This can also be found in the quality of material coming from the analysts, which is also very encouraging.  Hopefully this trend continues as it is the sharing of this end-to-end experience across all dimensions that will allow other organizations to be more productive in their SOA efforts and accelerate our maturity as an industry.