I've been thinking a lot lately about the gap between "what we have" and "what we need" in the Enterprise SOA space.  I think I have a need that is not yet filled by software.  (that I'm aware of).

I put up a post back in June about the difficulty in creating a common information model in a large enterprise, especially one with a federated environment like ours.  (CISR coordination and diversification models).  The feedback I got back was telling.  The majority of respondants told me what I suspected: developing an enterprise-wide common information model was so difficult that many folks considered it unfeasable.  (Actual words included things like "utopian" and "big design up front."  I'll stick to "unfeasable").

That said, I have also stated in public that I believe, firmly, that SOA at the enterprise level requires some levels of agreement on the way that information is understood and coordinated.  Each business unit can own information that is specific to that unit, but in the areas of coordination, where there is value, the business needs to be able to communicate.

So, we have something that is hard to do (build a CIM and keep it up to date).  That something is useful (to get the benefits of Enterprise SOA).  So, why not take the software approach?  After all, I do work at Microsoft.

What business scenario would this tool need to support?

Basically: each business unit would submit their information model to a common repository.  The submitters are architects, and they MUST align the models in some way, even if it is just to show that there are differences.  Services are posted to the same repository, and must be lined up under specific information models.  In order to consume a service, the business has to agree to the information model.  Multiple competing models are allowed to co-exist.  What do we get?  An economic model of self-governance that produces an optimum information model: one where agreement is reached only where agreement makes financial sense.

Specific capabilities:

• A business unit architect can consume part of an information model without consuming the entire thing.
   
• A business unit architect can assert part of an information model without proposing the entire thing.
   
• A business unit architect can offer services tied to their information model.
   
• A business unit architect can assert that a portion of an information model is "standard" for their use
   
• A developer, writing software for that business unit, can easily find and adopt their information model.
   
• A project team can provide a report to a governance committee proving that they are conformant to local standards
   
• An enterprise architect can run reports to isolate "points of difference" between business unit information models.
   
• A system designer, intent upon consuming services from another business unit, can begin a workflow to insure that the consuming business unit agrees to the information model of the presenting business unit. (an "information adoption workflow").
   
• The workflow needed for a consuming business unit to agree can be custom-tailored to the organization.
   
• Organizations that present a service have an automatic measurement mechanism built in allowing them to "charge back" the businesses that consume the service.  Various financial models must be supported (one time fee, pay per use, annual licenses).  This provides economic incentive for sharing of code, as well as the incentive to create services that align to commonly needed information models.
   
• Built in support for the versioning of information models over time (including both past and future versions) allows the business to change their minds, and even chart their course.

That's what my gut tells me.  This has some pretty interesting effects:

1. Information architects have a clearly defined and critical role at the earliest stages of a project: get consensus on information model changes needed to allow the consumption of existing, lower cost services.
    
2. Economics will drive good behavior.  No need for an Enterprise Architect to "design" good behavior. 
    
3. Less emotion.  There will not be consensus on everything, and this model doesn't require it.  Consensus will be reached surprisingly easily on some key areas, and it will happen without any architect looking to make it happen.  This helps remove politics from the picture as well.
    
4. It is easier to adopt existing code than build new code if services, offered by other groups, aligned with the information standards of the consuming business, are already clearly identifiied. 

We may be closer than we think.  With bits from various MS products, and with Oslo coming, this vision is getting closer to reality.  It's an end-to-end idea. 

Something to consider.  Comments?

Not long ago, I was asked an interesting question about our Enterprise Architecture team.  The question was "Does Microsoft provide the internal support to create an excellent Enterprise Architecture program?"

The answer is "yes" but it got me thinking: what qualifies as "excellent?"  That term is subjective.  In our business, what does it mean to be "excellent" and how might that differ from another business?

Excellent, to me, means that the effort is tailored to the needs of the business.  That includes business strategy, business structure, and corporate culture.  Our business, in Microsoft, is the business of developing and distributing software.  We are pretty good at it, although we have our critics.  

So our EA program is just that: tailored to the needs of Microsoft.  We don't do more than Microsoft needs, or less than Microsoft demands.  We push the envelope, as change agents and thought leaders, but we don't crimp creativity... let's face it: we make money on applied creativity.  If one idea out of 1,000 makes money, we earn back the investment.  It's a unique space to try to operate an EA program in.  We are excellent, but probably not typical.

I can only conjecture about what "excellent" would look like in another company.  We pay industry analysts and attend conferences, just like many of you do.  Part of the reason: to listen and learn about how practitioners in different companies do what they do.  Basically, we are trying to find out how our peers describe excellence for their own enterprise.

How do you define excellence?  Are you there yet?

This is the story of four people named Everybody, Somebody, Anybody, and Nobody.

There was an important job to be done and Everybody was asked to do it.

Anybody could have done it, but Nobody did it.

Somebody got angry about that, because it was Everybody's job.

Everybody thought Anybody could do it, but Nobody realized that Everybody wouldn't do it.

Consequently, it wound up that Nobody told Anybody, so Everybody blamed Somebody.

A Use case is a cool thing.  A little too cool.  The term has been occasionally misused, and in some respects, that misuse diminishes the value of a use case.  To succeed, we have to know what a use case is.   When you are done reading this post , you will still know what a use case is... but you will also know what a use case isn't.

What a use case is

The following section is a direct excerpt from “Writing Effective Use Cases” by Alistair Cockburn.

“A use case captures a contract between stakeholders of a system about its behavior. The use case describes the system’s behavior under various conditions as the system responds to a request from one of the stakeholders, called the primary actor. The primary actor initiates an interaction with the system to accomplish some goal. The system responds, protecting the interests of all the stakeholders. Different sequences of behavior, or scenarios, can unfold, depending on the particular requests made and the conditions surrounding the request. The use case gathers those different scenarios together.” (Cockburn, 2001)

With all due respect to Cockburn, his discussion doesn't so much define a use case as describe one.  There are very few formal definitions available in the public domain or in reference works.  Here is my attempt at a more formal definition:

A use case is a formal description of an interaction between an actor (usually a person) and a system for a specific purpose or goal.

Many of the discussions of use cases in the literature go into great detail about the requisite parts of this formal description.  Most include the concept of 'actors', 'use case scenarios,' 'preconditions,' 'postconditions,' and a stated 'goal.' 

Things to notice

1. In a use case, there are always at least two actors, and one of them is a system.  The use case is a description of system level interaction... in rich detail. "Enter name and address and click the 'enter' button."  There is very little about a use case that is abstract or high level.
    
2. The amount of formality is not part of the definition.  In fact, Cockburn specifies that you should create a use case in a fairly informal way at first, when the system is still being understood.  Only in a later iteration of the requirements, when the project is funded and the scope is reasonably well understood, should the specifics of the use case be added.

What a use case is not

As I mentioned before, the term "use case" has been used in many ways, and it has been applied in some pretty unusual things.  To be effective, we should recognize that a use case is a tool that is tailored to one purpose, and using it for a different purpose may not be optimal. 

1. A use case is not a description of a business process.  The use case describes the interaction between a single actor and a system.  At best, that interaction can be considered a single (atomic) activity in a business process.  A business process is much more than that, including many activities from inputs to outputs in support of a goal.  Let's not pretend that use cases describe business processes.  One activity, perhaps two... that I will buy.  Rarely, if ever, anything more.
    
2. A use case is not decomposable into other use cases.  It is the atom.  Break it down and you have parts that are not atoms.  Combine use cases and you have composites (molecules) that are not atoms.  A use case is the description of an interaction between person and machine.  That is all.
    
3. A use case is an inappropriate tool to describe system to system interaction.  Certainly you CAN use a use case this way, just as you CAN drive a screw into wood with a hammer.  But it is not optimal to do so.  A much better set of tools include UML Interaction diagrams, protocol descriptions, standard identifier formats, and WSDL.   
    
4. A use case is used to elicit requirements but it is not the requirement itself.  Requirements need to be collected and called out as statements.  A couple of noted authors have weighed in on the skills needed to describe and understand requirement statements.  Both analysts and developers should learn these skills.  
    
5. It is optional to use the use case approach.  While I'm a fan of use cases, I'm also in a role where we have to draw clear distinctions between the work that someone must do and the work that someone should do.  The requirements must be collected.  Use cases should be used.  If you can collect requirements in a different way, that is not wrong.  That said, I'm fairly comfortable stating that the use case approach is a 'best practice' for describing requirements to software developers.
    
6. For traceability and requirements validation, use cases are not the source of requirements.  Requirements come from the business needs, and most of the business needs are fairly easy to connect to specific stages of the business processes (with some fascinating exceptions).  As I pointed out in my prior post, I view the source of most business requirements to be the business processes and customer experience scenarios that software must support.  
   
   Therefore, if you want to determine if a requirement is needed, or provides value, or has been completely met, it is better to trace the requirement back to the business process.  The use case is an abstraction along the way.  (This is my opinion, of course, and your mileage may vary).
   
   (note to contributors: the distinction between functional and non-functional requirements is too vague to clearly delineate the requirements that are not easily traced back to business process or user experience scenarios.  There's another blog post in there somewhere.)

In short, a use case is an essential and valuable tool in the Business Analysts' toolkit.  Let's use it wisely.

Requirements elicitation is a critical, yet under-appreciated, activity.  A core capability of business analysts, the ability to get the customers to describe what they want, and need, is both a science and an art.  Requirements elicitation requires equal measures of careful planning, situational awareness, acute listening skills, business acumen, and a kind of optimistic tenacity. 

I have long taken for granted two basic principles of requirements elicitation. 

First: a rule: The goal of requirements elicitation is to first understand, and then document, the needs of the business for a software solution.

Secondly, a process: if you want to understand a system (be it a software system or a system of business processes), look first at the business problem that it solves.  Look second as how people use that system to solve the problem.  Look next at the information that moves from one point to another.  Look last at the technologies that have been identified and consumed by the system.

To be honest, I don't know where these notions came from.  I don't remember reading them in any book, although I suppose that I may have done just that.  They are assumptions that I have used, for a long time, to organize my thinking about software requirements.

Yet when I went looking for industry sources that discuss the need to trace a requirement from the business problem, through business process, I could not find any. 

The "Guide to the Business Analysis Body of Knowledge (BABOK)" published by the International Institute of Business Analysis goes into some depth about requirements elicitation.  in the BABOK, the authors discuss many techniques for getting users to talk, including Brainstorming, Focus Groups, Job Shadowing, and JAD sessions, to name a few.  Unfortunately, their description of the traceability matrix, where a requirement is traced to something valuable, is sparse IMHO (This is true for BABOK version 1.6.  Perhaps the upcoming version 2.0 will provide examples?)

To my surprise: there is no mention of using a model of the business process as the source for software requirements!  As far as the BABOK is concerned, requirements come from people... not from a process model that may have required the consensus of six or sixteen people to help build it.  (Perhaps I missed it.  If there is such a reference, in v1.6 of the BABOK, please reply with the section where I can find it).

Of course, this only meant that I needed to search out other sources of information... Someone, somewhere, must be talking about using business processes as the source for IT software requirements. 

Now, many of you are probably saying: what about use cases?  Aren't they simply descriptions of a business process, specifically for describing software requirements?

The answer is: go up a few layers of abstraction... away from the software... I'm interested in seeing how the subprocesses, from an enterprise viewpoint, are driving requirements.  I'd like to see how specific activities in a BPMN business process model drive requirements.  The requirements can be captured and shared in the form of a use case, or a big list, or a database in Visual Studio, for all I care.  But we don't start with the use case.  We start with the process.

Or at least, I do. 

Do you trace your requirements back to business process models?  If so, why?  If not, why not?  I'd love to hear from others.  

Building teamwork, at the enterprise level, is a tricky thing.

As a project team comes together to solve a problem, hopefully you find yourself in the same position that I've found myself in many times: with smart experienced people, all motivated to succeed.  Microsoft IT is chock-full of folks like this... and it's a big organization.  Couple of thousand folks.  So, it's pretty normal that when I start working with a project, there's always one or two smart motivated people on the team that I've not had the pleasure to work with before.

Thus begins the 'relationship-building' aspect of teamwork.  Meet a new person.  Figure out what motivates them.  Communicate.  Share.  Build trust.

Nice thing about a project team: your success is managed.  The majority of the team members are measured by the success of the project, and often they are full time on the project.  People can work together to accomplish things fairly quickly.  This is not usually the case at the enterprise level.

When working with a distributed organization, virtual teams become more important.  Here, you find smart people, motivated to succeed, but they are nearly never full time.  Getting consensus and buy-in on common goals becomes a high-order problem.  Without it, there is no traction.  And with people contributing a few hours a week, or month, to your deliverables, a lack of traction can be the difference between delivering in June and delivering in October.

And so the 'relationship-building' aspect of teamwork, at the enterprise level, is an entirely different game.  At this level, you need to harness the things that people are passionate about.   You need to find the influencers, and influence them.  You need to make sure that subject matter experts feel engaged, and empowered, and heard.

Building a conceptual model is a great way to get that to occur.  Getting agreement on the concepts, and business rules, for a business can bring people together.  It becomes a way to build common ground, establish relationships, and get different people, in different parts of the organization to see value in working together.  It is a work product that people can feel good about, and that will be useful.

When you build the conceptual model, you build relationships with the people whose ideas you are capturing.  Which is more important... the work product or the relationships? 

Both.