I ran into a friend today and, as friends often do, we let our conversation wander over the different "broken things" in IT in general (and a few in Microsoft in specific).  One thing that I'd like to share from that conversation: a truly bizarre assumption that we teach, over and over, to new programmers... the assumption that simply following a "functional decomposition" process, a well-trained programmer will naturally end up with a good design.

Now, I'm no great expert on product design or graphic design or industrial design... but one thing I can say for certain: creating a good design is not the natural outcome of a simple process.  Only an excellent design process can produce excellent design.

Let me provide an example of good design from the world of products.  This picture is a picture of a footrest.  You read that right: a place to rest your feet.  Mundane, right? 

You tell me.  Does this product LOOK mundane to you?  How about the fact that it promotes movement and blood flow while serving it's primary function? (special call out to the design firm, humanscale, for creating this beautiful product.)

Design is not just a process of decomposing a problem into its constituent parts.  Nor is it a process of creating objects that contain functionality and data... yadda-yadda-yadda.  There are dozens of techniques.  Don't read my blog post as a slam on any one of them.  I'm slamming anyone who thinks that they should reach a conceptual architecture, derive one solution, and go... without considering alternatives.

Design is a process where you consider the problem and then propose multiple, competing, wildly creative solutions.  You then narrow down your brainstorm and weed out the bits that won't work... and then you propose multiple, competing, wildly creative refinements... and the cycle continues.  This happens a couple of times, until you have refined your solution by being creative, then logical, then creative, then... you get the picture.

When was the last time you were in a design review and the team being reviewed came in with multiple solutions, asking for help to narrow it down?  Really?

In no other field is the word 'design' so misused as it is in software development.

I have not met many folks that use a process whereby they design multiple, competing, wildly creative solutions in software and then evaluate them, select one or two, and go after the problem again at a finer level of abstraction. 

Not many folks at all.

Why is that?  We are living with the myth: that you can follow a simple process, produce a simple and "pretty good" solution architecture that represents a "good design".  No alternatives.  Very little creativity.

How's that working for ya?

I've seen various taxonomies of requirements.  Like all taxonomies, any set of requirement types exists to classify or partition requirements into coherent groups for further analysis.  Most break down the list of requirements into things reminiscent of "who or where the requirement comes from."

For example, one taxonomy I've seen recently described:

• Business Requirements - high level business goals
• User Requirements - the user experience needs
• Functional Requirements - business process or functionality needs
• Non Functional Requirements - all other requirements (like quality attributes)

Another taxonomy may be:

• Information requirements - needs for information storage
• Functional Requirements - needs for functionality
• Non-functional requirements - all other requirements

I've seen others as well.  Most will have a category that contains "non-functional" requirements.  And there's where my heartburn lay. 

When creating classifiers of a type, whether in OO, or in taxonomy efforts, it is a very good idea to avoid creating a type called "All-Other."  If you create a type called "All-Other," that tells me that you don't really know enough about your domain, and you don't know why you have things in your domain that you cannot classify, but you do, so you create a category for "everything I cannot classify" and throw all elements in. 

How do you know you have one of these types in your taxonomy?  If the definition of the class contains a negative, as in Non-Functional or Non-Testable or Non-useful.

Basically, the category of 'non-functional requirements' is an "all-other" category.

Over the years, software development has matured to the point where we have categories for most requirements, and they are well understood, so the stuff that falls into the "non-functional requirements" category is very constrained.  We have a coherent set because we have identified all of the elements that don't belong there.  Yet the name remains.

I'd like to suggest that we kill off the classification of "non-functional" requirements, and replace the name with "quality metric requirements."  Basically, that's all that is left in the modern "All-Other" requirements class: those requirements that reflect a measurable goal of system quality, usually expressed as a metric.  For example: "the online store must be available for browsing of the product catalog 24 hours per day, reliably 99.99% of the time."  Availability is a notorious 'non-functional requirement.'

But if we replace the category of 'non-functional requirements' and call it a quality metric requirement, then we get three benefits:

1. We can make the statement that all 'quality metric' requirements are actually derived from a measurable goal, not a fiction.  The business should not say 'I want 2 second response time' without explaining why that is important.  A reasonable requirement, like a 2 second response time, can be connected to the customer expectations or the business competitive strategy. 
   
2. An less obvious relationship may be drawn when he business says "I need this system to be operating 99.999% of the time."   Anyone who has seen a requirement like this one knows that a "5-nines" requirement will definitely affect the cost of the solution, and probably the amount of time needed to test and deliver it.  If the customer needs this kind of reliability, they should be asked the answer "why."  By classifying this requirement as a quality metric, and by requiring that each quality metric must be defined, it should be much easier to catch those situations where the business has gold-plated their list of requirements.
   
3. By removing the 'All-Other' classification, we lose the temptation to use it to toss in "other" requirements that we have no real understanding of.  This forces a level of quality into the requirements gathering process.  

So my suggestion for a requirements type taxonomy would be:

• a) Business Ability Requirements-- high level or "one liner" requirements that identify the high level statements of functionality we can expect to come directly from a business user.
   
• b) Data Relationship Requirements -- the understanding of logical data entities and their relationships, expressed as software requirements to model and store information that matches those data entities.
   
• c) Reporting requirements - the understanding of the contents of documents, reports, and artifacts that are either generated by or consumed by the business processes themselves, often in the form of process artifacts.  Basically, any time your software facilitates communication between two people, or outward from a person to an external party, you would capture reporting requirements.
   
• d) Functional interaction requirements - the requirements most easily drawn from an understanding of the processes that a user or customer will use when interacting with the software, functional requirements specify conditions and behaviors that must be met.
   
• e) Quality Metric Requirements - the requirements that are drawn directly from business strategy or goals, including those that recognize customer expectations for software of a particular type, and those that establish or recognize a competitive position for the company in the marketplace.  This includes the software quality attributes like reliability, availability, usability, and flexibility.

It is time to get rid of the 'all-other' category of software requirements. 

My friends in the Agile community have succeeded in drilling a concept into my thick skull so deeply that the concept shows up in other things I do.  What is that concept: don't try to build the perfect app.  Build the least complicated app that will do the job.  Let the customer tell you when you are done. 

Makes sense.  Too bad there are so many developers who still insist on making this bit or that bit of code "really solid" or "reusable" when no one is paying for anything more than "functional" and "bug free." 

So that bit of agile philosophy tends to get repeated a lot, even by me.  There are a lot of people to reach, so we hammer home the concept: do the simplest thing possible... to the point where I use it even when I'm doing the ultimate BDUF exercise: Enterprise Architecture.

We tend to say things, in EA, like this: "we are not just about building apps right.  We are about building the right apps."

But to be really honest, no one really knows what the "right" apps are.  There are no tablets of stone that contain a perfect list of applications that should be funded or that should remain in the portfolio.  We are humans and we make human judgements, using the best tools we have. 

So the trick is to remember this: "Correct" is a point of view.  If you think that a particular list of applications should exist, or should be funded, it doesn't matter if you think you are correct.  That is your point of view.  Another person, with another point of view, may believe him or herself to be just as correct.  You have to sell your concepts (and yourself) to be impactful. Help others to see your principles and how you used them to pick your list.  Help them to share your point of view. 

The challenge is not to do the "right" or "perfect" things, but to do a good job... and not to 'gold plate' the decision process with tons of special justifications or long meetings.  Make a 'functional' decision, dare I say, 'agile' decision.  Use the minimum amount of fuss that produces a good result. 

That, my friends, is Agile Enterprise Architecture.

The (ISO Standard) RM-ODP model is a powerful and well reasoned mechanism for creating Architectural descriptions ("architectures").  Leveraging the IEEE-1471 taxonomy, and building out a visual style and standardized approach, there is tremendous value in learning and using this the RM-ODP (Reference Model for Open Distributed Processing), and I'm getting to the point of recommending it.

That said, there is a gap in one of the most fundamental areas of the RM-ODP model.  RM-ODP specifies exactly five viewpoints.  This term is defined by the authors of RM-ODP as:

A viewpoint (on a system) is an abstraction that yields a specification of the whole system related to a particular set of concerns.

In other words, if you want to communicate with Joe, and Joe cares about 19 things, you'd better cover those 19 things when discussing the system with Joe.  He has 19 concerns, and we can group those concerns together into 3 viewpoints (for example), and provide a "visual language" (my phrase) that can be used to communicate those concerns.

The next sentence, from the RM-ODP Overview (ISO/IEC 10746-1), is the problem:

Five viewpoints have been chosen to be both simple and complete, covering all the domains of architectural design.

Simple?  Not so fast.  The views produced by RM-ODP are far from simple... but...

Complete?  I disagree.  Big disagreement.

For those readers who are unfamiliar with the RM-ODP, this model describes five viewpoints:

• the enterprise viewpoint, which is concerned with the purpose, scope and policies governing the activities of the specified system within the organization of which it is a part;
• the information viewpoint, which is concerned with the kinds of information handled by the system and constraints on the use and interpretation of that information;
• the computational viewpoint, which is concerned with the functional decomposition of the system into a set of objects that interact at interfaces – enabling system distribution;
• the engineering viewpoint, which is concerned with the infrastructure required to support system distribution;
• the technology viewpoint, which is concerned with the choice of technology to support system distribution

(There are many things wrong with this list.  I'm only describing one of them here.  Another will be described in a later post).

One viewpoint that the RM-ODP forgot is the one that matters the most to Enterprise Architecture: Alignment.  I propose that the RM-ODP be extended to include the Alignment viewpoint.

The alignment viewpoint is concerned with the strategic and measurable purpose for the existence of a process or business system, and the justification for changing the features, structure, and operational profile of a process or business system, at a particular time, and in a particular organizational and technical environment.

The key concepts here:

Process or Business System -- This viewpoint is NOT limited to describing a technology or line-of-business application.  The views may (and typically do) limit themselves to business and informational views that do not illustrate any specific "ODP" system at all.

Purpose -- The notion that a system comes into existence for the sake of fulfilling a purpose.  That purpose is described in terms of business capabilities, business processes, business policies, business quality metrics, and business information.  These explicit concepts describe the environment in which a system adds value. 

Justification -- The notion that any change to a system has to be justified.  Enterprise Architecture is the business function concerned with insuring that all justifications align to the business changes implied by business strategy.  Insuring that justification is based on strategy is an activity frequently referred to as 'alignment.' 

Note that, in a mature organization, alignment must be demanded to justify a change in any system, not just a software system.  Changing a system requires care to prevent negative consequences on routine business operations, whether it is a system of people behaving with common goals, or teams behaving in a process, or applications behaving in a sequenced activity.

Some examples of Architectural Views that fall into the Alignment viewpoint:

Business Capability View -- An illustration of the business capabilities of a particular business unit (at any level of a business hierarchy where clear responsibilities and accountabilities exist).  Concerns for a business capability view may include the importance to strategy, staff readiness, process uniformity, cost of IT maintenance, flexibility of IT change, uniformity of information, maturity of process measurement, and scorecard value of process performance.

Business Process View -- An illustration of the business processes and process activities necessary to support one or more business capabilities.  May be demonstrated at varying levels of granularity, with the highest level representing abstract business processes (like "Marketing") down to task level processes (like "Deliver Marketing Brochure to Publishing Team").  Concerns may include role and responsibility clarity, efficiency, effectiveness, uniformity, delegation of authority, and service level management.

Enterprise Project Portfolio View -- An illustration of the business change programs in place throughout the business and how they align to meet the strategic and operational needs of the business.  Concerns for an Enterprise Project Portfolio View may include overlaps by feature area, relative budget estimates by category, interdependencies by time, and feature delivery by time.

Enterprise Application Portfolio View -- An illustration of the operational systems and processes in place to run the business, and how well those systems are prepared to adapt to potential risks and strategically driven projects.  Concerns for an Enterprise Application Portfolio view may include applications that are impacted by multiple projects in close temporal proximity, applications that share similar functionality but different data stores, applications that share data stores but remain inconsistent in their implementation of rules, and applications that with high risk metrics (risk of failure * impact of failure).

Enterprise Information Federation View -- An illustration of the core informational elements of the enterprise (like "customer," "product," "market offer," "shipment," and "service request").  Concerns for this view include addressability (the ability to find a unique data entity, using a readily accessible identifier, anywhere in the enterprise), consistency (the ability to insure that information that is shared by many people has consistent use throughout those who use it), and federation (the ability to apply the control of key data elements to the level that is closest to the team that needs or uses it).

Enterprise Technology Portfolio View -- An illustration of the supporting business capabilities required by the business, and the use of shared technology platforms to meet those capability requirements. 

While I have not met anyone who had told me that these views should be in the "enterprise viewpoint" as described by RM-ODP, I'm prepared to defend the notion that the enterprise viewpoint does not, in fact, cover this space.  (A different post, perhaps, if it is necessary).

Todd Biske, whom I respect for his writings on SOA, seemed to miss the mark in his recent blog post about SOA Governance and Decision Rights.  In that post, he said:

if you focus on education, you can allow individual teams to make decisions, because you’ve given them the necessary information to make the right decisions. If they don’t have the information to make decisions that will lead toward the desired behavior, it turns into a guessing game. Not only can there be confusion in what the correct decisions are, there can also be confusion on what decisions should be escalated up the chain. If we instead focus on creating policies and making those policies known so that anyone in the organization can make the right decision, we’re in a much better state.

Was Todd wrong?  Nope.  He was right on. 

In describing governance, Todd just described a useful, and workable, set of decision rights.  I don't think he realized it, because the blog was essentially trying to refute the concept of Governance as decision rights!

What were those decision rights he describes?

• "You've given [individual teams] the necessary information to make the right decisions" -- Implied in that statement is the notion that the individual teams, having made the right decisions, will not have those decisions taken away from them by someone else.  In order for a team to make a decision, they must clearly have the right to make it and, here is the kicker: Management Must Respect That Right.  Want that to happen?  Be explicit.  Tell everyone what decisions we will let the team make, and then hold them responsible for making them.
   
• "There can also be confusion on what decisions should be escalated up the chain" -- to avoid that confusion, we must be clear, as Todd correctly points out, about who has the right to make which decision.  Where does a decision stop?  That clarity avoids red tape.  That explicit clarity is called "decision rights."
   
• "If we focus on creating policies" -- And here is really the confusion.  What are those policies called?  They are called "decision rights." 
   

SOA Governance is a subset of IT Governance, on all three aspects of Governance: design time, deploy time, and run time.  SOA decision rights are a subset of IT decision rights, which are a subset of overall IT policies. 

What we have here, is a failure to communicate.

Imagine a time when building architecture meant "sketches" that would vary from one architect to another, one type of building to another.  It must have been quite difficult for the skilled tradesmen to build anything more than individual excellence... to deliver repeatable quality... because the "instructions" they were working from were so wildly different from one situation to the next. In some cases, the blueprints may have been vague, or even inaccurate.  In other cases, they must have been maddeningly specific.

Software Architecture is still at that stage.  We have the UML, thank goodness, but we don't yet have a good, reliable, standard set of 'documents' that we can use to describe an architecture in a consistent way... or do we?

One effort focused on moving Software Architecture to the next level of maturity is the ISO (International Standards Organization), working to create and sustain the RM-ODP (the Reference Model for Open Distributed Processing).  (Actually, the standard comes from a joint effort of three different standards bodies: ISO, IEC and ITU-T).

In May of 2007, the good folks behind the RM-ODP released a set of profiles for UML called UML4ODP.  This is a major move, and one that all architects should take note of.

The RM-ODP is cool because it attempts to describe the "set of models" that can be used to specify a software system.  A good analogy: The RM-ODP tells you what diagrams a set of blueprints should contain.  If you are an architect, and you use the RM-ODP, you can provide a "standard" set of blueprints. 

This is cool, because developers can then receive a standard set of blueprints from the architect.  The goal: to help the Software Development profession rise to a level of 'repeatable' quality. 

Using standard architectural descriptions, we can better estimate the cost of software.  We can encourage specialization on specific components, much as home builders can subcontract electrical wiring to one speciality, and the heating system to another.  We can use agile techniques on some parts of the system, and "spiral-iterative" techniques on others (if that appeals to you). 

We can even begin developing system services that can simply be 'plugged in' to a standardized architectural description (much like a building architect can use a stencil to draw a picture of a bathtub, knowing that it will be the right symbol, and size, regardless of which manufacturer is ultimately chosen to supply the actual tub).  (We are not there yet).

The RM-ODP indicates that a system should be described from five viewpoints: "Enterprise," "Information," "Computational," "Engineering," and "Technology."  

Personally, I don't think that these five viewpoints cover the gamut.  As I dig, I will surface more viewpoints: perhaps two more, from what I can tell now.  Regardless, this is the place to start, and I highly recommend that folks do a little digging on their own, especially if you are more than a few years out of college (like me).

When software architecture does make the move to "profession" from "craft," which side do you want to be standing on?