Just a quick note.  After reading through some of the feedback on my recent post on "the root causes of poor software requirements," I had to agree with some of the respondents: I had forgotten a branch in the analysis. 

So, for the past week, I've been stealing a few minutes here and there to review the analysis and add updates.  I just posted an updated analysis, adding a new top-level branch (at the end) and adding a sub-branch about half-way through.

Feel free to check it out. 

Now, to take this to the next step: anyone can figure out the steps and costs involved with improving your requirements-gathering practices.  Here are the steps.

a) Steal the list right from the post.

b) Load it into an Excel spreadsheet.  Add the following columns: Score, Mitigation type, Mitigation Tactic

c) For each item, add a score, from 0-5.  See the weighted ranking below.

Score 0 if this cause does not happen in your environment
Score 1 if this cause occurs but is easily overcome in your environment
Score 2 if this cause occurs but it doesn't occur frequently or drive substantial quality problems
Score 3 if this cause occurs occasionally, and the business analyst or program manager finds the flaws
Score 4 if this cause occurs frequently, and/or the software developers or testers find the flaws
Score 5 if this cause occurs frequently and the flaws are found in deployment or production

d) For each item that ranked 3,4, or 5, write a statement for how you think you should mitigate it in the Mitigation Tactic column.  Feel free to brainstorm many possible ways, but then select the way that you believe is most feasible.

e) For each mitigation, add a value in the Mitigation Type column.  use one of the types below or create your own list.

Use type 'Training' if your mitigation is focused on improving the skills of a participant
Use type 'Automation' if your mitigation is focused on moving information efficiently or using workflow tools
Use type 'Data' if your mitigation is focused on improving the information collected or making it available
Use type 'Process' if your mitigation is focused on analyzing workflow, allocating responsibility, or insuring accountability
Use type 'Staffing' if your mitigation is focused on adding staff to a role or creating a role that doesn't already exist

f) Sort your analysis by type and score.  For each group of mitigations by type, you have a workstream (or sub-project).  That effort can be estimated and an approximate cost established.  For the 'Data' and 'Automation' groups, you have a list of objectives that can be used to select software for requirements management.

g) Pull together a project proposal from your cost estimates and propose a project to your management that outlines the steps you intend to take, the problems you intend to solve, the amount of money that you believe it will cost, and (here is the hard part) the value of the quality gains that you expect to reap.

Good Luck!

If I had a nickel for every time I've heard a developer complain about poor quality requirements, I'd... well... have a lot of nickels.  Let's look, for a moment, at the root causes of poor requirements and business rules.  While I consider this to be a business problem, and not a technology problem per se', I'll use root cause analysis (discussed in a prior post) to organize the analysis.

The problem, as perceived by developers, can be quoted this way:

Problem Statement: The requirements for software, as delivered by typical business analysts, is not sufficiently clear, insightful, or well understood to develop software systems that meet the needs of business users. 

There is an assumption here.  One that I won't challenge (today) but one that probably should be qualified:

Assumption: Improving the quality of software requirements will have a net positive effect on the quality, reliability, applicability, usability, and value of custom software as perceived by the business users who use it.

Interesting assumption, that one.  Perhaps fodder for a later blog post.

So, how do we help software developers deliver timely valuable code without driving the business users crazy with technical details that they'd rather not bother with or understand?  Let's look at the causes of our problem. 

What follows is a (long) root cause analysis of "poor" software requirements.  This can be used to understand the problem, which is the first step to addressing it.

How to read this analysis:

Each entry is seen as a "cause" of the problem.  Indented below each are the answers to the question "why" or "what is the cause of this cause?"  As you read this list, insert the word "because" at the start of every sentence.

The text may look a little odd because of the indenting.  An analysis like this looks good on an Ishikawa diagram, but that's pretty tough to do on a blog.  The ultimate goal is to ask why five times, or to get to five levels deep.  The numbering does not go from 1. to 1.1 to 1.1.1 as the levels increase.  That is due to a limitation of my blogging tool.  I also don't go to "five levels deep" in many areas when there is no value in doing so.

Note: A SME is a Subject Matter Expert.  An Analyst will gather requirements from a SME, but may then need to review those requirements with other business stakeholders.

Problem: The requirements for software, as delivered by typical business analysts, is not sufficiently clear, insightful, or well understood to develop software systems that meet the needs of business users. 

1. Business stakeholders may not believe that it is important to invest time in creating clear, unambiguous requirements.
   1. Business stakeholders perceive "requirements collection" as a time consuming activity
      1. When an analyst starts to collect requirements, the analyst requires a LOT of hand-holding.
         1. Analysts may not routinely understand key aspects of the business.
            1. The business may change in ways that the analyst is not aware.
                
            2. The analyst may not remember key aspects of the business that they may have seen in the past.
                
            3. The analyst may not have made an effort to learn or understand the business.
                
            4. The analyst may not be able to get sufficient face-to-face time with business SMEs due to geography, language, time zones, or security policy.
                
         2. Business SMEs may find themselves explaining the same things, over and over, to different analysts.
            1. Analysts need deeper understanding than is available from orientation courses.
                
            2. Analysts are unable to document the business with sufficient clarity to benefit other analysts.
                
            3. Analysts may not be motivated to create documentation for other analysts.
                
            4. Turnover in the analysis team may be high, causing the need to "re-explain" key concepts.
                
         3. Analysts often ask questions where the answers have not been decided yet, especially for new business programs or innovative new products. 
             
         4. Business SMEs may not trust Analysts in meetings with high-ranking executive SMEs
            1. Analysts may suggest a change that will cause a business leader to lose power or prestige.
                
            2. Analysts may question a leader's pet project or assumption.
                
            3. Analysts may suggest a change to business that, if performed, would drive a great deal of effort and cost on the part of a person who does not want to do that work.
                
      2. When an analyst collects requirements, s/he takes a long time to write things down.
         1. The steps involved in documenting requirements may seem arcane or unclear to the business stakeholder.
             
         2. Business stakeholders may underestimate the amount of time it takes to write good requirements.
             
         3. Business stakeholders may not be aware of the contributions (and delays) of other business stakeholders in the requirements gathering process.
             
         4. The analyst may be using his or her time in an inefficient or ineffective way.
             
         5. The analyst may be working on many projects at one time, may be working part time, may be playing many roles, or may have some other reason why they cannot dedicate full time to documenting requirements.
             
      3. When a business stakeholder reviews the requirements, it takes a long time to validate them.
         1. Requirements are written in a "funny way" that takes getting used to.
             
         2. The business stakeholder may delay the start of the review process
            1. Requirements documents may be quite lengthy, and the business stakeholder may feel the need to try to tackle the document in one sitting. 
                
            2. The business stakeholder may benefit from delaying or sabotaging the analysis effort.
                
            3. The business stakeholders' immediate superior may request a delay in the start of the review effort.
                
         3. Business stakeholders may not place a high priority on validating requirements, which slows the process down.
            1. Placing a high priority on reviewing requirements may cause friction for a business stakeholder that is involved in "competing" activities.
                
            2. The review process may be perceived as an ineffective use of the stakeholder's time.
                
         4. Business stakeholders may delegate the effort of validating requirements to a person that is not qualified to make decisions, thereby delaying the validation process.
             
   2. Time spent in developing requirements does not seem to improve business operations in other ways. 
      1. No other part of the business gets value when a business SME spends time talking to an analyst.
         1. Business SMEs may feel that providing requirements does not add value to the stakeholder's other responsibilities. 
             
         2. Business SMEs are often interrupted in their daily activities to provide requirements.
             
         3. Business SMEs may feel unprepared to provide useful information.
             
         4. Business SMEs may fear reprisals if they provide a bad requirements, so it may be politically risky to share a requirement.
             
      2. No other part of the business gets value out of reading the written software requirements.
         1. The detail required by software people is not interesting to others.
             
         2. Requirements are written in long terse sentences that will put an insomniac to sleep.
             
         3. Collected software requirements are not useful for driving other business efforts.
             
   3. The business stakeholder cannot tell when to invest time or resources in producing good requirements and when the investment is wasted.
      1. The connection between poor requirements and defects is poorly understood.
         1. Business stakeholders may not believe that a defect is caused by a poor requirement.
             
         2. Developers have a difficult time determining if a defect will be easy to fix.  It can take days, or weeks, just to figure out how hard a fix will be. 
             
         3. Developers may argue that a defect that is caused by a poor requirement is not a defect, but rather a change request, forcing the issue to be addressed in a different way altogether.
             
         4. When presented with a defect, a business stakeholder is rarely able to see the requirement that led to it. 
             
         5. Business stakeholders do not understand how to identify which kinds of requirements need the most attention in order to have the best impact on defect rates.
             
      2. IT professionals do a poor job of guiding business stakeholders to focus on the requirements that will most impact the cost and quality of a solution.
         1. Business analysts often do not understand which requirements will "drive costs" and which will not.
            1. Simple cost analysis tools do not exist.
                
            2. Simple formulas, methods, or procedures do not exist for this kind of outcome.
                
            3. Where tools methods and processes do exist, few analysts know about them.
                
         2. Business analysts present information in a manner that does not highlight "cost driving" requirements
            1. The most common tools for collaborating on requirements are Excel and Word, which do not have features specific to requirements gathering.
                
            2. Most business stakeholders have no way of easily reshuffling, highlighting, sorting, or prioritizing requirements, so they cannot use their own concerns to select the requirements to review.
                
2. Business analysts may not add sufficient value in the collection and analysis process to drive down software defects.
   1. Many business analysts may not be aware of how to add value to business requirements.
      1. The business analyst may be poorly trained or may not have sufficient experience.
         1. The organization may not have invested in training and support for the business analysis role.
             
         2. The analyst may be new in his or her role.
             
         3. The analyst may not have sufficient basic skills in business or computing to be able to perform the role at all.
             
      2. Effective training and practices may not be available to the analyst.
         1. Long term studies on specific best practices in business analysis are lacking.
             
         2. Shared training and leadership in the business analysis profession is just getting started.   
             
   2. Many business analysis efforts are understaffed, have poor tools, or insufficient time to add substantial value.
      1. The project manager who planned the analysis may not understand the amount of effort needed to collect proper requirements.
          
      2. It may take longer than estimated, or be more difficult than expected, to collect requirements, causing a budget squeeze on the amount of time and money that can be spent on them.
          
      3. The project manager who planned the analysis effort may benefit in other ways by intentionally undercutting the time, resources, or tools for analysis.
          
   3. Many business analysis efforts do not have sufficient access to key decision makers to add the right amount of value.
      1. Interaction with business analysts may be delegated to people who cannot make decisions.
          
      2. Business analysts may be seen as outsiders, causing the organization to keep them at bay.
          
      3. Key decision makers may have prior bad experiences with IT professionals, creating a culture of "Avoid IT"
          
   4. Business stakeholders may oppose the effort of business analysts to add value to the requirements
      1. Adding value to the requirements may increase the cost to the business.
         1. The business may need to commit more resources to validate the requirements if the analyst adds value to them.
             
         2. The analyst may need to access more stakeholders and SMEs, for a longer period of time, to add value to the requirements. 
             
         3. The analyst may need to use data gathering means like surveys or travel that incur costs in order to add value to the requirements.
             
      2. Business stakeholders may oppose the efforts of an "outsider" to analyze or document their business beyond basic software requirements gathering.
         1. An existing resource within the business may be responsible for business analysis, business modeling, strategic development, or process improvement who would perceive additional analysis efforts as "overlapping" or wasteful.
             
         2. Business leaders may fear the political consequences of allowing an outsider to see, and document, how their business operates.
             
      3. The business stakeholders may not trust the interpretation of requirements written by an analyst.
         1. The analyst may use different words than the business does.  The business stakeholder may not trust or understand those terms.
            1. The analyst may have chosen to use different words than the business stakeholder to improve consistency between various requirements, between stakeholders of the project, or with other projects in the same line of business.
                
            2. Analysts may elect (or be forced) to use standardized terminology that the business stakeholder is not familiar with.
                
         2. The analyst may draw conclusions about the business model or business processes that the business stakeholder does not agree with.
            1. The analyst may lack the authority to draw the cited conclusion.
                
            2. The analyst may be recommending a change in the business that the business stakeholder believes will not work, or will work against current company policy or strategy.
                
            3. The analyst may lack the sufficient business acumen to explain their conclusions well.
                
            4. The analyst may lack the sufficient situational awareness to recognize the political or organizational implications of delivering the conclusion.
                
            5. The analyst may be using a different underlying paradigm of business or economics than the business stakeholder.
                
         3. The analyst may elect (or be required) to use a requirements gathering methodology that the business stakeholder does not like or agree with, leading to a lack of trust in the conclusions.
            1. The methodology may ask questions that the business stakeholder is not prepared to answer or that the business stakeholder believes will produce an incorrect conclusion.
                
            2. The methodology may assume that the business stakeholder is more knowledgeable, empowered, or self aware than he really is, producing conclusions that the business stakeholder cannot agree with (or, in some cases, understand).
                
            3. The methodology may involve people (internal or external) that have a stake in the failure of the particular business stakeholder, or that part of the business.
                
      4. Business stakeholders may not want to see contradictions in their business highlighted in text by a business analyst.
         1. The business stakeholder may benefit from those contradictions for their continued success.  For example: A business stakeholder may want to protect a business program funding model that allows "shadow" projects to be funded, even if the system being described calls for accountability, visibility, openness, and a fair funding model.
             
         2. The business stakeholder may have been responsible for removing a problem in the past, and they may have declared the problem "solved."  Any conclusion by an analyst that suggests that the problem still exists will work against their credibility with their own manager.
             
      5. Business stakeholder may benefit from structure within their business that they do not want an analyst to call attention to.
         1. The business stakeholder may believe that their success rests on controlling a particular business capability, even if doing so is skirts company policy.  For example, a business leader may have created a marketing team or IT team within their own business, even when the executives have issued directives to remove these independent units and consolidate to a central function.
             
         2. The business stakeholder may have a responsibility that other parts of the business do not want him or her to have.  For example, the business stakeholder may be chartered to complete a project that their business unit is not, technically, supposed to be working on. 
   5. The business analyst may be unable to properly reconcile conflicting requirements from multiple SMEs (updated)
      1. The business analyst may not know that the requirements conflict.
      2. The business analyst may view the reconciliation process as "not my problem"
      3. The business analyst may lack the skills to negotiate a reconciled set of requirements
      4. The business analyst may reconcile the requirements improperly, resulting in requirements that are consistent, accurate, and wrong
          
3. Software developers may not invest sufficient time and effort in understanding the requirements.
   1. Requirements may be difficult for developers to read and understand.
      1. Developers may be daunted by a large requirements document.  (see related cause 3.5 below).
         1. The format, delivery method, or structure of the requirements document may force the developers to consume it in paper form, HTML form, or some format that the developer finds non-navigable.
             
         2. Security or access restrictions on the document may make it difficult for the developers to consume and remember the document of the size delivered.
             
      2. Developers may not understand the text of the requirements document.
         1. In some cases, the developers may not natively speak the language that the document is written in, reducing comprehension.  (example: developers in China reading a requirements document written in German).
             
         2. The requirements document may have been translated from one language to another, causing a loss in readability.
             
         3. The analyst may be writing the requirements in a language that they are not sufficiently skilled to write in, creating grammar and logic errors that reduce readability.  (example: an analyst from the USA who speaks French poorly, writing in French for a French IT group to use).
            
         4. The analyst may have written the text in a logically inconsistent manner.  In other words, errors in thinking or structuring the information may come through, making the business requirements difficult to understand.
            
         5. The writing style, grammar, punctuation, formatting, or layout used by the analyst may be difficult for the developers to accept or consume.  For example, the developers may prefer a particular numbering style or indention rules that the analyst did not follow.  Culture may play a role in the consumability of the requirements document.
             
      3. Developers may not understand the cultural context of the requirements.
         1. The software system may be intended for users in other parts of the world than the developers are familiar with.  As a result, statements in the document may be perplexing or vague when taken out of the context of the culture in which the software will be used. (example, an IT division in the USA building a system marketed to Japanese accountants).
             
         2. The software system may be intended for users in other parts of the enterprise itself where business cultural rules drive specific business policies.  If a developer is not familiar with those business cultural rules, even if the users live in the same country, the requirements themselves may be perplexing or vague.  (example: an IT division that provides services to an airplane manufacturer that builds both military and civilian aircraft).
             
      4. The business terms and concepts in the requirements document may be unfamiliar, inconsistent, non-standard, or poorly defined.
         1. The document may not contain sufficient information about what the terms are, what they mean, or how one term relates to another.  (Example: a requirements document may describe the need for the CCC division to access a particular report, without either defining what the acronym means or making it clear that "account managers" may work for both the CCC division and the LJM division).
             
         2. The developers may have used some of the same terms in a different way, requiring them to unlearn their prior understanding in order to adopt the unique definitions in the requirements document.
             
         3. The terms and concepts may be inconsistently described or applied.  Example: one part of the document may describe requirements for an "administrator" while another part of the document may describe an "account manager" when the business intended to describe only one role, or the business process calls for only one role.
             
         4. The terms and concepts may have more than one meaning and it is not clear which meaning is being implied.  For example, the requirements may refer to "services" provided to customers and "service levels" applied to system reliability, without differentiating the use of the term "service."
             
      5. The business context of the requirements may be unfamiliar to the developers.
         1. The developers may not understand how the business itself makes money, how the partners make money, or how the products/services of the business are positioned in the marketplace.
             
         2. The developers may not understand what externally generated rules or regulations are influencing the business decisions that show up in the requirements.
             
         3. The developers may not understand the business strategies being implemented or how those strategies are supposed to produce results for the company.  For example, if a business leader decides that their products should be introduced at a particular price point in order to beat a competitor, that may lead to the need to keep the costs of a particular customer service process very low, driving specific software requirements.  Without tracability to the business strategy, the developer may not understand the need to automate the entire process to the exclusion of a small number of potential customers.
             
      6. The business processes described in the requirements may be incomplete, poorly described, or inconsistent.
         1. The business processes themselves are chaotic, inconsistent, ad hoc, or flawed.
             
         2. The business may not have a consistent set of roles and responsibilities defined.
             
         3. The analyst was unable to get a consistent explanation of the business process.
            1. Business may not have experts that understand the business processes from end to end.
                
            2. Business SMEs may not be available to the analyst to explain or validate the business process models.
                
            3. The business stakeholders may benefit from keeping the business process hidden or obscured.
                
         4. The business analyst may not have sufficient skill or training to produce effective business process models.
             
         5. The business analyst may not have had sufficient time or resources to describe business processes.
             
         6. The inclusion of business processes in a requirements document may not have been adopted as a practice within the particular enterprise.
             
      7. Developers may not understand the diagrams used in the requirements document.
         1. The developers may not be familiar with the diagramming notation used by the analyst.
            1. The analyst may not have used a standard diagramming language like BPMN or UML to capture information.
                
            2. The analyst may have used a standard notation in a novice or inconsistent manner.
                
            3. The developers may not have sufficient skill or training to read the diagramming notation.
                
         2. The developers may not find the diagrams to be useful for understanding software requirements.
            1. The diagrams may illustrate information that is not important to the developer who is consuming it.
                
            2. The diagram may illustrate information from a viewpoint that the developer does not understand.
                
            3. The diagrams may be include objects of different types that do not make sense to combine.
                
            4. The diagrams may introduce concepts that are not explained in the text.
                
            5. The diagrams may actively contradict the accompanying text or may not be explained at all in the accompanying text.
                
         3. Important content within the diagrams may be obscured, missing, or difficult to find.
            1. Analysts may have made poor use of color, losing content when printed in black and white, or when consumed by color-blind developers.
                
            2. Analysts may have included too much or too little text on the diagrams.
                
            3. Analysts may have mixed many concerns onto a single diagram, making the diagram needlessly complex.
                
            4. Analysts may use a metaphor in the diagrams that the developers are not familiar with, or find confusing or offensive.
                
   2. Requirements are boring and tedious to understand
      1. Analysts may adopt a "special language" for describing the requirements that feels unnatural or fails to illuminate the requirements.
         1. The "special language" may require sentences to written in a restricted subset of the language.  Flaws in the methodology may produce some "requirements statements" that are convoluted or difficult to understand using that restricted subset. 
             
         2. The software developer may not be familiar with the need for accuracy in requirements, causing them to reject "special languages" as unnecessary overhead, reducing the effectiveness of communication.
             
         3. Analysts may have adopted a "special language" in an inconsistent or novice manner, making it difficult for the intent of the requirement to be understood because the developer stumbles on flaws in the writing.
             
      2. Requirements are usually written in terse, humorless prose with few diagrams
         1. Embedded jokes may offend a stakeholder.
             
         2. Light-hearted or pleasantly written requirements text may be perceived by stakeholders as "not taking the problem seriously."
             
         3. Analysis may fear that developers will dismiss a requirement as "nonsense" if it is pleasantly written.
             
         4. Analysts may lack the skills to write requirements in a more engaging manner.
             
         5. Analysts may lack the skills to create useful diagrams to illustrate the requirements.
            
         6. Existing diagramming methods may not have been adopted by the organization.
             
      3. Requirements documents may place emphasis on sections of text that do not illuminate the problem.
         1. Requirements may be written in a manner that reflects the political desires of the business decision makers that need to approve it, causing emphasis to be placed on elements that the business leader considers valuable, rather than elements that developers would consider valuable.
             
         2. Requirements may be written in a manner that intentionally obscures flaws or defects in the business processes or business structure.  Obscuring these details may require that specific business rules or descriptions are written in a vague or incomplete manner.
             
   3. It may be difficult to describe how requirements specifically translate into design
      1. The design process may be seen as a creative process that is inspired by, but not tied to, requirements.
          
      2. The developers may find it difficult to tie specific aspects of the design to requirements.
         1. The developers may use reference architectures or patterns that have features that exceed the requirements.
             
         2. The developers may not understand which feature of a reference architecture specifically suits a requirement.
             
         3. Processes, methods, and formulas for deriving design from requirements may not be commonly accepted or widely followed within the enterprise.
             
      3. Standards of the organization may introduce requirements that must be met, but which the developer feels that they cannot justify to the business stakeholder.
          
      4. Developers may feel that the analyst has missed a requirement and therefore may add a design element to meet the "shadow requirement" without documenting the requirement itself.
          
   4. It is difficult and time consuming to insure that a design will meet requirements.
      1. The tools in use by the organization make it difficult or impossible to tie specific requirements to specific aspects of the design.
          
      2. Organizations may not have adopted standard methods or practices for connecting requirements to design.
          
      3. Developers may not be rewarded for spending time connecting requirements to aspects of the design.
         1. Tracing design to requirements may not be an activity expected within the organization.
             
         2. Developers may not view the activity as valuable.
             
         3. Project managers and project leaders may not be aware of the effort or value of the activity.
             
         4. Business stakeholders may not be aware of the value of demonstrating tracability in design.
             
         5. Project leaders may view the activity as "more expensive than beneficial."
             
      4. More than one design may have been considered, driving up the cost of tracability for each design considered.
          
      5. Developers may have made an intentional tradeoff that allow some requirements to go unmet.
          
      6. Requirements may be changing as the design process, making it frustrating for the developer to tie requirements to design.
          
      7. Iterative design processes allow some requirements to be left out of consideration until a later time.  At the end of iterative design timeframe, the project team may not have funds to deal with requirements that remain unmet by any design candidates.
          
   5. Requirements may contain details that any particular software developer is not interested in
      1. The organization's requirements management tools may not allow the software developer to select the requirements that are salient for him to understand.
          
      2. The organization may have adopted the practice of "write down everything" in an effort to improve requirements, triggering information overload.
          
   6. The project manager may not have allocated sufficient time or resources for developers to understand requirements
      1. The project manager may not understand the process or the difficulty involved with understanding requirements.
          
      2. The budget for the project may not allow for sufficient time spent on understanding requirements.
          
      3. The organization may not culturally value the effort of understanding requirements.
          
      4. The developers may be using inefficient means to understand requirements.
          
      5. The tools and techniques in use by the enterprise may make it more difficult for the developers to understand the requirements.
          
   7. Software developers may resist reading or understanding a particular requirement or set of requirements
      1. Since requirements change, developers may feel that it is a waste of time to invest in understanding them.
          
      2. Developers may want to substitute their own judgement for the judgement of the business stakeholders and analysts 
         1. Some developers may recognize common problems that the analyst did not describe, or may wish to bring insight from other experiences.
             
         2. If a developer does not understand a requirement, it is often simpler to substitute a different requirement (or ignore it) than to invest in understanding it.
             
         3. The developer may not trust the analyst, or the business stakeholder, to correctly describe the problem.
             
      3. A developer may not see the value in investing time and effort in understanding a requirement.
         1. Software developers may not have been trained on the implications of poorly understood requirements.
             
         2. The organization may inadvertently reward developers for not spending time to read requirements.
            1. Software developers may not be measured on the number of defects they produce.
                
            2. Many organizations place an emphasis on the test team, not the development team, to insure that a system meets requirements.
                
4. The analyst may not have acquired sufficient information to create a complete list of requirements. (updated)
1. The analyst may not have searched sufficient sources of information
   1. The analyst may not have spoken with the right people
      1. The most knowledgeable SME exists in the business, but the analyst does not interview him or her
         1. He or she has changed jobs or been promoted, and the new manager does not view their contribution as "part of their job"
         2. He or she has lost favor, so the analyst is encouraged not to speak with him or her
         3. The SME does not announce his knowledge or experience to the analyst
         4. The business does not know who the most knowledgeable SME is, and does not direct the analyst to him or her
      2. The most knowledgeable SME no longer exists in the business
         1. The SME has retired
         2. The SME has been laid off to cut costs
         3. The SME has been fired
         4. The SME was a contractor and their contract has ended
         5. The SME was a consultant and their project has completed
      3. The analyst may not be able to communicate effectively with the most knowledgeable SME
         1. The SME may speak a different language than the analyst
         2. The SME may work in a different location than the analyst
   2. The analyst may not have collected requirements from existing software
      1. Existing requirements may not be documented
      2. Existing design, constraints, and business rules may not be documented for analysts to use
   3. The analyst may not have collected requirements from external regulations
      1. The analyst may lack awareness of the regulations that apply
      2. The analyst may not have the necessary expertise to collect requirements from regulations
      3. The analyst may fail to get the business to share regulatory requirements
         1. The business may be relying on external experts that are not available.
         2. The business may be relying on external experts that are not sufficiently skilled.
         3. The business may not have analyzed the regulations for their requirements
         4. The business' analysis of regulatory requirements may be flawed or incomplete
2. Some of the information may not be available to the business
   1. The business may be taking on a new venture
   2. The business may be merging with another business that is sufficiently different from the existing business models.
   3. The business may be insourcing an existing function
   4. The business may be split off of an existing business (for example: IBM split off Lexmark as an independent printer business).
   5. The business may not know what existing requirements must change
      1. Existing requirements may not be searchable
      2. Existing requirements may not be documented
      3. Existing requirements may not be understood by anyone
      4. No one in the business is accountable for analyzing existing requirements for change
3. The analyst may collect conflicting requirements from different SMEs (see related section 2.5)
   1. SMEs may have differing personal opinions of how to solve specific problems
   2. SMEs may have different philosophies and/or business models in mind for the future of their business.
   3. Business leaders may have provided conflicting guidance to business SMEs.
   4. SMEs may have personal or political reasons for suggesting conflicting requirements
4. The business users may not know how to share requirements correctly
1. The business may not have trained their SMEs on "how to work with IT"
2. The business users may not have consistent terms or processes
3. The business users may not be aware of IT services
4. The business users may not be aware of the kinds of features that software can provide

Whew.  That list is long, and probably missing a few things, but it is a fairly good attempt at describing the root causes of "poor software requirements."  (There are 149 184 listed 'causes' of poor software requirements in the list above, only counting the leaf levels of the analysis).

Some takeaways from this effort:

1. Skills and training show up many times.  Select or develop standards and then train staff members to use them.
2. Creating useful requirements that get used... easy to say, hard to pull off, especially with so many potential roadblocks. 
3. Good, inexpensive, well thought out, requirements management tools can fix many of these problems. 

There is a joke that I sometimes like to refer to, more as an allegorical story than anything else.  This version is from AJokeADay.com:

A man in a hot air balloon realized he was lost. He reduced altitude and spotted a woman below. He descended a bit more and shouted,” Excuse me, can you help? I promised a friend I would meet him an hour ago, but I don't know where I am."

The woman below thought carefully for five minutes, and then replied, "You are in a hot air balloon hovering approximately 30 feet above the ground. You are between 40 and 41 degrees north latitude and between 59 and 60 degrees west longitude."

"You must be an engineer," said the balloonist.

"I am," replied the woman. "How did you know?"

"Well," answered the balloonist, "you took a long time to respond, and everything you told me is technically correct, but it is of no value to my problem.  I am still lost.  Frankly, you've not been much help so far."

The woman below responded, "You must be in management."

"I am," replied the balloonist, "but how did you know?"

"Well," said the woman, "you don't know where you are or where you are going. You have risen to where you are, due to a large quantity of hot air. You made a promise which you have no idea how to keep, and you expect people beneath you to solve your problems. The fact is you are in exactly the same position you were in before we met, but now, somehow, it's my fault!"

So it’s funny, but it is a useful story as well. 

All architecture, in any real sense, is an attempt to communicate a complex set of ideas.

Architecture is an answer to a question.  So many architects strive for accuracy in their “answers” (the architectural diagrams they produce), and we see countless discussions of the “correct” way to model this thing or that… but while accuracy is great, usefulness is so much more important. 

In some ways, that is what the IEEE 1471 / ISO 42010 standard is all about.  For those of you not familiar with IEEE 1471, it is a metamodel for all architecture.  This simple document frames architecture as an attempt to communicate, using the language of architectural models. 

But what is more important in the standard is not that architecture communicates… it is the fact that architecture, in order to succeed, must communicate to the specific concerns of specific stakeholders.  In other words, you must consider the needs of the audience before delivering the requested information, and then deliver what they need in a clear manner, even if it is not technically what they asked for.

In the joke, an engineer responds to the question from stranded businessman that he is 30 feet off the ground.  Accurate but unhelpful.  An architect would consider the businessman to be a stakeholder, and would take his concerns into account. 

Instead of replying with data that is of no value, the architect would toss up a cell phone to allow the businessman to call his friend and reschedule.  The businessman would still be lost, and hovering in a balloon… but at least his pressing concerns would be met. 

Honestly, what else could you ask for?

Well, it is February 2nd, and today, the Open Group is announcing the general availability of TOGAF version 9.0.  For those of you not familiar with TOGAF, it is an ambitious and maturing Enterprise Architecture Framework created by the members of the Open Group.

I’ve had the good fortune to be allowed early access to the TOGAF 9.0 specification, so while most of the readers of this blog will be seeing TOGAF for the first time today, I’ve been typing these notes for about a week now.  This is going to be a bit of a random discussion of good points and opportunities to improve the TOGAF, now that their new direction is clear.

Overview and Impressions of TOGAF 9.0

I’m not a certified TOGAF practitioner.  That caveat aside, I am someone who has studied multiple frameworks, and created an architectural framework, so I can draw on a good bit of experience when reviewing one.  I spent some time reviewing the prior version of TOGAF, and my first impression of the new version is this: Substantial and Deep Improvements.

The new version has a comprehensive model for the content, insuring that the stakeholders for architecture have separate sections of the framework dedicated to each of their varied concerns.  This makes navigation and adoption considerably easier.  In addition, substantially new content, along with new models and a richer set of descriptions, have been added.  The new framework is more readable and more usable than it’s predecessor.

The focus of TOGAF 9.0 has been sharpened, drawing the distinctions between different concepts and activities into the light.  This was needed for strategic architecture to be successful in to TOGAF model.  The terminology is more consistently described and used, and the attention to detail is refreshing.

The TOGAF is, and probably always will be, a work in progress.  It is a community effort, and the community that worked on this version have expended considerable effort.  That effort shows.  I really like where this framework is going.

Here is my call to action: for the first time, I’m willing to say this: The TOGAF is enterprise ready.  If your organization does not have a framework, or if you are using Zachman with some home-grown methods, I recommend that you serious consider TOGAF 9 for adoption. 

Specific things that I appreciate

The use of a metamodel

While the TOGAF metamodel is new and untested, I strongly appreciate that the framers of TOGAF were aware of metamodels at a sufficient level to include one at all.  This is a huge improvement.  In the coming weeks, I may get a chance to review the TOGAF metamodel in detail and provide insight into specific elements. 

Partitioned Architecture 

For years, the Zachman framework (ZF) has stood as the de-facto partitioning model for architecture.  In many organizations, the only thing that the CIO knew about enterprise architecture was a single word: Zachman. 

So teams around the world adopted John Zachman’s framework.  The problem is: the framework is not universally useful.  It is an implementation of a core concept: separate the different attributes of an architectural description and categorize your model.  Zachman not only demonstrates the concept, he implements it, and as a result, obscures the right of others to implement the same concept in a different way.

But there are other viewpoints than the rows represented in the Zachman Framework, and other subject areas than the columns he chose.  The concept is good, but the ZF implementation is not the only rational view. 

The TOGAF 9 takes a “metamodel” view of Zachman, providing a set of attributes that can be used to partition architectural models.  The ZF can easily be viewed as one implementation within the Partitioned Architecture mechanism described in TOGAF. 

In this way, the framers of TOGAF have given us the language to get past the ZF and allow many more views to emerge, without being bound to the ZF partitioning model, while remaining respectful of the ZF implementation as a valuable contribution to Enterprise Architecture.

Clarification of “Service”

TOGAF defines Information System Services as being different from Business Services.  As my prior blog points out, I couldn’t be happier!

The addition of Guidance

The good folks who worked on this current version are clearly in touch with the pain points that any adopting organization would feel when attempting to use the TOGAF, especially if they are not familiar with Enterprise Architecture already.  One of those pain points goes like this:  This is a great way to “think” about architecture, and “develop” architecture, but how do I “use” the framework in my business?  A new guidance section attempts to begin to answer that question. 

The guidance section is currently small, and I think it should be quite large (suggestions for improvement follow).  However, it is a start, and I’m glad to see that the start is there.

Opportunities to improve

1. Use process models to illustrate processes: A big chunk of TOGAF is dedicated to the methods used to develop architectural artifacts, called the ADM (Architectural Development Method).  It contains processes and terms and descriptions… good stuff.  However, when we, as architects, are called upon to describe methods and processes for our customers, we use diagrams, not just text.  The ADM does not use diagrams to explain the processes.  BPMN, please.
    
2. Include all the core terms in the metamodel: The Content Framework specifies a wide set of concepts.  In addition, various models in the TOGAF provide even more.  The metamodel does not cover all of them, nor does it provide any view that actually matches the other diagrams.  This is a point of incompleteness, and is not a criticism, since this is a new area of the TOGAF.  It is simply a suggestion for improvement.  I’m happy to help.
    
3. Adopt existing standard definitions – It is one thing to call an object by two names.  We can refer to a “business objective” by many different phrases, as long as we use the same definition.  It is another thing altogether to take an existing word, like Project, Business Process, Architecture, and API, and create an alternative definition for it.  (All three of these terms have definitions that are at variance with established standards).  That means creates an increasingly difficult problem for organizations that adopt TOGAF.  Writing your own definitions to common terms is a bad habit.  
    
4. Branch out into IT Management Processes – It should be no surprise that a framework, ostensibly targeting Enterprise Architecture, would be used by the strategic planning organization within an IT shop.  Yet, there is no specific description of a process where a planning organization would actually use an architectural model to fulfill their plans.  This is where the guidance section can go, and should go.
    
   That is not to say that TOGAF doesn’t have processes.  The ADM is basically a large set of processes.  There are plenty of discussions of the process of creating an artifact or model, but minimal descriptions of how an organization uses any of them.
   
   In the past, leaving out these business processes was a good thing.  Not any more.  I’d like to see processes specifically describing IT Project Portfolio Management, IT Application Portfolio Management, IT innovation funding, IT standards governance, and IT operations management.
    
5. Clarify the distinction between a generic business capability and an  architecture capability – The TOGAF is often adopted by organizations that are in the early stages of adopting Enterprise Architecture or attempting to improve EA maturity.  That adoption process requires that the adopting organization must turn “the lens of architecture” upon themselves, and when they do, they need to rationally discuss the business capabilities needed, by the architecture team, in order to perform a strategic enterprise architecture function.  This is a subset of overall business capabilities, specific for EA.  I call them architecture capabilities. 
   
   However, at some point, the newly minted architecture team must turn that architectural lens back to the business.  When what occurs, we use the term “business capability” in a more generic sense, to refer to the capabilities needed by any of the core business functions like Sales, Manufacturing, Fulfillment, and Customer Service (among others). 
   
   Unfortunately, the framework text frequently uses the word “capability” without any qualifier, leaving the reader to figure out which context the word is being used in.  In some places, the authors meant ‘architecture capability’ while in others, they meant ‘business capability.’  The concept of capabilities can be confusing to many business leaders as it is.  If the architect cannot eloquently describe the concept, adoption can be slowed.
    
   We need to make sure that we empower the adopting organization with a clear distinction between these two uses of the term “capability.” One suggestion is to add a strong differentiation into the text of the framework itself.  Another is by consistently avoiding the use of the unadorned term “capability” and instead using the more accurate qualified terms of ‘architecture capability’ or ‘business capability.’

Conclusion

Use TOGAF 9.0.  It is major step forward in architectural development.  While I had a few suggestions, they are not obstacles.  There is nothing in the TOGAF that prevents TOGAF v.9 from being useful as it stands today.