Inside Architecture

Notes on Enterprise Architecture, Business Alignment, Interesting Trends, and anything else that interests me this week...

February, 2007

  • Inside Architecture

    The importance of a name


    Enterprise Architecture is a young science.  I was reminded of that today as I sat through a presentation by a Garner analyst, listening to details of "EA Best Practices".  Part of the presentation was this interesting statistic: of the companies surveyed with EA programs, 75% of those programs were less than five years old.

    That doesn't mean that five years ago, there were 75% fewer programs.  It means that most programs don't stick around for more than five years, because EA is difficult to do well, and it is pretty difficult to keep business and IT stakeholders actively engaged through the years where the program shows progress by keeping bad things from happening (which is pretty tough to measure).

    So, making an EA program is hard, and keeping it running is hard, and you often find yourself just trying to keep the "value" moving. 

    So you really can't waste time making things up.  You really can't waste time trying to invent things that already exist.  There isn't any time.  You have to show value.  Put up or shut up.  The people in EA are valuable to the organization.  It is painful, to the organization, and sometimes to the architect, to take these super talented people out of roles where they create systems and put them into roles where they keep systems from being created incorrectly. 

    So when you find an artifact or heirarchy or model or tool that you can use, use it.  If it makes any sense at all, and can save you a week or an hour, adopt it.  It is easier to adopt than create.  It is easier to buy than to build.  Creating a model from scratch should be a last resort. 

    So if your company has adopted Zachmann or TOGAF or Gartner or one of the other EA frameworks, and an artifact can be pulled out of a different framework and used, don't worry about the source.  As long as the taxonomy doesn't conflict, and the model is useful, just use it.   

    Continuously improve (refactor) your program in small increments.  Deliver value in short regular intervals.  Listen to your customers, write up stories that describe what they need, and deliver it. 

    Sounds like agile development?  It is. Only EA style.  Agile EA. 

    Now, that's a name I can live with.

  • Inside Architecture

    Top 100 patterns: Building from a nascent catalog


    My respect for Grady Booch has grown over the past few days, as I've investigated the idea of building a list of the 100 patterns that every developer should know.  As I've surfed around looking at pattern-oriented sites, including those in MS-land, I've come to recognize and appreciate the amazing amount of work that Grady Booch has put into his 'software architecture handbook' site.

    He even has a heirarchy of systems by type, a better list than the one on Wikipedia.  I will work on the Wikipedia pages to clean them up.  They needed a reference source anyway.

    For those of you who haven't taken a look at his site yet, visit and sign up (free). It's worth a look.  It is still a work in progress, and to be fair, I believe that he is hindering his own progress by using a traditional site instead of a wiki, but it is a fairly good start on the herculean tasks he has set out for himself.

    I am not so patient.  When I consider the fact that there are clearly over 2000 published patterns, many overlapping in intent, I've realized that the work of selecting 100 core patterns will be really tough to do without much of the basic cataloging already done (as Grady Booch has started to do). 

    As most scientists recognize, you cannot effectively analyze a body of knowledge until that knowledge has been cataloged.  Naturalists of the 19th century discovered this and spent countless hours of labor before the major breakthroughs in understanding were reached.

    (Yes, I've been to the Portland Patterns Repository.  With all due respect to WardC, the PPR is a self-organized site created by intensely creative people, and as such, there's not a lot of cataloging going on.  As a result, a good navigation of patterns, or even a good comprehensive list, is not readily available there. Booch's work, while still in its infancy, is already more comprehensive and better organized than the self-organized PPR.  That doesn't mean that the PPR should be dismissed.  It just doesn't solve the initial problem.)

    Alas, I cannot do this by myself.  I don't believe my 25 years in software to be sufficient, by itself, to this task.

    In academia, when you need help, you seek out a graduate student eager to pick a thesis topic, and you convince him or her that your research idea is worth their while. Not sure if the same tactic would work for this space, but I'm mulling it over. 

  • Inside Architecture

    Top 100 patterns: how to pick the list


    In my previous posts, I suggested that we should create a list of the top 100 patterns that every developer should know.  Of course, the challenge is this: how to pick the list?  There are thousands!

    Let me start with criteria and see if we can approach it logically (alas, logic is the developers greatest weapon and most bothersome weakness.)

    So, the question I want to ask this group is two-fold.

    1. What are the criteria for considering a pattern as part of the set of "the most important patterns so far," and
    2. What patterns do you believe should be members of this set.

    The goal here is communication.  We are building a common "base of concepts", so for the sake of this set, we want patterns that explain or demonstrate fundamental ideas, or which other patterns are composed of, so that we can communicate amonst ourselves.  Therefore, we may elect to include a pattern in the set that we don't expect to use very often, simply because it does a fine job of explaining a key concept or being useful for understanding other patterns.

    Some ground rules:

    1. No patterns that describe the behavior of humans reacting to one another, collecting requirements, or performing the act of writing, deploying or maintaining software.   These patterns belong to the areas of organizational psychology and process management.  For the sake of this list: the pattern must be implementable in code.
    2. No algorithms, even if the algorithm has been documented as a 'pattern' in a peer-reviewed work, this list needs to differentiate between patterns that establish, demonstrate, or communicate fundamental principles, not ones that implement specific algorithms.
      For example, I do not believe that 'circular buffer' is a good candidate.  It is an algorithm (and a good one), and perhaps a useful pattern in its own right, but I don't believe that it is a pattern that belongs in this list.  Of course, we are going to have to have a debate on the patternity test at some point.  We may also need to decide that we will keep a seperate list of 'algorithms that every developer should know.'  (then again, I'm inclined to leave that work to Dr. Knuth).
    3. No business rules or best practices, even if the business rule has been documented as a 'pattern' in a peer-reviewed work.  A pattern that simply describes a 'common business rule' that applications in a particular space should implement may not be fundamentally useful for creating a basis of communication among developers.

    I'm thinking the set could break down this way:

    25 low level design patterns
    15 mid-level application architecture patterns
    20 high-level system integration patterns
    10 patterns related to performance and real-time systems
    10 patterns related to data management, persistence, and retrieval
    10 patterns related to system components that behave in a particular manner
    10 misc patterns useful for describing key concepts

    Opinions?  (The weakest part of this post is the breakdown above.  I don't think I got that part right.  Your feedback would be great).

  • Inside Architecture

    Why create a list of the top 100 patterns?


    I posted a blog entry about creating a list of the top 100 patterns that every developer should know, and got a veritable slew of responses.  In this post, I will try to do a better job of explaining WHY we need a top 100.

    Concern 1: Is this a grammar or jargon? 

    It is neither.  With all due respect, the word jargon is perjorative to my intent. defines Jargon as:

    1. the language, esp. the vocabulary, peculiar to a particular trade, profession, or group: medical jargon.
    2. unintelligible or meaningless talk or writing; gibberish.
    3. any talk or writing that one does not understand.
    4. pidgin.
    5. language that is characterized by uncommon or pretentious vocabulary and convoluted syntax and is often vague in meaning.

    Of the five definitions, four are clearly negative in tone and intent.  The first definition, while valid, is not reflective of my intent when taken in context with the other four.

    On the other hand, this is not grammar either.  The same source defines grammar as:

    1. the study of the way the sentences of a language are constructed; morphology and syntax.
    2. these features or constructions themselves: English grammar.
    3. an account of these features; a set of rules accounting for these constructions: a grammar of English.

    I am not talking about a set of the rules of discussion.  I am simply talking about creating a taxonomy of patterns that forms a basis for discussion between professionals.  It is, in effect, a jargon, but without the sense that it should be unintelligible, meaningless, or difficult to understand.

    Concern 2: Different people need different patterns

    I am amazed at the efforts of Grady Booch and his co-authors on his Handbook of Software Architecture.  There are over 2000 patterns there.  However, in the works he cites as inspiration (my favorite being Elements of Style by Strunk and White), there are far fewer concepts cataloged.  Is that because the Software profession is that much more widely used than the English language?  no. 

    It is because the authors whittled down the list to a core set.  A key set.  A set of elements that everyone should know. 

    My computer science degree required me to learn a great many things that I have not directly used since. Why?  Because being able to speak to someone whose job is different from mine, but related, is a key attribute of being literate.  It allows developers to be mobile, moving from sector to sector.  I started out writing retail software for the hospital market.  I jumped to firmware.  Then worked on the guts of an operating system.  The patterns were different each time.  But the ability to be literate was key.

    A core set of 100 patterns goes a long way towards that literacy.  The number doesn't have to end up at 100, but it should be close.  Too many means overload for folks to learn the basics of other concerns.  Too few means that we are all literate only in our shared areas, which doesn't foster mobility of knowledge.

    Concern 3: We aren't ready for that in this industry

    Nonsense.  You become ready by forging a path for those folks who want to be ready.  That's step 1.  Early adopters will follow by themselves.  College campuses can be convinced.  The rest of us have to be marketed to and sold to.  We can never reach critical mass until we try.

    Concern 4: do only the principles, doing the patterns is a waste of time.

    The principles are well known.  That clearly isn't enough.  The practices need to be shared too.  Not everyone in the world is a budding Grady Booch.  Some coders are just coders.  They like writing code.  They think in code.  I am still a little bit like that.  It's where I came from.  I didn't mine patterns or submit papers to a PLoP conference.  I would not have known how, but I have benefited from the patterns movement and literature. 

    Robert Martin shared a great set of principles over a decade ago.  They are still linked on his site. 

    Conclusion: we need the top 100 patterns.

    Using a top 100, we force the conversation.  What are the elements we should all know about EACH OTHER'S work? What are the key learnings that we should be able to share and discuss?  What elements, if shared, have the greatest potential for impacting other areas?  What principles, when expressed in code, change the mind and the thinking in the most elemental way?

    The rest is gravy.

  • Inside Architecture

    What are the top 100 patterns that every developer must know?


    It has been 10 years since the seminal publication of the Gang-of-Four Design Patterns book.  Since then, at least a dozen major works have followed, each with its own viewpoint, its own set of repeatable problems.  Some took on Architecture, others Real Time Systems, and so on. 

    Now, the goal of the Patterns work was not to create a bunch of noise that dies down.  It was to change the way we communicate with one another.  The language that we use. The forces that we consider.  The patterns themselves, even when not fully understood, can lead us to what Christopher Alexander called "the quality."  A design that is beautiful, elegant, balanced, and completely in context with its environment.

    We have the words.  We have too many of them.  Now, it's time to make the language.

    Will you help me? I'd like to start to create a consensus list.  I'd like to answer the question: what are the top 100 patterns and idioms that every developer should learn by the end of their second year of professional practice? 

    Is 100 the right number?  Only by attempting to create the language will we actually know what the right number is.  Scientists in other disciplines have settled on common terms for their common field.

    It is our turn.

    If someone has a suggestion for the best Wiki site to do this on, please reply. 

    (Please note: I followed this post with a follow-up post that attempts to answer some of the questions from the responses below.  See here.)

  • Inside Architecture

    The minimum amount of architecture needed for Test Driven Design


    My good friend Malcolm posted a response to my IFaP article and asked, in essence, "what is the minimum amount of architecture needed for a system built with Test Driven Design."

    I had to stare at that.  For me, TDD meant Test Driven Development, not Test Driven Design.  Search.Live.Com brought up Scott Ambler's site where he clearly states his formula for Test Driven Design:

    Test Driven Design = Test First Development (TFD) + Refactoring

    Hmmm.  My training by one of the original authors of JUnit clearly calls the technique Test Driven Development, so it appears to me (perhaps unfairly) that Scott is redefining an acronym to mean more than it originally meant.  He is not the first to do this, of course. 

    I truly dislike this aspect of our industry.  If an idea catches on, it becomes an acronym.  Then if someone has another idea they want to glue to the side of the first, they just redefine the acronym and say "it's been this way all along."  No.  It wasn't.

    Test Driven Development is a fairly low level coding activity.  Don't count me as a critic.  I really believe that a combination of Pair Programming and Test Driven Development can really make for writing good code fairly quickly.  However, neither handles design.  Design has to be done first.

    Mr. Ambler does have an answer to "when design happens," called Agile Model Driven Development.  Unfortunately, it's a waterfall process, so it's hardly agile.  According to his site, there is a set of iterations on the models, just enough to get them 'good enough' and then you start coding using TDD.  Note that you never go back to the models.  If they were wrong, you fix it with refactoring.  You cannot iterate on the design itself.  No iterations based on feedback.  No incremental improvement.  It's not agile.

    I'm not calling this incorrect.  I'm calling this "not-agile."  I don't believe it is incorrect.  Perhaps our tools are just not ready for this kind of post-coding refactoring of the model.

    I do believe that a project requires design up front, but it doesn't have to be big design.  I don't believe that you have to create every little detail in the diagrams, or even define much more than the interface and responsibilities for each independent component.  However, you DO need to define the following key elements right up front. 

    This is the minimum agile architecture.

    For each independent component:

    • In one or two sentences, describe the responsibility of the component.  Add one more sentence to clarify, if necessary, what the component will NOT do. 
    • Describe every endpoint, both inside the system and externally that your system calls.  For each endpoint, describe the Protocol used to communicate.  Describe every Unique Identifier that your system will use to look data up.  Describe the format of the data being passed to and from the endpoint (in a services-model, this is the canonical schema).  (Identifier + Format + Protocol = IFaP)
    • Describe the priority quality attributes that the endpoint must exhibit.  If an endpoint must be reliable, but should be performant, then you'd say "reliable first, performant second." 

    That should take less than three days, and should be done with two or three people working closely together, and then reviewed with the team to find things you missed.

    Agile design is light.  It is enabled by tools.  It is defined well enough to get people to work independently of one another.  Describing the IFaP allows tests to be written to the interface first, so it can be coded using Test Driven Development.  Hopefully, if you do this well, the need for refactoring, at least before the first full release is out the door, is pretty slim. 

Page 1 of 2 (10 items) 12