J.D. Meier's Blog : Architecture

Now Available: patterns & practices Application Architecture Book

J.D. Meier — Thu, 05 Nov 2009 18:58:00 GMT

The Microsoft Application Architecture Guide, 2nd edition, is now available on Amazon and should be available on the shelf at your local bookstores soon. The PDF was downloaded ~180,000 times. This is the Microsoft platform playbook for application architecture. You can think of it as a set of blueprints, and as your personal mentor for building common types of applications on the Microsoft platform: mobile, RIA, services, and Web applications.

The backbone of the guide is an information model for the application architecture space. It’s a durable and evolvable map to give you a firm foundation of principles, patterns, and practices that you can overlay the latest technologies. It’s your “tome of know-how.” While it’s not a step-by-step for building specific applications, it is a pragmatic guide for designing your architecture, with quality attributes, key software principles, common patterns, and architectural styles in mind. It’s holistic and focused on the key engineering decisions where you face your highest risks and most important choices.

Key Features of the Book
The book has several compelling features for slicing and dicing the application architecture body of knowledge:

Canonical Frame. This describes at a meta-level, the tiers and layers that an architect should consider. Each tier/layer will be described in terms of its focus, function, capabilities, common design patterns and technologies.
Application Types. These are canonical application archetypes to illustrate common application types: Mobile, Rich Client, RIA, Services, and Web applications. Each archetype is described in terms of the target scenarios, technologies, patterns and infrastructure it contains. Each archetype is mapped to the canonical app frame. They are illustrative of common application types and not comprehensive or definitive.
Quality attributes. This is a set of qualities and capabilities that shape your application architecture: performance, security, scalability, manageability, deployment, communication, etc.
Cross-cutting concerns. This is a common set of categories for hot spots for key engineering decisions: Authentication, Authorization, Caching, Communication, Configuration Management, Exception Management, Logging and Instrumentation, State Management, and Validation.
Step-by-Step Design Approach.
Principles, patterns, and practices. Using the application types, canonical frame, and cross-cutting concerns as backdrops, the guide provides an overlay of relevant principles, patterns, and practices.
Technologies and capabilities. The guide provides an overview and description of the Microsoft custom application development platform and the main technologies and capabilities within it.

Contents at a Glance
The full Microsoft Application Architecture Guide is available for free on MSDN in HTML. This is the contents of the guide at a glance:

Chapters

Appendices

The Team
Here is the team that brought you the guide:

Core Dev Team: J.D. Meier, Alex Homer, David Hill, Jason Taylor, Prashant Bansode, Lonnie Wall, Rob Boucher Jr, Akshay Bogawat
Test Team - Rohit Sharma, Praveen Rangarajan, Kashinath TR, Vijaya Jankiraman
Edit Team - Dennis Rea
External Contributors/Reviewers - Adwait Ullal; Andy Eunson; Brian Sletten; Christian Weyer; David Guimbellot; David Ing; David Weller; Derek Greer; Eduardo Jezierski; Evan Hoff; Gajapathi Kannan; Jeremy D. Miller; John Kordyback; Keith Pleas; Kent Corley; Mark Baker; Paul Ballard; Peter Oehlert; Norman Headlam; Ryan Plant; Sam Gentile; Sidney G Pinney; Ted Neward; Udi Dahan
Microsoft Contributors / Reviewers - Ade Miller; Amit Chopra; Anna Liu; Anoop Gupta; Bob Brumfield; Brad Abrams; Brian Cawelti; Bhushan Nene; Burley Kawasaki; Carl Perry; Chris Keyser; Chris Tavares; Clint Edmonson; Dan Reagan; David Hill; Denny Dayton; Diego Dagum; Dmitri Martynov; Dmitri Ossipov; Don Smith; Dragos Manolescu; Elisa Flasko; Eric Fleck; Erwin van der Valk; Faisal Mohamood; Francis Cheung; Gary Lewis; Glenn Block; Gregory Leake; Ian Ellison-Taylor; Ilia Fortunov; J.R. Arredondo; John deVadoss; Joseph Hofstader; Koby Avital; Loke Uei Tan; Luke Nyswonger; Manish Prabhu; Meghan Perez; Mehran Nikoo; Michael Puleio; Mike Francis; Mike Walker; Mubarak Elamin; Nick Malik; Nobuyuki Akama; Ofer Ashkenazi; Pablo Castro; Pat Helland; Phil Haack; Rabi Satter; Reed Robison; Rob Tiffany; Ryno Rijnsburger; Scott Hanselman; Seema Ramchandani; Serena Yeoh; Simon Calvert; Srinath Vasireddy; Tom Hollander; Wojtek Kozaczynski

Application Architecture Knowledge Base
The guide was developed in conjunction with our Application Architecture Guide v2.0 Knowledge Base Project. The knowledge base project was used to inform and steer the guide during its development. The Application Architecture Knowledge Base includes a large amount of material that expands on specific topics in the main guide. It also includes draft material from the main guide that is targeted and packaged for more specific audiences, such as the Pocket Guide series.

Key Links at a Glance
Here are the key links at a glance:

Lessons in Software from Alok Srivastava

J.D. Meier — Mon, 29 Jun 2009 21:20:48 GMT

I have a guest post, Lessons in Software from Alok Srivastava, on Shaping Software. Alok is a solution architect at Microsoft with several years of experience in large scale, distributed systems. In this post, he shares his lessons learned in software. Here is a summary of his lessons:

Lesson 1. Software development is a team sport.
Lesson 2. More lines-of-code does not mean better software.
Lesson 3. The Cloud is an inflection point.
Lesson 4. Scalability, performance and diagnostic ability are better achieved at design time.
Lesson 5. User experience and user expectation change continuously that is why UI projects are never done.
Lesson 6. Software maintainability is a key to longer life for any software.
Lesson 7. Development process should help development produce good quality software, if it comes in your way change it.
Lesson 8. Take agility with a grain of salt; result –oriented software development is what agility should help you gain.
Lesson 9. A great software engineer never stops working.
Lesson 10. Know the keys to writing great software; magic isn’t one of them.

You can read an explanation of the lessons in his post, Lessons In Software from Alok Srivastava.

A Language for Architecture

J.D. Meier — Sat, 25 Apr 2009 22:11:19 GMT

My Architecture Journal article is live, A Language for Architecture. I wrote the article to share the map of application architecture we created during our patterns & practices Application Architecture Guide 2.0 project. It's a simple language for helping you get in the ballpark when you're traversing the very large space of software architecture. By framing and naming the space, we can more effectively share our principles, patterns, and practices for application architecture. This also helps consolidate all the great information spread over time and space and threads and heads. More importantly, if we simplify how we talk about architecture, we can move up the stack as well as pave paths for others and help mentor others in our field. Instead of asking basic questions like what is architecture, we can ask things like how do we define archetypes for the cloud or how do improve product line engineering for common systems and application types? In our case, we're using the language to help rationalize our portfolio of assets in our patterns & practices product line.

Why the Map
There's an explosion of concepts in the architecture space. While working on the Application Architecture 2.0 Guide, we needed a simple, but effective bird's-eye view of the space. By framing and naming the space, we created a shared vocabulary, helped avoid information overload, and made it easier to find, organize, and share principles, patterns, and practices with customers, field, and product teams. It's good for the ecosystem.

Usage Scenarios
Here's some usage scenarios:

Heat Map. If you do product planning or competitive analysis you can use the map and language to create heat maps and create a "consumer reports" for the space you are in. You can identify key hot spots and quickly drill in or evaluate the level of maturity in that domain. You can use the map to find the most important areas to invest. That's key, especially in today's economic landscape.
Hot Spots. By mapping out hot spots you build a catalog of your key engineering decisions. These become your focus points. By focusing efforts you improve your results. For example, if cloud computing is a hot spot, you might identify manageability within that and then you might invest your energy in reducing friction or creating opportunity in that hot spot.
Solution Engineering. If you build applications you can use the map and language for quickly traversing key engineering decisions. You can use it to quickly identify your key risks as well as explore potential solution options. It helps you identify what you know, don't know and need to know next. It also helps you quickly rip through technology decisions.
Architectural Exploration. You can use the map as a way to explore existing systems. By chunking up architectures into things like archetypes, qualities, hot spots ... etc. you can drill into them more effectively. For example, you might go on an architectural expedition to build a catalog of common application patterns for your group.
Whiteboarding. Effective whiteboarding is an art. That said, whiteboarding is one of the most important tools in our industry when it comes to sharing knowledge and getting people on the same page. You can use the language, maps, and frames to improve your whiteboarding skills by using common elements from the map.
Product-line engineering. If you build software, you can use the map to get more precision over your product line by defining the scenarios, the requirements, the archetypes and hot spots. You can also use these to differentiate.
Slideware. If you have to build slides, you can use the map, the language, and the visuals to more effectively share what's important.
Prescriptive Guidance. If you're in the decision improvement business, then using a common map and frames will help you organize and share patterns and practices more effectively. What if we could easily traverse the common data access patterns for RIA applications from a security, performance or manageability perspective? We get closer to the capability when we share common maps.

Key Concepts
Here's some key concepts behind the map and language:

Hot Spot driven. Rather than exhaustive, the maps, language, and frames are hot spot driven. If you can see the forest from the trees, you have a better vantage point and can more selectively drill down or elaborate as needed. Hot spots are also where the action is. See Security Hot Spots and Performance Hot Spots.
Expandable by design. The maps are expandable by design. You can unfold, elaborate or expand each area as needed. You can tailor it for your context. The maps gives you a baseline of hot spots to start from so you don't have to start from scratch.
Overlays. The maps, language, and frames are backdrops. This means we can overlay principles, patterns, and practices. This also means we can overlay products or technologies or solution assets, such as reusable code.
Whiteboard oriented. The maps, language and frames are oriented towards whiteboard usage, simply because it's one of the most universal tools in our field. By optimizing around whiteboard usage and humans over tools, we reduce friction around sharing basic information.
Plays well with others. No matter what your favorite framework or mental models are, you can leverage the maps, language and frames. We didn't set out to build an exclusive set. Instead, we set out to map out an inclusive set of concepts that can easily stretch to fit. It's platform agnostic and if anything, it's most tightly bound to the knowledge areas of software architecture. In the words of Bruce Lee, "absorb what is useful."
Real-world over academic. We chose to optimize around the language, maps, and frames we see in practice. We needed it to be simple to explain whether we're in a team full of developers or a doing a presentation for business decision makers. It's a simple way to look at the legos and organize them in a meaningful way.

The Map
The key components of the language include:

User, Business, and System Perspective. Architecture is a constant balance of user, business, and technical concerns. For example, the user wants a response time of x, but the cost and impact on the system side have to make business sense. When you group by user, business, and system stories you get more precision around the drivers and goals and you can include the right people for the right decisions, as well as keep checks and balances. See What are the User, Business, and System Goals.
Architecture Frame. This is the main map that includes the context, the app types, the architectural styles, and the application feature frame. The context is expressed with scenarios, requirements / constraints, and quality attributes.
Application Archetypes (App Types). Application Types are simply blueprints for classes of applications. In our Application Architecture Guide 2.0, we called out mobile, RIA, Web, Rich Client, and Services. Those are technical types. You could also call out a set of business types. or a set of core system archetypes. The key is to identify a set of types for your business so you can think in terms of a product-line. See Application Types.
Quality Attributes. This includes system, run-time, design, and user qualities. For example, a run-time quality would be availability, while a design quality would be conceptual integrity. See Quality Attribute List.
Scenarios. You can't design or evaluate an architecture in a vacuum. Scenarios give you the context. The scenarios are a really important element. In terms of architecture, the architecturally significant use cases are vital to making the right trade-offs. You can find architecturally significant scenarios by looking at the intersections of quality attributes or hot spots with application features. See App Types, Verticals, and Scenarios and What's a Scenario?.
Architectural Styles. Architectural styles are simply sets of principles that shape the software. We grouped styles by big decision areas: communication, deployment, domain, interaction and structure.The beauty of architectural styles is we can talk about higher-level goals before getting into specific technologies. For example, if you choose SOA as your architectural style for your communication, you can then evaluate whether WCF or ASMX helps you implement those principles. Every app is a mash up of styles.
Requirements and Constraints. Requirements and constraints are simply what's required of the system. Again, this is where user, business, and system perspectives matter. We called out functional, non-functional, technological, and constraints. Non-functional would include quality attributes and constraints would include organizational and industry compliance. See Requirements Types and User Requirements vs. System Requirements.
Application Feature Frame. This is a set of hot spots focused on a common set of key engineering decisions. We called out Caching, Communication, Concurrency and Transactions, Configuration Management, Coupling and Cohesion
Data Access, Exception Management, Layering, Logging and Instrumentation, State Management, Structure, Validation, and Workflow. These are the types of decisions you don't want to make up on the fly and you want to avoid do-overs. You'll also notice our industry is rich with principles, patterns, and practices in these categories, as well as reusable components. For example, you can overlay Enterprise Library on several of these areas. See Application Infrastructure Frame.
Application Frames. These are hot spots based on a given application type. For example, for Web applications, we defined Authentication, Authorization, Caching, Exception Management, Logging and Instrumentation, Navigation, Page Layout (UI), Page Rendering, Presentation Entity, Request Processing, Service Interface, Session Management, Validation. Notice that the map has a lot of similar categories to the core application infrastructure frame. The key is context. Now we are talking about validation within Web applications. Also notice that some hot spots are more Web centric such as page layout and page navigation. Also notice that we can overlay key Web patterns on these hot spots. Instead of a laundry list of patterns, we can find, organize, and share key patterns for these hot spots.
Layers, Components, and Tiers. We called out layers, components, and tiers so that we could have a common backdrop with the various existing bodies of work. We kept layers logical and tiers physical (a precedent set back in 2001 to help untangle confusion.) We called out the presentation, business, data, and service layers. You can extend that for things like productivity layers or manageability layers. Keep in mind that layers are fractal, in other words you might have a service that has a presentation, business, and data layer within it. Effectively you can think in terms of macro and micro levels. For components we called out presentation components (user interface, user process), business components (application facade, business, business entity, business workflows), data layer components (data access logic, data helpers/utilities, service agents), and service layer components (service interfaces, message types). For tiers, we called out 2-tier, 3-tier, and N-tier.
Pattern Maps. We created pattern maps based on the frames. For example, for each application type (Web, RIA, Mobile ... etc.) we used the hot spots to overlay relevant patterns. This helped us surface up the most important patterns for a given topic and avoid information overload.
Product and Technology Maps. We created product and technology maps organized by scenarios, application types, and hot spots. The key here was making it easy to traverse the problem space and overlay the solution space.

Agile Architecture Method Revisited

J.D. Meier — Mon, 02 Mar 2009 18:49:23 GMT

I posted an update to the Agile Architecture method on Shaping Software. When I originally posted about the Agile Architecture method, I took some things for granted. I thought the mapping to agile practices was more obvious than it turned out to be. After taking more customers through the approach, I realized some things are worth more elaboration. Here's some of the key things that stood out among the conversations:

Whiteboard technique. You can use the Agile Architecture method as a simple technique to quickly sketch out your solutions on a whiteboard. It's a guided process for rapid prototyping. The value is that it helps you identify hot spots for technical risks more quickly than ad-hoc or random approaches.
Hot Spots. Thinking of the cross-cutting concerns and quality attributes as hot spots seems to stick. The intersection of the hot spots and stories is a key way to drive architectural concerns.
Prioritized scenarios. The user, business, and system stories are a way to help focus and prioritize which hot spots matter most.
Mapping to agile practices. You can use the Agile Architecture method to shape your candidate architecture, as well as prioritize technical risks and guide your architectural spikes for iteration 0. For iterations 1 - N, you can use the Agile Architecture method to guide your design, implementation, and deployment inspections as well as identify hot spots in your design. You can also use it to help you periodically refactor your design to meet your objectives for your various quality attributes, such as performance, security ... etc. Most importantly, you can use the Agile Architecture method as a structured approach for addressing cross-cutting concerns and baking quality throughout the life cycle.
Just in Time Architecture. One of the most important points about the Agile Architecture method is that it's just enough architecture and just in time architecture. When you're creating your baseline or candidate solutions, it's a structured approach to help you prioritize technical risk and find the key hot spots. During iterations, it's a way to provide just in time architecture by mapping relevant hot spots against your prioritized stories.

Architectural Styles

J.D. Meier — Tue, 10 Feb 2009 19:23:22 GMT

I wrote a post on Architectural Styles on Shaping Software. I tried to distill what I've learned on presenting architectural styles to various folks. Architectural styles are basically sets of principles that shape an application. By using architectural styles, you can abstract the conversation from the technologies. For example, you can talk about SOA and key principles, before getting into debates over whether WCF or ASMX is the right technology. In fact, I would argue that architectural styles are the backdrop that you overlay technologies on.

Grady Booch on the Microsoft Application Architecture Guide 2.0

J.D. Meier — Wed, 31 Dec 2008 23:10:50 GMT

Grady Booch blogged about our patterns & practices Application Architecture Guide 2.0, which is our Microsoft playbook for the application platform. It's a thoughtful post and he obviously took the time to figure out the structure of the guide. The guide is probably particularly relevant for him since he's working on his Handbook of Software Architecture.

Booch on the Application Architecture Guide 2.0
Here's what Booch has to say:

"I find this work to be very interesting (and useful). Architecture is all about making significant design decisions, and this guide focuses on a number of such decision points, including caching, communication, concurrency and transactions, configuration management, coupling and cohesion, data access, exception management, layering, logging and instrumentation, state management, structure, validation, and workflow (collectively, Microsoft calls these "architectural frames"). Full of best practices and patterns, I particularly liked the enumeration of architectural styles the authors have collected: client-server, component-based, layered, message-bus, model-view-controller, n-tier, object-oriented, and service-oriented. Congruent with these styles is their concept of application archetypes, which include mobile, rich client, rich internet, services, and web. Combine the these styles and archetypes, and you have an interesting language for describing a large class of applications. While I don't necessarily agree that these styles and archetypes are orthogonal (nor are the lists complete) for the general domain of software architecture, for Microsoft's purposes, these styles offer an excellent operating model into which one can apply their patterns and practices."

For whatever reason, I can't figure out how to link to the individual post, so I'm linking to the page that includes Grady's post on the Microsoft Application Architecture Guide.

My Take
I agree with Grady that the list of application archetypes and arch styles is not complete (at least the versions that we've shared online). We only shared a subset. We do have some pretty rich Mind Maps that are more exhaustive, but they're somewhat unwieldy. It's a trade-off.

We basically forked the app types. The app types that we shared are optimized around technical types that customers quickly identify with. We also have a set of business types, but the problem is these are more long-tail and there was less agreement on naming. Worse, customers couldn't quickly identify with them. There was a slight learning curve. I think we can potentially tune them and share them downstream, more as a visual catalog.

On the arch styles, they're not necessarily orthogonal, but we found it helpful to first identify the overall app type (web, RIA, ... etc.) and then shape the app with the arch styles. For precision, I think of the architecture styles as sets of principles that shape an important dimension of the application (such as deployment, structure, domain, communication ... etc.) It's like a bunch of martial arts systems for different styles of fighting. They have their pros and cons, but you can only evaluate them in context and measure effectiveness.

Key Take Aways
Here's my key take aways:

Architecture Frames. The arch frames are useful for cataloging the patterns and practices into actionable categories: caching, communication, concurrency ... etc.
Architecture Styles. The arch styles include client-server, component-based, layered, message-bus, model-view-controller, N-tier, object-oriented, and service-oriented.
Application Archetypes. The application archetypes are a language for describing classes of apps: mobile, rich client, RIA, services and Web.

Someday, I'd like to synch with Grady and show him all the stuff we haven't exposed at this time, including our technical capability maps, pattern capability maps, application patterns, quality attribute maps, ... etc. While a lot of it is half-baked, we have some good foundations for elaboration. Unfortunately, there's only so much we could share in a six month project where the dominant focus is writing a book.

Scenario-Based Architecture Evaluation Methods

J.D. Meier — Thu, 25 Sep 2008 22:19:53 GMT

I'm a fan of scenarios. Whether you use scenarios for scenario-driven development, scenario-based evaluations, competitive assessments or for shaping products ... scenarios are where the rubber meets the road. I think scenarios are particularly effective for evaluating architectures and for making architecture trade-off decisions. I wrote a short post on Shaping Software that reiterates a mantra I frequently use ... you can't evaluate architecture in a vacuum.

Model-Driven Approaches

J.D. Meier — Fri, 01 Aug 2008 19:05:00 GMT

When people ask me my take on model-driven approaches, I think of two ends of the spectrum -- human and the machine.

Key Points

Model for humans. For humans, I find using a whiteboard (whiteboard modeling) works well -- it's universal.
Model for machines. For machines, I find speaking closer to the code is a good thing and design is rarely a clean model.
Model to learn. I've found modeling more useful when you throw it away -- it's a learning tool. Model so you get it, then go to it. I think the key is that the model is a map, not the actual territory. The usefulness of a map is actually decoupling from the complexity and creating a simpler lens.
Model to share. I think the real value of models is when you create a way to share the problem or solution in a simplified way. This helps for sharing a vision of the end in mind, as well as getting more sustained thinking around the problem.

Model-Driven Code
I've never experienced an effective modeling approach that turns visuals of systems into code, where the model doesn't get in the way. At some point, the model stops being useful for humans or stops being useful to the machine. As a result, I've never really been a fan of model-driven approaches that are coupled to code in practice, although they're always interesting in theory. While I'm open to the idea, I just haven't seen it. Am I missing out?

Effective Modeling for Shaping Software
While I'm not a fan of most visual modeling tools, there’s some very real modeling approaches I find to be effective (which is more about modeling for the humans to understand what matters.) I find that light-weight, human-oriented models are particularly effective for shaping software around quality attributes. For example:

Performance. For performance design, I've found Performance Modeling to be effective. See patterns & practices Performance Modeling
Security. For security design, I've found Threat Modeling to be effective. See patterns & practices Threat Modeling
Reliability. For reliability design, I've found Fault-tree modeling to be effective.

My Related Posts