J.D. Meier's Blog

Software Engineering, Project Management, and Effectiveness

  • J.D. Meier's Blog

    Agile Architecture Method

    • 11 Comments
    AgileArchitecture

    I presented our new patterns & practices Agile Architecture Method for the first time at the patterns & practices Summit.   Our Agile Architecture Method is an iterative and incremental approach for designing architectures. 

    To summarize, it’s a technique that:

    • Scopes and focuses your architecture exercise.
    • Uses scenarios to drive the design and evaluate potential solutions.
    • Helps you think through your choice of application type, deployment, architectural style and technologies.
    • Helps you quickly iterate through potential solutions.
    • Helps you map potential patterns.

    I’ve summarized the approach below, and we’ve posted a step-step how to on CodePlex:

    Input
    Here’s the key input into the process:

    • Use cases and usage scenarios
    • Functional requirements
    • Non-functional requirements (quality attributes such as performance, security, and reliability)
    • Technological requirements
    • Target deployment environment
    • Constraints

    Output
    Here’s the key output of the process:

    • Architecturally significant use cases
    • Architecture hot spots
    • Candidate architectures
    • Architectural spikes

    Summary of Steps

    • Step 1. Identify Architecture Objectives.
    • Step 2. Identify Key Scenarios.
    • Step 3. Create an Application Overview.
    • Step 4. Analyze Key Hot Spots.
    • Step 5. Create Candidate Solutions.

    Step 1. Identify Architecture Objectives
    This is a scoping exercise.  The purpose of this step is to figure out how much time and energy to spend on subsequent steps as well as guide your overall effort.   You should know what you want in terms of outcomes.  Here’s an example of potential goals:

    • Build a prototype
    • Identify key technical risks
    • Test potential paths
    • Share models and understanding

    Step 2. Identify Key Scenarios
    Identify relevant scenarios to focus your design on what matters most, and to evaluate your candidate solutions.    In this case, you want to identify architecturally significant use cases.  Architecturally significant use cases are those that meet the following criteria:

    1. They are important for the success and acceptance of the deployed application.
    2. They exercise enough of the design to be useful in evaluating the architecture.

    You can draw key scenarios from your user stories, business stories and system stories.

    Step 3. Create an Application Overview
    Create an application overview.  The application overview serves to make your architecture more real, connecting it to real-world constraints and decisions. 

    WhiteboardingYourDesign

    An application overview consists of the following steps:

    • Determine your application type. First, determine what type of application you are building. Is it a mobile application, a rich client, a rich internet application, a service, a Web application, or some combination?
    • Understand your deployment constraints. Next, understand your targeted deployment environment and determine what impact this will have on your architecture.
    • Identify important architectural styles. Determine which architectural styles you will be using in your design. Will you build a service oriented architecture, client/server, layered, a message bus, or some combination?
    • Determine relevant technologies. Finally, identify the relevant technology choices based on your application type, architectural styles and deployment constraints.

    A good test of an application overview is whether you can whiteboard it.

    Step 4. Analyze Key Hot Spots
    Identify key hotspots based on quality attributes and the architecture frame. These are the areas where mistakes are most often made when designing an application.

    Quality Attributes Frame
    Understand the quality attributes that are important for your application and scenarios. For instance, most applications need to address security and performance and will be traded against usability, flexibility and other attributes that may be more or less important to you depending on your scenarios and requirements.  You can use the following frame to identify key quality attributes to consider:

    Category Considerations
    Availability
  • How to design for failover support
  • How to design a redundant site
  • How to plan for backup and recovery How to design for runtime upgrades
  • Conceptual Integrity
  • How to isolate from external dependencies
  • How to create a migration path from legacy technologies
  • How evolve the system without breaking clients
  • Flexibility
  • How to handle dynamic business rules How to handle dynamic UI
  • How to handle changes in data and logic processing
  • How to handle changes in business requirements
  • Interoperability
  • How to allow applications to interoperate while still evolving separately
  • How to isolate systems through the use of service interfaces
  • How to isolate systems through the use of mapping layers
  • Maintainability
  • How to reduce dependencies between layers and components
  • How to implement a pluggable architecture
  • How to choose an appropriate communication model
  • Manageability
  • How to understand the key types of failure
  • How to monitor system operation and health
  • How to modify system behavior based on load
  • Performance
  • How to determine a caching strategy
  • How to design high performance communication between layers
  • How to design high performance data access
  • How to manage resources effectively
  • Reliability
  • How to handle unreliable external systems
  • How to audit requests and jobs
  • How to redirect load
  • How to handle failed communication
  • How to handle failed transactions
  • How to handle exceptions
  • Reusability
  • How to reduce duplication between components and layers
  • How to share functionality across systems
  • How to share functionality across components and layers
  • Scalability
  • How to design layers and tiers for scalability
  • How to scale-up or scale-out
  • How to handle spikes in traffic and load
  • Security
  • How to address authentication and authorization.
  • How to protect against malicious input.
  • How to protect sensitive data
  • Supportability
  • How to design auditing and logging
  • How to design usable error messages
  • Testability
  • How to design for testability
  • How to design unit tests
  • How to design for UI automation
  • Usability
  • How to design for user empowerment
  • How to improve responsiveness
  • How to avoid common user experience pitfalls
  • Architecture Frame
    The architecture frame represents cross cutting concerns that will impact your design across layers and tiers. These are also the areas in which high impact design mistakes are most often made. Use the architecture frame to identify hot spots in your design that require additional attention to get right.  You can use the following architecture frame to identify cross cutting concerns in your design:

    Category Considerations
    Authentication and Authorization
  • How to choose an authentication strategy.
  • How to choose an authorization strategy.
  • How to flow identity across layers and tiers.
  • How to store user identities when not using Active Directory.
  • Caching and State
  • How to choose an appropriate caching technology.
  • How to determine what data to cache.
  • How to determine where to cache the data.
  • How to determine the expiration policy.
  • Communication
  • How to choose appropriate protocols for communication across layers and tiers.
  • How to design loose coupling across layers.
  • How to perform asynchronous communication.
  • How to pass sensitive data.
  • Composition
  • How to choose a composition pattern for the user interface (UI).
  • How to avoid dependencies between modules in the UI.
  • How to handle communication between modules in the UI.
  • Concurrency and Transactions
  • How to handle concurrency between threads.
  • How to choose between optimistic and pessimistic concurrency.
  • How to handle distributed transactions.
  • How to handle long running transactions.
  • Configuration Management
  • How to determine what information needs to be configurable.
  • How to determine where and how to store configuration information.
  • How to protect sensitive configuration information.
  • How to handle configuration information in a farm/cluster.
  • Coupling and Cohesion
  • How to choose an appropriate layering strategy for separation of concerns.
  • How to design highly cohesive components and group them within layers.
  • How to determine when loose coupling is appropriate between components within a layer.
  • Data Access
  • How to manage database connections.
  • How to handle exceptions.
  • How to improve performance.
  • How to handle binary large objects (blobs).
  • Exception Management
  • How to handle exceptions.
  • How to log exceptions.
  • How to provide notification when required.
  • Logging and Instrumentation
  • How to determine which information to log.
  • How to make the logging configurable.
  • How to determine what level of instrumentation is required.
  • User Experience
  • How to improve task efficiency and effectiveness.
  • How to improve responsiveness.
  • How to improve user empowerment.
  • How to improve look and feel. </>
  • Validation
  • How to determine where and how to perform validation.
  • How to validate for length, range, format, and type.
  • How to constrain and reject input.
  • How to sanitize output.
  • Workflow
  • How to choose the appropriate workflow technology.
  • How to handle concurrency issues within a workflow.
  • How to handle task failure within a workflow.
  • How to orchestrate processes within a workflow.
  • Step 5. Create Candidate Solutions
    Create a candidate architecture and along with architectural spikes and evaluate it against your key scenarios, hot spots, and deployment constraints.  The outcomes of this step are:

    • Baseline / Candidate Architectures
    • Architectural Spikes

    Iterative and Incremental Design
    You can iteratively flesh out your architecture as you work through your design and discover more details that impact your architecture.  You don’t have to design your architecture in a single iteration. Do not get lost in the details; focus on the big steps and build a framework on which you can base your architecture and design.

    My Related Posts

  • New Release: patterns & practices App Arch Guide 2.0 Beta 1
  • patterns & practices App Arch Guide 2.0 Project
  • App Arch Guide 2.0 Overview Slides
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Components
  • Key Software Trends
  • Cheat Sheet: patterns & practices Catalog at a Glance Posted to CodePlex
  • Cheat Sheet: patterns & practices Pattern Catalog Posted to CodePlex

  • J.D. Meier's Blog

    New Release: patterns & practices SharePoint Guidance

    • 1 Comments

    patterns & practices SharePoint Guidance is now available.

    Abstract 
    patterns & practices SharePoint Guidance helps architects and developers build SharePoint intranet applications. The guidance contains a reference implementation (RI) that demonstrates solutions to common architectural, development, and lifecycle management challenges. This guidance discusses the following:

    • Architectural decisions about patterns, feature factoring, and packaging.
    • Design tradeoffs for common decisions many developers encounter.
    • Implementation examples demonstrated in the RI and in the QuickStarts.
    • How to design for testability, create unit tests, and run continuous integration.
    • Set up of development, build, test, staging, and production environments.
    • Managing the application life cycle including upgrade.
    • Team-based intranet application development.

    Out of Scope

    • Content-oriented sites that use Web content management.
    • Internet and enterprise-scale SharePoint applications.
    • Multilingual SharePoint applications.
    • Scale or security testing of SharePoint applications.

    Key Links

  • J.D. Meier's Blog

    New Release: Distributed Agile Development at Microsoft patterns & practices

    • 6 Comments

    Distributed Agile Development at Microsoft patterns & practices (PDF) is now available.

    Abstract
    Distributed development is a fact of life for many teams. Unfortunately most agile methodologies or approaches assume that the team is located in a single team room. Until recently there has been little guidance about how to apply these approaches with a geographically dispersed team.   Microsoft’s patterns & practices group has been following an agile, distributed development approach for the past five years. During this time teams within the group have experimented extensively with different approaches to best address the challenges of distributed agile development. This paper outlines the challenges faced by geographically distributed agile teams and details some proven practices to address these issues and build successful distributed teams.

    Contents at a Glance

    • Introduction
    • Why Colocate?
    • The Reality of Distributed Teams
    • Distributed Teams at patterns & practices
    • Challenges
    • Proven Practices
    • Focus on Communication
    • Plan to Travel
    • Team Distribution
    • Focus on Coaching the Team
    • Distribution of Work
    • Build the Team Over Time
    • Provide the Right Tools
    • Conclusions
    • Acknowledgements
    • About the Author

    Key Links

  • J.D. Meier's Blog

    New Release: patterns & practices Acceptance Test Engineering Guide (BETA1)

    • 2 Comments

    patterns & practices Acceptance Test Engineering (Beta 1) is now available.

    Key Scenarios

    Here’s the key scenarios the guide addresses:

    • How to Plan for Acceptance Testing
    • What Kinds of Acceptance Tests to Run
    • How to Create and Run Acceptance Tests
    • Defining What “Done” Means
    • How to Justify Your Approach

    How The Guide is Organized
    The guide is organized in 3 parts:

    • PART I – THINKING MODELS: an overview of acceptance testing and explains several models that are useful in conceptual thinking about acceptance testing.
    • PART II – THUMBNAILS: a thumbnail is a short overview of a practice that explains what it is, when you may want to use it, the risks that it mitigates, and an overview of how to perform the practice.
    • PART III – SAMPLES: a collection of sample artifacts generated by applying different practices in a fictional real-world situation for Global Bank.

    Key Links

  • J.D. Meier's Blog

    New Release: Unity Application Block

    • 2 Comments

    Unity Application Block 1.2 is now available.

    What's New

    • Added an interception mechanism (extension) and integration of the Policy Injection Application Block with the Unity Application Block.
    • Added two instance interceptors (TransparentProxyInterceptor, InterfaceInterceptor) and one type interceptor (VirtualMethodInterceptor).
    • Improved support for generics.
    • Added support for arrays.
    • Registered names are now available as an ObjectBuilder policy so that you can do a ResolveAll from within the strategy chain. The container automatically registers itself with itself.
    • Added PerThreadLifeTimeManager.
    • Bug fixes.
    • Performance improvements.

    Key Features
    The Unity Application Block (Unity) is a lightweight, extensible dependency injection container. It facilitates building loosely coupled applications and provides developers with the following advantages:

    • Simplified object creation, especially for hierarchical object structures and dependencies
    • Abstraction of requirements; this allows developers to specify dependencies at run time or in configuration and simplify management of crosscutting concerns
    • Increased flexibility by deferring component configuration to the container
    • Service location capability; this allows clients to store or cache the container
    • Instance and type interception (via an extension introduced in this release)

    Key Links

  • J.D. Meier's Blog

    New Release: Enterprise Library 4.1

    • 2 Comments

    Enterprise Library 4.1 is now available.

    What's New?

    • Unity interception mechanism and integration of the Policy Injection Application Block with the Unity Application Block
    • Added support for generics in the Unity Application Block
      Added support for arrays in the Unity Application Block
    • Performance improvements
    • Usability improvements to the configuration tool
    • Visual Studio 2008 Service Pack 1 support
    • Bug fixes

    Key Features

    What Is Enterprise Library?
    Enterprise Library is a collection of reusable software components (application blocks) designed to assist software developers with common enterprise development challenges (such as logging, validation, caching, exception handling, and many others). Application blocks are a type of guidance encapsulating Microsoft recommended development practices; they are provided as source code, tests plus documentation that can be used "as is," extended, or modified by developers to use on complex, enterprise-level line-of-business development projects.

    Key Links

  • J.D. Meier's Blog

    3 Great Results for Today

    • 1 Comments

    What are 3 great results for today?  That's the question I ask to bootstrap my day.  As simple as it sounds, I find it's the most effective way to cut through the fog each day.  There's a lot of things I can do and there's lots of activities I'll be doing, but what are 3 great outcomes for today.  That's it.

    Example
    This is an example of my 3 great results for this past Monday:

    1. Ship Beta 1 of App Arch
    2. Beta 1 Post for App Arch
    3. Draft of Designing Your Architecture

    Great is relative.

    Why 3 things?  It forces me to prioritize among a sea of potential results.  Also, I can remember 3 things without writing them down, so throughout my day, I know what I'm working towards.   I can say it in the hall.

    If you manage endless to do lists and work on a bunch of stuff but don't actually get anything done, try focusing on 3 great results each day.  It works.

  • J.D. Meier's Blog

    New Release: patterns & practices App Arch Guide 2.0 Beta 1

    • 16 Comments
    AppArchGuidev2

    Today we released our patterns & practices App Arch Guide 2.0 Beta 1.  This is our guide to help solution architects and developers make the most of the Microsoft platform.  It's a distillation of many lessons learned.  It’s principle-based and pattern-oriented to provide a durable, evolvable backdrop for application architecture.  It's a collaborative effort among product team members, field, industry experts, MVPs, and customers.  Keep in mind it’s Beta so there’s still moving parts and we’re processing quite a bit of feedback across the guide.  Now’s the time to bang on it.

    5 Parts

    • Part I, “Fundamentals”
    • Part II, “Design”
    • Part III, “Layers”
    • Part IV, “Quality Attributes”
    • Part V, “Archetypes – Design and Patterns”

    Chapters

    • Chapter 1 - Fundamentals of Application Architecture
    • Chapter 2 - .NET Platform Overview
    • Chapter 3 - Application Archetypes
    • Chapter 4 - Deployment Patterns
    • Chapter 5 - Arch Styles
    • Chapter 6 - Quality Attributes
    • Chapter 7 - Layers and Tiers
    • Chapter 8 - Designing Your Architecture
    • Chapter 9 - Architecture and Design Guidelines
    • Chapter 10 - Designing Services
    • Chapter 11 - Communication Guidelines 
    • Chapter 12 - Presentation Layer Guidelines
    • Chapter 13 - Business Layer Guidelines
    • Chapter 14 - Data Access Layer Guidelines
    • Chapter 15 - Service Layer Guidelines
    • Chapter 16 - Performance Engineering
    • Chapter 17 - Security Engineering
    • Chapter 18 - Mobile Application
    • Chapter 19 - Office Business Application (OBA)
    • Chapter 20 - Rich Client Application
    • Chapter 21 - Rich Internet Application (RIA)
    • Chapter 22 - Service Archetype
    • Chapter 23 - SharePoint LOB Application
    • Chapter 24 - Web Application

    Key Scenarios
    The guide helps you address the following scenarios:

    • Choose the right architecture for your application.
    • Choose the right technologies
    • Make more effective choices for key engineering decisions.
    • Map appropriate strategies and patterns.
    • Map relevant patterns & practices solution assets.

    Key Features

    • Canonical app frame - describes at a meta-level, the tiers and layers that an architect should consider. Each tier/layer is described in terms of its focus, function, capabilities, common design patterns and technologies.
    • App Types.  Canonical application archetypes to illustrate common application types.  Each archetype is described in terms of the target scenarios, technologies, patterns and infrastructure it contains. Each archetype will be mapped to the canonical app frame. They are illustrative of common app types and not comprehensive or definitive.
    • Arch Frame - a common set of categories for hot spots for key engineering decisions.
    • Quality Attributes - a set of qualities/abilities that shape your application architecture: performance, security, scalability, manageability, deployment, communication, etc.
    • Principles, patterns and practices - Using the frames as backdrops, the guide overlays relevant principles, patterns, and practices.
    • Technologies and capabilities - a description/overview of the Microsoft custom app dev platform and the main technologies and capabilities within it.

    Conceptual Framework
    At a high level, the guide is based on the following conceptual framework for application architecture:

    ArchMetaFrame2

    Reference Application Architecture
    We used the following reference application architecture as a backdrop for explaining how to design effective layers and components:

    RefAppArch

    Key Links

    Core Dev Team

    • J.D. Meier , Alex Homer, David Hill, Jason Taylor, Prashant Bansode , Lonnie Wall, Rob Boucher, Akshay Bogawat
    Contributors / Reviewers
    • Test team: Rohit Sharma, Praveen Rangarajan
    • Edit team: Dennis Rea.
    • External Contributors/Reviewers. Adwait Ullal; Andy Eunson; Christian Weyer; David Guimbellot; David Weller; Derek Greer; Eduardo Jezierski; Evan Hoff; Gajapathi Kannan; Jeremy D. Miller; Kent Corley; Mark Baker; Paul Ballard; Norman Headlam; Ryan Plant; Sam Gentile; Udi Dahan
    • Microsoft Contributors / Reviewers. Ade Miller; Anoop Gupta; Bob Brumfield; Brad Abrams; Brian Cawelti; Bhushan Nene; Burley Kawasaki; Carl Perry; Chris Keyser; Chris Tavares; Clint Edmonson; David Hill; Denny Dayton; Diego Dagum; Dmitri Martynov; Dmitri Ossipov; Don Smith; Dragos Manolescu; Elisa Flasko; Eric Fleck; Erwin van der Valk; Faisal Mohamood; Francis Cheung; Gary Lewis; Glenn Block; Gregory Leake; Ilia Fortunov; J.R. Arredondo; John deVadoss; Joseph Hofstader; Koby Avital; Loke Uei Tan; Mehran Nikoo; Michael Puleio; Mike Walker; Mubarak Elamin; Nick Malik; Nobuyuki Akama; Ofer Ashkenazi; Pablo Castro; Pat Helland; Phil Haack; Reed Robison; Rob Tiffany; Ryno Rijnsburger; Scott Hanselman; Serena Yeoh; Srinath Vasireddy; Tom Hollander; Wojtek Kozaczynski

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • App Arch Guide 2.0 Overview Slides
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Components
  • Key Software Trends
  • Cheat Sheet: patterns & practices Catalog at a Glance Posted to CodePlex
  • Cheat Sheet: patterns & practices Pattern Catalog Posted to CodePlex
  • J.D. Meier's Blog

    Cheat Sheet: patterns & practices Pattern Catalog Posted to CodePlex

    • 2 Comments

    As part of our patterns & practices Application Architecture Guide 2.0 project, we've been hunting and gathering our patterns from across our patterns & practices catalog.  Here's an initial draft of our patterns & practices Patterns Catalog at a Glance:

    Pattern Catalog
    To collect the patterns, we first identified the key projects that focused on patterns:

    Next, we organized the patterns and summarized in tables.  You can browse the tables below to see which patterns are associated with which project.

    Composite Application Guidance for WPF

    Category Patterns
    Composite User Interface Patterns Adapter; Command; Composite and Composite View;
    Modularity Event Aggregator; Facade; Separated Interface and Plug In; Service Locator
    Testability Inversion of Control; Dependency Injection; Separated Presentation; Supervising Controller; Presentation Model

    Data Patterns

    Category Patterns
    Data Movement Patterns Data Replication; Master-Master Replication; Master-Slave Replication; Master-Master Row-Level Synchronization; Master-Slave Snapshot Replication; Capture Transaction Details; Master-Slave Transactional Incremental Replication; Master-Slave Cascading Replication
    Pattlets Maintain Data Copies; Application-Managed Data Copies; Extract-Transform-Load (ETL); Topologies for Data Copies

     Enterprise Solution Patterns

    Category Patterns
    Deployment Patterns Deployment Plan ; Layered Application; Three-Layered Services Application; Tiered Distribution; Three-Tiered Distribution
    Distributed Systems Broker; Data Transfer Object; Singleton
    Performance and Reliability Server Clustering; Load-Balanced Cluster; Failover Cluster
    Services Patterns Service Interface; Service Gateway
    Web Presentation Patterns Model-View-Controller; Page Controller; Front Controller; Intercepting Filter; Page Cache; Observer
    Pattlets Abstract Factory; Adapter; Application Controller; Application Server; Assembler; Bound Data Control; Bridge; Command(s); Decorator; Façade; Four-Tiered Distribution; Gateway; Layer Supertype; Layers; Mapper; Mediator; MonoState; Observer; Naming Service; Page Data Caching; Page Fragment Caching; Presentation-Abstraction-Controller; Remote Façade; Server Farm; Special Case; Strategy; Table Data Gateway; Table Module; Template Method

    Integration Patterns

    Category Patterns
    Integration Layer Entity Aggregation; Process Integration; Portal Integration.
    System Connections Data Integration; Function Integration; Service-Oriented Integration; Presentation Integration
    Additional Integration Patterns Pipes and Filters; Gateway

    Web Services Security Patterns

    Category Patterns
    Authentication Brokered Authentication; Brokered Authentication: Kerberos; Brokered Authentication: X509 PKI; Brokered Authentication: STS; Direct Authentication
    Authorization Protocol Transition with Constrained Delegation; Trusted Subsystem
    Exception Management Exception Shielding
    Message Encryption Data Confidentiality
    Message Replay Detection Message Replay Detection
    Message Signing Data Origin Authentication
    Message Validation Message Validator
    Deployment Perimeter Service Router

    Pattern Summaries
    The following pattern summaries are brief descriptions of each pattern, along with a link to the relevant MSDN pages.

    Composite Application Guidance for WPF

    Modularity

    • Service Locator. Create a service locator that contains references to the services and that encapsulates the logic to locate them. In your classes, use the service locator to obtain service instances.

    Testability

    • Dependency Injection. Do not instantiate the dependencies explicitly in your class. Instead, declaratively express dependencies in your class definition. Use a Builder object to obtain valid instances of your object's dependencies and pass them to your object during the object's creation and/or initialization.
    • Inversion of Control. Delegate the function of selecting a concrete implementation type for the classes' dependencies to an external component or source.
    • Presentation Model. Separate the responsibilities for the visual display and the user interface state and behavior into different classes named, respectively, the view and the presentation model. The view class manages the controls on the user interface and the presentation model class acts as a façade on the model with UI-specific state and behavior, by encapsulating the access to the model and providing a public interface that is easy to consume from the view (for example, using data binding).
    • Separated Presentation. Separate the presentation logic from the business logic into different artifacts. The Separated Presentation pattern can be implemented in multiple ways such as Supervising Controller or Presentation Model, etc.
    • Supervising Controller. Separate the responsibilities for the visual display and the event handling behavior into different classes named, respectively, the view and the presenter. The view class manages the controls on the user interface and forwards user events to a presenter class. The presenter contains the logic to respond to the events, update the model (business logic and data of the application) and, in turn, manipulate the state of the view.

    Data Movement Patterns

    • Data Replication.  Build on the data movement building block as described in Move Copy of Data by adding refinements that are appropriate to replication.
    • Master-Master Replication. Copy data from the source to the target and detect and resolve any update conflicts that have occurred since the last replication (due to changes to the same data on the source and target). The solution consists of a two replication links between the source and the target in opposite directions. Both replication links transmit the same replication set in both directions). Such a pair of replication links is referred to as related links in the more detailed patterns.
    • Master-Slave Replication. Copy data from the source to the target without regard to updates that may have occurred to the replication set at the target since the last replication.
    • Master-Master Row-Level Synchronization. Create a pair of related replication links between the source and target. Additionally, create a synchronization controller to manage the synchronization and connect the links. This solution describes the function of one of these replication links. The other replication link behaves the same way, but in the opposite direction. To synchronize more than two copies of the replication set, create the appropriate replication link pair for each additional copy.
    • Master-Slave Snapshot Replication. Make a copy of the source replication set at a specific time (this is known as a snapshot), replicate it to the target, and overwrite the target data. In this way, any changes that may have occurred to the target replication set are replaced by the new source replication set.
    • Capture Transaction Details. Create additional database objects, such as triggers and (shadow) tables, and to record changes of all tables belonging to the replication set.
    • Master-Slave Transactional Incremental Replication. Acquire the information about committed transactions from the source and to replay the transactions in the correct sequence when they are written to the target.
    • Master-Slave Cascading Replication. Increase the number of replication links between the source and target by adding one or more intermediary targets between the original source and the end target databases. These intermediaries are data stores that take a replication set from the source, and thus act as a target in a first replication link. Then they act as sources to move the data to the next replication link and so on until they reach the cascade end targets (CETs).

    Enterprise Solution Patterns

    Deployment Patterns

    • Deployment Plan. Create a deployment plan that describes which tier each of the application's components will be deployed to. While assigning components to tiers, if it’s found that a tier is not a good match for a component, determine the cost and benefits of modifying the component to better work with the infrastructure, or modifying the infrastructure to better suit the component.
    • Layered Application. Separate the components of your solution into layers. The components in each layer should be cohesive and at roughly the same level of abstraction. Each layer should be loosely coupled to the layers underneath.
    • Three-Layered Services Application. Base your layered architecture on three layers: presentation, business, and data. This pattern presents an overview of the responsibilities of each layer and the components that compose each layer.
    • Tiered Distribution. Structure your servers and client computers into a set of physical tiers and distribute your application components appropriately to specific tiers.
    • Three-Tiered Distribution. Structure your application around three physical tiers: client, application, and database.

    Distributed Systems

    • Broker. Use the Broker pattern to hide the implementation details of remote service invocation by encapsulating them into a layer other than the business component itself.
    • Data Transfer Object. Create a data transfer object (DTO) that holds all data that is required for the remote call. Modify the remote method signature to accept the DTO as the single parameter and to return a single DTO parameter to the client. After the calling application receives the DTO and stores it as a local object, the application can make a series of individual procedure calls to the DTO without incurring the overhead of remote calls.
    • Singleton. Singleton provides a global, single instance by making the class create a single instance of itself, allowing other objects to access this instance through a globally accessible class method that returns a reference to the instance. Additionally declaring the class constructor as private so that no other object can create a new instance.

    Performance and Reliability

    • Server Clustering. Design your application infrastructure so that your servers appear to users and applications as virtual unified computing resources. One means by which to achieve this virtualization is by using a server cluster. A server cluster is the combination of two or more servers that are interconnected to appear as one, thus creating a virtual resource that enhances availability, scalability, or both.
    • Load-Balanced Cluster. Install your service or application onto multiple servers that are configured to share the workload. This type of configuration is a load-balanced cluster. Load balancing scales the performance of server-based programs, such as a Web server, by distributing client requests across multiple servers. Load balancing technologies, commonly referred to as load balancers, receive incoming requests and redirect them to a specific host if necessary. The load-balanced hosts concurrently respond to different client requests, even multiple requests from the same client.
    • Failover Cluster. Install your application or service on multiple servers that are configured to take over for one another when a failure occurs. The process of one server taking over for a failed server is commonly known as failover. A failover cluster is a set of servers that are configured so that if one server becomes unavailable, another server automatically takes over for the failed server and continues processing. Each server in the cluster has at least one other server in the cluster identified as its standby server.

    Services Patterns

    • Service Interface. Design your application as a collection of software services, each with a service interface through which consumers of the application may interact with the service.
    • Service Gateway. Encapsulate the code that implements the consumer portion of the contract into its own Service Gateway component. Service gateways play a similar role when accessing services as data access components do for access to the application's database. They act as proxies to other services, encapsulating the details of connecting to the source and performing any necessary translation.

    Web Presentation Patterns

    • Model-View-Controller. The Model-View-Controller (MVC) pattern separates the modeling of the domain, the presentation, and the actions based on user input into three separate classes. The model manages the behavior and data of the application domain, responds to requests for information about its state (usually from the view), and responds to instructions to change state (usually from the controller). The view manages the display of information. The controller interprets the mouse and keyboard inputs from the user, informing the model and/or the view to change as appropriate.
    • Page Controller. Use the Page Controller pattern to accept input from the page request, invoke the requested actions on the model, and determine the correct view to use for the resulting page. Separate the dispatching logic from any view-related code. Where appropriate, create a common base class for all page controllers to avoid code duplication and increase consistency and testability.
    • Front Controller. Front Controller solves the decentralization problem present in Page Controller by channeling all requests through a single controller. The controller itself is usually implemented in two parts: a handler and a hierarchy of commands. The handler receives the HTTP Post or Get request from the Web server and retrieves relevant parameters from the request. The handler uses the parameters from the request first to choose the correct command and then to transfers control to the command for processing. The commands themselves are also part of the controller. The commands represent the specific actions as described in the Command pattern.
    • Intercepting Filter. Create a chain of composable filters to implement common pre-processing and post-processing tasks during a Web page request.
    • Page Cache. Use a page cache for dynamic Web pages that are accessed frequently, but change less often.
    • Observer. Use the Observer pattern to maintain a list of interested dependents (observers) in a separate object (the subject). Have all individual observers implement a common Observer interface to eliminate direct dependencies between the subject and the dependent objects.

    Integration Patterns

    Integration Layer   

    • Entity Aggregation. Introduce an Entity Aggregation layer that provides a logical representation of the entities at an enterprise level with physical connections that support the access and that update to their respective instances in back-end repositories.
    • Process Integration. Define a business process model that describes the individual steps that make up the complex business function. Create a separate process manager component that can interpret multiple concurrent instances of this model and that can interact with the existing applications to perform the individual steps of the process.
    • Portal Integration. Create a portal application that displays the information retrieved from multiple applications in a unified user interface. The user can then perform the required tasks based on the information displayed in this portal.

    Integration Topologies   

    • Message Broker. Extend the integration solution by using Message Broker. A message broker is a physical component that handles the communication between applications. Instead of communicating with each other, applications communicate only with the message broker. An application sends a message to the message broker, providing the logical name of the receivers. The message broker looks up applications registered under the logical name and then passes the message to them.
    • Message Bus. Connect all applications through a logical component known as a message bus. A message bus specializes in transporting messages between applications. A message bus contains three key elements, set of agreed-upon message schemas, set of common command messages, and shared infrastructure for sending bus messages to recipients.
    • Publish / Subscribe. Extend the communication infrastructure by creating topics or by dynamically inspecting message content. Enable listening applications to subscribe to specific messages. Create a mechanism that sends messages to all interested subscribers.

    System Connections   

    • Data Integration. Integrate applications at the logical data layer by allowing the data in one application (the source) to be accessed by other applications (the target).
    • Function Integration. Integrate applications at the business logic layer by allowing the business function in one application (the source) to be accessed by other applications (the target).
    • Service-Oriented Integration. To integrate applications at the business logic layer, enable systems to consume and provide XML-based Web services. Use Web Services Description Language (WSDL) contracts to describe the interfaces to these systems. Ensure interoperability by making your implementation compliant with the Web Services family of specifications.
    • Presentation Integration. Access the application's functionality through the user interface by simulating a user's input and by reading data from the screen display.

    Web Service Services Security Patterns

    Authentication

    • Brokered Authentication. The Web service validates the credentials presented by the client, without the need for a direct relationship between the two parties. An authentication broker that both parties trust independently issues a security token to the client. The client can then present credentials, including the security token, to the Web service.
    • Brokered Authentication: Kerberos. Use the Kerberos protocol to broker authentication between clients and Web services.
    • Brokered Authentication: X509 PKI.  Use brokered authentication with X.509 certificates issued by a certificate authority (CA) in a public key infrastructure (PKI) to verify the credentials presented by the requesting application.
    • Brokered Authentication: STS.  Use brokered authentication with a security token issued by a Security Token Service (STS). The STS is trusted by both the client and the Web service to provide interoperable security tokens.
    • Direct Authentication. The Web service acts as an authentication service to validate credentials from the client. The credentials, which include proof-of-possession that is based on shared secrets, are verified against an identity store.

    Authorization   

    • Protocol Transition with Constrained Delegation.  Use the Kerberos protocol extensions in Windows Server. The extensions require the user ID but not the password. You still need to establish trust between the client application and the Web service; however, the application is not required to store or send passwords.
    • Trusted Subsystem.  The Web service acts as a trusted subsystem to access additional resources. It uses its own credentials instead of the user's credentials to access the resource.

    Exception Management   

    • Exception Shielding.  Sanitize unsafe exceptions by replacing them with exceptions that are safe by design. Return only those exceptions to the client that have been sanitized or exceptions that are safe by design. Exceptions that are safe by design do not contain sensitive information in the exception message, and they do not contain a detailed stack trace, either of which might reveal sensitive information about the Web service's inner workings.

    Message Encryption   

    • Data Confidentiality.  Use encryption to protect sensitive data that is contained in a message. Unencrypted data, which is known as plaintext, is converted to encrypted data, which is known as cipher-text. Data is encrypted with an algorithm and a cryptographic key. Cipher-text is then converted back to plaintext at its destination.

    Message Replay Detection   

    • Message Replay Detection.  Cache an identifier for incoming messages, and use message replay detection to identify and reject messages that match an entry in the replay detection cache.

    Message Signing   

    • Data Origin Authentication.  Use data origin authentication, which enables the recipient to verify that messages have not been tampered with in transit (data integrity) and that they originate from the expected sender (authenticity).

    Message Validation   

    • Message Validator.  The message validation logic enforces a well-defined policy that specifies which parts of a request message are required for the service to successfully process it. It validates the XML message payloads against an XML schema (XSD) to ensure that they are well-formed and consistent with what the service expects to process. The validation logic also measures the messages against certain criteria by examining the message size, the message content, and the character sets that are used. Any message that does not meet the criteria is rejected.

    Deployment   

    • Perimeter Service Router.  Design a Web service intermediary that acts as a perimeter service router. The perimeter service router provides an external interface on the perimeter network for internal Web services. It accepts messages from external applications and routes them to the appropriate Web service on the private network.

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • App Arch Guide 2.0 Overview Slides
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Components
  • Key Software Trends
  • Cheat Sheet: patterns & practices Catalog at a Glance Posted to CodePlex
  • J.D. Meier's Blog

    Effectiveness Post Roundup

    • 11 Comments

    At Microsoft, I regularly mentor some fellow softies.   It can be tough to navigate the waters, find your strengths, figure out worklife balance, and deal with the stresses of the job, all while making things happen.  I help my mentees learn  the most effective ways for getting results in a tough, competitive environment.  It's challenging.  It's rewarding.  I've had several great mentors throughout my life at Microsoft, so mentoring is a way that I give back, sharing my lessons learned and helping others grow.  While my 1:1 sessions are the most effective, I try to share key practices more broadly in posts.  Here's a roundup of my various posts for improving effectiveness at work and life.  I organized them by meaningful buckets and provided an A-Z list at the end.  Enjoy.

    Career
    Learn how to find your path and get more from your career.  Work with the right people on the right things making the right impact.  These posts focus on career and worklife balance:

    Communication
    Communication is among the most important skills for getting results in work and life.  Empathic listening is the most important communication skills.  Improve the quality of your communication, and improve the quality of your life.  These posts focus on communication skills:

    Email
    Don't be a slave to your mail.  With the right approach, you can spend less time in your inbox and enjoy an empty inbox on a regular basis.  These posts focus on email management:

    Intellectual Horsepower
    Thinking is asking and answering questions.  Learn ways to improve your thinking through question-driven techniques and changing perspectives:

    Leadership
    Leadership is influence.  Amplify your results by improving your sphere of influence.  Leadership starts with self-leadership.  These posts focus on leadership skills: 

    Learning
    Learning is a life-long process.  Adopt practices that help you grow.  These posts focus on improving your learning:

    Motivation
    Motivation is your energy or desire to make something happen.  It's also the energy or desire for others to make something happen.  Learn how to improve your own passion for results as well as how to influence and motivate those around you.  These posts focus on motivation:

    Personal Development
    Personal excellence is a path, not a destination.  In life you're either climbing or sliding.  One key is to find ways to climb faster than you slide.  Another key is balancing your multiple demands and growing in your mind, body, career, emotions, financial, relationships and fun.  These posts focus on personal development: 

    Personal Productivity
    Make stuff happen.  Drive or be driven.  With the right approaches, you can carve out time for what's important and prioritize more effectively.  This is the key to getting results.  These posts focus on personal productivity:

    Project Management
    If you need to get something done, make it a project.   Whether it's a small-scale, personal project or a large, team-based project, there's patterns and practices you can use to be more successful.  These posts focus on project management:

    Teamwork
    Effective teamwork is a key skill in today's workplace.  Learn how to get more done with your colleagues.  These posts focus on improving your teamwork:

    Time Management
    You can't add more hours to the day, but you can spend your time more effectively.  You can also add more power hours to your day.  These posts focus on time management:

    Questions
    Questions are a powerful way to shape your thinking and mindset.  Ask better questions and get better answers.  These posts focus on asking and answering better questions:

    A-Z
    Here's the posts organized in a flat A - Z list for easy scanning:

    Sources of Insight
    If that's not enough for you, check out my project blog: Sources of Insight.  Sources of Insight is a browsable KB of insights and actions for work and life.  It's where I share my lessons learned from books, heroes and quotes.  You can read more about the mission and vision in the About page.

  • J.D. Meier's Blog

    Scenario and Feature Frame

    • 3 Comments

    A Scenario and Feature Frame is a matrix that shows incremental value and dependencies.  It's a simple way to show management your baseline release, your nice to have scenarios, and your vNext planning.  It's also a great way for your development team to see how their effort fits into the big picture, as well as recognize any dangerous cross-dependencies.

    I summarized how to create a Scenario and Feature Frame on ShapingSoftware.com.  Enjoy.

  • J.D. Meier's Blog

    Cheat Sheet: patterns & practices Catalog at a Glance Posted to CodePlex

    • 5 Comments

    As part of our patterns & practices Application Architecture Guide 2.0 project, we've been hunting and gathering our patterns & practices solution assets.  Here's our initial draft of our catalog at a glance:

    You can use it to get a quick sense of the types and range of solution assets from blocks to guides.

    Architecture Meta-Frame 
    We used our Architecture Meta-Frame (AMF) as a lens to help slice and dice the catalog:

    ArchMetaFrame

    Here's some examples to illustrate:

    • App Types - Factories such as Web Client Software Factory and Web Services Software Factory map to the application types.  You can think of them as product-line engineering.
    • Quality Attributes - Various guides address quality attributes such as security, performance and manageability. 
    • Architecture Frame - Enterprise Library assets such as the Validation block and the Logging block map to the Architecture Frame.  The Architecture Frame represents hot spots and common cross-cutting concerns when building line of business applications.

    The frame is easily extensible.  For example, if we include our Engineering Practices Frame, we can group our process, activity, and artifact related guidance.

    Catalog at a Glance
    Here's a quick list of key patterns & practices solution assets at a glance:

    Product Line Solution Assets
    Enterprise Library
  • Enterprise Library
  • Caching Application Block
  • Cryptography Application Block
  • Data Access Application Block
  • Exception Handling Application Block
  • Logging Application Block
  • Policy Injection Application Block
  • Security Application Block
  • Unity Application Block
  • Validation Application Block
  • Composite Application Guidance for WPF
  • Individual Blocks
  • Composite Application Guidance for WPF
  • Smart Client – Composite UI Application Block
  • Unity Application Block
  • Archived Blocks
  • Asynchronous Invocation Application Block
  • Aggregation Application Block for .NET
  • Smart Client Offline Application Block
  • Updater Application Block – Version 2.0
  • User Interface Application Block for .NET
  • User Interface Process (UIP) Application Block – Version 2.0
  • Web Service Façade for Legacy Applications
  • Factories
  • Mobile Client Software Factory
  • Smart Client Software Factory
  • Web Client Software Factory
  • Web Service Software Factory
  • Guides
  • Application Architecture for .NET: Designing Applications and Services
  • Application Interoperability: Microsoft .NET and J2EE
  • Authentication in ASP.NET: .NET Security Guidance
  • Building Secure ASP.NET Applications: Authentication, Authorization, and Secure Communication
  • Caching Architecture Guide for .NET Framework Applications
  • Deploying .NET Framework-based Applications
  • Describing the Enterprise Architectural Space
  • Design and Implementation Guidelines for Web Clients
  • Designing Application-Managed Authorization
  • Designing Data Tier Components and Passing Data Through Tiers
  • Guidelines for Application Integration
  • Improving .NET Application Performance and Scalability
  • Improving Web Application Security: Threats and Countermeasures
  • Microsoft .NET /COM Migration and Interoperability
  • Microsoft ESB Guidance for BizTalk Server 2006 R2
  • Monitoring in .NET Distributed Application Design
  • Performance Testing Guidance for Web Applications
  • Production Debugging for .NET Framework Applications
  • Security Engineering Explained
  • Smart Client Architecture and Design Guide
  • Team Development with Visual Studio .NET and Visual SourceSafe
  • Team Development with Visual Studio Team Foundation Server
  • Testing .NET Application Blocks - Version 1.0
  • Threat Modeling Web Applications
  • Upgrading Visual Basic 6.0 Applications to Visual Basic .NET and Visual Basic 2005
  • Archived Guides
  • .NET Data Access Architecture Guide
  • Exception Management Architecture Guide
  • Testing Software Patterns
  • Patterns
  • Data Patterns
  • Enterprise Solution Patterns Using Microsoft .NET
  • Integration Patterns
  • Web Service Security Guidance: Scenarios, Patterns, and Implementation Guidance for Web Services Enhancements (WSE) 3.0
  • Reference Implementations
  • Global Bank Scenario
  • WS-I Basic Security Profile 1.0 Reference Implementation: Final Release for .NET Framework 2.0
  • Archived Reference Implementations
  • Applied Integration Baseline Reference Implementation

  • Application Types
    Guidance assets listed by application type.

    Category Solution Assets
    Mobile
  • Mobile Client Software Factory
  • Rich Client
  • Composite Application Guidance for WPF
  • Smart Client Architecture and Design Guide
  • Smart Client Software Factory
  • Service
  • Improving Web Services Security: Scenarios and Implementation Guidance for WCF
  • Web Service Security Guidance: Scenarios, Patterns, and Implementation Guidance for Web Services Enhancements (WSE) 3.0
  • Web Service Software Factory
  • Web Client
  • Design and Implementation Guidelines for Web Clients
  • Improving .NET Application Performance and Scalability
  • Improving Web Application Security: Threats and Countermeasures
  • Web Client Software Factory


  • Quality Attributes
    Guidance assets listed by quality attributes.
    Category Solution Assets
    Integration
  • Enterprise Solution Patterns Using Microsoft .NET
  • Guidelines for Application Integration
  • Integration Patterns
  • Interoperability
  • Application Interoperability: Microsoft .NET and J2EE
  • Enterprise Solution Patterns Using Microsoft .NET
  • Microsoft .NET /COM Migration and Interoperability
  • Flexibility
  • Policy Injection Application Block
  • Unity Application Block
  • Manageability
  • Deploying .NET Framework-based Applications
  • Enterprise Solution Patterns Using Microsoft .NET
  • Monitoring in .NET Distributed Application Design
  • Production Debugging for .NET Framework Applications
  • Performance
  • Improving .NET Application Performance and Scalability
  • Performance Testing Guidance for Web Applications
  • Scalability
  • Improving .NET Application Performance and Scalability
  • Security
  • Designing Application-Managed Authorization
  • Improving Web Application Security: Threats and Countermeasures
  • Improving Web Services Security: Scenarios and Implementation Guidance for WCF
  • Security Engineering Explained
  • Security Guidance for .NET Framework 2.0
  • Threat Modeling Web Applications
  • Web Service Security Guidance: Scenarios, Patterns, and Implementation Guidance for Web Services Enhancements (WSE) 3.0

  • Engineering Practices
    Guidance assets organized by engineering practices.
    Category Solution Assets
    Deployment
  • Deploying .NET Framework-based Applications
  • Enterprise Solution Patterns Using Microsoft .NET
  • Monitoring in .NET Distributed Application Design
  • Production Debugging for .NET Framework Applications
  • Performance Engineering
  • Performance Testing Guidance for Web Applications
  • Security Engineering
  • Security Engineering Explained
  • Threat Modeling Web Applications
  • Team Development
  • Team Development with Visual Studio Team Foundation Server
  • Testing
  • Performance Testing Guidance for Web Applications
  •  My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • App Arch Guide 2.0 Overview Slides
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Components
  • Key Software Trends
  • J.D. Meier's Blog

    App Arch How Tos Posted to CodePlex

    • 6 Comments

    As part of our patterns & practices App Arch Guide 2.0 project, we'll be creating step-by-step How Tos.  The How Tos help turn some of the concepts into action.  They are particularly helpful for some areas where there's friction or complexity.  Here's our initial batch of draft How Tos on CodePlex:

    They're still works in progress, but this way you can start to bang on them and give feedback.

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • App Arch Guide 2.0 Overview Slides
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Components
  • Key Software Trends

  • J.D. Meier's Blog

    App Arch Guide 2.0 Overview Slides

    • 8 Comments

    I added a brief over deck of our patterns & practices App Arch Guide 2.0 project to codeplex:

    It's actually a pretty fast slide deck.  The first handful of slides give you a quick overview of the project goals and scope.  The appendix is all visual.  You can flip through and see how some of the key parts of our story are unfolding.

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Components
  • Key Software Trends
  • J.D. Meier's Blog

    Key Architecture Resources

    • 4 Comments

    As part of our patterns & practices App Arch Guide 2.0, I put together a short list of resources I want to make sure my team is spending time in:

    As far as some community sites, here's a short list of ones I've found helpful:

    What's your favorite haunts for architecture nuggets?

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Tiers
  • Layers and Components
  • Services Layer
  • Scenario Frames for Presentation, Business, Data and Services
  • patterns & practices Security Engineering
  • patterns & practices Performance Engineering
  • Key Software Trends
  • J.D. Meier's Blog

    Scenario-Based Architecture Evaluation Methods

    • 0 Comments

    I'm a fan of scenarios.  Whether you use scenarios for scenario-driven development,  scenario-based evaluations, competitive assessments or for shaping products ...  scenarios are where the rubber meets the road.  I think scenarios are particularly effective for evaluating architectures and for making architecture trade-off decisions.  I wrote a short post on Shaping Software that reiterates a mantra I frequently use ... you can't evaluate architecture in a vacuum.

  • J.D. Meier's Blog

    Arch Frame Posted to CodePlex

    • 1 Comments

    Today we posted our Arch Frame to CodePlex.  Wednesdays are ship days (I don't ship on Fridays.)  The App Arch Frame is a collection of hot spots you hit when building line of business (LOB) applications.  The key to the buckets is that they organize actionable principles, patterns, and practices.  They also help us overlay patterns & practices solution assets.  You give feedback either in the comments here, or on the CodePlex page:

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • Abstract for Application Architecture Guide 2.0
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Tiers
  • Layers and Components
  • Services Layer
  • Scenario Frames for Presentation, Business, Data and Services
  • patterns & practices Security Engineering
  • patterns & practices Performance Engineering
  • Key Software Trends
  • J.D. Meier's Blog

    Abstract for Application Architecture Guide 2.0

    • 12 Comments

    A few people have asked me for an abstract on the patterns & practices Application Architecture Guide 2.0 project (a work in progress).  Here it is.

    Conceptual Framework
    A picture is worth a thousand words.  Here's the conceptual framework for the guide:

    AppArchMetaFrame

    Tagline

    "How to put the legos together"

    Abstract
    The purpose of the Application Architecture Guide 2.0 is to  improve your effectiveness building applications on the Microsoft platform.   The primary audience is solution architects and developer leads.  The guide will provide design-level guidance for the architecture and design of applications built on the .NET Framework. It focuses on the most common types of applications, partitioning application functionality into layers, components, and services, and walks through their key design characteristics.

    Key Features

    • Canonical app frame - describes at a meta-level, the tiers and layers that an architect should consider. Each tier/layer will be described in terms of its focus, function, capabilities, common design patterns and technologies.
    • App Types - 5-7 canonical application archetypes to illustrate common application types. Each archetype will be described in terms of the target scenarios, technologies, patterns and infrastructure it contains. Each archetype will be mapped to the canonical app frame. They are illustrative of common app types and not comprehensive or definitive.
    • Arch Frame - a common set of categories for hot spots for key engineering decisions.
    • Quality Attributes - a set of qualities/abilities that shape your application architecture: performance, security, scalability, manageability, deployment, communication, etc.
    • Principles, patterns and practices - Using the frames as backdrops, the guide will overlay relevant principles, patterns, and practices.
    • Technologies and capabilities - a description/overview of the Microsoft custom app dev platform and the main technologies and capabilities within it.

    Key Scenarios

    • Help you choose the right architecture for your application.
    • Help you choose the right technologies
    • Help you make more effective choices for key engineering decisions.
    • Help you map appropriate strategies and patterns.
    • Help you map relevant patterns & practices solution assets.

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • App Arch Meta-Frame
  • App Types
  • Architecture Frame
  • App Arch Guidelines
  • Layers and Tiers
  • Layers and Components
  • Services Layer
  • Scenario Frames for Presentation, Business, Data and Services
  • patterns & practices Security Engineering
  • patterns & practices Performance Engineering
  • Key Software Trends
  • J.D. Meier's Blog

    Sources Of Insight is One Month Old

    • 2 Comments

    A few of my mentees mentioned that I'm not posting enough about effectiveness on my MSDN blog.  That's true.  I'll be posting more on effectiveness in the near future.  Meanwhile, I've been posting insights and actions for work and life on Sources of Insight.  It's how I scale myself and how I create reusable nuggets for my mentees.  Sources Of Insight is one month old.  Although the blog is only a month old, I ported my nuggets from my earlier blog, The Bookshare.  There are more than 250 nuggets that will help you improve your life.  Each nugget is carefully crafted to help you turn insights into action and improve your effectiveness.

  • J.D. Meier's Blog

    Architecture Frame

    • 10 Comments

    As part of our patterns & practices App Arch Guide 2.0 project, we've put together an arch frame.  The arch frame is simply a collection of hot spots.  These aren't just any hot spot though.  These hot spots represent key engineering decisions, anti-patterns, and opportunities for improved designs for more effective technical architectures.  This Arch Frame is part of the larger App Arch Meta Frame.  Think of it as an important branch off the main tree.  It serves as a lens to cut through a lot of information to get to the most meaningful and actionable guidance.

    Categories
    The following categories are the "hot spots" in the architecture frame:

    • Authentication and Authorization
    • Caching and State
    • Communication
    • Composition
    • Concurrency and Transactions
    • Configuration Management
    • Coupling and Cohesion
    • Data Access
    • Exception Management
    • Logging and Instrumentation
    • User Experience
    • Validation
    • Workflow

    What you might notice about the hot spots is that they map to common cross-cutting concerns when building applications.  You also might notice that the hot spots map to various patterns & practices solution assets.  For example, Enterprise Library includes blocks for caching, data access, exception management, logging, validation ... etc.  The categories also map to very actionable decisions where you there's relevant principles, patterns, and practices.  These buckets also contain many anti-patterns.  The worst anti-patterns are the "do overs."  Nobody wants a "do over" architecture.

    Key Engineering Decisions
    This table summarizes the key engineering decisions within each hot spot:

    Category Key Engineering Decisions
    Authentication and Authorization
  • How to store user identities.
  • How to authenticate callers.
  • How to authorize callers.
  • How to flow identity across layers and tiers.
  • Caching and State
  • How to choose effective caching strategies.
  • How to improve performance with caching.
  • How to improve security with caching.
  • How to improve availability with caching.
  • How to keep the cached data up to date.
  • How to determine when and why to use a custom cache.
  • How to determine what data to cache.
  • How to determine where to cache the data.
  • How to determine the expiration policy and scavenging mechanism.
  • How to load the cache data.
  • How to monitor a cache.
  • How to synchronize caches across a farm.
  • How to determine which caching technique provides the best performance and scalability for a specific scenario and configuration.
  • How to determine which caching technology complies with the application's requirements for security, monitoring, and management.
  • Communication
  • How to communicate between layers / tiers.
  • How to perform asynchronous communication.
  • How to pass sensitive data.
  • Composition
  • How do design for composition.
  • How to design loose coupling between modules.
  • How to handle dependencies in a loosely coupled way.
  • Concurrency and Transactions
  • How to handle concurrency between threads.
  • How to choose between optimistic and pessimistic concurrency.
  • How to handle distributed transactions.
  • How to handle long running transactions.
  • How to determine appropriate transaction isolation levels.
  • How to determine when compensating transactions are required.
  • Configuration Management
  • How to determine which information needs to be configurable.
  • How to determine where and how to store configuration information.
  • How to handle sensitive information.
  • How to handle configuration information in a farm/cluster.
  • Coupling and Cohesion
  • How to separate concerns
  • How to structure the application.
  • How to choose an appropriate layering strategy.
  • How to establish boundaries.
  • Data Access
  • How to manage database connections.
  • How to handle exceptions.
  • How to improve performance.
  • How to improve manageability.
  • How to handle blobs.
  • How to page records.
  • How to perform transactions.
  • Exception Management
  • How to handle exceptions.
  • How to log exceptions.
  • Logging and Instrumentation
  • How to determine which information to log.
  • How to make the logging configurable
  • User Experience
  • How to improve task efficiency and effectiveness.
  • How to improve responsiveness.
  • How to improve user empowerment.
  • How to improve look and feel.
  • Validation
  • How to determine where and how to perform validation.
  • How to validate for length, range, format, and type.
  • How to constrain and reject input.
  • How to sanitize output.
  • Workflow
  • How to handle concurrency issues within a workflow
  • How to handle task failure within a workflow
  • How to orchestrate processes within a workflow
  •  

    Key Issues
    This table summarizes the key issues within each hot spot:

    Category Key Issues
    Authentication and Authorization
  • Clear text credentials in configuration files
  • Passing clear text credentials over the network
  • Over-privileged accounts
  • Long sessions
  • Mixing personalization with authentication
  • Reliance on a single gatekeeper
  • Failing to lock down system resources against application identities
  • Failing to limit database access to specified stored procedures
  • Inadequate separation of privileges
  • Caching and State
  • Cache misses.
  • Failure to expire items
  • Poor cache design
  • Lack of a cache synchronization mechanism for scaling out.
  • Communication
  • Increased network traffic and latency due to chatty calls between layers.
  • Inappropriate transport protocols and wire formats.
  • Large data volumes over limited bandwidth networks.
  • Composition Tightly coupled modules Duplication of code
    Concurrency and Transactions
  • Blocking calls
  • Nongranular locks
  • Misuing threads
  • Holding onto locks longer than necessary
  • Inappropriate isolation levels
  • Configuration Management
  • Insecure administration interfaces
  • Insecure configuration stores
  • Clear text configuration data
  • Too many administrators
  • Over-privileged process accounts and service accounts
  • Coupling and Cohesion
  • Limited scalability due to server and resource affinity.
  • Mixed presentation and business logic, which limits your options for scaling out your application.
  • Lifetime issues due to tight coupling.
  • Data Access
  • Per user authentication and authorization when not required.
  • Chatty calls to database
  • Intersperse of business logic
  • Exception Management
  • Leaving system / application in unstable state
  • Revealing sensitive information to end users.
  • Using exceptions for logic
  • Not logging enough details about the exception.
  • Logging and Instrumentation
  • Lack of logging and instrumentation
  • Too fine grained logging and instrumentation
  • Not making logging and instrumentation configurable option at runtime
  • Failure to log business critical functionality.
  • User Experience
  • Inefficient or ineffective task support.
  • Poor responsiveness.
  • Disempowered users.
  • Validation
  • Application-only filters for malicious input
  • Non-validated input in the Hypertext Markup Language (HTML) output stream
  • Non-validated input used to generate SQL queries
  • Reliance on client-side validation
  • Use of input file names, URLs, or user names for security decisions
  • Workflow
  • Tight coupling
  • Inflexible processes
  • Race and deadlock issues
  •  

    Key Guidelines
    This table summarizes the key guidelines within each hot spot:

    Category Key Guidelines
    Authentication and Authorization
  • Consider single-sign on requirements.
  • Separate public and restricted areas.
  • Use account lockout policies for end-user accounts.
  • Support password expiration periods.
  • Be able to disable accounts.
  • Do not store passwords in user stores.
  • Require strong passwords.
  • Do not send passwords over the wire in plaintext.
  • Protect authentication cookies.
  • Use multiple gatekeepers.
  • Restrict user access to system-level resources.
  • Consider authorization granularity.
  • Caching and State
  • Avoid caching per-user data.
  • Avoid caching volatile data which is required by the user to be accurate and updated in real time.
  • Cache data that does not change very frequently or is completely static.
  • Do not cache shared expensive resources.
  • Cache transformed data, keeping in mind the data use.
  • Evaluate stateful versus stateless design.
  • Consider your state store options.
  • Minimize session data.
  • Free session resources as soon as possible.
  • Avoid accessing session variables from business logic.
  • Communication
  • Choose the appropriate remote communication mechanism.
  • Design chunky interfaces.
  • Consider how to pass data between layers.
  • Minimize the amount of data sent across the wire.
  • Batch work to reduce calls over the network.
  • Reduce transitions across boundaries.
  • Consider asynchronous communication.
  • Consider message queuing.
  • Consider a "fire and forget" invocation model.
  • Cut call chains with queues and caches as much as possible. Doing so will enhance the scalability and availability of the overall solution.
  • Push out asynchronous boundaries close to the user, service interfaces, and service agents, to isolate your service from external dependencies.
  • If you need to expose functionality as a synchronous operation, evaluate whether you can wrap an internally asynchronous operation as described in the following discussion
  • Composition
  • Avoid using dynamic layouts that are difficult to load and maintain.
  • Be careful with dependencies between components. Use abstraction patterns when possible to avoid issues with maintainability.
  • Consider creating templates with placeholders. For example use the Template View pattern to compose dynamic web pages to ensure reuse and consistency.
  • Consider composing views from reusable modular parts. For example use the Composite View pattern to build a view from modular, atomic component parts.
  • Use well-known design patterns to implement a composite interface containing separate modules or user controls where appropriate.
  • Modules should not directly reference one another or the application that loaded them.
  • Modules should use services to communicate with the application or with other modules.
  • Modules should not be responsible for managing their dependencies.
  • Modules should support being added and removed from the system in a pluggable fashion.
  • Concurrency and Transactions
  • Treat threads as a shared resource.
  • Pool shared or scarce resources.
  • Acquire late, release early.
  • Consider efficient object creation and destruction.
  • Consider resource throttling.
  • Reduce contention by minimizing lock times.
  • Balance between coarse- and fine-grained locks.
  • Choose an appropriate transaction isolation level.
  • Avoid long-running atomic transactions.
  • Configuration Management
  • Protect your administration interfaces.
  • Protect your configuration store.
  • Maintain separate administration privileges.
  • Use least privileged process and service accounts.
  • Coupling and Cohesion
  • Partition application functionality into logical layers.
  • Choose the proper locality for your objects based on your reliability, performance, and scalability needs.
  • Design for loose coupling.
  • Design for high cohesion.
  • Use early binding where possible.
  • Evaluate resource affinity.
  • Data Access
  • If your application uses a single database, use the database-specific data provider.
  • If you need to support multiple databases, you generally need to have an abstraction layer, which helps you transparently connect to the currently configured store.
  • Consider resource throttling.
  • Consider the identities you flow to the database.
  • Separate read-only and transactional requests.
  • Avoid unnecessary data returns.
  • Exception Management
  • Avoid revealing sensitive data to users.
  • Do not use exceptions to control application flow.
  • Use validation code to avoid unnecessary exceptions.
  • Do not catch exceptions that you cannot handle.
  • Be aware that rethrowing is expensive.
  • Preserve as much diagnostic information as possible in your exception handlers.
  • Logging and Instrumentation
  • Instrument your code up front.
  • Make your logging configurable.
  • User Experience
  • Measure effectiveness against scenarios.
  • Improve user responsiveness where possible.
  • Model from effective user design.
  • Validation
  • Validate for length, range, format, and type.
  • Constrain and reject input.
  • Sanitize output.
  • Don’t rely on client-side validation.
  • Workflow
  • Determine management requirements. If a business user needs to manage the workflow, you’ll need a solution that provides an interface that the business user can understand.
  • Determine how exceptions will be handled.
  • With human workflow, you need to consider the un-deterministic nature of humans. In other words, you can’t determine when a task will be completed, or if it will be completed correctly.
  • Use service interfaces to interact with external workflow providers.
  • If supported, use designers and metadata to define the workflow instead of code to define the workflow.
  • How To Provide Feedback
    We're still banging through the frame.  There's some rough spots.  We want to make sure we can map problems, principles, patterns, assets, and technologies to the right hot spots.  We want a prioritized list over a laundry list so we're still deciding what's in and what's out.  We'll add it to CodePlex shortly.  You can either comment here on my blog or wait until the frame is on CodePlex, and then provide comments in the Wiki.

    Additional Resources


    My Related Posts

  • J.D. Meier's Blog

    App Types

    • 6 Comments

    As part of our patterns & practices App Arch Guide 2.0 project, we've created a set of application archetypes ("app types" for short.)   You can see our app archetypes on our Application Types (Archetypes) Index page.  To keep it simple, we focused on a small set of common application types that customers identify with.  Here's the cool part ... for each application type, we're taking a principle-based approach, then showing relevant patterns, p&p solution assets, and Microsoft technologies.  Keep in mind we're still hashing it out.

    Application Types

    • Mobile
    • Rich Client
    • Rich Internet Application
    • Service
    • Web Application

    Mobile
    A mobile application will normally be structured as a multi-layered application consisting of user experience, business and data layers.

    AppTypeMobile

    When developing a mobile application you may choose to develop a thin web-based client or a rich client. If you are building a rich client, the business and data layers are likely to be on the device itself. If you are building a thin client the business and data layers will be on the server.

    See Mobile Application Type (CodePlex)

    Rich Client
    Rich client user interfaces can provide high performance, interactive, and rich user experiences for applications that must operate in stand-alone, connected, occasionally connected, and disconnected scenarios.

    AppTypeRichClientApp

    Windows Forms, WPF, and Office Business Application (OBA) development environments and tools are available that allow developers to quickly and easily build Rich Client applications. While these technologies can be used to create standalone applications, they can also be used to create applications that run on the client machine, but communicate with services exposed by other layers (both logical and physical) that expose operations the client requires. These operations may include data access, information retrieval, searching, sending information to other systems, backup, and related activities.

    See Rich Client Archetype (CodePlex).

    Rich Internet Application (RIA)
    A Rich Internet Application (RIA) runs in the browser in a sandbox.  The benefits of a RIA application include richer user experience, better responsiveness and network efficiency over traditional Web applications.

    AppTypeRIAApp

    See Rich Internet Application Archetype (CodePlex).

    Service
    A service is a public interface that provides access to a unit of functionality. Services literally provide some programmatic ‘service’ to the caller who consumes them.

    AppTypeServicesApp

    Services are loosely coupled and can be combined from within a client or within other services to provide more complex functionality. Services are distributable and can be accessed from a remote machine as well as from the local machine on which they are running. Services are message oriented, meaning that service interfaces are defined by a WSDL file and operations are called using xml-based message schemas that are passed over a transport. Services support a heterogeneous environment by focusing interoperability at the message/interface definition. If components can understand the message and interface definition they can use the service regardless of their base technology.

    See Service Application Archetype (CodePlex).

    Web Application
    The core of a Web application is its server-side logic.  The Web application layer itself can be comprised of many distinct layers.  The typical example is a three-layered architecture, comprised of presentation, business, and data layers.

    AppTypeWebApp

    See Web Application Archetype (CodePlex).

    Template for Application Type Guidance
    Here's the approach we're using to organize the guidance for each archetype:

    • Overview
    • Design considerations
    • Frame of Hot Spots / Categories
    • Category X
    • Category Y
    • Category Z
    • Presentation Layer Considerations
    • Business Layer Considerations
    • Data Layer Considerations
    • Services Layer Considerations
    • Performance Considerations
    • Security Considerations
    • Deployment Considerations
    • Pattern Map
    • Technology Considerations
    • patterns & practices Solution Assets
    • Additional Resources

    You can walk the Web application example to see it in action.

    Additional Resources

    My Related Posts

  • J.D. Meier's Blog

    Key Software Trends

    • 7 Comments

    What you don’t know can hurt you. Sometimes the world can change under your feet and you never saw it coming.  I like to anticipate and stay ahead of the curve where I can. As part of our patterns & practices Application Architecture Guide 2.0 project, I’ve been hunting and gathering trends that influence software development. Rather than make this exhaustive, I wanted to share "good enough" for now, and leave you room to tell me what I've missed and share what you’re seeing in your world.

    Key Notes On Trends

    • Trends aren’t fads.  Trends tend to have depth and staying power, whereas fads tend to be short-lived.
    • Some fads are future trends in disguise.
    • In my experience, consumer trends influence Enterprise trends.
    • Use trends to help you avoid surprises and to sound hip at the water cooler when you too can speak the buzz.

    Trend "Hot Spots"
    Rather than distinguish between trends and fads, I decided to focus on “hot spots” and simply identify the topics that keep showing up in various contexts with customers, in Microsoft, in the industry, … etc.

    Applications
  • Business Process Management (BPM)
  • Composite / Mash Ups (Server-side, Client-side)
  • Dynamic Languages
  • Functional Programming
  • Health
  • Model-Driven
  • Representational State Transfer (REST)
  • Software plus Services / Software as a Service / Platform as a Service (S+S / SaaS / PaaS)
  • Service Oriented Architecture (SOA)
  • Rich Internet Applications (RIA)
  • Testability
  • User Empowerment (shift in power from business and tech to the user)
  • User Experience (not to be confused with UI)
  • Infrastructure
  • Cloud Computing
  • Green IT
  • Virtualization
  • Very Large Databases
  • Performance
  • Grid
  • High Performance Computing (HPC)
  • Many-core / Multi-core
  • Parallel Computing
  • Software Development
  • Application Life-Cycle Management (ALM)
  • Distributed Teams
  • Lean
  • Scrum
  • User-Lead
  • XP
  • I know the list looks simple enough, but it actually took a bit of vetting to spiral down on the ones that have wood behind the arrow and have signs of a trajectory.

    David Chou on Trends ...
    In this post, Cloud Computing and Microsoft, David Chou identifies the following trends:

    • Consumerization of the Web, and use of browsers
    • Application development efforts shifting towards thin clients and server-side programming
    • Improvements in network bandwidth, anywhere wireless access, etc.
    • Increased maturity in open source software
    • Proliferation and advancement of mobile devices
    • Service Oriented Architecture
    • Software-as-a-Service
    • Utility/Grid Computing

    Simon Guest on Trends ...
    In his talk, An Architectural Overview of Software + Services, Simon Guest outlines the following industry trends:

    • Trend 1: Service Oriented Architecture (SOA)
    • Trend 2: Software as a Service (SaaS)
    • Trend 3: Web 2.0
    • Trend 4: Rich Internet Applications (RIA)
    • Trend 4: Cloud Computing

    Simon connects the dots from the trends to how they support Software + Services:

    • SOA - Reuse and agility
    • RIA: Rich Internet Applications - Experience
    • SaaS: Software as a Service - Flexible pricing and delivery
    • Cloud Computing - Service Utility
    • Web 2.0 - Network Effect

    Phillip Winslow on Trends …
    In an Interop Keynote Panel on Current Software Trends, Phillip Winslow responded to the to please predict the biggest IT story for 2008 as follows:

    “ … decoupling of the end user platform. Virtualization and desktop virtualization and layering on SAAS… Salesforce.com, gmail, etc how we think about the desktop sitting on our desk will be much different.”

    TrendWatching.com on Trends ...
    Here's a snapshot of interesting tidbits from a an earlier version of this trend report page on TrendWatching.com:

    • HappyNomics - In fact, this year may be a good time to move 'HAPPYNOMICS' from the academic world to your ideation team.
    • 12 Themes - The 20+ trends covered in the report are part of bigger themes:
      'REAL', 'BEST', 'STORY', 'UNREAL', 'UNFIXED', 'TIME', 'GREEN', 'DOMAIN', 'ONLINE', '(R)ETAIL', 'ASSIST' and 'PARTICIPATE'.   These themes are all about what will EXCITE consumers in the near future, and can be read/presented independently, or as a coherent 'story'.

    Here's the cool part.  I saw GREEN on their page well before I noticed any of the Green IT initiatives show up.  It struck me as an example of consumer influencing Enterprise.

    Key Posts
    Here's some of the posts that I found to be useful for understanding the impact and influences behind some of the trends:

    Additional Resources
    I know this looks like a laundry list of links but these are actually helpful to quickly get what some of the topics are about:

    My Related Posts

  • J.D. Meier's Blog

    The Story of the Engineering Practices Frame

    • 2 Comments

    I shared the story of our patterns & practices Engineering Practices Frame on Shaping Software.  In a nutshell, the Engineering Practices Frame is a set of categories to organize software development knowledge.  The idea behind the frame is to help collect and share principles, patterns, and practices for life cycle activities and artifacts.  It's meant to play well with SWEBOK, various Microsoft efforts around life cycle practices, and our customers and field in the trenches.  The Engineering Practices Frame is also the foundation for our ALM Frame.

  • J.D. Meier's Blog

    patterns & practices Performance Engineering

    • 4 Comments

    As part of our patterns & practices App Arch Guide 2.0 project, we're consolidating our information on our patterns & practices Performance Engineering.  Our performance engineering approach is simply a collection of performance-focused techniques that we found to be effective for meeting your performance objectives.  One of the keys to the effectiveness is our performance frame.   Our performance frame is a collection of "hot spots" that organize principles, patterns, and practices, as well as anti-patterns.  We use the frame to perform effective performance design and code inspections.  Here's a preview of our cheat sheet so far.  You'll notice a lot of similarity with our patterns & practices Security Engineering.  It's by design so that you can use a consistent approach for handling both security and performance.

    Performance Overlay
    This is our patterns & practices Performance Overlay:

    PerformanceEngineering

    Key Activities in the Life Cycle
    This Performance Engineering approach extends these proven core activities to create performance specific activities.  These include:

    • Performance Objectives. Setting objectives helps you scope and prioritize your work by setting boundaries and constraints. Setting performance objectives helps you identify where to start, how to proceed, and when your application meets your performance goals.
    • Budgeting. Budget represents your constraints and enables you to specify how much you can spend (resource-wise) and how you plan to spend it.
    • Performance Modeling. Performance modeling is an engineering technique that provides a structured and repeatable approach to meeting your performance objectives.
    • Performance Design Guidelines. Applying design guidelines, patterns and principles which enable you to engineer for performance from an early stage.
    • Performance Design Inspections. Performance design inspections are an effective way to identify problems in your application design. By using pattern-based categories and a question-driven approach, you simplify evaluating your design against root cause performance issues.
    • Performance Code Inspections. Many performance defects are found during code reviews. Analyzing code for performance defects includes knowing what to look for and how to look for it. Performance code inspections to identify inefficient coding practices that could lead to performance bottlenecks.
    • Performance Testing. Load and stress testing is used to generate metrics and to verify application behavior and performance under normal and peak load conditions.
    • Performance Tuning.  Performance tuning is an iterative process that you use to identify and eliminate bottlenecks until your application meets its performance objectives. You start by establishing a baseline. Then you collect data, analyze the results, and make configuration changes based on the analysis. After each set of changes, you retest and measure to verify that your application has moved closer to its performance objectives.
    • Performance Health Metrics.   Identity the measures, measurements, and criteria for evaluating the health of your application from a performance perspective.
    • Performance Deployment Inspections. During the deployment phase, you validate your model by using production metrics. You can validate workload estimates, resource utilization levels, response time, and throughput.
    • Capacity Planning. You should continue to measure and monitor when your application is deployed in the production environment. Changes that may affect system performance include increased user loads, deployment of new applications on shared infrastructure, system software revisions, and updates to your application to provide enhanced or new functionality. Use your performance metrics to guide your capacity and scaling plans.

    Performance Frames
    Performance Frames define a set of patterns-based categories that can organize repeatable problems and solutions. You can use these categories to divide your application architecture for further analysis and to help identify application performance issues. The categories within the frame represent the critical areas where mistakes are most often made.

    Category Description
    Caching What and where to cache? Caching refers to how your applications caches data. The main points to be considered are Per user, application-wide, data volatility.
    Communication How to communicate between layers? Communication refers to choices for transport mechanism, boundaries, remote interface design, round trips, serialization, and bandwidth.
    Concurrency How to handle concurrent user interactions? Concurrency refers to choices for Transaction, locks, threading, and queuing.
    Coupling / Cohesion How to structure your application? Coupling and Cohesion refers structuring choices leading to loose coupling, high cohesion among components and layers.
    Data Access How to access data? Data Access refers to choices and approaches for schema design, Paging, Hierarchies, Indexes, Amount of data, and Round trips.
    Data Structures / Algorithms How to handle data? Data Structures and Algorithm refers to choice of algorithms; Arrays vs. collections.
    Exception Management How to handle exceptions? Exceptions management refers to choices / approach for catching, throwing, exceptions.
    Resource Management How to manage resources? Resource Management refers to approach for allocating, creating, destroying, and pooling of application resource
    State Management What and where to maintain state? State management refres to how your application maintains state. The main points to consider are Per user, application-wide, persistence, and location.


    Architecture and Design Issues
    Use the diagram below to help you think about performance-related architecture and design issues in your application.

    PerformanceIssues

    The key areas of concern for each application tier are:

    • Browser.  Blocked or unresponsive UI.
    • Web Server.  Using state affinity.  Wrong data types.  Fetching per request instead of caching.  Poor resource management.
    • Application Server.  Blocking operations.  Inappropriate choice of data structures and algorithms.  Not pooling database connections.
    • Database Server.  Chatty instead of batch processing.  Contention, isolation levels, locking and deadlock.

    Design Process Principles
    Consider the following principles to enhance your design process:

    • Set objective goals. Avoid ambiguous or incomplete goals that cannot be measured such as "the application must run fast" or "the application must load quickly." You need to know the performance and scalability goals of your application so that you can (a) design to meet them, and (b) plan your tests around them. Make sure that your goals are measurable and verifiable. Requirements to consider for your performance objectives include response times, throughput, resource utilization, and workload. For example, how long should a particular request take? How many users does your application need to support? What is the peak load the application must handle? How many transactions per second must it support? You must also consider resource utilization thresholds. How much CPU, memory, network I/O, and disk I/O is it acceptable for your application to consume?
    • Validate your architecture and design early. Identify, prototype, and validate your key design choices up front. Beginning with the end in mind, your goal is to evaluate whether your application architecture can support your performance goals. Some of the important decisions to validate up front include deployment topology, load balancing, network bandwidth, authentication and authorization strategies, exception management, instrumentation, database design, data access strategies, state management, and caching. Be prepared to cut features and functionality or rework areas that do not meet your performance goals. Know the cost of specific design choices and features.
    • Cut the deadwood. Often the greatest gains come from finding whole sections of work that can be removed because they are unnecessary. This often occurs when (well-tuned) functions are composed to perform some greater operation. It is often the case that many interim results from the first function in your system do not end up getting used if they are destined for the second and subsequent functions. Elimination of these "waste" paths can yield tremendous end-to-end improvements.
    • Tune end-to-end performance. Optimizing a single feature could take away resources from another feature and hinder overall performance. Likewise, a single bottleneck in a subsystem within your application can affect overall application performance regardless of how well the other subsystems are tuned. You obtain the most benefit from performance testing when you tune end-to-end, rather than spending considerable time and money on tuning one particular subsystem. Identify bottlenecks, and then tune specific parts of your application. Often performance work moves from one bottleneck to the next bottleneck.
    • Measure throughout the life cycle. You need to know whether your application's performance is moving toward or away from your performance objectives. Performance tuning is an iterative process of continuous improvement with hopefully steady gains, punctuated by unplanned losses, until you meet your objectives. Measure your application's performance against your performance objectives throughout the development life cycle and make sure that performance is a core component of that life cycle. Unit test the performance of specific pieces of code and verify that the code meets the defined performance objectives before moving on to integrated performance testing.  When your application is in production, continue to measure its performance. Factors such as the number of users, usage patterns, and data volumes change over time. New applications may start to compete for shared resources.

    Design Guidelines
    This table represents a set of secure design guidelines for application architects. Use this as a starting point for performance design and to improve performance design inspections.

    Category Description
    Caching Decide where to cache data. Decide what data to cache. Decide the expiration policy and scavenging mechanism. Decide how to load the cache data. Avoid distributed coherent caches.
    Communication Choose the appropriate remote communication mechanism. Design chunky interfaces. Consider how to pass data between layers. Minimize the amount of data sent across the wire. Batch work to reduce calls over the network. Reduce transitions across boundaries. Consider asynchronous communication. Consider message queuing. Consider a "fire and forget" invocation model.
    Concurrency Design for loose coupling. Design for high cohesion. Partition application functionality into logical layers. Use early binding where possible. Evaluate resource affinity.
    Coupling / Cohesion How to structure your application? Coupling and Cohesion refers structuring choices leading to loose coupling, high cohesion among components and layers.
    Data Structures / Algorithms Choose an appropriate data structure. Pre-assign size for large dynamic growth data types. Use value and reference types appropriately.
    Resource Management Treat threads as a shared resource. Pool shared or scarce resources. Acquire late, release early. Consider efficient object creation and destruction. Consider resource throttling.
    Resource Management How to manage resources? Resource Management refers to approach for allocating, creating, destroying, and pooling of application resource
    State Management Evaluate stateful versus stateless design. Consider your state store options. Minimize session data. Free session resources as soon as possible. Avoid accessing session variables from business logic.

    Additional Resources

    My Related Posts

  • J.D. Meier's Blog

    Top 100 Blogs for Software Development Managers

    • 2 Comments

    Jurgen Appelo put together an impressive list of what he calls the Top 100 Blogs for Software Development Managers (Q3 2008).  I think it's a great way to find out about blogs you might not know about.

    The Top 10
    Here's the top ten blogs on the list:

    No, I didn't make the top 10.  I'm number 20.

    Blogs That Made the List
    In Jurgen's words:

    "The idea of this list is to promote popular blogs that are interesting to software development managers. Therefore, I ignored the blogs that did not have any (recent) content directly related to either requirements, design, coding, testing, management, process improvement or methodologies. I also discarded generic project management blogs without topics on software development practices. Likewise, I discarded the pure coding and technology blogs when they did not elevate to the level of "managing" or "ruminating about" software development. Last of all, I only considered blogs in the English language."

  • Page 29 of 42 (1,042 items) «2728293031»