J.D. Meier's Blog

Software Engineering, Project Management, and Effectiveness

  • J.D. Meier's Blog

    Services Layer


    In my previous posts I showed layers and components, and layers and tiers.  In this post, I'll show the services layer in a layered architecture.

    Services Layer
    Here's a visual example of a services layer, where the application is exposing services:


    Note that services don't need to be "web" services.

    Key Services Layer Components
    Here's the key components of a services layer:

    • Service Interfaces. The service interface acts as a facade that exposes the business logic to potential consumers.  The definition of the set of messages that must be exchanged with a service in order for the service to perform a specific business task is the contract.
    • Message Types. Message Types are “message contracts” for communication between service consumers and providers.  They wrap business entity components into specific messages for the calling service.

    Additional Resources

    My Related Posts

  • patterns & practices App Arch Guide 2.0 Project
  • Layers and Components
  • Layers and Tiers
  • App Arch Meta Frame
  • Scenario Frames for Presentation, Business, Data and Services
  • J.D. Meier's Blog

    Layers and Components


    In my previous post, I summarized layers and tiers.  In this post, I'll walk through the key components of the layers.  This exercise is part of our patterns & practices App Arch Guide 2.0 project.

    Layers and Components
    Here's a visual of a layered architecture and relevant components:


    Note that this is just an example of common components and layers.  Your scenarios may vary.

    Presentation Layer Components
    Here's typical presentation layer components:

    • User interface (UI) components. Your user interface components (controls) provide a way for users to interact with your application. They render and format data for users and acquire and validate data coming in from them.
    • User process components. Your user process components help synchronize and orchestrate user interactions. This way the process flow and state management logic is not hard-coded in the user interface elements themselves, and the same basic user interaction patterns can be reused by multiple user interfaces.

    Business Layer Components
    Here's typical business layer components:

    • Application Facade. (Optional).  Your application facade maps your business logic to your application.  It's optional and it depends on how reusable and generic your business logic is.  If your business logic was written specifically for your application, then you probably don't need an application facade.
    • Business components. Your business components implement business rules and perform business tasks.
    • Business workflows. Your business workflows define and coordinate long-running, multi-step business processes.  They can be implemented using business process management tools.
    • Business entity components. Your business entity components represent actual business entities (for example products or orders.)  You use them to pass data between components.  They're usually data structures (such as DataSets, DataReaders, or XML streams), but they can also be implemented using custom object-oriented classes.

    Data Layer Components
    Here's typical data layer components:

    • Data access logic components. Data access components abstract the logic necessary to access. Doing so centralizes data access functionality and makes it easier to configure and maintain.
    • Data Helpers / Utilities.   
    • Service agents. Your service agents help you call external services.  They do so by mapping the format of the data exposed by the service to the format your application uses, as well as help manage the semantics of calling the services.


    • Cross-cutting components.  This includes components  components to perform exception management, to authorize users to perform certain tasks, and to communicate with other services and applications.

    Does this match what you see in practice?

    Additional Resources
    Here's some relevant links:

    My Related Posts

  • J.D. Meier's Blog

    Layers and Tiers


    As part of the patterns & practices App Arch Guide 2.0 project, we needed to nail down layers and tiers.   

    Layers vs. Tiers
    The original App Arch Guide distinguished between layers and tiers:

    "This guide uses the term layer to refer to a component type and uses the term tier to refer to physical distribution patterns."

    In other words, layers are logical and tiers are physical (two-tier, three-tier, N-tier).  This distinction is helpful, particularly when you want to talk about where you run your layers (which tier).

    Presentation Layer, Business Layer and Data Layer
    While there's some variations in layer terms, many people that build application will identify with presentation, business, and data layers.  Here's an example:


    • Presentation Layer - provides interactive access to the application.  (You could argue that user interaction layer might be more appropriate.)
    • Business Layer - the logical grouping of components and services that provide the business functionality in your application.
    • Data Layer - the logical grouping of the components and services that provide data access functionality in your application.  (You could argue to call it your resource access layer.)

    Two-Tier, Three-Tier, and N-Tier
    As mentioned earlier, you can think of tiers as physical distribution patterns.   Here are some visual examples:







    Additional Resources
    Here's some links you might find useful:

    My Related Posts

  • J.D. Meier's Blog

    Scenario Frames for Presentation, Business, Data, and Services


    As part of our App Arch Guide 2.0 project, we're creating scenario frames to organize customer problems into meaningful lists.   These particular frames are an elaboration of our App Arch Meta Model.  This helps scope our guidance.   We also use them to test effectiveness.  The value of the guidance is the value of the problems solved.

    Scenario Frames on CodePlex
    You can review and contribute to our scenario frames on CodePlex:

    Presentation Layer Scenarios Frame
    Heres' the key hot spots for our presentation layer frame:

    • Caching
    • Composition
    • Exception Management
    • Input
    • Layout
    • Navigation
    • Presentation Entities
    • UI Components
    • UI Process Components
    • Validation

    Business Layer Scenarios Frame
    Here's the key hot spots for our business layer frame:

    • Authentication
    • Authorization
    • Business Components
    • Business Entities
    • Caching
    • Concurrency and Transactions
    • Data Access
    • Exception Management
    • Logging
    • Service Interface
    • Validation
    • Workflow

    Data Access Layer Scenarios Frame
    Here's the key hot spots for our data access layer frame:

    • General
    • BLOB
    • Batching
    • Connections
    • Data Format
    • Exception Management
    • Security Considerations
    • Stored Procedures
    • SQL Commands
    • Validation
    • XML

    Services Layer
    Heres's the key hot spots for our services layer frame:

    • General
    • Authentication
    • Authorization
    • Communication
    • Exception Management
    • Messaging Channels
    • Message Construction
    • Message Endpoint
    • Message Protection
    • Message Routing
    • Message Transformation
    • Message Exchange Patterns
    • REST
    • SOAP
    • Validation

    My Related Posts

  • J.D. Meier's Blog

    App Arch Meta-Frame


    As part of the App Arch Guidance project, we've created an organizing frame to help think about application architecture:


    Anatomy of the App Arch Meta Frame
    You can see from the figure, we have a few parts that work together:

    • Scenarios - You can't evaluate an architecture in a vacuum.  Scenarios are the backdrop and the context. 
    • Quality Attributes / Qualities - This is your classic set of reliability, security, performance, flexibility, maintainability ... etc.
    • Requirements / Constraints - These are the user, business, and technical rules that shape your architecture.
    • App Types - This is your overall shape.  This includes Web app, Rich Client, Mobile, ... etc.  While the line may blur, there's important distinctions among application types.
    • Architecture Styles - This includes architectural patterns such as N-tier, client-server, SOA, ... etc.  You can see shifts in styles over the years such as from object-orientation to message-orientation.
    • Architecture Frame - These are the architectural "hot spots."  This is where your key engineering decisions happen.

    How We Use the Frame
    We use the frame to explore and gain insight into different aspects of application architecture.  App arch is a big space.  We'll be using the frame to catalog and organize our various principles, patterns, practices, and assets.

    Keep in mind that this is a meta-frame (so it's a frame of frames.)  We'll have a collection of frames that shine the spotlight on more focused areas.

    What do you think? ...

    Additional Resources

  • J.D. Meier's Blog

    patterns & practices App Arch Guide 2.0 Project


    It's long overdo, but we're kicking off a project for v2 of the patterns & practices Application Architecture Guide.  You might be familiar with Tom Hollander's post "Application Architecture for .NET" 2 - The Revenge.  Well, now it's time for results.

    Solution Architects / Dev Leads / Developers
    Our primary audience is solution architects, developer leads, and developers.

    Principles, Patterns, and Practices
    I'm a principles, patterns, and practices kind of a guy, so expect the guide to be principle-based, durable, and evolvable.

    CodePlex Community Site
    It's a work in progress and it's early, but you can follow along here:

    Here's some of the areas we're looking at for the guide:


    • App Types
    • Arch Styles
    • Patterns
    • Layers, Tiers, and Components
    • Architectural Hot Spots
    • Presentation Layer
    • Business Layer
    • Data Access Layer
    • Services Layer
    • Services Design
    • Quality Attributes
    • Trends
    • Security Engineering
    • Performance Engineering
    • Baseline Architectures
    • Deployment Patterns


    • Choosing Presentation Technologies
    • Choosing Data Access Technologies
    • Choosing Workflow Technologies
    • Inline SQL vs. sprocs
    • MVC vs. MVP
    • Domain model driven vs. structure driven

    Your Feedback
    What would you like to see in the guide?

  • J.D. Meier's Blog

    SourcesOfInsight.com is Now Live


    Sources of Insight.com is now live.  It's a blog I use to share insights and actions for improving work and life.  I have a lot of mentees at Microsoft so it helps me scale.

    I have a very simple goal for the blog.  It's to share the best insight and action to help you be your best in any situation.

    It's a work in progress, but I treat it as a living KB of the best insight I can find from books, people, and quotes.  While books are great mentors, I also draw from a lot of people.  Some mentors have a knack for saying just the right things at just the right times.   Sometimes the right quote says it best.

    To make the most of the sight, first, start with the About.  That will give you a pretty good sense of what the blog is about.  Next, visit the Archives.   Unfortunately, I haven't fixed all my links yet, so the Archives is the fastest way to hop around.

    I'm still experimenting with format, style, and voice so that will continue to evolve.  Sometime's it's tough for me to switch gears from writing technical guidance to writing life changing insight.  The engineer in me wants precision.  The writer in me wants impact.  In the meantime, I can say that my mentees, friends, family, and colleagues have been using the nuggets to improve the quality of their lives.


  • J.D. Meier's Blog

    My Favorite Software Books


    Aside from people, I think books are the best mentors.  Ajoy mentioned to me that people like to know what books our team actually uses on the job.  I can't speak for the patterns & practices team, at large, but I thought I would help bootstrap by creating a list of my favorite software books.  The list includes the books that I continue to mine for principles, patterns, and practices for shaping software.

  • J.D. Meier's Blog

    Lessons Learned from Peaceful Warrior


    Have you seen Peaceful Warrior?  It's a pretty simple movie, but it's full of lessons.  Probably the most important lesson is how to keep getting back up when life knocks you down and how to continue to follow your dreams, even after your world is shattered.   I shared my lessons learned from Peaceful Warrior with some colleagues and they suggested I post them.  Here it goes ...

    Where Are You, What Time is It, What Are You
    This theme echoed throughout the movie:

    • Where are you? .... HERE.
    • What time is it? ... NOW.
    • What are you? ... THIS MOMENT.

    Favorite Lessons

    • There are no ordinary moments
    • A Warrior acts; only a fool reacts.
    • What do you do if you can’t do what you were born to do?  Everything has a purpose -- it’s up to you to find it.
    • Make every move about the move -- that one moment in time.
    • Don't fall into the trap -- If only I had this, I'd be ...  If only ... , I'd get to be happy. 
    • If you don't get what you want, you suffer.  If you get what you want, you still suffer.
    • You control you.  Master you.
    • Don't pin your success on outcomes.
    • The people that are the hardest to love, probably need it the most.
    • Take out the trash.  Clear you mind of everything you don't need (doubt, past failures, future victories, ... etc.)
    • Develop the wisdom to use the right leverage at the right time.
    • Knowledge is not the same as wisdom.  Knowledge is knowing -- wisdom is doing.
    • You'll never be better than anybody, the same way you'll never be any less.

    Additional Lessons Learned 
    Here’s my lessons from peaceful warrior:

    • Experience the moment.  Be fully engaged in the moment.
    • Limiting beliefs.  Don't become a victim of your own limiting beliefs.  Stay adaptable.
    • Don't miss out on what's going on by letting your mind fill up with noise.
    • Trash is anything that keeps you from the only thing that matters which is right here and now.
    • Be right here, right now.  Don't be in the past, gloating.  Be in the present living.
    • All you have is right now.
    • Let got of attachments.
    • Meditate.  Be able to clear you mind and focus on the moment.
    • Don't let your emotions control you.
    • Enjoy the here and now -- that's the secret.
    • Savor the moment.
    • Other people's perspectives don't matter as much if you have your own perspective.
    • Find your purpose.
    • Assign meaning.  You're the most important meaning maker in your life.
    • There's never nothing going on (take a look around.)
    • Don't give up the one thing you control -- your response.
    • The habit is the problem.  Wake up and experience the moment.
    • Be conscious of your choices.
    • Be responsible for your actions.
    • Every action has its price and it's pleasure.
    • Most people don't live at all.
    • When you feel fear, use this sword.
    • Don't take for granted what you can do; don't be sloppy with your life.
    • Devote life to a higher purpose; service to others.
    • The warrior doesn't give up what they love.
    • The warrior finds the love in what they do.
    • The warrior is not about invincibility.
    • The warrior chooses to act; the coward reacts.
    • Life is about 20 seconds in front of the judges.
    • A warrior does what they love.  A warrior does the thing they are put here to do.
    • Don't live in fear of failure
    • It's the journey, not the destination.
    • Warrior is not about perfection.  Warrior is about absolute vulnerability.
    • Life is about choice.
    • Actions speak for themselves.
  • J.D. Meier's Blog

    Software Methodologies at a Glance


    I like to draw from a variety of sources for software engineering principles, patterns, and practices.  To be able to do this, I usually need to create information models that let me quickly scan bodies of knowledge.  Once I can frame out a space, it's a lot easier to drill into areas looking for gold.  I can also step back and see across approaches and this helps me see underlying principles or key distinctions between approaches.

    XP, MSF Agile, RUP, and Microsoft Solution Framework
    To compare process methodologies, here are some skeletal information models I've used:

    While the frames aren't entirely consistent, I can still quickly scan the methodologies and get a good sense of what the key ideas, activities, artifacts, and practices are.

    My Related Posts

  • J.D. Meier's Blog

    10 Success Patterns for PMs


    Here's a brief set of success patterns I've shared with a few colleagues.  These are the patterns I see that make a difference in getting results.

    10 Success Patterns

    1. Empathic listening.
    2. Rapport before influence
    3. Character trumps emotion trumps logic
    4. Match their style
    5. Ask WIIFY
    6. Distinguish between responsibility and authority
    7. Turn chickens into pigs
    8. Adapt, adjust, or avoid situations
    9. Know the system.
    10. Analyze it over time.

    Success Patterns Explained
    Here's the essence of each:

    • Empathic listening.  Listen until the other person "feels" they've been heard.  Once they feel heard, they're more likely to listen to you.  You can do this 1:1 or in a large meeting.  Covey uses an "Indian Talking Stick."  The person with the stick talks until they feel heard.  A former Softie told me his team used an eraser as "the mutex."   See Stephen Covey Speaks at Microsoft.
    • Rapport before influence.  This is true whether it’s a presentation, interview … etc.. For example, go to a comedy club and see how the comedian gets the crowd laughing only  after they have rapport.  See How Might That Be True?
    • Character trumps emotion trumps logic.  If you base all your arguments on logic, but fail to persuade, now you know.  See Win the Heart, the Mind Follows.
    • Match their style.  You don't have to go overboard, but a little bridge can go along way.  If somebody is visual, could you whiteboard it for them?  If somebody's detail oriented, can you provide the details?  If somebody needs to hear action, can you turn your ideas into action?
    • Ask WIIFY.  Ask the question What's In It For You?  If you're a marketer, this might come natural for you.  If you're an engineer, this might feel weird.  It's about shifting the focus from the thing to the person. If nobody shows up to your meetings, tailor the invite to be explicit about what's in it for the attendees.
    • Distinguish between responsibility and authority.  Know whether you influence a decision or own it.  When you don't have authority, but you need to get results, leverage the model in Influencing Without Authority.
    • Turn chickens into pigs.  A pig's committed while a chicken's involved.  Don't let a chicken have a controlling vote, without turning them into a pig.  See Turning Chickens into Pigs.
    • Adapt, adjust, or avoid situations.  Learn how to read situations. Some situations you should just avoid.  Some situations you should adapt yourself, as long as you play to your strengths.  Some situations you should adjust the situation to set yourself up for success.
    • Know the system.   Analyze the problem from a system standpoint.  What are the components and subsystems?  What are the inputs and outputs?  Who are the players?   What levers can you pull that make the most impact?  If you don't know, who does?
    • Analyze it over time.  Look at the problem or solution over time. Build your temporal skills.  The more you play "what ifs" in the future, the easier it gets to anticipate.

    Do you have any favorite success patterns to share?

    My Related Posts

  • J.D. Meier's Blog

    Lessons Learned from Per


    I like to learn from everyone around me.  One of my most influential mentors has been my manager, Per.  Here’s a highlight of some of the lessons I learned from Per over the years:


    • Consolidate communication.   Chunky over chatty.  Rather than a random stream of ideas, consolidate down to a theme.  Rather than a bunch of mini-mails, consolidate to a more meaningful message. 
    • Distinguish between phases of communication.  Walt Disney used three stages for ideas: Dreamer, Realist, and Critic.  Dreamer is the vision, Realist is the steps, Critic is the constraints and limits.  In practice, Per encourages first brainstorm, then critique.

    Cutting Questions

    • Do you influence or do you own?   Distinguish between when you own the decision versus when you influence.  If you influence, but don’t own, reset your own expectations.  Responsibility without authority is a common pitfall. 
    • Does it matter?   In the grand scheme of things, does this particular issue matter.
    • Five customers under your belt?  If you don’t have five customers you can point to that stand behind what you’re doing, it’s a flag.  As simple as this test sounds, I’ve used it many times to ensure project success.
    • Is it the right thing to do?  First figure out the right thing to do, then figure out what you can do based on the circumstances.
    • Is it working?  If you keep taking the same approach, but it’s not working, you need to change your approach.  It’s simple, but you’d be surprised how many people get stuck.
    • Next steps?   Asking “next steps?” at the end of your meeting is a pretty crisp way of turning talk into action.
      What’s the agreement?  When you’re not sure why things aren’t happening as you expect, check the agreement.  You might find you don’t really have one.
    • What’s the solution?  If you’re spending too much time on the problem, switch by asking, “what’s the solution?”
    • What’s their story?  If you don’t get why another person or team isn’t doing what you want, figure out what their side of the story is.  (see What’s Their Story)
      What’s your gut say?  If you find a decision isn’t sitting right, check your gut.   While the numbers may say one thing, your intuition may be telling you a deeper story.


    • Argue the data.  Rather than argue your opinion, argue the data.  Arguing the data keeps it more objective.  It can also be more convincing.
    • Don’t state the conclusion.  Rather than state your conclusions, show the data that leads to your conclusions.  Even if you know the “right answer,” showing the data can help others get on board.
    • Lead the horse to water.   Let them connect the dots.  You can draw a dotted line, but let your audience connect the dots. 
      Mental Judo.  This is Per’s phrase for Ward Cunninham’s uncanny ability to lead people to a conclusion through rhetoric and stories.
    • Set the frame.   Frame the problem or solution so that you have a common backdrop to share understanding.  This will help set boundaries, create agreements, and improve communication.
    • They’re not engaged.  If you notice a lack of engagement throughout your meeting, there’s a good chance that you don’t have rapport.  If you don’t have rapport, you won’t effectively influence.
    • They’re not on board.  You should be able to tell whether somebody is on board.  If they aren’t, then don’t kid yourself.
      You didn’t agree on values.  Recognize when you don’t have the same values.  Do they value time or do they value a level of results?  If you don’t share the values, you can expect problems in agreements, prioritization, expectations, and results.
    • You didn’t have buy in.  Just because somebody shakes their head yes, doesn’t mean they are bought in.  You don’t want somebody to tell you yes, just to drag their feet.

    Organizational Prowess

    • Call it an experiment.  In a risk adverse environment, it can help to call a project an experiment.  It’s a simple but effective metaphor to help people think about trying something new.
    • Do you influence or do you own?  If you don’t own the decision, you don’t own it.  Period.  You can attempt to influence, but at the end of the day, you’re not the owner.   That doesn’t mean don’t try your best.  Just don’t be surprised if it’s not the decision you wanted and don’t take it personally.
    • Expose the approach.   People may not agree on the implementation or the results, but sharing the approach is a good place to start.  Exposing the approach gives you a chance to improve the approach.  Exposing the approach can also build trust, particularly if everybody buys in.
    • Expose the thinking.  This is similar to showing your homework.  If you expose the thinking, then you can improve the thinking.  You can also help build trust.  You may not get to the right answer, but showing how you got there helps bring people along with you.
    • Get consensus.  Where possible and time permits, get folks on board.  If there’s resistance, this can be a flag that you’re off in the wrong direction.  If it’s a decision where you need everybody’s skin in the game, then you’ll want them on board helping push the ball forward versus dragging their feet.
    • Get smart folks on the bus.  This is just a general belief that if you have smart people around, the right things will happen.  It’s about people over process.
    • Slides are a forcing function.    Slides are a great way to get folks on the same page, or at least start the dialogue.  If you can’t show your ideas in a slide, then you don’t have a well-formed message for others.  Doing slides helps create a strawman of the thoughts that can be reviewed and improved.  Slides can help turn thoughts into action.


    • Change your approach.  If it’s not working; change your approach.
    • Change yourself first.  You can change yourself faster than you can change others.  Seriously.
    • Chunk it down.   If you can’t release something in a healthy rhythm, chunk it down.
    • Divide and Conquer.  If you have an overwhelming challenge, divide and conquer.(See Divide and Conquer - One Step at a Time)
    • Drive or Be Driven.  If you don’t drive it, you’ll be driven by somebody else.
      Get it out and get feedback.  Good ideas often die because there’s no action or momentum.  If you get something out, you can get feedback and start to improve it.
    • Give attribution.  In our industry, acknowledgement is important.  Be sure to give attribution where it’s due.
    • Focus.   If there’s one place where projects fail or people fail, it’s a lack of focus.  Focus is your friend.  If you’re not getting results, narrow the focus. 
    • Follow your passion.  Where there’s passion, there’s strength.
    • Forcing functions.  Use meetings, slides, and reviews as forcing functions to drive results.
    • Model the best.   This is Per’s phrase for my approach of finding the best people and modeling their success. 
    • Play to your strengths.  Per’s a fan of the book, Good to Great, and he believes in focusing on your strengths rather than your weaknesses.
    • Rattle the cage.  Challenge the thinking.  Challenge ideas.  Challenge yourself.  Don’t fall into the trap of the status quo.
    • Ship it!  This is one of Per’s favorite sayings which characterizes the emphasis on getting results and making impact.   He’s learned that a rhythm of shipping produces results while failure to ship is how teams fail at Microsoft. 

    My Related Posts

  • J.D. Meier's Blog

    Project Life Cycles at patterns & practices


    Periodically I like to revisit our project life cycle in patterns & practices. I like to see how it's shape-shifted over the years.  (Note - our project life cycle wraps our product cycle)  

    patterns & practices Project Life Cycle Circa 2005
    Here's a snapshot of our patterns & practices project life cycle circa 2005:


    I used this as a baseline to reflect against.  Here are the phases, stages, and milestones:

    Projects cycled through the following phases:

    • Planning
    • Design
    • Implementation
    • Stabilization
    • Release

    Stages included:

    • Requirements
    • Specifications
    • Iteration 1
    • Iteration N
    • Final Test and Edit Pass
    • Production

    The milestones included:

    • Proposal Approved
    • Vision Scope Approved
    • M0 (Milestone Zero) / Specifications Approved
    • Technical Review and Solution Approved
    • Test and Edit Complete
    • Go / No Go
    • Customer Availability

    Three Things That Worked Well
    Here's three things that worked well with the original project cycle:

    • There were clear phases, stages, milestones, and deliverables, along with criteria.
    • The project cycle was decoupled from the product cycle.  This gave management a simple frame for understanding projects.  This also gave each project flexibility to choose the most appropriate software development methodology depending on the product.
    • There was sufficient time between key milestones to provide a frame + air-cover.  This helped avoid randomizing engineering and being able to see the forest from the trees.

    Additionally, the key milestones such as Vision Scope and MO were something of a ceremony and tended to include the right representation across the p&p team.

    Three Things That Needed Improvement
    Here's three things that needed improvement:

    • It was a lot of overhead for smaller projects.  It worked well for larger programs (collections of projects), but it was tougher for individual projects.
    • It was tough to bootstrap projects.  M0 and Vision/Scope could be especially tough.  In retrospect, there were two key issues: 1) asking the right questions at the wrong time (premature) 2) chickens with controlling votes over pigs. (See Turning Chickens Into Pigs.)
    • There was too much agreement up front, with not enough ability to coarse correct in the later stages/phases (needed more agility)
  • J.D. Meier's Blog

    Model-Driven Approaches


    When people ask me my take on model-driven approaches, I think of two ends of the spectrum -- human and the machine.

    Key Points 

    • Model for humans.  For humans, I find using a whiteboard (whiteboard modeling) works well -- it's universal.
    • Model for machines.  For machines, I find speaking closer to the code is a good thing and design is rarely a clean model.
    • Model to learn.  I've found modeling more useful when you throw it away -- it's a learning tool.  Model so you get it, then go to it.  I think the key is that the model is a map, not the actual territory.  The usefulness of a map is actually decoupling from the complexity and creating a simpler lens.
    • Model to share.  I think the real value of models is when you create a way to share the problem or solution in a simplified way.  This helps for sharing a vision of the end in mind, as well as getting more sustained thinking around the problem.

    Model-Driven Code
    I've never experienced an effective modeling approach that turns visuals of systems into code, where the model doesn't get in the way.  At some point, the model stops being useful for humans or stops being useful to the machine.  As a result, I've never really been a fan of model-driven approaches that are coupled to code in practice, although they're always interesting in theory.   While I'm open to the idea, I just haven't seen it.  Am I missing out?

    Effective Modeling for Shaping Software
    While I'm not a fan of most visual modeling tools, there’s some very real modeling approaches I find to be effective (which is more about modeling for the humans to understand what matters.)  I find that light-weight, human-oriented models are particularly effective for shaping software around quality attributes.  For example:

    My Related Posts

  • J.D. Meier's Blog

    Agile Guidance


    When I ramp new folks on the team, I find it helpful to whiteboard how I build prescriptive guidance.  Here's a rough picture of the process:


    I've used the same process for Performance Testing Guidance, Team Development with Visual Studio Team Foundation Server, and WCF Security.

    Here's a brief explanation of what happens along the way:

    The dominant focus here is identifying candidate problems, candidate solutions, and figuring out key risks, as well as testing paths to explore.  The best outcome is a set of scenarios we can execute against.

    • Research - finding the right people, the right problems, and the right solutions.
    • Prototypes - experiment and test areas of high risk to prove the path.  This can include innovating on how we build prescriptive guidance.  We also use these to test with customers and get feedback on the approach.
    • Question Lists - building organized lists of one-liner user questions.
    • Task Lists - building organized lists of one-liner user tasks.
    • Scenario Frames - organizing scenarios into meaningful buckets.  See Scenario Frame Example.
    • Information Models - framing out the problem space and creating useful ways to organize, share, and act on the information.  See Web Services Security Frame.
    • Guidance Types  - testing which guidance types to use (how tos, checklists, guidelines, patterns, ... etc.)

    The dominant focus here is product results.  It's scenario-driven.  Each week we pick scenarios to execute against.

    • Development - building prescriptive guidance, including coding, testing, and writing.
    • Backlog - our backlog is a prioritized set of scenarios and guidance modules.
    • Iterations - picking sets of scenarios to focus development on and test against.
    • Refactoring - tuning and pruning the guidance to improve effectiveness.  This includes breaking the content up and rewriting it.  For example, a common refactoring is factoring reference information from action.  We try to keep reference information in our Explained modules and action information in our How Tos.
    • Testing -  step through the guidance against the scenario.  The first pass is about making sure it works.  It should be executable by a human.  Next, it's about reducing friction and making sure the guidance really hits the what, why and how.  We test the guidance against objectives and scenarios with acceptance criteria so we know when we're done.
    • Problem Repros - creating step by step examples that reproduce a given problem.
    • Solution Repros - creating step by step examples that reproduce a given solution.

    We produce a Knowledge Base (KB) of guidance modules and a guide.  The guidance modules are modular and can be reused.   The guide includes chapters in addition to the guidance modules.  Here's examples from our WCF Security Guide:

    Agile Publishing
    We release our guidance modules along the way to test reactions, get feedback and reshape the approach as needed.

    • CodePlex - we publish our guidance modules to CodePlex so we can share early versions of the guidance and get customer feedback, as well as to test the structure of the guidance, and experiment with user experience.
    • Guidance Explorer - we publish guidance modules to Guidance Explorer so users can do their own guidance mash ups and build their own personalized guides.  Our field also uses this to build customized sets of guidance for customers.

    Stable Reference
    Once we've tested and vetted the guidance and have made it through a few rounds of customer feedback, we push the guidance to MSDN.

    • MSDN - this is the trusted site that users expect to see our prescriptive guidance in final form.
    • Visual Studio/ Visual Studio Team System - once we're a part of the MSDN distribution, we can automatically take advantage of the VS/VSTS documentation integration.

    My Related Posts

  • J.D. Meier's Blog

    Software Guidance Share


    I'm testing another version of the home page on Software Guidance Share.  Software Guidance Share is a perpetual work in progress.  I think of it as my quick-and-dirty guidance KB for developers and solution architects.  I continuously refactor information into reusable nuggets.  I also test ways to browse the guidance and find relationships among the nuggets.

    Here's a couple of example scenarios:

    I haven't fleshed out some of the areas, but the Wiki gives me a lot of flexibility and it's easy to course-correct.  In other words, it's more adaptable than adapted.

  • J.D. Meier's Blog

    Designing an Authentication and Authorization Strategy


    What are the key steps to designing an effective authentication and authorization strategy?  The keys are knowing your user stores, role stores, and who need to access what or perform which operations.   In this post, I share the approaches we've used in two of our patterns & practices guides.  These are the approaches we've used to help customers design successfully design their authentication and authorization approaches.

    Designing an Authentication and Authorization Strategy - v1
    When we first wrote Building Secure ASP.NET Applications, here's the meta-process we came up with for working through your authentication and authorization strategies:

    1. Identify resources
    2. Choose an authorization strategy
    3. Choose the identities used for resource access
    4. Consider identity flow
    5. Choose an authentication approach
    6. Decide how to flow identity

    For elaboration, see Authentication and Authorization.

    Designing an Authentication and Authorization Strategy - v2
    When we recently wrote Improving Web Application Security, we made some revisions:

    1. Identify your user stores.
    2. Identify your role stores.
    3. Identify resources you need to access and operations you need to perform.
    4. Identify which identities need to access the resources and perform the operations.
    5. Choose your authentication and authorization strategies.

    Personally, I've found it really cuts to the chase if you start with your user stores and role stores, since they tend to be somewhat fixed. 

    When you think through the identities, I've found it helpful to think in terms of who needs to access which resources or perform which actions.  Consider the following:

    • Original caller
    • Process identity
    • Service account
    • Custom identity
    • Role

    Resource Types
    When you think through the resource types, I find it helpful to think in terms of:

    • System
    • Application
    • User

    Authorization Strategies
    When thinking through the authorization strategies, I find it helpful to consider:

    • Role-based
    • Resource-based
    • Operation-based

    Resource Access Patterns
    When thinking through the resource access patterns, I find it helpful to consider:

    • Trusted subsystem model
    • Impersonation/delegation model

    Designing authentication and authorization can be a gnarly topic.  I hope the scaffolding above helps you find a path that works for you.

  • J.D. Meier's Blog

    WCF Security Guide is Now Available in HTML


    Our guide, patterns & practices Improving Web Services Security:Scenarios and Implementation Guidance for WCF is now available in HTML.

  • J.D. Meier's Blog

    New Release: patterns & practices WCF Security Guide (BETA)


    Today we released our WCF Security guide, patterns & practices Improving Web Services Security: Scenarios and Implementation Guidance for WCF.  This is our Microsoft playbook for Windows Communication Foundation (WCF - "Indigo".)  It shows you how to build secure Web services using WCF.  It's a compendium of proven practices, product team recommendations and insights from the field.

    Download the guide

    Contents at a Glance

    • Part I, "Security Fundamentals for Web Services"
    • Part II, "Fundamentals of WCF Security"
    • Part III, "Intranet Application Scenarios"
    • Part IV, "Internet Application Scenarios"


    • Ch 01 - Security Fundamentals for Web Services
    • Ch 02 - Threats and Countermeasures for Web Services
    • Ch 03 - Security Design Guidelines for Web Services
    • Ch 04 - WCF Security Fundamentals
    • Ch 05 - Authentication, Authorization and Identities in WCF
    • Ch 06 - Impersonation and Delegation in WCF
    • Ch 07 - Message and Transport Security in WCF
    • Ch 08 - WCF Bindings Fundamentals
    • Ch 09 - Intranet – Web to Remote WCF Using Transport Security (Original Caller, TCP)
    • Ch 10 - Intranet – Web to Remote WCF Using Transport Security (Trusted Subsystem,HTTP)
    • Ch 11 - Intranet – Web to Remote WCF Using Transport Security (Trusted Subsystem TCP)
    • Ch 12 - Intranet – Windows Forms to Remote WCF Using Transport Security (Original Caller, TCP)
    • Ch 13 - Internet – WCF and ASMX Client to Remote WCF Using Transport Security (Trusted Subsystem, HTTP)
    • Ch 14 - Internet – Web to Remote WCF Using Transport Security (Trusted Subsystem, TCP)
    • Ch 15 - Internet – Windows Forms Client to Remote WCF Using Message Security (Original Caller, HTTP)


    • WCF Security Checklist
    • WCF Security Guidelines
    • WCF Security Practices at a Glance
    • WCF Questions and Answers (Q&A)
    • How Tos
    • WCF Security Resources

    Contributors and Reviewers

    • External: Andy Eunson; Anil John; Anu Rajendra; Brandon Bohling; Chaitanya Bijwe; Daniel Root; David P. Romig, Sr.; Dennis Rea; Kevin Lam; Michele Bustamante; Parameswaran Vaideeswaran; Rockford Lotka; Rudolph Araujo; Santosh Bejugam
    • Microsoft: Alik Levin; Brandon Blazer; Brent Schmaltz; Curt Smith; David Bradley; Dmitri Ossipov; Don Smith; Jan Alexander; Jason Hogg; Jason Pang; John Steer; Marc Goodner; Mark Fussell; Martin Gudgin; Martin Petersen-Frey; Mike de Libero; Mohammad Al-Sabt; Nobuyuki Akama; Ralph Squillace; Richard Lewis; Rick Saling; Rohit Sharma; Scott Mason; Sidd Shenoy; Sidney Higa; Stuart Kwan; Suwat Chitphakdibodin; T.R. Vishwanath; Todd Kutzke; Todd West; Vijay Gajjala; Vittorio Bertocci; Wenlong Dong; Yann Christensen; Yavor Georgiev
  • J.D. Meier's Blog

    Web Services Security Frame


    The key to making principles, patterns, and practices more effective is to have an organizing frame.  While working on our patterns & practices WCF Security Guidance Project, we created the Web Services Security Frame for just such a purpose.  We use the frame throughout the guidance to organize threats, attacks, vulnerabilities and countermeasures, as well as to organize principles, patterns, and practices. 

    Web Services Security Frame

    Here's a snapshot of the frame (the power of the frame is that it's a durable, evolvable backdrop -- in other words, you can shape it to your own purposes.)  You'll see this frame used throughout our upcoming guide.  Notice that the categories serve as a pivot that we can hang other viewpoints (threats/attacks, vulnerabilities, countermeasures.)

    Category Description
    Auditing and Logging Auditing and logging refers to how security-related events are recorded, monitored, and audited.
    Authentication Authentication is the process where an entity proves the identity of another entity, typically through credentials, such as a user name and password.
    Authorization Authorization is how your service provides access controls for resources and operations.
    Configuration Management Configuration management refers to how your service handles database connections, administration and other configuration settings.
    Exception Management Exception management refers to how you handle exceptions within your application, including fault contracts.
    Impersonation/Delegation Impersonation and delegation refers to how your service impersonates users and passes identity information downstream for authorization purposes.
    Message Encryption Message encryption refers to protecting a message by converting the contents to cipher-text using cryptographic methods.
    Message Replay Detection Message replay detection refers to identifying and rejecting messages that are re-submitted.
    Message Signing Message signing refers to signing a message with a digital signature using cryptographic methods, to confirm the source of the message and detect if the contents have been tampered with (i.e. authentication and integrity of the message.)
    Message Validation Message validation refers to how you verify the message payload against schema, as well as message size, content and character sets. This includes how your service filters, scrubs and rejects input and output before additional processing. Input and output includes input from clients consuming the service as well as file-system input, as well as input from network resources, such as databases. Output typically includes the return values from your service or disk / database writes among others.
    Sensitive Data Sensitive data includes data integrity and confidentiality of your user and application data that you need to protect. This includes how you protect sensitive data from being stolen from memory, from configuration files or when transmitted over the network.
    Session Management A session refers to a series of related interactions between a client and your service.


    Threats / Attacks Organized By the Web Services Security Frame


    Category Threats / Attacks
    Auditing and Logging
  • Repudiation
  • Denial of services
  • Disclosure of confidential information
  • Authentication
  • Network eavesdropping
  • Brute force attacks
  • Dictionary attacks
  • Cookie replay attack
  • Credential theft
  • Authorization
  • Elevation of privilege
  • Disclosure of confidential data
  • Data tampering
  • Luring attacks
  • Token stealing
  • Configuration Management
  • Unauthorized access to configuration stores
  • Retrieval of clear text configuration secrets
  • Exception Management
  • Information disclosure
  • Denial of service
  • Elevation of privilege
  • Impersonation/Delegation
  • Elevation of privilege
  • Disclosure of confidential information
  • Message Encryption
  • Stealing sensitive data.
  • Theft of encryption keys.
  • Man in the middle attack.
  • Message Replay Detection
  • Session replay
  • Message Singing
  • Data tampering.
  • Message Validation
  • XPath injection
  • XML Bombs
  • Canonicalization issues
  • Cross-site scripting
  • SQL injection
  • Sensitive Data
  • Memory dumping
  • Network eavesdropping
  • Configuration file sniffing
  • Session Management
  • Session hijacking
  • Session replay
  • Man in the middle attack
  • Inability to logout successfully
  • Cross-site request forgery
  • Session fixation
  • Load balancing and session affinity

    Vulnerabilities Organized by the Web Services Security Frame


    Category Vulnerabilities
    Auditing and Logging
  • Failing to audit failed logons
  • Failing to secure log files
  • Storing sensitive information in log files Failing to audit across application tiers Failure to throttle log files
  • Authentication
  • Using weak passwords
  • Storing clear text credentials in configuration files
  • Passing clear text credentials over the network
  • Permitting prolonged session lifetime
  • Mixing personalization with authentication
  • Using weak authentication mechanisms (For example, using basic authentication over an untrusted network.)
  • Authorization
  • Relying on a single gatekeeper (e.g. relying on client-side validation only)
  • Failing to lock down system resources against application identities
  • Failing to limit database access to specified stored procedures
  • Using inadequate separation of privileges
  • Permitting over-privileged accounts
  • Configuration Management
  • Using insecure custom administration interfaces
  • Failing to secure configuration files on the server
  • Storing sensitive information in the clear text
  • Having too many administrators Using over-privileged process accounts and service accounts
  • Exception Management
  • Failing to use structured exception handling (try/catch)
  • Revealing too much information to the client
  • Failure to specify fault contracts with the client
  • Failure to use a global exception handler
  • Impersonation / Delegation
  • Failure to revert to a lower privilege after using impersonation
  • Improper use of global impersonation across the entire service
  • Message Encryption
  • Failure to encrypt messages
  • Using custom cryptography
  • Distributing keys insecurely Managing or storing keys insecurely
  • Message Replay Detection
  • Failure to implement message replay detection feature
  • Message Signing
  • Unsigned messages that don't confirm the source
  • Unsigned messages that don't detect tampering
  • Message Validation
  • Using non-validated input used to generate SQL queries
  • Relying only on client-side validation Using input file names, URLs, or user names for security decisions
  • Using application-only filters for malicious input Looking for known bad patterns of input
  • Trusting data read from databases, file shares, and other network resources
  • Failing to validate input from all sources including cookies, SOAP headers, SOAP parameters, databases, and network resources
  • Session Management
  • Passing session identifiers over unencrypted channels
  • Permitting prolonged session lifetime Having insecure session state stores
  • Placing session identifiers in query strings

    Countermeasures Organized by the Web Services Security Frame


    Category Countermeasures
    Auditing and Logging
  • Identify malicious behavior.
  • Know your baseline (know what good traffic looks like)
  • Use application instrumentation to expose behavior that can be monitored
  • Throttle logging
  • Strip sensitive data before logging
  • Authentication
  • Use strong password policies
  • Do not store credentials in an insecure manner
  • Use authentication mechanisms that do not require clear text credentials to be passed over the network
  • Encrypt communication channels to secure authentication tokens
  • Use HTTPS only with forms authentication cookies
  • Separate anonymous from authenticated pages
  • Using cryptographic random number generators to generate session IDs
  • Authorization
  • Use least privilege accounts.
  • Authentication tied to authorization on the same tier
  • Consider granularity of access
  • Enforce separation of privileges
  • Use multiple gatekeepers
  • Secure system resources against system identities
  • Configuration Management
  • Use ACLs.
  • Encrypt sensitive sections of configuration files
  • Use secure settings for various operations of web services using configuration files
  • Exception Management
  • Use structured exception handling (by using try/catch blocks)
  • Catch and wrap exceptions only if the operation adds value/information
  • Do not reveal sensitive system or application information
  • Implement a global exception handler
  • Do not log private data such as passwords
  • Impersonation / Delegation
  • Use Using statement to automatically revert impersonation
  • Granularly impersonate only those operations that need it
  • Message Encryption
  • Use message security or transport security to encrypt your messages
  • Use platform-provided cryptography
  • Use platform features for key management
  • Periodically change your keys
  • Message Replay Detection
  • Cache an identifier for incoming messages, and use message replay detection to identify and reject messages that match an entry in the replay detection cache
  • Message Signing
  • verify messages have not been tampered with in transit (data integrity)
  • verify messages originate from the expected sender (authenticity)
  • Message Validation
  • verify the message payload against schema
  • verify the message message size, content and character sets
  • filter, scrub and reject input and output before additional processing
  • Sensitive Data
  • Do not store secrets in software
  • Encrypt sensitive data over the network
  • Secure the channel
  • Encrypt sensitive data in configuration files
  • Session Management
  • Partition site by anonymous, identified, and authenticated users
  • Reduce session timeouts
  • Avoid storing sensitive data in session stores
  • Secure the channel to the session store
  • Authenticate and authorize access to the session store
  • Thanks
    Special thanks to Rudy Araujo and ACE Team members, Richard Lewis and John Steer for their contribution toward helping shape a better frame.

    My Related Posts

  • J.D. Meier's Blog

    WCF Security Resources


    If you're building Web services or if you're implementing SOA on the Microsoft platform , then you're probably either working with or exploring WCF (Windows Communication Foundation.)   When we started our patterns & practices WCF Security Guidance project, one of the first things I did was compile a list of WCF security resources for our team.  This helped us quickly ramp up and as well as see gaps.  One thing that surprised me is how much is available in the product documentation, if you know where to look.  Here's a preliminary look at our WCF Security resources index which we'll include in our WCF Security Guide: 

    Getting Started













    Documentation (MSDN Product Documentation)




    Threats and Countermeasures


    How Tos






    patterns & practices

    Product Support Services (PSS)





    Web Casts

    MSDN Support WebCasts

  • J.D. Meier's Blog

    patterns & practices WCF Security Practices at a Glance Now Available


    For this week's release in our patterns & practices WCF Security Guidance project, we released our first version of our WCF Security Practices at a Glance.  Practices At a Glance gives you a bird's-eye view of how to perform common tasks.  They are scannable and outcome-driven so that you can quickly browse the problem/solution pairs.  Rather than a laundry list of granular tasks, we organize them by our Web Services Security frame (still evolving.)

    Here's how we grouped our WCF Security Practices at a Glance so far:

    • Auditing and Logging
    • Authentication
    • Authorization
    • Configuration Management
    • Deployment Considerations
    • Exception Management
    • Hosting
    • Impersonation/Delegation
    • Input Validation
    • Message Security
    • Proxy Considerations
    • Sensitive Data
    • Transport Security

    Here's a snapshot of the problems solved from our Practices At a Glance, but you can see our answers explained at our WCF Security Guidance project site.

    Auditing and Logging

    • How to audit authentication events
    • How to audit authorization events
    • How to enable WCF message logging
    • How to enable WCF tracing
    • How to use Health Monitoring in WCF
    • How to view log information
    • How to view trace information
    • How to log traces to a WMI provider
    • How to turn off audit failure suppression


    • How to authenticate users against the SQL Membership Provider
    • How to authenticate users against Active Directory
    • How to authenticate users against Active Directory without windows authentication
    • How to authenticate users with certificates
    • How to map certificates with windows accounts
    • How to authenticate users against a custom user store
    • How to authenticate users with Kerberos direct to support non-WCF clients with windows authentication


    • How to authorize imperatively
    • How to authorize declaratively
    • How to authorize users against Windows groups
    • How to authorize users against Windows groups using the AspNetWindowsTokenRoleProvider
    • How to authorize users against the SQL Role Provider
    • How to authorize users against the ASP.Net Role Provider
    • How to assign the current principal with IAuthorizationPolicy to allow authorization using custom authentication

    Configuration Management

    • How to encrypt sensitive data in your configuration files
    • How to run your service under a specific identity
    • How to create a service account for your WCF service
    • How to stop clients from referencing your service
    • How to protect against message replay attacks

    Deployment Considerations

    • How to configure certificates to enable SSL in IIS
    • How to map Windows accounts with certificates
    • How to create a Service Principle Name (SPN)
    • How to configure WCF for NATs and Firewalls
    • How to create an X.509 certificate

    Exception Management

    • How to shield exception information with fault contracts
    • How to create an error handler to log details of faults for auditing purposes
    • How to handle unhandled exceptions in downstream services
    • How to throw an exception with complex types or data contracts with a fault exception
    • How to handle unknown faults in a service
    • How to implement a data contract to propagate exception details for debugging purposes
    • How to implement fault contracts in call back functions


    • How to host WCF in IIS
    • How to host WCF in a Windows service
    • How to self-host WCF
    • How to configure a least-privilege account to host your service


    • How to choose between trusted subsystem and impersonation/delegation
    • How to impersonate the original caller when using Windows authentication
    • How to impersonate programmatically in WCF
    • How to impersonate declaratively in WCF
    • How to delegate the original caller to call backend services when using Windows authentication
    • How to impersonate the original caller without Windows authentication
    • How to impersonate the original caller using S4U Kerberos extensions.
    • How to delegate the original caller using S4U Kerberos extensions.
    • How to impersonate and delegate using LogonUser Windows API
    • How to flow the original caller from an ASP.NET client to WCF
    • How to control access to a remote resource based on the original callers identity.

    Input Validation

    • How to protect your service from malicious messages
    • How to protect your service from malicious input
    • How to protect your service from denial of service attacks
    • How to validate parameters with parameter inspectors
    • How to validate parameters with message inspectors using schemas
    • How to validate data contracts with message inspectors using schemas
    • How to validate message contracts with message inspectors using schemas
    • How to use regular expressions validate format, range and length in schemas
    • How to validate inbound messages on a service
    • How to validate outbound messages on a service
    • How to validate outbound messages on the client
    • How to validate inbound messages on the client
    • How to validate input parameters
    • How to validate output parameters

    Message Security

    • How to use message security
    • How to partially encrypt a message
    • How to use out-of-band credentials with message security

    Proxy Considerations

    • How to avoid proxy spoofing
    • How to expose service metadata for your clients
    • How to create a proxy to a service hosted in IIS that requires certificate authentication and transport security

    Sensitive Data

    • How to encrypt sensitive data in configuration files
    • How to protect sensitive data in memory
    • How to protect sensitive data on the network

    Transport Security

    • How to use transport security
    • How to use secure conversations in WCF

    X.509 Certificates

    • How to create a temporary X.509 certificate for transport security
    • How to create a temporary X.509 certificate for message security
    • How to create a temporary X.509 certificate for certificate authentication

    My Related Posts

  • Page 34 of 46 (1,140 items) «3233343536»