J.D. Meier's Blog

Software Engineering, Project Management, and Effectiveness

July, 2010

  • J.D. Meier's Blog

    What is a PM?

    • 2 Comments

    What is a PM (Program Manager)?  While the Program Manager role seems unique to Microsoft, in general, when you map it to other companies, it’s a product manager, or a project manager, or a combination of the two.  At Microsoft, there are various flavors of PMs (“design” PM, “project” PM, “process” PM, etc.) and the PM discipline can be very different in different groups.  I’ve also seen the PM title used as a general job title, in the absence of something more specific.

    At Microsoft it’s a role that means many things to many people.  In general though, when you meet a PM at Microsoft, you expect somebody who has vision, can drive a project to completion, can manage scope and resources, coordinate work across a team, bridge the customer, the business, and the technology, act as a customer champ, and influence without authority.  From a metaphor standpoint, they are often the hub to the spokes, they drive ideas to done, they take the ball and run with it, or find out who should run with the ball.  Some PMs are better at thought leadership, some are better at people leadership, and the best are great at both. 

    Here is a roundup of some of my favorite points that elaborate, clarify, and distill what a PM is, what a PM does, and how to be one.

    Attributes and Qualities of a PM
    Here is a list of key attributes from Steven Sinofsky’s post -- PM at Microsoft:

    • “Strong interest in what computing can do -- a great interest in technology and how to apply that to problems people have in life and work”
    • “Great communication skills -- you do a lot of writing, presenting, and convincing”
    • “Strong views on what is right, but very open to new ideas -- the best PMs are not necessarily those with the best ideas, but those that insure the best ideas get done”
    • “Selfless -- PM is about making sure the best work gets done by the team, not that your ideas get done. “
    • “Empathy -- As a PM you are the voice of the customer so you have to really understand their point of view and context “
    • “Entrepreneur -- as a PM you need to get out there and convince others of your ideas, so being able to is a good skill”

    Here is an example of PM qualities from Sean Lyndersay’s post -- Exchange team defines a Program Manager:

    • You are passionate about working directly with customers, able to clearly articulate customers requirements and pains to executives, architects and developers.
    • You have experience building strong teams and have a passion for mentoring.
    • The ability to work with multiple teams to develop a plan to provide this value to the customers.
    • You understand how software features will impact and/or modify the marketplace once they are shipped.
    • You have a love of innovation and the ability to think through big, long term ideas.
    • Demonstrated expertise at prioritization and project management.
    • You are hands-on with software development, and passionate about user experience, both as an end-user and administrator.

    Microsoft Careers site on Program Manager
    Here is a description of a Program Manager from the Microsoft Careers site:
    “As a Program Manager, you’ll drive the technical vision, design, and implementation of next-generation software solutions. You’ll transform the product vision into elegant designs that will ultimately turn into products used by Microsoft customers. Managing feature sets throughout the product lifecycle, you’ll have the chance to see your design through to completion. You’ll also work directly with other key team members including Software Development Engineers and Software Development Engineers in Test. Program Managers are advocates for end-users, so your passion for anticipating customer needs and creating outside-the-box solutions for them will really help you shine in this role. As a Program Manager you will have the ability to lead within a product’s life cycle using evangelism, empathy, and negotiation to define and deliver results. You will also be responsible for authoring technical specifications, including envisaged usage cases, customer scenarios, and prioritized requirements lists.” 

    Chris Pratley on Program Manager 
    Here are some points on Program Management from Chris Pratley’s post -- Program Manager:

    • “One way to describe PMs is that they not only "pick up and run with the ball, they go find the ball". That really defines the difference between "knowing what to do and doing it", and "not knowing what to do, but using your own wits to decide what to do, then doing it". That means as a PM you are constantly strategizing and rethinking what is going on to find out if there is something you are missing, or the team is missing. You’re also constantly deciding what is important to do, and whether action needs to be taken.”
    • "... In Office, there are "design" PMs who mainly work on designing the products, "process" PMs who mainly work on driving processes that make sure we get things done, localization PMs who are sort of like process PMs but also sort of like developers in that they sometimes produce code and make bits that ship in the product..."
    • “This ‘jack of all trades’ or ‘renaissance man’ … acted as a hub of communication, and made the marketers job easier, since they had someone to talk to who could take the time to really understand the outside world, and the devs could talk to someone who spoke their language and was not ridiculous …”
    • “You’re also constantly deciding what is important to do, and whether action needs to be taken. The number of such decisions is staggering. I might make 50 a day, sometimes more than 100 or even 200. Most of the time there is not nearly enough hard data to say for certain what to do, and in any case almost all these decisions could never have hard data anyway - you need to apply concentration and clear thinking.”

    Here are the stages of your first year as a PM, according to Pratley:

    1. Start off with excitement and enthusiasm for the new job.
    2. About 4 weeks into the job, you start to feel strange. People keep asking you to decide things you don’t know anything about, as if you’re some kind of expert.
    3. By month two, you're convinced you are the dumbest person on the team by far.
    4. By month four, you have lived through a torture of feeling incompetent and a dead weight on your team.
    5. By month six, you have a great moment.
    6. By month 12, you have developed your network of contacts that pass information to you, you are a subject matter expert on your area, and people on the team are relying on you because you know lots of things they don't know. You have made it.

    Joel Spolsky on Program Manager
    Here are points from Joel’s post on How To Be a Program Manager:

    • “Having a good program manager is one of the secret formulas to making really great software”
      “According to Joel, Charles Simonyi invented the job “Program Manager”, but Jabe Blumenthal, a programmer on the Mac Excel team in the late 80s, reinvented the job completely.”
    • “What does a Program Manager do?  “Design UIs, write functional specs, coordinate teams, serve as the customer advocate.”
    • “How To Be a Program Manger - Making Things Happen, by Scott Berkun, Don't Make Me Thin, by Steve Krug's, User Interface Design for Programmers, by Joel Spolsky, and How To Win Friends and Influence People by Dale Carnegie.“

    Ray Schraff on Program Manager
    Here are point from the comments on Chris Pratley’s post, Program Management:

    • “Once they find the ball, PMs don't pick up every ball themselves... but they own the task of making sure that every ball is picked up and carried to the correct endzone by SOMEBODY. “
    • “PMs translate technology to English “
    • “PMs translate English to technology”

    Sean Lyndersay on Program Manager

    • “My favorite definition involves an analogy: A program manager is to a software project what an architect is to a building.”  See Reflections Program Management at Microsoft.
    • “… the PM occupies a unique position in most software engineering structures – sort of the hub of a bumpy wheel (with dev, QA/test, design, usability, marketing, planning, customer support, etc. being the spokes).  See Someone has an Interesting View of PMs (at least at Yahoo!)”  See Someone has an Interesting View of PMs (at least at Yahoo)
    • "You are the center of the hurricane, the eye of the product development storm. You have passion and more, you have vigor. You are fueled by pure energy and endless drive. You are reading this and wondering why it's not in bullet-points because that would have been more efficient. You are working on a game that merges poker with chess in your spare time because neither game uses your full capabilities and talents. You have no use for extraneous clutter or mundane activities and you wonder what is holding up the full-scale integration of robots into the home – its 2007 already and doing the dishes remains as mundane and inefficient as it ever was. You are thinking that this job description is taking too long to read, and you are right. So, here is the rest of it in bullet points."  See Exchange team defines a Program Manager.
    • “With regard to where ideas come from, what we like to say is that the job of program management is not to have all the great ideas but to make sure all the great ideas are ultimately picked. The best program managers make sure the best ideas get done, no matter where they come from.”  See The Job of Program Management.

    Steven Sinofsky on Program Manager
    Here are points from Sinfoksy’s post on PM at Microsoft:

    • “Learn, convince, spec, refine”
    • “PM is one equal part of a whole system.”
    • “PM is a role that has a lot of responsibility but you also define it--working in partnership with expert designers, clever developers, super smart testers, etc. you all work together to define the product AND the responsibilities each of you have”
    • “… the PM role at Microsoft is both unique and one that has unlimited potential -- it is PMs that can bring together a team and rally them around a new idea and bring it to market (like OneNote, InfoPath).”
    • “It is PMs that can tackle a business problem and work with marketing and sales to really solve a big customer issue with unique and innovative software (like SharePoint Portal Server).”
    • “Where developers were focused on code, architecture, performance, and engineering, the PM would focus on the big picture of "what are we trying to do" and on the details of the user experience, the feature set, the way the product will get used.”
    • “A key benefit of program management is that we are far more agile because we have program management.  That can be counter-intuitive (even for many developers at Microsoft who might be waiting for their PM to iron out the spec).  But the idea that you can just start writing code without having a clear view of the details and specification is a recipe for a poorly architected features.”
    • “A great PM knows when the details are thought through enough to begin and a great developer knows when they can start coding even without the details for sure.  But like building a house--you don't start without the plans.  “
    • “A good book that describes the uniqueness of PM at Microsoft is Michael Cussumano's book Microsoft Secrets or his new book, The Business of Software.”

    My Related Posts

  • J.D. Meier's Blog

    Azure Canonical Application Architecture

    • 2 Comments

    While working with a customer last week at our Microsoft Executive Briefing Center (Microsoft EBC), one of the things we need to do was find a simple way to whiteboard the architecture.  We walked through a lot of variations of application patterns, but the customer ultimately wanted to see one code base running on Azure that could support multiple clients (such as desktop, phone, browser, etc.)   We found a model that was pretty straightforward and allowed us to easily map the customer’s scenario onto the Azure platform.

    Once we had a model, it was a lot easier to drill down on things such as why SQL Azure vs. Azure Storage or how to flow claims or how to partition the database.  It’s like a backdrop that you can overlay or drill down on security, performance, manageability, etc.  It helped us quickly explore paths without getting into weeds and keeping the big picture in mind.  It also helped us figure out where target some of the key architectural spikes.

    Here are some of the models we walked through on the whiteboard:

    Baseline

    image

    Azure Storage Example

    image

    SQL Azure Example

    image

  • J.D. Meier's Blog

    Application Types (App Types) – The Early Years

    • 1 Comments

    Several years back, I did an exercise in mapping out families of application architectures and application types.  It was an extensive archeological expedition.

    Key Goals / Outcomes
    There were several goals of the exercise:

    • Identify canonical application architectures and app types
    • Figure out a useful way of sharing architectures
    • Figure out whether it's better to focus on the business architectures or the technical architectures
    • Find a simple way to overlay patterns work on top of technical architectures
    • Find a simple way to share architectural styles
    • Find a way to simplify and share visuals of end-to-end architectures

    The exercise fed into a number of later works years later, including:

    Key Activities
    Some of the key activities of this early exercise included:

    • Creating an exhaustive catalog of application types grouped and sliced in various ways
    • Interviewing many of our best application architects in the trenches
    • Sanity checking with many of our top architects at head quarters, including Rico Mariani, Pat Helland, and Jim Gray
    • Reverse engineering many available sample applications inside and outside of Microsoft
    • Checking with industry experts on their lessons learned about the types and the shapes of all the applications they've seen and built over the years
    • Creating an extensive inventory of application patterns from various sets of our  key customers that had a wide range of applications and platforms
    • Evaluating competitive efforts to index and catalog application shapes and types, and making sense of various patterns efforts

    Keep in mind, that going into this, I already had the benefit of doing more than 650 customer architecture and design reviews -- yet still, this was an overwhelming exercise.  It forced me to find new ways to deal with large bodies of data and information, and somehow turn them into  shared maps, browsable, reusable knowledge nuggets, and backdrops for deeper conversations and elaboration.

    Lessons Learned from Architectural Exploration
    Some of the surprises for me or things that I learned that I didn't expect include:

    • Focusing on business architectures and verticals was less effective and less reusable than focusing on technical architectures as a baseline (from there, we could overlay business architectures)
    • Knowing the deployment patterns was the fastest way to get a good sense of the application patterns (it's where the logical met the physical against the infrastructure)
    • Looking through quality attributes as a lens made it easy to find and explore patterns for security, performance, reliability, manageability, etc.
    • So many applications boiled down to simple CRUD (Create Read Update and Delete)
    • The biggest variation among applications was the business logic and workflow
    • IBMs simple model of user, process, and data was useful as one lens
    • Views and viewpoints were a powerful way to change perspective and drill into another aspect of applications
    • At a high-level, you could capture a customer's core system by capturing the data, workflows, and workloads (the workloads significantly shaped the design of the app)

    In retrospect, the simplicity and common denominators of CRUD make a lot of sense.  It's all about interacting with information at a fundamental level.  People are just trying to get things done and there's only so many things you can do with information -- find it, browse, save it, share it, transform it, etc. as part of your workflow.

    Early Map of App Types
    I included one of the many early maps of the application types that helped us figure out what to throw out and what to keep as we moved forward.   One of the key distinctions that Ward Cunningham helped me figure out was to distinguish between the shape of the application architecture and design, and the actual “purpose” of the application.  Some purposes were more business-oriented, while some were more technically oriented, and this helped me find and distinguish different families of apps.

    It's circa 2004, but the irony is how timeless the backdrop really has been.  It’s rough and raw, but like I said, it’s just one sampling of the many braindumps behind the scenes of mapping out app types.

     

    Category Items
    Base Archetypes
    • Rich Client
    • Web Client
    • Web Service
    • Mobile / Phone
    • Video Game
    • Framework
    • Component/Library
    • Subsystem
    • Service
    Pool of Stuff
    • OLAP
    • Data Mining
    • Data Warehouses
    • Data Management
    • Business Process Integration
    • Business Intelligence
    • BAM
    • Server-side
    • Desktop
    • Peer-to-peer
    • Mobile
    • Web-centric
    • Data-centric
    • Middle-tier
    • Transactions
    • Read-Only data
    • Workflow
    • Batch
    • Portal
    • Community
    • Commerce
    • Task-Tracking
    Business Applications
    • Accounting and Finance
    • Customer Service
    • Distribution and Warehousing
    • Education and Training
    • Engineering, Design and Drafting
    • Facilities Management
    • General Office Automation
    • Groupware and Collaboration
    • Human Resources
    • Knowledge Management
    • Legacy Systems Analysis and Upgrade
    • Logistics and Procurement
    • Manufacturing and Process Management
    • Other Business Application Tools
    • Project Management
    • Reporting, Analysis and Decision Support
    • Sales and Marketing
    Accounting and Finance
    • Account Activity Analysis
    • Accounts Payable
    • Accounts Receivable
    • Billing and Invoicing
    • Budgeting, Financial Planning and Analysis
    • Consolidation Statements and Performance Reporting (CS-PR, Aggregation)
    • Currency and Foreign Exchange Management
    • Electronic Funds Transfer (EFT)
    • Fixed Asset Management
    • General Ledger
    • Integrated Accounting Solutions
    • Integrated Financial Management Solutions
    • Inventory Accounting
    • Investment and Portfolio Management
    • Job Costing
    • Payment, Clearing and Settlement Systems
    • Payroll and Personnel Accounting
    • Purchasing
    • Quotation, Order Entry and Order Processing
    • Risk Management
    • Small Business Accounting
    • Tax Preparation and Reporting
    • Time and Billing
    Customer Service
    • Automatic Page-Voice Notification
    • Call Center Management
    • Customer Service
    • Help Desk and Call Management
    • Interactive Voice Response Systems
    • On-line Customer Support
    • Scheduling
    Distribution and Warehousing
    • Distribution and Warehousing (Integrated)
    • Freight Dispatch
    • Freight Handling
    • Other Distribution and Warehousing Industry Sectors
    • Route Scheduling
    • Truck-Fleet Management
    • Wholesale Distribution and Warehousing
    Education and Training
    • Computer-Based Training
    • Courseware Development
    • Web-Based Training
    Engineering Design and Drafting
    • Cartography-Mapping-Geographic Information Systems (GIS)
    • Computer-Aided Design (CAD)
    • Computer-Aided Manufacturing (CAM)
    • Electronics Design Automation (EDA)
    • Materials Analysis
    • Mechanical Computer-Aided Design (MCAD)
    • Mechanical Computer-Aided Engineering (MCAE)
    • Mechanical Computer-Aided Manufacturing (MCAM)
    • Modeling
    • Sampling and Testing
    • Structural Analysis
    Facilities and Management
    • Building Security
    • Communications Management
    • Computer Maintenance Management (CMMS)
    • Energy Analysis and Management
    • Enterprise Asset Management (EAM)
    • Equipment Maintenance and Field Service
    • Facilities Maintenance
    • Facilities Management (Integrated)
    • Heating, Ventilation and Air Conditioning (HVAC)
    • Lighting Analysis and Control
    • Other Facilities Management Industry Sectors
    General Office Automation
    • Appointment Management
    • Calculators
    • Calendaring & Scheduling
    • Data Entry
    • Desktop Management
    • Desktop Publishing
    • Electronic Mail
    • Fax Management
    • Flowcharting
    • Graphics (Raster)
    • Graphics (Vector)
    • Mailing List Management
    • Multimedia and Animation
    • Office Suites (Integrated)
    • Presentation Applications
    • Spreadsheets
    • Time Management
    • Time and Expense Reporting
    • Voice Recognition Software
    • Web Browsers
    • Word Processors
    Groupware and Collaboration
    • Chat and Discussion Systems
    • Collaborative Writing Systems
    • E-mail Clients
    • Group Calendars
    • Integrated Groupware Solutions
    • Newsgroup Management
    • Resource Scheduling
    • Video Conferencing and IP-TV
    • Workflow Automation
    Human Resources
    • Benefits Administration
    • Employee Self-Service Software
    • Human Resource Management
    • Integrated Human Resources Utilities
    • Personnel Administration
    • Professional Services Administration
    • Recruiting
    • Resume Tracking
    • Time & Attendance
    • Travel Management
    Knowledge Management
    • Computer Output to Laser Disc (COLD)
    • Document Imaging Systems
    • Document Management Systems
    • Idea Management
    • Knowledge Base Management
    • Workflow
    Legacy Systems Analysis and Upgrade -
    Logistics and Procurement
    • Import and Export Management
    • Partnership Relationship Management
    • Requisitioning and Procurement
    • Supply Chain Management
    • Transportation Management
    • Warehouse Management Systems
    Manufacturing and Process Management -
    Capacity Requirements Planning (CRP)
    • Data Reduction
    • Distribution Management
    • Enterprise Resource Planning (ERP)
    • Equipment Maintenance and Management
    • Factory Automation and CIM
    • Factory Data Collection
    • Inventory Management
    • Machine Tools
    • Machine Vision
    • Maintenance, Repair and Operating Supplies (MRO)
    • Manufacturing Automation Protocol
    • Manufacturing Execution Systems (MES)
    • Manufacturing Simulation
    • Manufacturing Solutions (Integrated)
    • Material Requirements Planning (MRP)
    • Materials Manufacturing
    • Metrology
    • Numerical Control (NC)
    • Operations Planning
    • Plant Maintenance and Service
    • Process Control (HMI-SCADA)
    • Product Information Management (PIM)
    • Product Service
    • Production Planning
    • Production Scheduling
    • Quality Control
    • Reliability Modeling and Analysis
    • Robotics
    • Shop Floor Control
    • Test Development
    • Test Information Integration
    • Test Simulation
    • Work Load Control
    Project Management
    • Project Accounting
    • Project Estimating
    • Project Management (General)
    • Project Scheduling
    • Purchasing
    • Resource Planning
    Reporting, Analysis, and Decision Support
    • Business Planning
    • Business Process Reengineering
    • Competitive Analysis
    • Data Mining and OLAP
    • Decision Support Systems
    • End-User Query and Reporting
    • Executive Information Systems
    • Expert Systems
    • Mapping and Visualization
    • Multi-Dimensional Analysis
    • Needs Analysis Systems
    • Neural Networks
    • Statistics and Technical Data Analysis
    • Trend Analysis
    Sales and Marketing
    • Contact Management
    • Customer Relationship Management
    • Demographics
    • Enterprise Marketing Automation
    • Lead Distribution Management
    • Market Research
    • Marketing Management
    • Media Planning and Buying
    • Mobile Field Sales
    • Partnership Management
    • Product Configurators
    • Sales Analysis & Reporting
    • Sales Force Automation (SFA)
    • Sales Management
    • Telemarketing
    Industries
    • Aerospace & Defense
    • Automotive
    • Chemicals
    • Communications
    • Consumer Products
    • Education & Research
    • Energy
    • Engineering & Construction
    • Financial Services
    • Healthcare
    • High Technology
    • Industrial Manufacturing
    • Life Sciences
    • Professional Services
    • Public Sector
    • Retail
    • Travel & Transportation
    • Utilities
    Business
    • Contact Management
    • Customer Management
    • Accounting
    Data
    • Datawarehouse
    • Data mining
    • OLAP
    Desktop
    • Photo Editing
    • Graphic Design
    • Web Publishing
    Desktop Business Software
    • Email Client
    • Spreadsheet
    • Presentations
    • Word Processing
    Home and Hobbies
    • Cooking and Health
    • Fashion
    • Gardening and Landscape
    • Genealogy
    • Hobbies
    • Home Design
    • Home Publishing
    • Instrument Instruction
    • Legal
    • Mapping
    • Movies and Television
    • Music Appreciation
    • Personal Improvement
    • Script and Screenwriting
    Personal Finance
    • Investment Tools
    • Money Management
    • Tax Preparation
    Utilities
    • Virus Protection
    • PC Maintenance
    A
    • Batch
    • BPM (business process management)
    • BIM (business intelligence management)
    • EAI (enterprise architecture integration)
    • Document Management
    B
    • Operating System
    • Framework
    • Component/Library
    • Service
    • Subsystem
    C
    • Portal
    • Community
    • Commerce
    • Task-Tracking
    D
    • Transactions
    • Read-Only data
    • Workflow
    • Batch
    E
    • Server-side
    • Desktop
    • Peer-to-peer
    • Mobile
    • Web-centric
    • Data-centric
    • Middle-tier
    F
    • Content Management
    G
    • Web Server
    • Application Server
    • Database Server
    H
    • Collaboration Suite
    I
    • OLAP
    • Data Mining
    • Data Warehouses
    • Data Management
    • Business Process Integration
    • Business Intelligence
    • BAM
  • J.D. Meier's Blog

    Now Available: Microsoft Windows Azure Guide

    • 0 Comments

    The Microsoft patterns & practices Windows Azure Architecture Guide - Part I is now available on MSDN. 

    image

    This guide focuses on a migration scenario.  It walks through migrating a custom expense tracking and reimbursement system to the cloud.

    Chapters

    Key Topics

    • Asynchronous processing, blobs, and shared access signatures
    • Authentication and authorization
    • Automating deployment and managing environments
    • Background processing and performance considerations
    • Data access
    • Development life cycle
    • Pricing and cost considerations
    • Session management
    • Terminology and components
    • Using Windows Azure table storage

    Key Links

  • J.D. Meier's Blog

    SQL Azure Partitioning Pattern

    • 0 Comments

    While working with a customer at our Microsoft Executive Briefing Center (Microsoft EBC), one of the things we need to do was find a simple and effective way to partition SQL Azure.

    The problem with SQL Azure is that depending on your partitioning strategy, you can end up allocating increments that you don’t need and still pay for it.  The distinction is that for Azure Storage, you pay for storage used, but with SQL Azure you pay for the allocated increments.  See Windows Azure Pricing.

    After checking with our colleagues, one of the approaches we liked is to partition by customer ID ranges.  By adding this level of indirection, you can optimize or reduce unused increments because you already know the ranges and usage, and you can change the ranges and allocations if you need to.   Here is a visual of the approach:

    image

    Thanks to our colleague Danny Cohen for the insight, clarification, and suggestions.

  • J.D. Meier's Blog

    Workshop Planning Framework – The Six P’s

    • 0 Comments

    Workshops are one of the best ways to rapidly brainstorm ideas, capture scenarios, mine for patterns, etc.  The challenge can be designing an effective workshop. 

    In the book Scenarios, Stories, Use Cases: Through the Systems Development Life-Cycle, Ian F. Alexander and Neil Maiden present The Six P’s Workshop Framework.  Here is a summary of the approach:

    Category Question Concerns
    Purpose Why are we doing the workshop?
    • Goals
    • Need
    • Motivation
    Participants Who is involved?
    • Roles
    • Stakeholder
    • Experts
    Principles How should we function as a group?
    • Guidelines for participation
    • Working agreements
    • Ground rules
    • Group norms
    Products What should the workshop produce?
    • Work products
    • Dependencies
    • Models
    • Decisions
    • Next steps
    • Issues for resolution
    Place Where should we gather and share space?
    • Location
    • Time
    Process When should things happen and in what order?
    • Steps
    • Activities
    • Order
    • Concurrency

    As you can see, the six P’s (Purpose, Participants, Products, Place, and Process) create the overall frame.   Each category within the frame has a driving question.  For example, “Purpose” contains the question, “Why are we doing the workshop?”  Each category within the frame also has a list of “Concerns.”  For example, “Purpose” contains the concerns, “goals", “need”, and “motivation.”

    By addressing the question and the concerns, you can design and shape a more effective workshop.

  • J.D. Meier's Blog

    Types of Scenario Tools

    • 0 Comments

    I use scenarios all the time for anything from designing a user experience to evaluating architecture. Scenario is an overloaded term though. There are lots of types of scenario tools. If you know the types of scenario tools, you can use the right one for the job. For example, exception scenarios are useful for assessing robustness.  Misuse cases are helpful for figuring out potential threats and attacks.

    In the book Scenarios, Stories, Use Cases: Through the Systems Development Life-Cycle , Ian F. Alexander and Neil Maiden write about the types of scenarios and when to use them in software development.

    • Alternative World, Situation, Snapshot
    • Concept of Operations
    • Exception(-handling) Scenario
    • Negative Scenario, Misuse Case
    • Roleplay, Playthrough (of an Acted Scene)
    • Scenario Sequence, Script
    • Scripted Sequence Diagram
    • Simulation, animation
    • Story
    • Storyboard
    • Swim lanes Diagram
    • Test Case
    • Use Case
    • User Story

    Here is a brief explanation of each:

    • Alternative World, Situation, Snapshot - A scenario that describes a more or less static picture of an imagined future business situation
    • Concept of Operations - A set of scenarios describing in more or less design-independent terms how a system is expected to be used in practice.
    • Exception(-handling) Scenario - A scenario that describes how to handle an undesired event.  Useful for reliability analysis.
    • Negative Scenario, Misuse Case - A scenario that is desired not to occur by the stakeholders responsible for a system. Useful for identify security threats or security issues.
    • Roleplay, Playthrough (of an Acted Scene) - A scenario acted out to illustrate a desired behavior of or a possible problem with a future System.
    • Scenario Sequence, Script - A simple list of steps ordered by time, possibly numbered, representing a Course of Events. Each step is an action taken by a named role (Actor).
    • Scripted Sequence Diagram - Shows the steps of a user with the system.
    • Simulation, animation - Narrates a scenario using images in action.
    • Story - A text that narrates a Scenario with little or no formal structure.
    • Storyboard - A sequence of diagrams or other images that narrates a Scenario.
    • Swim lanes Diagram - A flowchart arranged with a “swimming pool lane” for each role, showing both the actions of each role and interaction between roles over time.
    • Test Case - A sequence of steps to be run to verify that a system works as you intend it.
    • Use Case - A UML scenario format that consists of an Actor, a Goal, and text that contain a set of related scenarios representing different combinations of sequences of the Use Case, depending on Events.
    • User Story - A scenario format consisting of a brief, informal narrative text (a Story) that explains how a user can achieve a goal.
  • J.D. Meier's Blog

    Short-Burst Work

    • 0 Comments

    This is a walk down memory lane.  It’s an idea I was circulating circa 2005 so the words will seem stale now.  Periodically I like to flip through my old ideas to see how I’ve used them, what impact they made, and what I learned in the process.

    When I originally pitched the idea, there were two basic reactions – either “this will never fly” or “we have to make this happen.”  The funny thing was, when Amazon launched its Mechanical Turk, several months later, some of the original naysayers, came back to me and said, “Oh, I guess you were on to something.”

    I wasn’t passionate about the idea, so I didn’t pursuit it.  I was also busy with other game changers.  However, I did share the idea with some key folks, and then I wrote it up in a simple way so that whoever might need to see it, could quickly understand the scenarios and the value.  However, the thinking behind this did end up shaping some interesting efforts.  It also helped me articulate a principle that kept showing up – Human Shepherds and the Law of Relevancy

    As an aside, the big thing I learned from having clusters of ideas at a time … (I filed several patents that year, created a few key software prototypes that internal/external customers built offerings around, and helped shape some key product-line paths) … is that you can actually get ahead of the innovation eight-ball, if you know how to combine key principles and patterns with the trends and the maturity cycles of markets.  The patterns and principles help you see things that might not otherwise be obvious.  There are even tricks to getting ahead of the mainstream market.   Someday I’ll share the patterns and practices for innovation that I’ve learned from the school of hard knocks (part of our group acted like applied research so we have the battle scars of making innovation stick against real-world scenarios.)

    Here is the original idea of “Short-Burst Work”, in raw form.  Note that one of the other things I learned since then is that choosing a sticky name for an idea, makes all the difference in the world – alliteration and fun, helps a ton!

    Idea: Short Burst Work

    Elevator Pitch
    ”Connect short-burst human power to the relevant short-burst work opportunities ...”

    Summary
    Here's an idea that could be a game changer ...
    What if you could connect a world-wide market of job seekers to short-burst opportunities through "adverstising" in the corp space? (consumer works too, but let's play w/corp scenarios for now)
    This is nearly the "long tail" of job opportunities issue.   Imagine all the short-burst work opportunities in your day job.   What's interesting is this model is happening on SecondLife, with real people and real exchange of money.
    Imagine the world of pooled resources, available 24x7.  Imagine the world of editors, coders, great thinkers …. etc.  Imagine how many former dot-Com'ers that would love to lend short-burst help.  Imagine all the great professors of the world you can suddenly involve when you can virtualize your work items and collaborate with a simple platform.

    Situation

    • Fine-grained work items (short-burst) can be tough to fill whether temp agency, Monster ...
    • Find-grained opportunities (mentor me for 1/2 hr Wed, build me code to do X ... etc.) are a challenge to find the short-burst service/provider (yet the help exists ... it's just displaced or tough to make the match or there's workflow hurdles)
    • Untapped  or displaced resource pools in the US
    • Untapped resource pools world-wide
    • Lots of challenges connecting a world-wide resource pool to short-burst, relevant work
    • 24/7 availability of a distributed world-wide resource pool, waiting to be leveraged

    Opportunity

    • Connect services/providers to relevant short-burst work
    • Improve the ability for consumers to find relevant labor pools

    Recommendation
    From a mental model, think of it as a highly relevant "match" service for short-burst work opportunities and resource pools. 

    • Create a platform for a "Live Help/Services" model.
    • Create a pluggable "service" model for providers (pluggable experts/skills:  mentoring, professional counseling, health, psychology, education,  software engineers ... anything)
    • Simplify the ability for a service provider to plug in from anywhere in the world

    Corporate Scenarios
    Imagine live services to connect your short-burst work items to:

    • Build me a strawman presentation for X
    • Write me a doc on X that explains Y
    • Write me a piece of code to do Z
    • Show me how to build a solution for scenario N
    • I need advice on today's big task
    • Coach my on ....
    • Tech edit my specs ...
    • Help me create a demo to show ....
    • Advise me on ....

    Home Scenarios

    • Write me a book report ...
    • Help me with my math homework
    • Help with my presentation ...
    • Mentor my niece ...

    Why is this feasible? 
    It's happening on SecondLife.  Folks around their world are selling their services (scripting, image consulting, music dj …. You name it)  It's happening because the barrier to entry is low:

    To sell your services/skills:

    • You don't need to figure out how to exchange money; SecondLife figured out the exchange and abstracted (like PayPal on the net)
    • You don't need to go through a bunch of hoops to advertise your services.

    To consume the services/skills:

    • You search/browse
    • Can't find what you need?  Then you bring in Live Help and a concierge helps you find it.

    There's reasons why existing services like this haven't taken off.  Connecting the consumer/provider to the "long tail" of the short-burst opportunities is a relevancy algorithm issue and a platform hurdle (services platform with good consumption/provider models).
    Your engine can help folks find the short-burst services they need.  And when the relevancy is beyond organic search, you broker a "concierge" type service (Live Concierge) to connect a consumer to a provider … or a provider to a consumer.  It's a game changer.
    How to make it happen?

    Imagine you the user wants to connect to "live" help.  Think (Google (relevancy) + Monster (labor pool) + Temp Agencies (labor pool and short-burst work) + (plug in your favorite Web 2.0 model)

  • J.D. Meier's Blog

    Success Pattern for V-Teams

    • 0 Comments

    What is the success pattern for leading a successful v-team?  I asked one of my mentors, a seasoned softie for his take on what are the keys to a successful v-team.  Here is what he had to say:

    • you're building a coalition -- not just a vteam
    • make sure everyone is committed ... not just involved (see Turning Chickens into Pigs)
    • get commitments and support from their leadership
    • do an "exec" vision scope and milestone reviews
    • give the team members the credit
    • show up as a leader who is worth following - defend them, work for them, help them succeed -- you are working with people that you may want to follow you one day
  • J.D. Meier's Blog

    Cloud Security Threats and Countermeasures at a Glance

    • 0 Comments

    Cloud security has been a hot topic with the introduction of the Microsoft offering of the Windows Azure platform.  One of the quickest ways to get your head around security is to cut to the chase and look at the threats, attacks, vulnerabilities and countermeasures.  This post is a look at threats and countermeasures from a technical perspective.

    The thing to keep in mind with security is that it’s a matter of people, process, and technology.  However, focusing on a specific slice, in this case the technical slice, can help you get results.  The thing to keep in mind about security from a technical side is that you also need to think holistically in terms of the application, network, and host, as well as how you plug it into your product or development life cycle.  For information on plugging it into your life cycle, see the Security Development Lifecycle.

    While many of the same security issues that apply to running applications on-premise also apply to the cloud, the context of running in the cloud does change some key things.  For example, it might mean taking a deeper look at claims for identity management and access control.  It might mean rethinking how you think about your storage.  It can mean thinking more about how you access and manage virtualized computing resources.  It can mean thinking about how you make calls to services or how you protect calls to your own services.

    Here is a fast path through looking at security threats, attacks, vulnerabilities, and countermeasures for the cloud …

    Objectives

    • Learn a security frame that applies to the cloud
    • Learn top threats/attacks, vulnerabilities and countermeasures for each area within the security frame
    • Understand differences between threats, attacks, vulnerabilities and countermeasures

    Overview
    It is important to think like an attacker when designing and implementing an application. Putting yourself in the attacker’s mindset will make you more effective at designing mitigations for vulnerabilities and coding defensively.  Below is the cloud security frame. We use the cloud security frame to present threats, attacks, vulnerabilities and countermeasures to make them more actionable and meaningful.

    You can also use the cloud security frame to effectively organize principles, practices, patterns, and anti-patterns in a more useful way.

    Threats, Attacks, Vulnerabilities, and Countermeasures
    These terms are defined as follows:

    • Asset. A resource of value such as the data in a database, data on the file system, or a system resource.
    • Threat. A potential occurrence – malicious or otherwise – that can harm an asset.
    • Vulnerability. A weakness that makes a threat possible.
    • Attack. An action taken to exploit vulnerability and realize a threat.
    • Countermeasure. A safeguard that addresses a threat and mitigates risk.

    Cloud Security Frame
    The following key security concepts provide a frame for thinking about security when designing applications to run on the cloud, such as Windows Azure. Understanding these concepts helps you put key security considerations such as authentication, authorization, auditing, confidentiality, integrity, and availability into action.

    Hot Spot Description
    Auditing and Logging Cloud auditing and logging refers to how security-related events are recorded, monitored, audited, exposed, compiled & partitioned across multiple cloud instances. Examples include: Who did what and when and on which VM instance?
    Authentication Authentication is the process of proving identity, typically through credentials, such as a user name and password. In the cloud this also encompasses authentication against varying identity stores.
    Authorization Authorization is how your application provides access controls for roles, resources and operations. Authorization strategies might involve standard mechanisms, utilize claims and potentially support a federated model.
    Communication Communication encompasses how data is transmitted over the wire. Transport security, message encryption, and point to point communication are covered here.
    Configuration Management Configuration management refers to how your application handles configuration and administration of your applications from a security perspective. Examples include: Who does your application run as? Which databases does it connect to? How is your application administered? How are these settings secured?
    Cryptography Cryptography refers to how your application enforces confidentiality and integrity. Examples include: How are you keeping secrets (confidentiality)? How are you tamper-proofing your data or libraries (integrity)? How are you providing seeds for random values that must be cryptographically strong? Certificates and cert management are in this domain as well.
    Input and Data Validation Validation refers to how your application filters, scrubs, or rejects input before additional processing, or how it sanitizes output. It's about constraining input through entry points and encoding output through exit points. Message validation refers to how you verify the message payload against schema, as well as message size, content and character sets. Examples include: How do you know that the input your application receives is valid and safe? Do you trust data from sources such as databases and file shares?
    Exception Management Exception management refers to how you handle applications errors and exceptions. Examples include: When your application fails, what does your application do? Does it support graceful failover to other application instances in the cloud? How much information do you reveal? Do you return friendly error information to end users? Do you pass valuable exception information back to the caller?
    Sensitive Data Sensitive data refers to how your application handles any data that must be protected either in memory, over the network, or in persistent stores. Examples include: How does your application handle sensitive data? How is sensitive data shared between application instances?
    Session Management A session refers to a series of related interactions between a user and your application. Examples include: How does your application handle and protect user sessions?

     

    Threats and Attacks

     

    Category Items
    Auditing and Logging
    • Repudiation. An attacker denies performing an operation, exploits an application without trace, or covers his or her tracks.
    • Denial of service (DoS). An attacker overwhelms logs with excessive entries or very large log entries.
    • Disclosure of confidential information. An attacker gathers sensitive information from log files.
    Authentication
    • Network eavesdropping. An attacker steals identity and/or credentials off the network by reading network traffic not intended for them.
    • Brute force attacks. An attacker guesses identity and/or credentials through the use of brute force.
    • Dictionary attacks. An attacker guesses identity and/or credentials through the use of common terms in a dictionary designed for that purpose.
    • Cookie replay attacks. An attacker gains access to an authenticated session through the reuse of a stolen cookie containing session information.
    • Credential theft. An attacker gains access to credentials through data theft; for instance, phishing or social engineering.
    Authorization
    • Elevation of privilege. An attacker enters a system as a lower-level user, but is able to obtain higher-level access.
    • Disclosure of confidential data. An attacker accesses confidential information because of authorization failure on a resource or operation.
    • Data tampering. An attacker modifies sensitive data because of authorization failure on a resource or operation.
    • Luring attacks. An attacker lures a higher-privileged user into taking an action on their behalf. This is not an authorization failure but rather a failure of the system to properly inform the user.
    • Token stealing. An attacker steals the credentials or token of another user in order to gain authorization to resources or operations they would not otherwise be able to access.
    Communication
    • Failure to encrypt messages. An attacker is able to read message content off the network because it is not encrypted.
    • Theft of encryption keys. An attacker is able to decrypt sensitive data because he or she has the keys.
    • Man-in-the-middle attack. An attacker can read and then modify messages between the client and the service.
    • Session replay. An attacker steals messages off the network and replays them in order to steal a user's session.
    • Data tampering. An attacker modifies the data in a message in order to attack the client or the service
    Configuration Management
    • Unauthorized access to configuration stores. An attacker gains access to configuration files and is able to modify binding settings, etc.
    • Retrieval of clear text configuration secrets. An attacker gains access to configuration files and is able to retrieve sensitive information such as database connection strings.
    Cryptography
    • Encryption cracking. Breaking an encryption algorithm and gaining access to the encrypted data.
    • Loss of decryption keys. Obtaining decryption keys and using them to access protected data.
    Exception Management
    • Information disclosure. Sensitive system or application details are revealed through exception information.
    • Denial of service. An attacker uses error conditions to stop your service or place it in an unrecoverable error state.
    • Elevation of privilege. Your service encounters an error and fails to an insecure state; for instance, failing to revert impersonation.
    Input and Data Validation
    • Canonicalization attacks. Canonicalization attacks can occur anytime validation is performed on a different form of the input than that which is used for later processing. For instance, a validation check may be performed on an encoded string, which is later decoded and used as a file path or URL.
    • Cross-site scripting. Cross-site scripting can occur if you fail to encode user input before echoing back to a client that will render it as HTML.
    • SQL injection. Failure to validate input can result in SQL injection if the input is used to construct a SQL statement, or if it will modify the construction of a SQL statement in some way.
    • Cross-Site Request Forgery: CSRF attacks involve forged transactions submitted to a site on behalf of another party.
    • XPath injection. XPath injection can result if the input sent to the Web service is used to influence or construct an XPath statement. The input can also introduce unintended results if the XPath statement is used by the Web service as part of some larger operation, such as applying an XQuery or an XSLT transformation to an XML document.
    • XML bomb. XML bomb attacks occur when specific, small XML messages are parsed by a service resulting in data that feeds on itself and grows exponentially. An attacker sends an XML bomb with the intent of overwhelming a Web service’s XML parser and resulting in a denial of service attack.
    Sensitive Data
    • Memory dumping. An attacker is able to read sensitive data out of memory or from local files.
    • Network eavesdropping. An attacker sniffs unencrypted sensitive data off the network.
    • Configuration file sniffing. An attacker steals sensitive information, such as connection strings, out of configuration files.
    Session Management
    • Session hijacking. An attacker steals the session ID of another user in order to gain access to resources or operations they would not otherwise be able to access.
    • Session replay. An attacker steals messages off the network and replays them in order to steal a user’s session.
    • Man-in-the-middle attack. An attacker can read and then modify messages between the client and the service.
    • Inability to log out successfully. An application leaves a communication channel open rather than completely closing the connection and destroying any server objects in memory relating to the session.
    • Cross-site request forgery. Cross-site request forgery (CSRF) is where an attacker tricks a user into performing an action on a site where the user actually has a legitimate authorized account.
    • Session fixation. An attacker uses CSRF to set another person’s session identifier and thus hijack the session after the attacker tricks a user into initiating it.
    • Load balancing and session affinity. When sessions are transferred from one server to balance traffic among the various servers, an attacker can hijack the session during the handoff.

    Vulnerabilities

    Category Items
    Auditing and Logging
    • Failing to audit failed logons.
    • Failing to secure log files.
    • Storing sensitive information in log files.
    • Failing to audit across application tiers.
    • Failure to throttle log files.
    Authentication
    • Using weak passwords.
    • Storing clear text credentials in configuration files.
    • Passing clear text credentials over the network.
    • Permitting prolonged session lifetime.
    • Mixing personalization with authentication.
    • Using weak authentication mechanisms (e.g., using basic authentication over an untrusted network).
    Authorization
    • Relying on a single gatekeeper (e.g., relying on client-side validation only).
    • Failing to lock down system resources against application identities.
    • Failing to limit database access to specified stored procedures.
    • Using inadequate separation of privileges.
    • Connection pooling.
    • Permitting over privileged accounts.
    Configuration Management
    • Using insecure custom administration interfaces.
    • Failing to secure configuration files on the server.
    • Storing sensitive information in the clear text.
    • Having too many administrators.
    • Using over privileged process accounts and service accounts.
    Communication
    • Not encrypting messages.
    • Using custom cryptography.
    • Distributing keys insecurely.
    • Managing or storing keys insecurely.
    • Failure to use a mechanism to detect message replays.
    • Not using either message or transport security.
    Cryptography
    • Using custom cryptography
    • Failing to secure encryption keys
    • Using the wrong algorithm or a key size that is too small
    • Using the same key for a prolonged period of time
    • Distributing keys in an insecure manner
    Exception Management
    • Failure to use structured exception handling (try/catch).
    • Revealing too much information to the client.
    • Failure to specify fault contracts with the client.
    • Failure to use a global exception handler.
    Input and Data Validation
    • Using non-validated input used to generate SQL queries.
    • Relying only on client-side validation.
    • Using input file names, URLs, or usernames for security decisions.
    • Using application-only filters for malicious input.
    • Looking for known bad patterns of input.
    • Trusting data read from databases, file shares, and other network resources.
    • Failing to validate input from all sources including cookies, headers, parameters, databases, and network resources.
    Sensitive Data
    • Storing secrets when you do not need to.
    • Storing secrets in code.
    • Storing secrets in clear text in files, registry, or configuration.
    • Passing sensitive data in clear text over networks.
    Session Management
    • Passing session IDs over unencrypted channels.
    • Permitting prolonged session lifetime.
    • Having insecure session state stores.
    • Placing session identifiers in query strings.

     

    Countermeasures

     

    Category Items
    Auditing and Logging
    • Identify malicious behavior.
    • Know your baseline (know what good traffic looks like).
    • Use application instrumentation to expose behavior that can be monitored.
    • Throttle logging.
    • Strip sensitive data before logging.
    Authentication
    • Use strong password policies.
    • Do not store credentials in an insecure manner.
    • Use authentication mechanisms that do not require clear text credentials to be passed over the network.
    • Encrypt communication channels to secure authentication tokens.
    • Use Secure HTTP (HTTPS) only with forms authentication cookies.
    • Separate anonymous from authenticated pages.
    • Using cryptographic random number generators to generate session IDs.
    Authorization
    • Use least-privileged accounts.
    • Tie authentication to authorization on the same tier.
    • Consider granularity of access.
    • Enforce separation of privileges.
    • Use multiple gatekeepers.
    • Secure system resources against system identities.
    Configuration Management
    • Using insecure custom administration interfaces.
    • Failing to secure configuration files on the server.
    • Storing sensitive information in the clear text.
    • Having too many administrators.
    • Using overprivileged process accounts and service accounts.
    Communication
    • Use message security or transport security to encrypt your messages.
    • Use proven platform-provided cryptography.
    • Periodically change your keys.
    • Use any platform-provided replay detection features.
    • Consider creating custom code if the platform does not provide a detection mechanism.
    • Turn on message or transport security.
    Cryptography
    • Do not develop and use proprietary algorithms (XOR is not encryption. Use established cryptography such as RSA)
    • Avoid key management.
    • Use the RNGCryptoServiceProvider method to generate random numbers
    • Periodically change your keys
    Exception Management
    • Use structured exception handling (by using try/catch blocks).
    • Catch and wrap exceptions only if the operation adds value/information.
    • Do not reveal sensitive system or application information.
    • Implement a global exception handler.
    • Do not log private data such as passwords.
    Sensitive Data
    • Do not store secrets in software.
    • Encrypt sensitive data over the network.
    • Secure the channel.
    • Encrypt sensitive data in configuration files.
    Session Management
    • Partition the site by anonymous, identified, and authenticated users.
    • Reduce session timeouts.
    • Avoid storing sensitive data in session stores.
    • Secure the channel to the session store.
    • Authenticate and authorize access to the session store.
    Validation
    • Do not trust client input.
    • Validate input: length, range, format, and type.
    • Validate XML streams.
    • Constrain, reject, and sanitize input.
    • Encode output.
    • Restrict the size, length, and depth of parsed XML messages.

    Threats and Attacks Explained

    1.  Brute force attacks. Attacks that use the raw computer processing power to try different permutations of any variable that could expose a security hole. For example, if an attacker knew that access required an 8-character username and a 10-character password, the attacker could iterate through every possible (256 multiplied by itself 18 times) combination in order to attempt to gain access to a system. No intelligence is used to filter or shape for likely combinations.
    2. Buffer overflows. The maximum size of a given variable (string or otherwise) is exceeded, forcing unintended program processing. In this case, the attacker uses this behavior to cause insertion and execution of code in such a way that the attacker gains control of the program in which the buffer overflow occurs. Depending on the program’s privileges, the seriousness of the security breach will vary.
    3. Canonicalization attacks. There are multiple ways to access the same object and an attacker uses a method to bypass any security measures instituted on the primary intended methods of access. Often, the unintended methods of access can be less secure deprecated methods kept for backward compatibility.
    4. Cookie manipulation. Through various methods, an attacker will alter the cookies stored in the browser. Attackers will then use the cookie to fraudulently authenticate themselves to a service or Web site.
    5. Cookie replay attacks. Reusing a previously valid cookie to deceive the server into believing that a previously authenticated session is still in progress and valid.
    6. Credential theft. Stealing the verification part of an authentication pair (identity + credentials = authentication). Passwords are a common credential.
    7. Cross-Site Request Forgery (CSRF). Interacting with a web site on behalf of another user to perform malicious actions. A site that assumes all requests it receives are intentional is vulnerable to a forged request.
    8. Cross-site scripting (XSS). An attacker is able to inject executable code (script) into a stream of data that will be rendered in a browser. The code will be executed in the context of the user’s current session and will gain privileges to the site and information that it would not otherwise have.
    9. Connection pooling. The practice of creating and then reusing a connection resource as a performance optimization. In a security context, this can result in either the client or server using a connection previously used by a highly privileged user being used for a lower-privileged user or purpose. This can potentially expose vulnerability if the connection is not reauthorized when used by a new identity.
    10. Data tampering. An attacker violates the integrity of data by modifying it in local memory, in a data-store, or on the network. Modification of this data could provide the attacker with access to a service through a number of the different methods listed in this document.
    11. Denial of service. Denial of service (DoS) is the process of making a system or application unavailable. For example, a DoS attack might be accomplished by bombarding a server with requests to consume all available system resources, or by passing the server malformed input data that can crash an application process.
    12. Dictionary attack. Use of a list of likely access methods (usernames, passwords, coding methods) to try and gain access to a system. This approach is more focused and intelligent than the “brute force” attack method, so as to increase the likelihood of success in a shorter amount of time.
    13. Disclosure of sensitive/confidential data. Sensitive data is exposed in some unintended way to users who do not have the proper privileges to see it. This can often be done through parameterized error messages, where an attacker will force an error and the program will pass sensitive information up through the layers of the program without filtering it. This can be personally identifiable information (i.e., personal data) or system data.
    14. Elevation of privilege. A user with limited privileges assumes the identity of a privileged user to gain privileged access to an application. For example, an attacker with limited privileges might elevate his or her privilege level to compromise and take control of a highly privileged and trusted process or account. More information about this attack in the context of Windows Azure can be found in the Security Best Practices for Developing Windows Azure Applications at http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=0ff0c25f-dc54-4f56-aae7-481e67504df6
    15. Encryption. The process of taking sensitive data and changing it in such a way that it is unrecognizable to anyone but those who know how to decode it. Different encryption methods have different strengths based on how easy it is for an attacker to obtain the original information through whatever methods are available.
    16. Information disclosure. Unwanted exposure of private data. For example, a user views the contents of a table or file that he or she is not authorized to open, or monitors data passed in plaintext over a network. Some examples of information disclosure vulnerabilities include the use of hidden form fields, comments embedded in Web pages that contain database connection strings and connection details, and weak exception handling that can lead to internal system-level details being revealed to the client. Any of this information can be very useful to the attacker.
    17. Luring attacks. An attacker lures a higher-privileged user into taking an action on his or her behalf. This is not an authorization failure but rather a failure of the system to properly inform the user.
    18. Man-in-the-middle attacks. A person intercepts both the client and server communications and then acts as an intermediary between the two without each ever knowing. This gives the “middle man” the ability to read and potentially modify messages from either party in order to implement another type of attack listed here.
    19. Network eavesdropping. Listening to network packets and reassembling the messages being sent back and forth between one or more parties on the network. While not an attack itself, network eavesdropping can easily intercept information for use in specific attacks listed in this document.
    20. Open Redirects. Attacker provides a URL to a malicious site when allowed to input a URL used in a redirect. This allows the attacker to direct users to sites that perform phishing attacks or other malicious actions.
    21. Password cracking. If the attacker cannot establish an anonymous connection with the server, he or she will try to establish an authenticated connection. For this, the attacker must know a valid username and password combination. If you use default account names, you are giving the attacker a head start. Then the attacker only has to crack the account’s password. The use of blank or weak passwords makes the attacker’s job even easier.
    22. Repudiation. The ability of users (legitimate or otherwise) to deny that they performed specific actions or transactions. Without adequate auditing, repudiation attacks are difficult to prove.
    23. Session hijacking. Also known as man-in-the-middle attacks, session hijacking deceives a server or a client into accepting the upstream host as the actual legitimate host. Instead, the upstream host is an attacker’s host that is manipulating the network so the attacker’s host appears to be the desired destination.
    24. Session replay. An attacker steals messages off of the network and replays them in order to steal a user’s session.
    25. Session fixation. An attacker sets (fixates) another person’s session identifier artificially. The attacker must know that a particular Web service accepts any session ID that is set externally; for example, the attacker sets up a URL such as http://unsecurewebservice.com/?sessionID=1234567. The attacker then sends this URL to a valid user, who clicks on it. At this point, a valid session with the ID 1234567 is created on the server. Because the attacker determines this ID, he or she can now hijack the session, which has been authenticated using the valid user’s credentials.
    26. Spoofing. An attempt to gain access to a system by using a false identity. This can be accomplished by using stolen user credentials or a false IP address. After the attacker successfully gains access as a legitimate user or host, elevation of privileges or abuse using authorization can begin.
    27. SQL injection. Failure to validate input in cases where the input is used to construct a SQL statement or will modify the construction of a SQL statement in some way. If the attacker can influence the creation of a SQL statement, he or she can gain access to the database with privileges otherwise unavailable and use this in order to steal or modify information or destroy data.
    28. Throttling. The process of limiting resource usage to keep a particular process from bogging down and/or crashing a system. Relevant as a countermeasure in DoS attacks, where an attacker attempts to crash the system by overloading it with input.

    Countermeasures Explained

    1. Assume all input is malicious. Assuming all input is malicious means designing your application to validate all input. User input should never be accepted without being filtered and/or sanitized.
    2. Audit and log activity through all of the application tiers. Log business critical and security sensitive events. This will help you track security issues down and make sense of security problems. Skilled attackers attempt to cover their tracks, so you’ll want to protect your logs.
    3. Avoid storing secrets. Design around storing secrets. If necessary, sometimes they can be avoided by storing them after using a one-way hash algorithm.
    4. Avoid storing sensitive data in the Web space. Anything exposed to the public Internet is considered “web space.” Sensitive data stored in a location that might be compromised by any member of the public places it at much higher risk.
    5. Back up and regularly analyze log files. Some attacks can occur over time. Regular analysis of logs will allow you to recognize with sufficient time to address them. Performing regular backups lowers the risk of an attacker covering his tracks by deleting logging of his activities.
    6. Be able to disable accounts. The ability to reactively defend an attack by shutting out a user should be supported through the ability to disable an account.
    7. Be careful with canonicalization issues. Predictable naming of file resources is convenient for programming, but is also very convenient for malicious parties to attack. Application logic should not be exposed to users in this manner. Instead, you use file names derived from the original names or fed through a one-way hashing algorithm.
    8. Catch exceptions. Unhandled exceptions are at risk of passing too much information to the client. Handle exceptions when possible.
    9. Centralize your input and data validation. Input and data validation should be performed using a common set of code such as a validation library.
    10. Consider a centralized exception management framework. Exception handling frameworks are available publically and provide an established and tested means for handling exceptions.
    11. Consider authorization granularity. Every object needs to have an authorization control that authorizes access based on the identity of the authenticated party requesting access. Fine grained authorization will control access to each resource, while coarse grained authorization will control access to groups of resources or functional areas of the application.
    12. Consider identity flow. Auditing should be traceable back to the authenticated party. Take note of identity transitions imposed by design decisions like impersonation.
    13. Constrain input. Limit user input to expected ranges and formats.
    14. Constrain, reject, and sanitize your input. Constrain, reject and sanitize should be primary techniques in handling input data.
    15. Cycle your keys periodically. Expiring encryption keys lowers the risk of stolen keys.
    16. Disable anonymous access and authenticate every principle. When possible, require all interactions to occur as an authenticated party as opposed to an anonymous one. This will help facilitate more effective auditing.
    17. Do not develop your own cryptography. Custom cryptography is not difficult for experts to crack. Established cryptography is preferred because it is known to be safe.
    18. Do not leak information to the client. Exception data can potentially contain sensitive data or information exposing program logic. Provide clients only with the error data they need for the UI.
    19. Do not log private data such as passwords. Log files are an attack vector for malicious parties. Limit the risk of their being compromised by not logging sensitive data in the log.
    20. Do not pass sensitive data using the HTTP-GET protocol. Data passed using HTTP GET is appended to the querystring. When users share links by copying and pasting them from the browser address bar, sensitive data may also be inadvertently passed. Pass sensitive data in the body of a POST to avoid this.
    21. Do not rely on client-side validation. Any code delivered to a client is at risk of being compromised. Because of this, it should always be assumed that input validation on the client might have been bypassed.
    22. Do not send passwords over the wire in plaintext. Authentication information communicated over the wire should always be encrypted. This may mean encrypting the values, or encrypting the entire channel with SSL.
    23. Do not store credentials in plaintext. Credentials are sometimes stored in application configuration files, repositories, or sent over email. Always encrypt credentials before storing them.
    24. Do not store database connections, passwords, or keys in plaintext. Configuration secrets should always be stored in encrypted form, external to the code.
    25. Do not store passwords in user stores. In the event that the user store is compromised, an attack should never be able to access passwords. A derivative of a password should be stored instead. A common approach to this is to encrypt a version of the password using a one-way hash with a SALT. Upon authentication, the encrypted password can be re-generated with the SALT and the result can be compared to the original encrypted password.
    26. Do not store secrets in code. Secrets such as configuration settings are convenient to store in code, but are more likely to be stolen. Instead, store them in a secure location such as a secret store.
    27. Do not store sensitive data in persistent cookies. Persistent cookies are stored client-side and provide attackers with ample opportunity to steal sensitive data, be it through encryption cracking or any other means.
    28. Do not trust fields that the client can manipulate (query strings, form fields, cookies, or HTTP headers). All information sent from a client should always be assumed to be malicious. All information from a client should always be validated and sanitized before it is used.
    29. Do not trust HTTP header information. Http header manipulation is a threat that can be mitigated by building application logic that assumes HTTP headers are compromised and validates the HTTP headers before using them.
    30. Encrypt communication channels to protect authentication tokens. Authentication tokens are often the target of eavesdropping, theft or replay type attacks. To reduce the risk in these types of attacks, it is useful to encrypt the channel the tokens are communicated over. Typically this means protecting a login page with SSL encryption.
    31. Encrypt sensitive cookie state. Sensitive data contained within cookies should always be encrypted.
    32. Encrypt the contents of the authentication cookies. In the case where cookies are compromised, they should not contain clear-text session data. Encrypt sensitive data within the session cookie.
    33. Encrypt the data or secure the communication channel. Sensitive data should only be passed in encrypted form. This can be accomplished by encrypting the individual items that are sent over the wire, or encrypting the entire channel as with SSL.
    34. Enforce separation of privileges. Avoid building generic roles with privileges to perform a wide range of actions. Roles should be designed for specific tasks and provided the minimum privileges required for those tasks.
    35. Enforce unique transactions. Identify each transaction from a client uniquely to help prevent replay and forgery attacks.
    36. Identify malicious behavior. Monitoring site interactions that fall outside of normal usage patterns, you can quickly identify malicious behavior. This is closely related to “Know what good traffic looks like.
    37. Keep unencrypted data close to the algorithm. Use decrypted data as soon as it is decrypted, and then dispose of it promptly. Unencrypted data should not be held in memory in code.
    38. Know what good traffic looks like. Active auditing and logging of a site will allow you know recognize what regular traffic and usage patterns are. This is a required step in order to be able to identify malicious behavior.
    39. Limit session lifetime. Longer session lifetimes provide greater opportunity for Cross-Site Scripting or Cross-Site Request Forgery attacks to add activity onto an old session.
    40. Log detailed error messages. Highly detailed error message logging can provide clues to attempted attacks.
    41. Log key events. Profile your application and note key or sensitive operations and/or events, and log these events during application operation.
    42. Maintain separate administration privileges. Consider granularity of authorization in the administrative interfaces as well. Avoid combining administrator roles with distinctly different roles such as development, test or deployment.
    43. Make sure that users do not bypass your checks. Bypassing checks can be accomplished by canonicalization attacks, or bypassing client-side validation. Application design should avoid exposing application logic, and segregating application logic into flow that can be interrupted. For example, an ASPX page that performs only validations and then redirects. Instead, validation routines should be tightly bound to the data they are validating.
    44. Pass Forms authentication cookies only over HTTPS connections. Cookies are at risk of theft and replay type attacks. Encrypting them with SSL helps reduce the risk of these types of attacks.
    45. Protect authentication cookies. Cookies can be manipulated with Cross-Site Scripting attacks, encrypt sensitive data in cookies, and use browser features such as the HttpOnly cookie attribute.
    46. Provide strong access controls on sensitive data stores. Access to secret stores should but authorized. Protect the secret store as you would other secure resources by requiring authentication and authorization as appropriate.
    47. Reject known bad input. Rejecting known bad input involves screening input for values that are known to be problematic or malicious. NOTE: Rejecting should never be the primary means of screening bad input, it should always be used in conjunction with input sanitization.
    48. Require strong passwords. Enforce password complexity requirement by requiring long passwords with a combination of upper case, lower case, numeric and special (for example punctuation) characters. This helps mitigate the threat posed by dictionary attacks. If possible, also enforce automatic password expiry.
    49. Restrict user access to system-level resources. Users should not be touching system resources directly. This should be accomplished through an intermediary such as the application. System resources should be restricted to application access.
    50. Retrieve sensitive data on demand. Sensitive data stored in application memory provides attackers another location they can attempt to access the data. Often this data is used in unencrypted form also. To minimize risk of sensitive data theft, sensitive data should be used immediately and then cleared from memory.
    51. Sanitize input. Sanitizing input is the opposite of rejecting bad input. Sanitizing input is the process of filtering input data to only accept values that are known to be safe. Alternatively, input can be rendered innocuous by converting it to safe output through output encoding methods.
    52. Secure access to log files. Log files should only be accessible to administrators, auditors, or administrative interfaces. An attacker with access to the logs might be able to glean sensitive data or program logic from logs.
    53. Secure the communication channel for remote administration. Eavesdropping and replay attacks can target administration interfaces as well. If using a web based administration interface, use SSL.
    54. Secure your configuration store. The configuration store should require authenticated access and should store sensitive settings or information in an encrypted format.
    55. Secure your encryption keys. Encryption keys should be treated as secrets or sensitive data. They should be secured in a secret store or key repository.
    56. Separate public and restricted areas. Applications that contain public front-ends as well as content that requires authentication to access should be partitioned in the same manner. Public facing pages should be hosted in a separate file structure, directory or domain from private content.
    57. Store keys in a restricted location. Protect keys with authorization policies.
    58. Support password expiration periods. User passwords and account credentials are commonly compromised. Expiration policies help mitigate attacks from stolen accounts, or disgruntled employees who have been terminated.
    59. Use account lockout policies for end-user accounts. Account login attempts should have a cap on failed attempts. After the cap is exceeded the account should prevent further login attempts. Lockout helps prevent dictionary and brute force attacks.
    60. Use application instrumentation to expose behavior that can be monitored: Application transactions that are more likely to be targeted by malicious interactions should be logged or monitored. Examples of this might be adding logging code to an exception handler, or logging individual API calls. By providing a means to watch these transactions you have a higher likelihood of being able to identify malicious behavior quickly.
    61. Use authentication mechanisms that do not require clear text credentials to be passed over the network: A variety of authentication approaches exist for use with web based applications some involve the use of tokens while others will pass user credentials (user name/id and password) over the wire. When possible, it is safer to use an authentication mechanism that does not pass the credentials. If credentials must be passed, it is preferable to encrypt them, and/or send them over an encrypted channel such as SSL.
    62. Use least privileged accounts. The privileges granted to the authenticated party should be the minimum required to perform all required tasks. Be careful of using existing roles that have permissions beyond what is required.
    63. Use least privileged process and service accounts. Allocate accounts specifically for process and service accounts. Lock down the privileges of these accounts separately from other accounts.
    64. Use multiple gatekeepers. Passing the authentication system should not provide a golden ticket to any/all functionality. System and/or application resources should have restricted levels of access depending on the authenticated party. Some design patterns might also enforce multiple authentications, sometimes distributed through application tiers.
    65. Use SSL to protect session authentication cookies. Session authentication cookies contain data that can be used in a number of different attacks such as replay, Cross-Site Scripting or Cross-Site Request Forgery. Protecting these cookies helps mitigate these risks.
    66. Use strong authentication and authorization on administration interfaces. Always require authenticated access to administrative interfaces. When applicable, also enforce separation of privileges within the administrative interfaces.
    67. Use structured exception handling. A structured approach to exception handling lowers the risk of unexpected exceptions from going unhandled.
    68. Use the correct algorithm and correct key length. Different encryption algorithms are preferred for varying data types and scenarios.
    69. Use tried and tested platform features. Many cryptographic features are available through the .NET Framework. These are proven features and should be used in favor of custom methods.
    70. Validate all values sent from the client. Similar to not relying on client-side validation, any input from a client should always be assumed to have been tampered with. This input should always be validated before it is used. This encompasses user input, cookie values, HTTP headers, and anything else that is sent over the wires from the client.
    71. Validate data for type, length, format, and range. Data validation should encompass these primary tenets. Validate for data type, string lengths, string or numeric formats, and numeric ranges.

    SDL Considerations
    For more information on preferred encryption algorithms and key lengths, see the Security Development Lifecycle at http://www.microsoft.com/security/sdl/ .

Page 1 of 1 (10 items)