Software Engineering, Project Management, and Effectiveness
While trying to create threat model template for customers, I analyzed many threat models inside and outside Microsoft. It was insightful to see the patterns of what was useful across threat models and what was noise.
A good threat model has the following components:
A good threat model serves the following purposes:
By far, the most tangible output of the threat modeling activity is a prioritized list of vulnerabilities. These are action items for your developers and input for your testers. The developer makes a call on whether and how to fix, and the tester will test the fix.
This sample Template for a Web Applications Threat Model comes very close to showing what I've empirically seen to be useful, though there's always a gap between reality and real-time.
How do you know which techniques to use to shape your software throughout the life cycle? Start with the high Return On Investment (ROI) activities as a baseline set. You can always supplement or modify for your scenario.
Most development shops have some variations of the following activities:
· Design guidelines
· Architecture and design review
· Code review
· Deployment review
Identifying design guidelines involves putting together a set of recommendations for the development team. The recommendations address key engineering decisions such as exception management and include input from software vendor recommendations, corporate policies, industry practices, and patterns. This is a high ROI activity because it helps creates the scaffolding for the project.
Architecture and design review is an exercise in evaluating the design against functional requirements, non-functional requirements, technological requirements and constraints. It can be as simple as a whiteboard sketch or it can involve multiple documents, diagrams, and presentations. An architecture and design review is most valuable when it’s performed early enough to help shape your design. It’s less valuable when it’s performed late enough that it’s only function is to find “do overs.”
What better way to find problems than at the source? Code review is a practical and effective way for finding quality issues. While some issues can be found through static analysis tools, the advantage of a manual code review is contextual analysis (e.g. you know the usage scenario and likely conditions).
Testing is executable feedback for your software. It works or it doesn’t. If it works “right” is a bit trickier to establish. The ideal case is where you establish an executable contract between your code and the requirements, including functional, non-functional and constraints. It also helps to know what you’re looking for so you can test against it. While testing is usually optimized around functional behavior, you can tune it for quality attributes depending on your ability to set measurable objective and to define what good looks like.
Deployment review is an evaluation of your application deployed to your infrastructure. It’s where the rubber meets the road. This is where you evaluate the impact of configuration settings against runtime behavior. A deployment review can be your last actionable checkpoint before going into production. Preferably you have a working build of software earlier vs. later in your development cycle so that early deployment reviews can help reduce surprises going into production.
What makes these activities high ROI activities is that, if applied properly, they directly impact the shape of the software throughout the process. I know these are high ROI activities because I’ve seen them transform my customers’ results for the better. These activities provide key feedback and action at high-leverage points in the development process.
As a software engineer, how do you cope with information overload? I suggest domain specific categories. If the basic idea of domain specific languages (DSL) is a software language targeted at a specific area of problems, then domain specific categories (DSC) are an idea to create categories specific to an area of problems.
Here's some practical usage for the categories:
Here's practical examples illustrating domain specific categories:
In the examples above, notice how the headings are carefully chosen categories. Each category that organizes recommendations is evaluated against both the problem space, such as security or performance, the application type, such as Web application, and then the specific technology at hand.
Also notice how the baseline categories in the Web Application Security Frame become more specific and relevant in two ways:
Are there down-sides to not using a one set of categories fits all approach? You bet ... but based on the results I've seen from practitioners, I'd bank on using more thoughtful and empirical categories that are tested against how actionable and relevant they are.
The Web Application Security Frame is a set of categories you can use to scope security and improve your effectiveness. It consists of the following categories:
We created these categories during Improving Web Application Security to represent two things:1. Where are the most common mistakes made2. Where are the most actionable improvements
How do you use these to be more effective? You use these categories to focus and prioritize your security work. For example, if you know the most prevalent security issues occur in the input validation, authentication and authorization categories, you can start there.
You can immediately put the Web Application Security Frame into action. when you perform Security Design Inspections or Security Code Inspections you can use the frame to walk categories of common security issues. To do so, see the following:
For more information on the Web Application Security Frame, see Cheat Sheet: Web Application Security Frame.
Today, I cleaned up my Security Wiki on Channel9 at http://channel9.msdn.com/Security
The purpose of this Wiki is to let me share information that may not be completely fit and finish like on MSDN. This comes in handy for a few things:
What's in store going forward for Security Wiki? Well, potentially a lot. I'm still thinking through some of the possibilities. If you have things you'd really like to see more of, let me know, and I'll see what I can do.