Michael Howard and Shawn Hernan did an analysis of our bulletins and some CERT and CVE data. Their goal was to validate work they'd done on threat tre es. (Covered in the SDL book.) They were looking for classes of things that would cause us to ship updates. That’s tremendously important, so I’ll repeat it. They were looking for classes of things that would cause us to ship an update. If we wouldn’t update for it, it doesn’t exist in the chart. That’s not to say it doesn’t exist. If there’s an elevation of privilege against an external entity, well, by definition, we can’t fix it. It’s external. So is there value in calling out that risk in threat modeling? Sometimes there might be.
Breach Disclosure Laws
Sometimes organizations lose control of data that’s been entrusted to them. You might find that risk as an instance of “information disclosure by external entity.” So should we have a check there? Well, on the one hand, we don’t have any control over it - unless we hand them that information by design. In which case, perhaps we could design to hand over less. At Microsoft, we haven't added it to the chart we ask people to use, and we'll revisit that over time.
Specific Elements
Another thing you might note is that the STRIDE chart is sorta vague. A process could be an exe, a .NET assembly, or an a.out executable running on Unix v7. Each of those will be vulnerable to different instantiations of threats. Your exe or a.out will be vulnerable to simple stack smashing overflows, but the .NET assembly won’t be. As you make your elements more specific, you can provide more prescriptive guidance as to what threats to look for, and how to effectively mitigate them.
Customizing the Chart for Your Threats
The chart is centered on our needs at Microsoft. Those may not be your needs. Perhaps you’re building a threat modeling process to focus on voting. You can get much more specific about what a process is, and what threats it might come under. You might have really specific threats against voter lists (a type of data store) or vote tallies (another store).
Being able to enumerate the assets in data stores helps you motivate threats, and assess their risk and importance of fixing them. When we build a threat modeling process for the SDL, it gets used in Windows, Office, SQL, Exchange, Dynamics, and a slew of other products. We need some degree of consistency, so we can deliver consistent products and messages to our customers. We also need to encourage customization and specificity, so that the process is as prescriptive as we can make it. Doing so allows you to make it more prescriptive, appropriate and evocative for your users.
Adam again. I hope you’re still enjoying this as we hit #5 in the threat modeling series.
In my last post, I talked about how almost everyone in software draws on whiteboards regularly, and this makes it an ideal first step. It’s an ideal first step because everyone can do it, see that they’ve done it, and feel like they’re making progress.
That wasn’t quite complete. Not only do we want people to see that they’ve done it, we want to give them a way to validate their work or other people’s work. So we ask them to tell a story. We’re not asking for Shakespeare here, we’re asking them to explain how their software will be used, and to make sure that their diagram supports that story, and that it relates to their actual software.
We also give them rules of thumb (lots of rules of thumb) about things we often see wrong in diagrams:
(Larry Osterman has some in his blog post, "Threat Modeling Rules of Thumb" I helped edit those, but want to suggest additional changes. In particular, “you need to be concerned” is not actionable. “Review this carefully,” or “Focus your attention here” are more actionable. People threat modeling are already concerned.)
Good “rules of thumb” encourage flow by empowering people to make a snap decision and move along.
Adam Shostack here, with part four of my threat modeling series. This post is a little less philosophical and a lot more prescriptive than the one about flow. It explains exactly how and why I changed a couple of elements of the process. The first is the brainstorming meeting, and the second is the way trust boundaries may be placed.
The brainstorming meeting is a mainstay of expert threat modeling. It’s pretty simple: you put your security experts in a room with system diagrams and a whiteboard. Usually, you put your system designers in there, and make them promise not to strangle your experts. Optionally, you can add beer or scotch. Sometime later, you get a list of threats. How long depends on how big the system is, how well its requirements are documented, and how well your experts work together.
We like having our developers threat model. There are a lot of reasons for this. Not only do they know the system better than anyone else, but getting people involved in a process helps ensure that they buy into it.
Now this desire is great, but it leads to some issues, first and foremost is that many of the people who are now involved aren’t security experts. This means that they lack both direct experience of the process and the background that informs it. This isn’t a slam on them. I lack experience in the database design process, and I don’t have years of experience to help orient me. So I’d make mistakes designing a database, and someone who isn’t a security expert may make mistakes in security. For example, someone might try to use encryption to mitigate tampering threats. (The SDL crypto requirements cover this, and I try to gently correct them to integrity mechanisms like MACs or signatures.) This is a reality that we have to account for at the process design level.
Adding Structure to Chaos
So how does this relate to the brainstorming meeting? It’s a dramatic increase in the need for structure. Where experts may think they do better threat modeling with scotch in hand, , it certainly doesn’t lead to beginners having a flow experience. So we need a structure, and we need to provide it.
We encourage people to get started by drawing a whiteboard diagram. Almost everyone in software draws on whiteboards regularly, and this makes it an ideal first step. It’s an ideal first step because everyone can do it, see that they’ve done it, and feel like they’re making progress.
The core mechanism we’ve used to provide it is the STRIDE/element chart. (I’ll talk a lot more about its origins and limits in a few posts, but for now, let’s pretend it’s gospel, and enumerates all possible threats.) Given this gospel, it becomes possible to step through the threat modeling diagram, “turn the crank,” and have threats come out. “Item 7 is a data flow? Let’s look for T,I and D.” (Tampering, Information disclosure, and Denial of service.)
Similarly, we have four ways of addressing threats – redesign, standard mitigations, new mitigations, and risk acceptance. We have training on mitigating threats, we have explanation of why and when to use each (and they’re presented in a preferred order).
Lastly, we provide advice about how to validate the threat model and it’s relation to reality.
Between these four steps and the hamster wheel which ties them together, we give people the structure in which they can take on the process. The other thing I wanted to address is how we respond to consistent “errors” that we see.
Where Trust Boundaries Show Up
We used to give people clear guidance that trust boundaries should only intersect with data flows. After all, you can’t really have a process that’s half-running as admin, and half as a normal user. Logically, you have two entities. And people kept drawing trust boundaries across processes and data stores. It drove me up the wall. It was wrong.
As people kept doing it, I decided to swallow my pride and accept it. I now tell people to put their trust boundaries wherever they believe one exists. And they’ve continued exactly as before, but I’m a lot happier, because I’ve found a way to help them draw more detailed diagrams where they need them. Which includes anywhere a trust boundary crosses a process or data store. They’re happier too. No one is telling them that they’re wrong.
I was going to title this post “Lord grant me the strength to change the things I can, the courage to accept what I can’t, and the wisdom to know the difference,” but, first, it’s too long, and second, if we started that way, it would be wrong to add beer or scotch.
Adam Shostack again, with the third in our series on threat modeling. In this post, I want to explain one of the ‘lenses’ that seemed to help us focus threat modeling, and how I’ve applied it.
The concept of flow originated with Mihaly Csikszentmihalyi. It refers to a state where people are energetically involved with what they’re doing. Seeing this a few times during threat modeling sessions made it obvious when it was missing, and it was missing often. I set out to address some of the elements that seemed to make threat modeling harder. The Wikipedia article (currently) has a good list, so I’ll focus in on a few of them:
Let’s take these one at a time.
Clear Goals
Giving people clear goals is important because it helps take them from worrying about what your goals mean to worrying about how to achieve them. Without clear goals, it’s very challenging to get into the spirit of anything, whether playing a game or shipping an operating system. As goals go, “think like an attacker” and “brainstorm” aren’t up there with “A PC on every desktop.” The lack of a clear answer to “how do I know I’m done with my diagrams” or “how many threats should I have,” or “what is a good threat model” made it hard for people to do well, and just as importantly, even people who were doing well weren’t sure that they were doing well. Which brings us directly to the next point:
Direct and Immediate Feedback
When people have direct and immediate feedback, they can adjust their activities relatively easily. If you’re cooking from a well written recipe, at each stage, you have some indicators that things are done. “Are the onions brown?” “Is the water boiling?” “Does the pasta stick to the wall?” This is true of any skill we’re learning. With threat modeling, when there is no expert in the room, feedback can take weeks or months to come back. As we rolled threat modeling out at Microsoft, it was possible for an entire threat model to be cooked without any course correction. We’re doing better now - there are more experienced people around, better documentation, and more focus on how to self-check. But at one time we asked people to engage in really complex tasks from the get go, without trying to achieve a…
Balance Between Ability and Challenge
Almost by definition, when you’re new at something, you don’t have a lot of skill around it. If the new activity is something highly complex, like speaking a foreign language, you’re likely to be anxious. And once you are skilled, simple tasks, like talking to a beginner, are often boring. Which brings us to the chart, which I’ve redrawn from Csikszentmihalyi’s book, Flow.
The ideal learning experiences involve a balance of challenge and reward, each of which grows as you learn. For example, when you buy Halo, there’s a training mission that allows you to run around doing simple tasks, learning the controls. If we dropped people into the live online game without any practice, they’d get slaughtered, and that’s bad for everyone. The newb is frustrated, and the experienced player is bored. (In fact, there’s pretty explicit acknowledgement of this in the Wired article on Halo.)
So looking at the chart of skills versus challenge, there’s a channel in the middle—it’s the flow channel. We’d like to guide people there, by giving them challenges that are in line with their skills. Once people are there, we want to prod them from point A to point D, where there are larger challenges, and skills that allow people to address those challenges. Sometimes, we want them to leave their comfort zone and learn new things. To take on new challenges (going to B) and learn from them, going to D. If we don’t challenge people, they get bored, and it’s less likely that they’ll learn. (This obviously isn’t a complete model of learning, but it flows out of the discussion.)
We want to do that for Halo, and we want to do that for threat modeling. To date, this has been implicit. We’ve simplified the process, in part in the interest of getting people out of the anxiety zone, and into a place where they can feel comfortable and then learn. Over time, we’ll be challenging and encouraging people to do more complex tasks as part of SDL threat modeling.
This assumes that threat modeling is a frequent enough activity that people develop skills around it. I think that’s an ok assumption, as long as we design good refresher and update training and documentation.
Focused Attention
If you’re reading this blog post while talking on the phone and petting your dog, you’re going to get less out of it than if you do nothing else while reading it. Similarly, when you’re learning a new skill or performing a hard task, you need to focus your attention. Related to this is moving around. Getting a little blood flow helps you be energetic and engaged. This has been one of the hardest elements of flow to add to threat modeling. Many people here multi-task intensively, with laptops or blackberries always at hand. Challenging that habit seems like a challenge to the way people work, and I’ve shied away from issuing that challenge.
Why flow?
One final point I’d like to address is that flow is one of many frameworks we could use to address problems in threat modeling. My choice to use it was a driven by its striking absence. Flow isn’t the only framework for thinking about these things, and there are some criticisms associated with trying to shoehorn everything into flow. For threat modeling, it just seemed to well…flow.
Adam Shostack here, with the second post in my series on the evolved threat modeling process. To summarize, what I’ve tried to achieve in changing the process is to simplify, prescribe, and offer self-checks. I’ll talk in the next post about why those three elements are so important to me. For now, let me describe the process.
One of the largest changes that we’ve made is to a simplified process (and diagram). I like to say that this looks pretty much like every other software process diagram you see today. That’s intentional. There’s only so much we can expect people to take away from a class, and making this simple and familiar helps ensure there’s room for the other important parts.
First, the “process hamster wheel,” (with apologies to Yankee Group analyst Andy Jaquith):
Now that you’ve seen the wheel, I’ll briefly describe the steps:
a. Draw a diagram of your software. We encourage use of the DFD formalisms, which Larry Osterman describes in this post. Essentially, the elements are External entities (anything outside your control) Processes (running code) Data stores (files, registry entries, shared memory, databases) Data flows (which connect all the other elements) b. Draw trust boundaries between components. You can do this on a whiteboard, in Visio, or in one of the specialized threat modeling tools we’ve built. (A trust boundary is anywhere that more than one principal can access an object, such as a file or process.) c. If your trust boundary crosses something which isn’t a data flow, you need to break it into two logical elements, or draw a sub-diagram with more details. (This is different advice: we used to tell people trust boundaries could only cross data flows. People drew them anywhere that felt right. We decided to go with what people were doing—there was important information in what they were expressing.) d. If you need more details to express where trust boundaries are, add an additional diagram. e. When you don’t have any more trust boundaries to draw, you’re done. f. If a diagram doesn’t have any trust boundaries, you may have drawn too many details. 3. Identify Threats using STRIDE/element
a. Draw a diagram of your software. We encourage use of the DFD formalisms, which Larry Osterman describes in this post.
Essentially, the elements are
b. Draw trust boundaries between components. You can do this on a whiteboard, in Visio, or in one of the specialized threat modeling tools we’ve built. (A trust boundary is anywhere that more than one principal can access an object, such as a file or process.)
c. If your trust boundary crosses something which isn’t a data flow, you need to break it into two logical elements, or draw a sub-diagram with more details. (This is different advice: we used to tell people trust boundaries could only cross data flows. People drew them anywhere that felt right. We decided to go with what people were doing—there was important information in what they were expressing.)
d. If you need more details to express where trust boundaries are, add an additional diagram.
e. When you don’t have any more trust boundaries to draw, you’re done.
f. If a diagram doesn’t have any trust boundaries, you may have drawn too many details.
3. Identify Threats using STRIDE/element
For each element in your diagram, consider threats of the types indicated in this chart. (We’ll come back to the chart’s origins in a later post.)
There’s an important mis-conception we often see, which is that STRIDE is appropriate for use as a classification system. It’s really hard to use STRIDE to describe attacks—the impacts blend together really quickly. The most valuable use of STRIDE is to help people think about how threats have impacted elements of a design in the past. That is, it’s a framework for finding threats, not for describing them. “What if someone spoofs this host?” 4. Mitigate Here on the SDL strategy team, we love threat modeling. We know that not everyone feels that way, and we ask teams to threat model so that they can find and mitigate threats. A threat model document with great diagrams and lots of threats isn’t very useful if you don’t take the key step of addressing the issues you find. There are four ways to do that: a. Redesign to eliminate threats. b. Use standard mitigations, such as those provided by OS features, to mitigate threats. c. Invent new mitigations, understanding that this is a subtle art. d. Accept risk, when allowed by the SDL
There’s an important mis-conception we often see, which is that STRIDE is appropriate for use as a classification system. It’s really hard to use STRIDE to describe attacks—the impacts blend together really quickly. The most valuable use of STRIDE is to help people think about how threats have impacted elements of a design in the past. That is, it’s a framework for finding threats, not for describing them. “What if someone spoofs this host?”
4. Mitigate
Here on the SDL strategy team, we love threat modeling. We know that not everyone feels that way, and we ask teams to threat model so that they can find and mitigate threats. A threat model document with great diagrams and lots of threats isn’t very useful if you don’t take the key step of addressing the issues you find. There are four ways to do that:
a. Redesign to eliminate threats.
b. Use standard mitigations, such as those provided by OS features, to mitigate threats.
c. Invent new mitigations, understanding that this is a subtle art.
d. Accept risk, when allowed by the SDL
5. Validate
There are two levels of validation. The first is within each stage, the second is a validation pass at the end of the process. That end of process validation entails:
a. Make sure that the diagrams are up-to-date and accurate
b. Ensure that you have STRIDE threats per data flow that crosses a trust boundary, and for each element that such a trust boundary connects to
c. Make sure you’re mitigating each threat
i. You have a bug filed per threat that you want to mitigate. The bug should be of the form “attacker can do X. Proposed fix: Y.” You might include tradeoffs you’re making, and possibly have test plans in the bug, if you include those.
ii. You have a valid reason for each non-mitigated threat not being mitigated.
iii. All threats are in class i or ii.
5.a. On change, re-validate
This hamster wheel has a very intentional brake on it: the word change, above validate. What that means is you want to go through the process again when you make changes that need to be on the diagram. Checking to see if your diagrams change is a relatively simple check that allows people to track changes against their threat model as their design iterates.
In the next post, I’ll talk about the reasoning behind the design, and offer up some process tools that anyone can use to make a process more user-friendly.