About 2.5 years ago, I wrote a series of articles about how we threat model at Microsoft, about 18 months ago, I made a couple of updates to it, including a post about why we threat model at Micrososoft, and a review of how the process has changed over the years.
It's threat modeling time again in my group (it seems to happen about once every 18 months or so, as you can see from my post history :)), and as the designated security nutcase in my group, I've been spending a LOT of time thinking about the threat modeling process as we're practicing it nowadays. It's been interesting looking at my old posts to see how my own opinions on threat modeling have changed, and how Microsoft's processes have changed (we've gotten a LOT better at the process).
One thing that was realized very early on is that our early efforts at threat modeling were quite ad-hoc. We sat in a room and said "Hmm, what might the bad guys do to attack our product?" It turns out that this isn't actually a BAD way of going about threat modeling, and if that's all you do, you're way better off than you were if you'd done nothing.
Why doesn't it work? There are a couple of reasons:
So how do we go about threat modeling?
Well, as the fictional Maria Von Trapp said in her famous introductory lesson to solfege, "Let's start at the very beginning, A very good place to start"...
One of the key things we've learned during the process is that having a good diagram is key to a good threat model. If you don't have a good diagram, you probably don't have a good threat model.
So how do you go about writing a good diagram?
The first step is to draw a whiteboard diagram of the flow of data in your component. Please note: it's the DATA flow you care about, NOT the code flow. Your threats come via data, NOT code. This is the single most common mistake that people make when they start threat modeling (it's not surprising, because as developers, we tend to think about code flow).
When you're drawing the whiteboard diagram, I use the following elements (you can choose different elements, the actual image doesn't matter, what matters is that you define a common set of elements for each type):
You build a data flow diagram by connecting the various elements by data flows, inserting boundaries where it makes sense between the elements.
Now that we have a common language, we can using it to build up a threat model.
Tomorrow: Drawing the DFD.
Threat modeling? Larry, I learned Windows system programming from hacking. Your first threat is that the code is running on a computer. No, I'm not being too paranoid or irrational. All data and code is suspect, including internal data that you think never touches the outside. Is your process secure? How about the one next to it? Did the sysadmin load a Sony CD? How about a CD that I made?
Threats come from outside and inside.
Wanna know real paranoia?
Here's some books in my library:
"The Art of Computer Virus Research and Defense" by Peter Szor
"Silence on the Wire" by Michal Zalewski
"Hacker Disassembling Uncovered" by Kris Kaspersky
"Rootkits : Subverting the Windows Kernel" by Greg Hoglund
"Exploiting Software" by Hoglund and McGraw
p.s. - Good to see you over at the building 86 cafeteria.
Brian, threats don't come from programs, they come from data. I'm not aware of a single exploit that wasn't spread by tainted data (I may be wrong, but I'm pretty sure about that). That's why threat modeling is so important.
I also have Silence on the Wire, it's ok (not great, but ok - i reviewed it couple of years ago). I don't have the others, but they're on my amazon wishlist.
You didn't mention Writing Secure Code and my current "waiting for the compile to finish" book, which is (I think) "Testing for security".
Looking forward to this. Go, Larry!
What are your opinions on the Microsoft TAM tool? I use it and find it quite good. It allows me to see who, when and where interactions with my application are happening. Also I get genuinely useful information out of it such as data flow diagrams, call flows etc. It's available for download at: http://www.microsoft.com/downloads/details.aspx?familyid=59888078-9daf-4e96-b7d1-944703479451&displaylang=en
"Your threats come via data, NOT code."
I disagree with this, one should be able to feel secure about running programs on their own computer without worrying what it could do to their settings, system, and files IMO. That any random program can install their own drivers, boot sectors and hooks from any account without having any notifications to the user or any way to stop seems to be something that can't be fixed with any amount of threat modeling. Yes, people are going to download h4xm3.exe from virus.com and run it, and they will enter their password into the escalation dialog and click OK. Forbidding programs to do dangerous things forbids them from doing clever things, yes, but these are the times where there is just as much software trying to do dangerous things for evil as there are for good. And until they are stopped, slashdot will never shut up.
Derek, the old tool's pretty good. I wasn't aware it was available for public download, or I'd have mentioned it, thanks for that info.
Triangle: You're totally right. But no amount of threat modeling or security will stop the user who downloads h4xm3.exe from virus.com. Allowing the user runs h4xm3.exe, it's NOT a security hole. The computer is doing exactly what the user asked it to do.
Threat modeling is a tool for finding security holes. It's not a tool to mitigate malicious programs. Having said that, the threat model for an FTP client or a Web browser might include the fact that it needs to sandbox the file downloaded from the internet (and in fact that's where the "Mark of the Web" came from - the threat models for IE indicated that there was a risk associated with downloading unsigned code from the internet, the mitigation was to add the MotW to all downloaded programs - it's a mild sandbox that IE applies to let the user know that there might be a risk).
Note to self: Make sure you include this point in the wrapup post.
You do know "The Sound of Music" was based on a true story don't you?
Granted, I imagine they sang fewer songs, but there was a real Von Trapp family. And a real Maria von Trapp ( http://en.wikipedia.org/wiki/Maria_von_Trapp )
Massif: Of course I know that Maria von Trapp is real (we actually have one of the von Trapp family albums in our music library).
That's why I refered to her as the "fictional" Maria von Trapp. I suspect that the real Maria never used a song like "Do Re Me" to teach Solfage.
In my last post , I listed off some of the elements that make up a threat model. Now that we have a common
Wait until you hear the singing of the fictional Larry Osterman.
Adam Shostack here. I've been meaning to talk more about what I actually do, which is help the teams
Nice post Larry. Good stuff and fun to read every 18 months ;)
We as an industry are still trying to figure this here threat-modeling thing out, and so it is good to get your perspective.
I've been writing a LOT about threat modeling recently but one of the things I haven't talked about is
Adam Shostack here. I said recently that I wanted to talk more about what I do. The core of what I do