Threat Modeling Again, Threat Modeling Rules of Thumb

MSDN Blog Postings » Threat Modeling Again, Threat Modeling Rules of Thumb

MSDN Blog Postings » Threat Modeling Again, Threat Modeling Rules of Thumb — Fri, 21 Sep 2007 21:02:26 GMT

PingBack from http://msdnrss.thecoderblogs.com/2007/09/21/threat-modeling-again-threat-modeling-rules-of-thumb/

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Bill Stepanov — Tue, 25 Sep 2007 01:52:06 GMT

I don't want to ruin the party, but the security model is flawed.

First, if there is any possibility of a flaw in a program, all the space (user space or kernel space) is wasted. At least that's how "the bad guys" (as you call them), do it. They first find one little function that didn't check the size of the array and overwrite your data. Then they find ways to execute the overwritten data.

You mention that you tell developers to "check". Sorry, but that won't cut it. You need to use abstraction layers (as used in Java) to remove from the language the very possibility of overwriting data. Educating developers is a lost battle, because they make mistakes all the time, and they are so tired and so full of ideas coming from the marketing department, that they can't retain in their heads all your suggestions.

Besides, microkernel technologies and safe languages like Java have been invented to solve these issues, so why spend so much time reinventing the wheel?

re: Threat Modeling Again, Threat Modeling Rules of Thumb

nksingh — Tue, 25 Sep 2007 02:46:08 GMT

@Bill:

Java and safe languages are not yet ready to be used for system-level software. They have hard-to-control failure conditions that make them unreliable in low-memory or other resource-constrained situations. At least this is true in the CLR (may or may not be true in Java).

Not to forget that the long legacy of systems out there have been written in C and other unsafe languages. Microsoft would be reinventing the wheel even more if they were to move to a microkernel or to a safe OS (MSR is doing this with Singularity).

Lastly, buffer overflows are checked for automatically in MS compilers. Look up PreFAST and the /analyze compiler switch sometime. These tools aren't perfect, but they do catch a number of bugs. For another interesting tool that is being applied at Microsoft, look up "Automated Whitebox Fuzz Testing" in your favorite search engine.

Threat modeling, as Larry describes it, is just a way for Microsoft to focus more manual resources on reviewing sensitive code for bugs which the automated tools may have missed.

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Jim Lyon — Mon, 01 Oct 2007 19:47:11 GMT

Bill's comments above in some ways reiterate Larry's rule number 2: If an attacker has already gotten you to run his code, you don't care what exactly that code can do -- you've already lost.

Therefore, you need to be really concerned (paranoid, actually) about threats that allow an attacker to run arbitrary code without your consent. But it's a losing battle to try to stop that arbitrary code, once running, from doing evil things.

Some final thoughts on Threat Modeling...

Larry Osterman's WebLog — Mon, 01 Oct 2007 19:54:31 GMT

I want to wrap up the threat modeling posts with a summary and some comments on the entire process. Yeah,

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Bill Gates — Mon, 01 Oct 2007 22:35:10 GMT

Larry, keep up the good work, I wish we had more people like you in Redmond.

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Ka-Ping Yee — Mon, 01 Oct 2007 23:47:50 GMT

Please stop using, stop quoting, and stop teaching "Immutable Law #1." It's simply wrong, and it's deceptively named.

One of the important jobs of an operating system is to isolate and protect applications from one another. To assume the so-called "Immutable Law #1" is to pretend that this responsibility doesn't exist. Yes -- it is an assumption, not a law, and to call it "immutable" is to mislead the reader into accepting a broken security model instead of demanding better.

I'm sure you've heard of encapsulation? Defense in depth? How about the principle of least privilege? The thinking behind Law #1 -- that when a user runs a program, that program automatically gets full rights to pull all the user's privileges out of thin air and exercise them in whatever way it wants -- runs counter to all of these fundamental security concepts.

A revision of your Law that's somewhat closer to the truth would be something like: "If a bad guy can persuade you to run his program on your computer, and your operating system allows that program to damage the system, other programs, or your data, it's not your computer anymore." The operating system does not get a free pass.

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Jeff Haynes — Tue, 02 Oct 2007 05:24:00 GMT

Although you mentioned the codec example in your last post, I somewhat fail to see how the security model addresses this case. And, it seems that in some respects, as we use more flexible technologies (e.g. XML) that this is the more prevalent case. I like to think of various components as being layer-agnostic in the sense that, of course, a media player shouldn't have to understand the stream that the codec does. To that end, it seems that the security model should be aspect oriented...but I don't think that it is. Maybe I'm missing something.

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Frank — Tue, 02 Oct 2007 08:19:22 GMT

Ka-Ping, you need to re-think your logic.

If an OS had zero bugs, then you couldn't write an exploit to make the OS do something that wasn't intended.

However, all OSs have at least one bug (perhaps unknown) that will permit a hacker to run arbitrary code at an elevated priv.

Thus, the statement as the author wrote it is accurate. I do agree that your statement makes sense in a 8th grade discussion on computer security however. That isn't a slam, it just glosses over real-world facts.

Every OS has at least one hole lurking somplace.

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Ka-Ping Yee — Tue, 02 Oct 2007 12:33:36 GMT

Frank: I believe you're making an invalid logical leap — from "no operating system is perfect" to "it doesn't matter whether operating systems provide isolation". If "all OSs have at least one" privilege escalation bug, then why provide privilege levels at all? Why even bother to provide any OS security? Clearly, because design intentions still matter — the possibility of an imperfect implementation doesn't imply that you can ignore them.

Larry is talking here about design. He's using "Law" #1 as a reason to ignore a class of threats, and thereby to ignore the possibility of defense in depth. It's as if he's forgotten that software components can be written using the principle of least privilege, to limit the damage that can happen when problems do occur.

You say that the statement of "Law" #1 is accurate. I propose "Law #11", then: "Whenever you choose to visit any website, you are making a decision to turn over control of your computer to Internet criminals." Now, is that accurate? Would you have any objections to this statement appearing, say, as a newspaper headline?

Since you could argue that every browser has at least one security weakness and every website can be defaced, Law #11 is precisely as accurate as Law #1. But obviously, Law #11 is not how browsers are supposed to work. It is infeasible to use a browser in a way that is consistent with Law #11. If you really believed it, you would never visit any websites. And likewise with Law #1. If you really believed it, you would hardly ever run any programs on your computer (which would include never visiting any websites, or disabling Flash, Java, and JavaScript).

Law #1 is accurate only in the same way that Law #11 is accurate — that is, in a way that is not useful to anyone. It neither gives users a model of computer operation that they can apply in practice, nor gives developers a technical fact that helps them design better programs.

But "Law" #1 is presented as an "Immutable Law" — a definition of how computers must and will always work. Culp introduces his "Laws" thus: "Don't hold your breath waiting for a patch... . It isn't possible for Microsoft—or any software vendor—to 'fix' them, because they result from the way computers work." Presenting it in such a way is exactly as misleading as presenting Law #11 as an "Immutable Law". Would you be satisfied if the Firefox security team responded like this to an arbitrary code execution vulnerability? "Don't hold your breath. It isn't possible for us to fix this problem, because it results from the way computers work."

re: Threat Modeling Again, Threat Modeling Rules of Thumb

Alan Karp — Fri, 05 Oct 2007 18:44:19 GMT

Ping is right. There is no reason every program I run needs all my privileges. I always launch my browser in a restricted user account using a variant of runAs. More than once a malicious script has run that didn't do anything worse than make me delete the account and create a new one. In other words, a bad guy persuaded me to run his program on my computer and it was still my computer. Sounds like Law #1 isn't so immutable after all.

(Full disclosure: Ping and I worked together for a while.)

Threat Modeling Self Checks and Rules of Thumb

The Security Development Lifecycle — Tue, 23 Oct 2007 00:04:02 GMT

Adam again. I hope you’re still enjoying this as we hit #5 in the threat modeling series. In my