Ralph here, I wanted to let everyone know that Crispin Cowan has just started his own blog. Keep an eye on it for some great posts in the future.
Adam Shostack here. I spoke at Toorcon this past weekend on "SDL Threat Modeling: Past, Present and Future." I wanted to share my slides to help clarify a bit about where SDL threat modeling is and why, and a bit about where we're going.
(Click on the post title, and you'll see an attachment in the per-post page.)
A colleague sent me a link to a blog post from a couple of days ago: Pete Lindstrom of Burton Group blogged that Microsoft's SDL has Saved the World!! raising concerns about Microsoft using vulnerability counts as a means to measure security improvement resulting from the SDL.
I've raised this topic before, in my blog post The First Step on the Road to More Secure Software is admitting you have a Problem. Here are two pertinent quotes from that blog post of Feb 21st:
"Let's face it, no-one can agree on any measurement of security without getting knotted up." "Measuring security is a real challenge, and while we may debate the merits of vulnerability counts, right now it's the only concrete metric we have."
"Let's face it, no-one can agree on any measurement of security without getting knotted up."
"Measuring security is a real challenge, and while we may debate the merits of vulnerability counts, right now it's the only concrete metric we have."
These comments are very important because there appears to be no more widely accepted security metric today, and while no perfect metrics exist, it's useful to have some objective data when trying to discuss this complex subject. Our customers constantly tell us to reduce the number of patches they need to apply to their products once in deployment. It costs them time and money to deploy security updates. The primary metric that matters to customers is the number of security updates they need to apply. And the only way to reduce the number of updates is to systematically reduce the number and severity of vulnerabilities in the code in the first place - that's the goal of the SDL.
In my mind there are two kinds of vulnerability metrics, and I alluded to both in my prior blog post. The first vulnerability metric compares Microsoft to Microsoft, in other words we compare Windows XP to Windows Vista, SQL Server 2000 to SQL Server 2005 and so on. We use this metric while a product is being built; we track incoming security bugs for the prior version of the product to see how we're faring with the current version in development. Fewer vulnerabilities in the product under development is a good sign that the product will fare better in the real world.
The second vulnerability metric compares Microsoft with other vendors. This is an interesting metric, but our group is full of engineers, so we pay little attention to the figures because there is very little we can influence. The post Mr. Lindstrom refers to cited those vulnerability figures I use to point out that other development organizations need to admit they have a secure development problem. Looking back at the figures we cited, it's pretty clear that the sheer volume of security vulnerabilities supports our assertion, regardless of the subtleties of security metrics. More about this below.
Mr. Lindstrom states:
"Microsoft has systematically hired and/or contracted with every one of their most vocal critics (and most seasoned bugfinders) to do the work behind the scenes and they don't count those vulns!"
But in making this assertion, he's saying the vulnerabilities we remove (and do not add to the code in the first place) as part of the SDL process should be counted as though they were part of the product after we shipped it. We don't count vulnerabilities that don't affect customers, regardless of the vendor.
We hire some security researchers to be part of our teams executing the SDL because they're among the best and brightest at performing component design reviews, code reviews, black box testing and other security procedures needed to make our products more secure. Everyone in the industry covets their expertise because it's in short supply, and so we've competed to bring in the most capable people - as employees, contractors and advisors. These experts, helping us execute the SDL, have helped Microsoft eliminate vulnerabilities before our products ship, which naturally means lower vulnerability counts and improved security for our customers. In addition, bringing in researchers helps us to better understand what the community is thinking about today so we can anticipate and head off the problems of tomorrow.
To put it bluntly, we hire security researchers to help protect customers. Period.
While we've made an effort to hire the right researchers to help us improve the security of our products, it's far from the case that we've hired "every one of their [our] most vocal critics." There are still plenty of security researchers who are looking at our products and reporting the vulnerabilities they find after the products ship.
We have found that the training and principles of SDL have indeed significantly improved the products Microsoft engineers create. You improve security by expending effort on improving security. We have seen the evidence of this in the fewer customer updates being released against that code. When applied correctly, the SDL development principles prevent vulnerabilities from entering the final code in the first place. This last point is very, very important: you can't count a bug that was never created; the goal of the SDL is to not create the bugs in the first place.
Some of the many SDL principles that reduce or mitigate security bugs include:
So, to answer Mr. Lindstrom's question:
"Could it really be that SDL has done nothing to help MS developers write better code?"
Without a doubt, the SDL has helped Microsoft developers write better and more secure code.
However, we are still faced with the question whether vulnerability-based metrics are a valid way to measure progress of the SDL. In my opinion, vulnerability counts are a useful metric, but imperfect. We'd welcome Mr. Lindstrom's (and anyone else in the security community) sharing with us the metrics they would use to measure security-related success and how to calculate them. While the notion of what constitutes a "real, objective metric" is often based on individual preference, I think both the efficacy of SDL and the industry as a whole would benefit from this discussion.
Interestingly, Mr. Lindstrom has at times pointed to vulnerability counts as an interesting (but not perfect) metric.
One final comment: If the Microsoft product security vulnerability trend was in the other direction, up and not down, would industry observers claim SDL is failing? I think so.
The SDL works; it's not perfect, we've never said it is, but it's making our customers happier because they have fewer security updates to apply. Not zero, but fewer. And we are always looking for ways to improve how we measure the progress of SDL.
As we've been saying all along, industry dialogue is key - so let us know what you think.
Hi folks, Eric Bidstrup here.
Last week at RSA, Microsoft Chief Research and Strategy Officer Craig Mundie spoke and outlined a proposed vision for “End to End Trust.” Much has and will be written on that, and additional information and discussions can be found at the End to End Trust portal http://www.microsoft.com/endtoendtrust. In many ways, Craig’s talk was very unusual for Microsoft’s presence at RSA in that it wasn’t a big new product announcement, nor was it evangelizing a new technology or platform to innovate upon. Rather, it was a aimed at kicking off a dialogue by describing some of the current challenges and barriers we see to achieving a more trusted and privacy enhanced Internet, and some of our ideas on how both industry and society might be able to start a productive dialog about collaborating toward that end. Make no mistake: this is tough stuff. This needs to be an industry-wide, long-term effort, and it’s about more than just technology. Enabling true End to End Trust will require that we continue to build on technology progress while aligning those innovations more closely with social, economic and political forces.
Along those lines, I wanted to take a few moments and comment on how SDL factors into that broader discussion on trust. Allow me to draw some analogies with some of my prior work…
In the late 1990’s, I was not yet working on computer security but on computer speech recognition and speech synthesis for Microsoft. Having an engineering background, I was (and still am) very interested in the opportunities and possibilities enabled by freeing people from computer keyboards and mice and allowing them to interact with computers in one of the same ways we interact with each other – by voice. Speech recognition was, and still is, largely assessed by a key metric of “what percentage of words spoken by a person did the computer correctly understand?” Nirvana for speech recognition is 100 percent accuracy (defined as “the computer correctly understood all of the words spoken”) with any audio stream (even with a microphone far away from a person in a noisy room) with an unlimited vocabulary (regardless if I am discussing sports using slang or detailed technical terminology) in any spoken language/dialect. State of the art of speech recognition technology today is not 100 percent accurate within the parameters I described, but let’s pretend for a minute that it is – then what? If you start thinking more deeply on this subject, you can quickly see that many other pieces of the puzzle are needed to realize the goal of “allowing people to interact with computers in one of the same ways we interact with each other – by voice. Natural Language Processing and designing an effective Voice User Interface (VUI) are two of the first major challenges encountered when trying to realize the broader vision of enabling people to interact with computers via voice. These are hard problems that I hope to see significant progress on in my lifetime. However, analyzing an audio stream and converting into some format (words or otherwise) is a fundamental requirement necessary for speech recognition. Yet, it’s also insufficient to realize the broader vision.
Some of you reading may be thinking “But wait Eric, this is a security blog so why are you rambling on about your former roles working on speech recognition?” Well, there is an analogy I’m trying to draw. The point I’ve been leading up to is that the SDL plays a similar role in the context of realizing the broader “End to End Trust” vision. Having software that operates securely without exposing systems or data to unnecessary risk is a fundamental requirement in order for people to trust their computers and software. Yet, that alone is insufficient to enable confidence and trust. As Scott Charney noted in the “End to End Trust Paper:”
“There remained, however, other more specific threats not well addressed by SD3 or Defense-in-Depth. For example, spam does not normally exploit vulnerabilities, nor would one turn off mail by default. There is also very little a specific user or enterprise can do to prevent a distributed denial-of service attack from a botnet. As a result, Microsoft started working on threat mitigations for specific issues. With regard to phishing and spam, for example, it engaged in broad consumer education campaigns and worked on developing technological solutions such as phishing filters and SenderID. For both phishing and botnets, Microsoft began working more extensively with law enforcement to identify phishers and botnet herders in an attempt to create deterrent to such activity, even though the deterrent effect is limited by the current environment because it is hard to find offenders, and criminal penalties may be applied without sufficient force.”
In the non-computing world, even if I keep my house, car, and other valuables under lock and key, I still am at risk of being victimized by criminal activity through no fault of my own. However, a broader set of societal constructs help offer improved assurances that if I don’t live careless or recklessly I will largely remain safe and secure. Note I said “improved.” Society is still not perfect; crime still exists and it always will! The online world is no different. The online world has not yet been around quite as long as human society, it too needs help in developing improved assurances – assurances that ensure I will largely remain safe and secure given I don’t live carelessly or recklessly. These assurances can’t be provided by any single vendor. They require collaboration from all of industry, and indeed society. Craig Mundie’s talk aimed to start a dialogue about how to evolve our online society to be a safer place, where devices and software enable people to make more effective trust decisions and take control over whom and what they trust online.
The creation of a more trustworthy Internet will benefit all of society, and an open dialogue among its members is critical component of achieving this. Feel free to go to http://forums.community.microsoft.com/en-US/EndToEndTrust/threads/ and chime in with your thoughts. As Scott Charney noted “"… if we want the internet to reach its full potential, we need a safer, more trusted online environment."
Hello all – Dave here…
I am currently at RSA and decided to take a few moments to blog about some updates to the Security Development Lifecycle. Admittedly, I have been “radio silent” on the blog for awhile – for those that know me, that’s usually a warning signal that I am cooking something up…
Anyway, back when we first started this blog we promised that you would see more about the particulars of the SDL – and I think we have done a reasonably good job. Michael Howard has written some pretty interesting pieces on a wide variety of subjects; bug post-mortems, philosophical notes and the like. Adam Shostack did a fabulous job on the threat modeling series; Eric Bidstrup took a deeper look at the perceived vs. real benefits of the Common Criteria and I have penned a moderately well received screed or two from time to time.
However, one of the common requests (complaints?) that I have heard is that we have been short on the real “guts” of the SDL – that is to say, a point by point examination of how to apply the SDL. I would argue that Michael and Steve’s book on the SDL is a good primer on how to get started. I think Jeremy Dallman added more momentum with his “Crawling toward SDL” post, giving some practical advice on how to approach the issue of secure software development from scratch.Despite these efforts I have heard that people still want more detail – some folks are curious about how an organization the size of Microsoft programmatically drives culture change; others are looking for guidance that can be repurposed for their own organizations and finally, some folks are convinced that we are deliberately holding back some security “secret sauce” for some reason. Go figure.
With that, let me cut to the chase. Today, we have made the Microsoft Security Development Lifecycle, version 3.2 available for your perusal on MSDN. This has been in the works for quite awhile and has involved a ton of folks in SEC and TWC putting in a lot of hours and resources into getting this published (props to Ziv Fass and Jed Pickel!).
As you can probably guess, this is not an exact duplication of the SDL for a number of reasons – but it’s pretty darn close. Given that caveat, allow me to illustrate a few points about this guidance...
Finally, in reference to the third point above, I am compelled to say the following. (LEGAL DISCLAIMER ALERT – those with weak constitutions should avert their eyes):
The following documentation on the Microsoft Security Development Lifecycle, version 3.2 is for illustrative purposes only. This documentation is not an exhaustive reference on the SDL process as practiced at Microsoft. Additional assurance work may be performed by product teams (but not necessarily documented) at their discretion. As a result, this example should not be considered as the exact process that Microsoft follows to secure all products.
This documentation should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented herein. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, OR STATEMENTS ABOUT APPLICABILITY OR FITNESS OF PURPOSE FOR ANY ORGANIZATION ABOUT THE INFORMATION IN THIS DOCUMENT.
For the morbidly curious: Yes, I wrote that; yes, it passes legal muster; no, I am not a lawyer, nor do I play one on TV. : )
So there you have it – Microsoft SDL 3.2.
There are a few sharp eyed souls that read the blog and will wonder about our publishing schedule for updates – it’s no secret that we examine the SDL every six months and either add new requirements to meet emerging threats or deprecate old guidance. It has been described by some as analogous to “changing tires on a moving vehicle.” Let me say now that we will NOT be publishing new SDL guidance on a six month schedule for the foreseeable future – we’ll settle on a reasonable publication frequency and hopefully accelerate over time.
I welcome your thoughts and comments...
Hi everyone, Bryan Sullivan here.
Here’s a quiz for you. Quick, tell me what page the following URL is going to take you to:
http://www.somebank.com/welcome.aspx?p=http%3A%2F%2Fwww.somebank.com%2Flogin.aspx
If you answered “www.somebank.com/welcome.aspx”, you’re right. But if you answered “www.somebank.com/login.aspx”, you’re also right. How can both of these be true? Because the page www.somebank.com/welcome.aspx redirects the user to whatever location is specified in the “p” parameter of the querystring, highlighted below:
This may look pretty innocent to you. But what if I sent you an email claiming to be from “SomeBank,” telling you that your account was under investigation, and that you needed to login at the following link to confirm your good standing: http://www.somebank.com/welcome.aspx?p=http%3A%2F%2Fwww.evilphishers.com%2F
Now you, being a technologically savvy reader of the SDL blog, probably wouldn’t fall for such a transparent phishing scheme. But it’s not hard to imagine that some people would see that the link takes them to www.somebank.com and believe the message to be legitimate.
This security vulnerability is commonly referred to as an “open redirector”, and we’re going to be proposing a requirement for the next version of the SDL that will help prevent these vulnerabilities. At the heart of this requirement is a new library we’ve adapted from the Windows Live Spaces team called SafeRedirect.
SafeRedirect is an alternative to the ASP.NET method System.Web.HttpResponse.Redirect (hence forth referred to as Response.Redirect). While Response.Redirect will redirect the client to any specified URL (as shown below):
Response.Redirect(Request.QueryString["p"]);
,calls to SafeRedirect.Redirect will only succeed if the specified URL belongs to a predefined “whitelist” of known good domains specified in the application’s configuration file. To continue our example, let’s say that “SomeBank” rewrites its application to use the SafeRedirect library and allows only redirects to the domain somebank.com. The new redirection code will look like this:
SafeRedirect.Redirect(Request.QueryString["p"]);
And while the following legitimate request will succeed:
,the following phishing attempt request will now fail:
http://www.somebank.com/welcome.aspx?p=http%3A%2F%2Fwww.evilphishers.com%2F
,and the user will instead be redirected to a safe warning page.
I’m personally very excited about this proposal, since it succeeds on two important levels: first, it addresses an important vulnerability that harms our users and undermines trust in our online services; and second, it is very quick and simple for development teams to implement. We can even take this a step further and write an FxCop rule that would detect usage of the old, vulnerable Response.Redirect and flag those calls as errors. For virtually no effort, product teams will be able to detect and fix phishing holes that could have potentially harmed their users. And that’s good news for everyone. Except the phishers.