Hello all... Dave here.
I have worked on security and privacy initiatives at Microsoft for a number of years, but it wasn’t until I came to the Security Engineering group to work on the Security Development Lifecycle that I realized I don’t actually work on security. To be clear, I do many of the tasks that one might associate with security – look at bugs, evaluate tools, provide guidance and the like - but it's more accurate to say that I (along with everyone else in Security Engineering and Communications) am in the culture change business.
At its heart, the SDL is a series of common sense processes and technologies woven into a lightweight, coherent framework that has produced real results for Microsoft. While it may pain some of my colleagues to read this, the SDL is not Unified Field Theory – much of what makes up the SDL has been in existence for years (although I would be remiss if I didn’t note that there is a big difference between a process that has been applied to millions of lines of production code and those that have not). Furthermore, many of the processes used by SDL (and other methodologies) are generally acknowledged as effective in identifying and mitigating security threats. Pondering this notion is what lead me to my realization about culture change, and prompted a question: If this stuff has been around for awhile in some shape or form, why aren’t more people doing it? Being a naturally curious type, I wanted to learn more about this apparent discontinuity. More often than not, when talking to our customers an interesting social phenomenon is presented – as I talk with technical employees (developers, project managers, testers and the like) the desire to implement secure development practices extends all the way up the technical food chain, including the CSO - then it stops…cold. I haven't done any empirical study of this behavior, nor do I plan to – however, in the process of conducting my informal probes for causality, more often than not I find that it hinges on two points: 1. lack of understanding at the CxO level about how security, privacy and reliability are contributors to corporate risk, and; 2. discomfort with the notion of redirecting resources to trustworthiness in the short term, in exchange for longer term gains in product quality and customer satisfaction.
Microsoft has two things going for it in this regard; strong executive support and the unfortunate experience gained by living through "tectonic" security events – which have snapped trustworthiness issues into razor-sharp focus and continue to provide motivation by the truckload. This clarity of thought at the executive level has allowed us to create, implement, and refine the SDL – providing resources and support at critical junctures has proven to be absolutely vital in driving culture change at Microsoft. Have we won the culture war? Absolutely not. Microsoft is constantly evolving; people come and go and the lessons of the past must be taught to future generations. Bottom line: our executives understand that we are in this for the long haul and that by addressing trustworthiness on a continuing basis, we will see tangible gains in both product quality and customer satisfaction.
As one might expect, we get a lot of requests from customers asking for help in "selling the concept to their execs" - and that’s okay because as we have learned, executive support for trustworthiness can be cultivated. It’s our intent to try to help illustrate the value of adopting secure development practices – watch this space over the next few months as we start to roll out some of this work.
A final note to help illustrate my point – for those of you that are old enough to remember, there was an old TV commercial for Fram Oil Filters that showed a mechanic working on the tear down of some old beater. At the end of the commercial, the mechanic turns to the camera and says, "You can pay me now, or you can pay me later..."
Seems to be oddly prophetic advice.
James Whittaker here
“You can’t test quality in.” It’s a truism coined long ago and an accepted fact of software development. Yet, for security, testing is arguably the most talked about aspect of the Security Development Lifecycle (SDL). When we get security wrong, the first criticism we almost always hear is, “Didn’t you guys test this thing?” It is no great stretch to say that many of the most famous industry security folks made their reputation by finding vulnerabilities (through, no doubt, testing). You simply can’t avoid the subject of testing when you talk about security, and you can’t be sure you’re secure without testing.
We often get questions about SDL-required security testing and too often these questions deal exclusively with fuzz testing. Equating fuzz testing and security testing couldn’t be further from the truth when it comes to how it is treated inside Microsoft. With this post, I want to shed some light on what Microsoft actually does for security testing. In a follow up post on this blog, Rob Roberts will talk about our privacy testing practices.
To begin, it is difficult to confine testing activity within a single SDL phase. At Microsoft, we don’t try. Testers are involved in architecture review, security design reviews, threat modeling, code reviews and many other things that happen both before and after the actual testing phase. In each of these instances, testers bring a valuable how-I-would-break-this slant to these endeavors. This contribution has been valuable enough to spawn a big push around the company to move testing activity to earlier phases of the lifecycle and, though some might not agree, I think the practice of threat modeling can be ascribed to this movement. The idea of thinking through threats and understanding attack vectors has been our focus in security testing for years, and threat modeling represents the extraction of this process as its own standalone entity.
Our overall goal is clear - whenever an engineer designs or writes code, we want that person to think about how the code might be exploited. When attack scenarios, threats and test cases are swirling around in a developer’s mind as they architect, design or write code, chances are he or she will write more secure code and plan better defenses. Clearly there is an overwhelming amount of stuff to think about, requiring a healthy amount of due caution and discourse with teammates and outside experts. Being careful and consulting colleagues is rarely a bad thing!
But, no matter how successful we are in spreading testing wisdom throughout the SDL, at some point we need to check that such wisdom actually made its way into the shipping product. I trust developers to do the right thing, but as a tester myself, you better believe I’m going to check that they actually did it.
Microsoft uses a three-pronged approach to security testing. During these tests we may refer to a threat model or security design review document, but we may also choose to ignore these documents for an independent assessment of an application or service. This decision is at the discretion of the security test lead, and depends on how independent he or she wants the test team to be.
1. Attacks against the application’s environment.
The environment, the sum total of all OS components, runtime libraries, environment variables, network activity, file system configurations, registry keys and so forth, is probably the biggest unknown when fielding an application. For some environments the application will work securely, for others it may fail miserably. We train our testers to map out the environment, identify components subject to modification or variation and test as many configurations of these as possible. These attack scenarios are recognition that our applications work in unpredictable environments where we have to work out the trust relationships very carefully. It takes only one insecure component to put an entire machine or network at risk. We need to ensure that our own applications work securely despite the presence of these environment insecurities.
2. Direct attacks against the application itself.
Inputs are dangerous and inputs that cross trust boundaries are crucial targets of this class of security testing. Our testers must build and maintain lists of illegal, ill-formed and improper inputs that are consumed by their application’s interfaces. Code, scripts, SQL queries, special characters, long strings and the like must be gathered in large numbers and used to pummel the application under test mercilessly. Large scale automated testing comes into play here in a big way., Our goal is for our applications to be able withstand targeted and sustained attacks – whether it’s a regression suite of past and potential exploits or fuzz testing using both random or format-aware logic. These tests are crucial to prevent repeat exploits and to test against targeted attacks scenarios.
3. Indirect attacks against the application’s functionality.
Application features need to be cataloged for potential bad effect. All features clearly have intended functionality for good effect or they wouldn’t be features, our concentration as security testers is to understand what ways those features can be misused to the misery or inconvenience of our users. We must look at our application’s functionality and ask whether any of it can be ‘turned against itself.’ Are there ways that the software can be easily misconfigured? Can security features be circumvented? Is there some function whose purpose is benign and even useful that under certain circumstances has undesirable consequences? A feature-by-feature assessment is necessary to ensure we’ve covered all the bases.
Security testing has been – and will always be – about assurance, that is, assuring that the product as built and shipped has been thoroughly tested for potential vulnerabilities. Bug detection and removal will continue to be important until a time in the future comes when we can deploy provably secure systems. However, we’re likely never to get to such a future without learning the lessons that testing is teaching right now. Anyone can write a system and call it secure – it’s only through watching real systems fail in real ways that we learn to get better. Moral of the story – testing is by far the best way to show us what we’re doing wrong in software development.
Adam Shostack here.
Last week, we held the 5th Blue Hat conference, focused on the “Paradox of Innovation.” BlueHat is a conference where Microsoft brings applied security researchers to campus to speak to executives and engineers. I have both personal and professional perspectives on BlueHat. On a personal level, I spoke on a panel at BlueHat 2 after filming a “Security 360” webcast with Microsoft’s Mike Nash. Conversations there and at BlueHat 3 were an important part of my decision to join Microsoft after a decade building security startups.
It’s fascinating to watch the interactions that happen between the speakers and the Microsofties, and consider how I see those conversations then and now. What I didn’t see then was how the conversations tie into larger, longer term initiatives. One of the recurring themes from speakers is “why we hack things.” A few BlueHats ago, H.D. Moore talked about why he created Metasploit, and this time around bunnie and Felix talked about why they hack hardware (“Your Tamper-Resistant Hardware Makes a Great Sport for Hackers.”) That’s an important message to hear from people who are investing effort in understanding hardware security. There are a lot of people who do research for fun, and a lot who research because they want to secure their systems. Developers also enjoy their work. Getting security bug reports can feel like a kick in the pants, especially the way some of those reports are commented. Hearing from the people doing research helps Microsofties understand that the researchers enjoy what they do, and the goal is usually the research and getting a problem fixed, not the kicking.”
In the MSRC blog, (“BlueHat V5 Opens!”) Andrew Cushman did a great job of explaining the goals:
- Expose senior product leaders and front line engineers to the threats and attack tools and methodologies used in the real world. Take the security threat from the theoretical/intellectual level of, ”I understand what a buffer overflow is”, to “OMG that’s what it’s like.” BlueHat connects with employees at a visceral in order and *really* brings the message home…
- Expose security researchers (and the security community) to Microsoft engineers and business leaders… BlueHat gives us a chance to open up on our home turf and gives the researchers an opportunity to interact with all levels of the organization. They too get to experience first-hand that Microsoft does have smart, passionate engineers that do care about security.
Meanwhile, Christopher Budd said
From my point of view, this is something that makes BlueHat unique among security conferences: I don’t know of any other venue where security researchers talk to an audience that’s mainly comprised of people who consider themselves first and foremost engineering professionals rather than an audience of security professionals.
This is a really important point. There’s a vibrant and engaged community of security researchers, and those who spoke got a chance to influence upwards of 1,200 engineers who otherwise might never hear those particular security messages. Some of the messages I was especially happy to hear were the talks [6] from RSnake and Rob Thomas. RSnake talked about “Death by 1,000 cuts” and Rob talked about the underground economy. Sometimes it’s easy to focus on the pain of having to install critical updates, and miss the larger pictures of how attackers put things together to cause “death by 1,000 cuts” and the economic motivations that cause them to do so. The SDL is focused on developing more secure products, and that includes how to make things secure by default, by design and in deployment. We want to help our engineers and execs remember the larger security picture. Giving them specifics of how the issues can complement each other helps catalyze that understanding.
The security research community gets to influence more change by engaging with engineers. The SDL involves changing the way people think about developing software, and even if BlueHat isn’t a formal stage of the SDL, it’s an important complement, and it’s reflective of the way our culture is changing.
(I was also really pleased to see Katie Moussouris, Richard Johnson and Mark Novak presenting at Toorcon. This sort of engagement with the researcher community benefits everyone but the underground.)
Tina Knutson here...
A few years back we integrated privacy into the SDL. Privacy and security often go hand-in-hand, but they are not the same thing. They often have the same objective, but the focus is different. When it comes to customer data, security focuses on keeping your data safe, while privacy focuses on giving you control.
At privacy conferences and trainings, I’ve run into what I believe is a disturbing trend. In a lot of the events and conversations I’ve experienced, privacy often ends up being used as a synonym for “data security.” Data security breaches are clearly a big concern and shouldn’t be taken lightly; but privacy training, policies, and processes should go much deeper than *just* safeguarding the data. Yes, data security is very important, but privacy should cover so much more.
Anytime we collect your data, we know that the experience can either increase your trust or destroy it. If you understand what’s being collected, why it’s being collected, what the benefits are (to you – not to Microsoft!), and how you can control it in the future, you are much more likely to trust us. In order to build trust when collecting data, we believe that clear and accurate communication is paramount. For example, when Windows Media Player collects information about a DVD you’re watching, it’s better to know up front that this information is used to provide you with media information such as DVD title and cover art. If you don’t know this and have to extrapolate why Microsoft might want to know the DVDs you’re watching, it could seem pretty creepy.
In addition to communication, another privacy concern is minimizing the data collected. It’s all too easy for a product or marketing team to collect data because “it could be useful” one day. My job, and the job of my colleagues in the privacy space, is to make sure that teams know that any use that hasn’t been disclosed in the initial capture of data is off limits. In integrating privacy considerations into the SDL, we’re spreading the word that all of the commitments made at the time of collection apply to that data until it is destroyed. Anyone who uses the data must understand and follow the parameters under which it can be used. When your data is collected specifically to provide a service to you, it shouldn’t be used for secondary purposes, like marketing, unless you were notified of the use and agreed to it when the data was collected. Yes, the data also needs to be kept safe, but that shouldn’t be the only focus.
Privacy is not just about protecting data once you have it; it’s also about minimizing the data collected, and making sure that you know what that data will be used for and consent to that use before your data is captured. This is one of the main reasons Privacy has been built into the SDL. Securing the data alone is not enough.
Read more about how we view privacy in our Privacy Guidelines for Developing Software Products and Services.
-Tina
Dave Ladd here...
There has been a lot of hoopla lately around "secure programming skills" – with not-so-thinly veiled condemnations of academicians and the role of the university in addressing the IT security problem.
While it’s tempting to view education as synonymous with training, that really does neither concept justice. I’ll argue that the role of education, (as provided by an accredited college or university) is threefold;
On the other hand, training assumes that the prior three conditions are already met, and as a result, focuses on time-bounded transfer of information - with an emphasis on the applied (rather than theoretical) particulars of a concept. Put differently, if I have a degree in physics this does not necessarily make me qualified to work on nuclear reactor design. I need training and practical experience on this subject before I put my education to effective use.
I have been a manager at Microsoft for a number of years, across many teams - given the opportunity, I hire for aptitude rather than skill set every time. That’s a firmly rooted hiring philosophy here at Microsoft. I want people working for me that “grok” security and have the capacity to learn, because this is a rapidly evolving field. Sometimes, I don’t have the luxury of hiring for aptitude and I need someone with specific skills - however, if I have the luxury of smart people, I can tell them “go learn how ‘foo’ works” and have a high degree of confidence that the end result will meet my expectations.
So how does this relate to computer security? Comprehensive system security consists of (among other things) solid design, coding and testing expertise – focusing on one at the expense of the others is simply an incomplete solution to a problem that demands our best efforts. Unfortunately, there's an assumption held by many in our (IT) community that the road to better security leads to “drinking from the fire hose” – that is to say, employees are rocketed through week long training classes, then drilled and tested on security topics. Without the necessary exposure to secure systems design and concepts, more often than not these classes simply become a blur. Moreover, without the proper theoretical basis, little understanding of core concepts is actually achieved. The notion that time bounded, “point” solutions like training and skills testing alone provide a “better” long-term result is simply a fallacy. We haven’t yet defined what a better long term result is. Is it fewer stack smashes? We could achieve that with managed code, and no security training at all.
It’s absolutely true that there are many applied security concepts (normally classified within the realm of “training”) that can and should be injected into IT focused curricula (CS, EE, SE, CE, MIS) at universities. Exposure to these concepts would not detract from the overall quality or effectiveness of a program of study – indeed many universities are already working on the problem. However, the university is not the place for skills enhancement or job re-training. It’s my view that universities are places to “tinker” – the environments best suited to ask “What if…?” type questions – precisely the type of activity that encourages exploration into things like architecture and system design.
We need both education and training to reach maximum potential in the shortest amount of time.
We require our SDL training to emphasize the basics of secure design, development and test – then allow employees and their management to select the training that meets the needs of their particular product or service. There is one other point that bears mentioning – our training is constantly being reviewed or embellished to make sure that emerging security or privacy issues are being addressed. It’s an approach that is delivering results. We do not skills test. Skills testing is a static process that simply establishes one’s specific conceptual understanding at a specific point in time. It does little to insure that the conceptual understanding is being applied when actually building software. We test our code and our products at various stages of the development process – because that’s what we deliver to our customers. It’s also where we believe validation processes are the most effective.
Circling back to the role of security education versus security training – my request of our colleagues in academia; don’t get distracted by the IT community noise – keep your “eye on the ball.” If you are presented with the opportunity to inject more security and privacy concepts into your teaching, do so – the computing ecosystem needs it and will be the better for it. But, for the sake of real computer security and a continued cycle of innovation – keep sending us the thinkers...
- Dave