Adam Shostack here...
Not too long ago, I was talking to a friend at a large company (not Microsoft). My friend has been in security a long time. He’s frustrated that he’s nicknamed Dr. No, because his co-workers expect him to say no to everything. He’s risk averse, convinced that all security failures, viruses or a breach will be attributed to him, and cause his career to dead-end.
What he doesn’t see is that his career is dead already. It’s not there because of a security flaw, but because of an all-too-common failing among security professionals. He’s not managing risk in the way his execs want him to manage risk. He’s not evaluating the benefits of a program, he just focuses on the downside. He’s not trying to make tradeoffs.
In the SDL, we help people and teams make more informed decisions on the trade-offs and balances associated with security decisions. There are some risks that we say are unacceptable because of their impact on customers. On behalf of those customers, we tell teams that they can’t ship software unless they’ve taken certain steps to reduce that risk, and that there are classes of issues that they must resolve before they ship (roughly, anything that would result in an MSRC bulletin).
Knowing up front what’s allowed and not allowed, teams can build it into their schedules. We keep the SDL on a twice-yearly change schedule, giving teams a chance to integrate the SDL recommendations and requirements into their planning. (We’ve reserved the right to issue emergency changes.)
By setting a clear and prescriptive bar for how to control risks, we’ve made the SDL a lot more acceptable than if we simply played Dr. No when projects are trying to get out the door.
This is an example of what John Boyd called an ‘orientation.’ It’s the result of our heritage, cultural traditions, training, and previous experiences. Orientation influences what we notice and how we act. If you’re oriented around blocking and risk avoidance, rather than ensuring that your concerns are discussed at the table, you might end up avoided, rather than having a conversation.
We try to orient the SDL around what our customers want, both explicitly and implicitly.
It’s a risk we’re cognizant of as we evolve the SDL.
Steve Lipner here.
A couple of weeks ago, I participated in a panel on the ethics of security vulnerability disclosure at Black Hat in Las Vegas. I believe that I was invited for my role in Microsoft’s Security Engineering and Community team and because I’m a former director of the Microsoft Security Response Center.
The focus of the panel was on handling software vulnerabilities discovered by security researchers, but the discussion really dived into a wide variety of ethical issues around software security. For example, one panelist asked whether it was ethical for a company with billions of dollars in the bank to ship a product with known classes of vulnerabilities. Buffer overruns were mentioned specifically. A member of the audience asked whether, in today's environment, it's ethical for a software vendor to release products without conducting an extensive security review.
At Microsoft, we hear these kinds of ethical questions more often than you would think. All of them tend to come down to two common themes: How much should a vendor do and how long should a vendor wait to make a release "secure enough?” Our answer is that we do as much as we can to make our products secure, but we’re always mindful of the need to ship customers a product that will not only improve security but be timely enough so that they’ll actually use it. It is not much more ethical to work forever on a secure product that you never ship and users never use than it is to ignore security altogether.
Back in the 1980s, I led a project to build a system that could be evaluated under class A1 of the Orange Book. We did things like strict layering of the design, writing and verifying formal specifications of the system, and characterizing and removing "covert channels" that could allow the leakage of information from one security classification to another. Boy, was it secure!
Unfortunately, the development process we followed was so rigorous that it took us several quarters to turn around a major design change to eliminate a new class of attacks or fix a performance problem. I eventually made the hard decision to cancel the project because feature enhancements to competing products were moving faster than we were able to design and build our product. By the time we were ready to ship the system, it wouldn’t have been competitive in the market. Even government agencies handling very sensitive information didn't want to pay the performance and feature penalty for a system that secure.
Some folks have also advocated "throwing away the legacy" and building a completely new system unburdened by the constraints of compatibility with legacy code. This is a nice idea in theory because eliminating the constraints of compatibility appears to allow a design and implementation that prioritize security over all. But I believe it's a loser in practice. Even our A1 system was constrained by compatibility with legacy applications (we designed it as a VMM for that exact reason). We were pretty sure that a "clean sheet" incompatible design would find no customer interest at all, and I still believe we were right on that score.
My experience with the A1 system has definitely influenced my work on the SDL. While we do the very best we can, we know that perfection is not achievable. What we do is add steps to a commercially viable development lifecycle that can be accomplished by real developers on a schedule that allows them to ship competitive products. We learn from our mistakes and update the processes as we go, but we never forget that it's important to ship.
What does all this have to do with ethics? Well, I think that given the choice between shipping perfectly secure software (whatever that means) that no customers will use and shipping software with continuously improved security that will actually help customers, the better ethical path is to ship. That's a controversial view in some circles, but it's the view I've reached after working in the field for the last 35 years or so.
Rob Roberts here…
We often fear what we don’t know. Take my mother’s casseroles, for example. The initial view scares you, but once you take that first bite, you realize not only that it’s edible, but sometimes, it’s even tasty.
When we meet with product teams in privacy reviews for the first time, we often encounter a team that’s on the defensive. This is typically caused by their fear that we’ll tell them they can’t do something because of privacy concerns. Once they describe what their application does and we begin to give advice, they come to learn that we aren’t out to kill their ‘cool’ software capability, but in fact, have ways for them implement it while at the same time increasing customer trust and confidence.
Designing Software for Privacy
As much as possible, we design our solutions to allow customers to gain the benefits of services without having to give up personal information. An example of this is our online advertising.
For customers with a Windows Live ID (WLID), advertising utilizes a one-way hash of the WLID called an Anonymous ID (AnID), which is stored in a cookie on the customer’s computer. This allows the Microsoft site to collect information about searches and to serve up targeted, user relevant ads without tying a customer’s profile to the searches or ad profile information. Customers gain the benefit of custom advertising without having to set special preferences to protect their privacy.
Inform and Give Control
Sometimes more user information is needed in order to deliver service or capability in a piece of software. Assuming we have user consent, we have a couple of privacy levers that we can adjust to address privacy in our products:
· Disclosure – informing the customer of the privacy impacting behaviors and how to address and control them whenever possible.
· Privacy Controls – settings that allows the user to modify the privacy impacting behavior directly.
Both of these controls can be presented in such a way as to address the varying needs of the three types of people that privacy expert Dr. Alan Westin described in his research, without overwhelming the user with information or choices.
The "Unconcerned" "Pragmatic" "Fundamentalist"
In his research on public privacy concerns, Westin classified the public into three categories: "Fundamentalists", who are distrustful of a company or organization’s collection of personal information; "Pragmatics", who are more willing to share information after weighing the benefits of doing so; and the "Unconcerned", who trust the company or organization’s collection of their information. Westin’s studies (1990-2003) determined that just over half of the people fall into the middle "Pragmatic" category (58%), while smaller percentages fall into "Fundamentalist" (25%) and "Unconcerned" (18%).
Though the "Fundamentalist", we prefer to call them "Privacy Advocate", group is not the majority of the public, their number is significant enough that it cannot be ignored when designing software for privacy. By designing with this group in mind we can build out Disclosures and Privacy Controls that are scalable depending upon the needs of the users at any point on the continuum.
· Privacy "Unconcerned" –
o Disclosure – A simple prominent notice with a link to a privacy statement may be given to assure that the user is aware of how the software may impact them from a privacy standpoint.
o Privacy Controls – Default controls may be set to allow flexible use of the product, such as in the case of IE7 – which is set to medium privacy settings that block certain risky cookie types for users.
· Privacy "Pragmatic" –
o Disclosure – Prominent notices typically include a link to a more detailed privacy statement which allows users in this category to further explore what the privacy impacts are and how they can change them. Layered privacy statements, such as the one for Windows Vista, allow customers to see a summary of the privacy impacting behaviors and give the option to drill deeper into aspects customers may want to learn more about.
o Privacy Controls – Where appropriate, adding variable privacy controls to software allows a user to nuance the privacy behavior of an application. Using the IE7 privacy control example above, this user may move the privacy slider from medium to a higher or lower setting, depending upon their level of concern.
· Privacy "Fundamentalist" –
o Disclosure – Sometimes prominent notice and privacy statements aren’t enough for people that fall into this category. For complex products such as Windows, we published supplemental information such as the Windows Controlling Communication with the Internet whitepaper. This was particularly important to customers in enterprises that must maintain a high level of security in their IT deployments.
o Privacy Controls – In addition to their desire for a detailed understanding of their software’s privacy behavior, a Privacy Fundamentalist typically wants more refined control over the behavior of the application. In the case of the IE privacy settings, going to the advanced options will allow specific control over the types of cookies that may be encountered (i.e. First-party vs. Third-party and session cookies).
It’s this continuum of preferences that helps us understand how we need to build out our software from a privacy perspective. By setting a privacy standard that considers these levers, and implementing them through a consistent repeatable process like the SDL, we can drive our products to be innovative, secure and privacy aware.