June, 2009

Hi, Michael here.

This is part one of a two part series of posts by myself and Bryan Sullivan; I will cover the static analysis tools we use at Microsoft (and make available publicly) for analyzing unmanaged (ie; Native) C and C++ code, and Bryan will cover managed code static analysis in a later post.

I’m a huge fan of static analysis tools; actually, I’m a fan of any tooling that beneficially automates any portion of the software development process. Software development is a complex business, and anything you can do to make the process more repeatable, predictable and reduces ‘friction’ is a big win in my book.

There are many benefits to using static analysis tools. The most important reasons include:

• Static analysis tools can scale: they can review a great deal of code very quickly; this is something humans cannot do very well.
• Static analysis tools don’t get tired. A static analysis tool running for four straight hours at 2:00 in the morning is just as effective as if it runs during business hours. You can’t say the same thing about human reviewers!
• Static analysis tools help developers learn about security vulnerabilities. Over the years, I’ve met a small number of developers who had bugs flagged in their code by static analysis tools, and they never knew what the bugs were until the tool posted a sign saying, “Security bug, right here!”

Before I dive into static analysis tools in detail, it’s worthwhile explaining what ‘static analysis’ is. Static analysis is a method of analyzing program code without actually running the code. Generally, the tool will build an internal model of the code and analyze potential program flow through the code making assumptions about the data. For example, the following code may or may not be a real vulnerability:

char foo[4];
foo[i] = 0;

because it depends on the value of ‘i’; if ‘i’ is in the range 0..3 and can only ever be in the range 0..3 then there is no security vulnerability, so the static analysis tool has to determine if this condition is possible. Clearly, it’s simple to determine that the following code is safe, because the index is constrained right next to the code that writes to the array:

char foo[4];
if (i>=0 && i<=3) foo[i] = 0;

But things get more complex if the index is validated in remote parts of the code. It is this level of analysis that determines if a tool is noisy: a tool that flags too many issues (false positives) because it missed a validity check will rapidly annoy a developer.

I want to point out that static analysis is not grep, static analysis tends to be more robust. That does not mean grep is not useful, for example, if you have a set of banned functionality such as banning MD4 and MD5 (as the SDL does, along with other crypto algorithms) then grep’ing for MD4 and MD5 is totally valid, probably low noise and requires next to zero engineering effort.

I also want to point out that the SDL focuses on using static analysis tools to find security vulnerabilities. Under the SDL umbrella, we would not require development teams use static analysis tools that didn’t find security issues. A tool that does not find security bugs is not a useless tool; we just would not make it an SDL requirement.

It’s important to point out that static analysis tools work in tandem with human code reviewing experts. Tools tend to find a lot of bugs quickly, but expert code reviewers are better at finding a smaller number of hard-to-find security bugs. I wrote an article for IEEE Security & Privacy a few years back describing the methods I use to review code for security bugs.

Static analysis tools have been used for many years at Microsoft. We started in earnest with a tool named PREfix when we acquired Intrinsa. PREfix is aimed at finding general code quality bugs in C and C++ and has proven very effective over the years. The main downside to PREfix is it is big, and generally is run centrally rather than each developers’ desktop. So PREfix begat PREfast, a smaller desktop version of PREfix. PREfast has the advantage of being relatively quick to run (it only doubles compile times!) but it suffers from only being intra-procedural; in other words, its view of your code is very small, while PREfix is inter-procedural and can evaluate conditions in far-flung regions of your code. If you need to know why that’s important, refer to the example code above!

PREfix and PREfast both support the Standard Annotation Language (SAL) which I have addressed a couple of times in the past. SAL allows you to describe function contract semantics to help tools like PREfix and PREfast find more security bugs. SAL is used throughout Visual C++.

PREfast is available in Visual C++ today as the /analyze option, it’s also freely available in the Windows Device Driver Kit (as prefast.exe) and Software Development Kit (as /analyze).

What You Should Do

If you write native C or C++ code, you should:

• Compile at least once a day with /analyze
• Use SAL to annotate your function prototypes, this will help the static analysis functionality in the compiler find many more bugs.

The following warnings should be analyzed, as they are probably security issues:

6029 6053 6054 6057 6059 6063 6067 6200 6201 6202 6203 6204 6209 6248 6277 6298 6305 6308 6383

Finally, for extra credit, look for the following warnings that are generated by the compiler and not by the static analysis tools:

4700 and 4701

Both of these relate to uninitialized data and to enable these warnings either compile with warning level 4 (/W4), or if you’re not daring enough, use /W3 augmented with the following:

/W3 /WX /we4701/we4700

The SDL Optimization Model and Static Analysis

If you’re following the SDL Optimization model, use of static analysis tools is deemed a requirement for the ‘Advanced’ maturity level.

Summary

In summary, the SDL mandates static analysis tools for C and C++ code. If you are currently not using static analysis tools in your development environment, you should. If you’ve never run static analysis tools then the chances are good you’ll find some ‘interesting’ bugs!

We wanted to take a minute to point out this good post from Gunnar Peterson.  He’s right, and it’s worth repeating: we threat model not to find threats, but to find and implement countermeasures.  We’re glad to see people building on our work like this.

Hey everyone, Jeremy Dallman here. Today I will be co-blogging with David Lenoe (Group Program Manager, Adobe Secure Software Engineering Team (ASSET)). Now, here’s the story behind the Microsoft and Adobe security pairing …

A couple of years ago, Microsoft and Adobe made a decision to work together on security rather than address our similar security goals within the vacuum of each company. Our security teams have since been working closely together with the clear goal of protecting our mutual customers. This collaborative relationship enables faster implementations of security protection through the lifecycle processes both companies offer (Microsoft’s Security Development Lifecycle - SDL, Adobe’s Secure Product LifeCycle - SPLC), and allows us to share best practices learned over the years. In turn, each company learns about new ways to apply their respective lifecycle plan, thereby helping to provide our customers with a more secure computing environment.

Through the last couple of years we have had conversations about defining and implementing security requirements, prioritizing security risk, threat modeling, the benefits of compiler/linker flag protections, fuzzing, and penetration testing. We’ve even shared data on security incidents and response.

Implement proactive engineering protections

With support from the security folks at Microsoft, ASSET helped the Adobe product teams set the security-related C++ compiler and linker flags such as /NXCOMPAT, /DYNAMICBASE (ASLR), /GS, and /SAFESEH. Working together, we were able to address compatibility issues and get these protections in place for both Adobe Flash Player and Adobe Reader. These protections have helped to mitigate entire classes of vulnerabilities in Microsoft products and will improve the security of Adobe products as well.

Encourage consistent security updating

Most recently, we worked together to publish some 2008 attack data on vulnerabilities affecting Microsoft and Adobe products in the Microsoft Security Intelligence Report. Our goal was to emphasize to our mutual customers that installing security updates for Microsoft, Adobe and other third-party applications is very important. Having customers update promptly when Microsoft or Adobe addresses vulnerabilities is the best way to avoid the rapid spread of attacks.

Adopt security tools

After the Microsoft Security Sciences team released !exploitable in March, some of Adobe’s security testing teams started using it on their own products along with WinDbg to analyze the results of fuzz testing. Microsoft and Adobe continue to work together to address questions and help improve the effectiveness of this tool.

Some of Adobe’s development teams also use static analysis tools like /analyze and FxCop to identify potential security vulnerabilities in source code.

Share response information

By collaborating amongst the teams at Microsoft and Adobe, the Microsoft Security Response Center (MSRC), Microsoft Vulnerability Research (MSVR) program, the Microsoft Security Research and Defense team, the Adobe Product Security Incident Response Team (PSIRT) and Adobe Secure Software Engineering Team (ASSET), respectively, we have also been able to identify security trends and more rapidly address vulnerabilities.

Continue working together

We consider the collaboration between Microsoft and Adobe to be a great success for both companies. We look forward to continuing to work together and discovering new and better ways that we can protect both Microsoft and Adobe customers in the future.

Hi, Michael here.

A while back I wrote a blog post explaining the Standard Annotation Language (SAL) which is a technology we use to help static analysis tools find more bugs, including security vulnerabilities, in C and C++ code. If you look closely at VC++ 2005 and VC++ 2008, you’ll notice that almost all function prototypes are SAL annotated, which means you get the benefit of all the SAL work we did. But you might have also notice that the annotation style between the two compiler versions is different.

For example, in Visual C++ 2005, realloc() is annotated like this:

__checkReturn __bcount_opt(_NewSize) 
    void * __cdecl realloc(
        __in_opt void * _Memory, 
        __in size_t _NewSize);

But in VC++ 2008, realloc() is annotated like this:

_Check_return_ _Ret_opt_bytecap_(_NewSize) 
    void * __cdecl realloc(
        _In_opt_ void * _Memory, 
        _In_ size_t _NewSize);

void Foo( 
    __in_bcount(cb) BYTE* pBuf, 
    size_t cb );

void Foo( 
    _In_bytecount_(cb) BYTE* pBuf, 
    size_t cb );

Aren’t you happy we created macros for the low-level primitives!? You should never have to use the low-level primitives in your code: the table is to show you why the two SAL formats got their names.

So why a new SAL syntax? I have good news and really good news. First, the good news: other than a simple macro syntax change, there is not a lot new to learn in part because the macros are similar (not identical, however) and the major difference, the low-level primitives, are abstracted away.

Now for the really good news. Attribute SAL is much more rigorous than declspec SAL, which means analysis tools can find more bugs with lower false positives (‘noise’). For example, declspec SAL is often silent in the face of an incorrect annotation.

The introduction of attribute SAL does not mean declspec SAL is dead, but it does mean that we will not be investing any more resources into declspec SAL, all our energy improving SAL and our analysis tools use of SAL will be in attribute SAL. At a pragmatic level, this means:

· If you have already invested in using declspec SAL you should migrate over to attribute SAL as time allows, and use new attribute SAL for new functions. Both syntaxes can co-exist.

· If you have never used SAL, you should use attribute SAL. As far as you’re concerned, declspec SAL never existed.

One noticeable difference in macro names is the use of declspec SAL’s “count” and attribute SAL’s “cap” and “count.” The former is a buffer size in elements or bytes, but the latter two are the buffer’s writing capacity and the size of the buffer for reading, respectively.

An important addition to attribute SAL is _Printf_format_string_ which can be used to find many printf-related format-matching ills.

The following table shows some of the major differences:

As noted in the table above, there is one minor drawback to using attribute SAL. If you use constant expressions as count or cap arguments, you must use a special set of macros, which is a little less elegant than declspec SAL:

void Foo( _In_count_c_( 8 ) int* rgInt );

versus

void Foo( __in_count( 8 ) int* rgInt );

Note the _c_ portion of the attribute syntax, which is not needed when using declspec macros. With that said, attribute syntax supports accept any C++ conformant constant expression including enums and template arguments, but decspec SAL supports only simple expressions.

An Example

To put his altogether, let’s look at some simple code, and see how the VC++ 2008 /analyze static analysis performs when faced with the different SAL types.

First, declspec SAL:

   1: #include "stdafx.h"

   2:  

   3: struct SomeStruct {

   4:     int x;

   5:     float f;

   6: };

   7:  

   8: bool FuncOne(__in_z_opt const char* filename);

   9: void FuncTwo(const char *pFormat, ...);

  10: void FuncThree( __in const SomeStruct* setup );

  11: void FuncFour(__in HWND h, __in char *sz); 

  12:  

  13: void TestWarnings() {

  14:     char b;

  15:     FuncOne(&b); 

  16:     FuncTwo("%d %p %d", 10.0, "Hello");

  17:     FuncThree(0); 

  18:     SomeStruct blah;

  19:     FuncThree(&blah);

  20:  

  21:     char buff[100];

  22:     FuncFour(NULL,buff);

  23: }

When compiled with /W4 /analyze, the compiler gives us:

warning C6309: Argument '1' is null: this does not adhere to function 
specification of 'FuncThree'
warning C6309: Argument '1' is null: this does not adhere to function 
specification of 'FuncFour'
warning C6387: 'argument 1' might be '0': this does not adhere to the 
specification for the function 'FuncThree': Lines: 14, 15, 16, 17
warning C6387: 'argument 1' might be '0': this does not adhere to the 
specification for the function 'FuncFour': Lines: 14, 15, 16, 17, 18, 19, 21, 
22

Now, let’s take the same code, but decorate the function prototypes with attribute SAL rather than declspec SAL.

   1: #include "stdafx.h"

   2:  

   3: struct SomeStruct {

   4:     int x;

   5:     float f;

   6: };

   7:  

   8: bool FuncOne(_In_opt_z_ const char* filename);

   9: void FuncTwo(_Printf_format_string_ const char *pFormat, ...);

  10: void FuncThree(_In_ const SomeStruct* setup );

  11: void FuncFour(_In_ HWND h, _In_ char *sz); 

  12:  

  13: void TestWarnings() {

  14:     char b;

  15:     FuncOne(&b); 

  16:     FuncTwo("%d %p %d", 10.0, "Hello");

  17:     FuncThree(0); 

  18:     SomeStruct blah;

  19:     FuncThree(&blah);

  20:     

  21:     char buff[100];

  22:     FuncFour(NULL,buff);

  23: }

warning C6273: Non-integer passed as parameter '2' when integer is required 
in call to 'FuncTwo': if a pointer value is being passed, %p should be used
warning C6064: Missing integer argument to 'FuncTwo' that corresponds to 
conversion specifier '3'
warning C6309: Argument '1' is null: this does not adhere to function 
specification of 'FuncThree'
warning C6309: Argument '1' is null: this does not adhere to function 
specification of 'FuncFour'
warning C6001: Using uninitialized memory 'b': Lines: 14, 15
warning C6387: 'argument 1' might be '0': this does not adhere to the 
specification for the function 'FuncThree': Lines: 14, 15, 16, 17
warning C6001: Using uninitialized memory 'blah': Lines: 14, 15, 16, 17, 18, 
19
warning C6387: 'argument 1' might be '0': this does not adhere to the 
specification for the function 'FuncFour': Lines: 14, 15, 16, 17, 18, 19, 21, 
22

As you can see, using attribute SAL found many more code bugs, and all of them are real. I’ll let you sift through the list to see what attribute SAL found over and above declspec SAL! There are some duplicate bugs, however.

If you want to learn more about SAL, I would recommend you simply open sal.h and read the comments and examples.

The Rosetta Stone

Below is a partial Rosetta Stone to help you convert between the two SAL syntaxes if you need to do so.

Acknowledgments

Hi all, Anmol Malhotra here… I’m a Senior Security Engineer with Microsoft’s ACE (Assessment, Consulting & Engineering) Team. We are part of Microsoft Information Security group and our mission is to enable secure and reliable business for Microsoft and its customers. ACE Team is responsible for security, privacy and performance for line-of-business (LOB) applications at Microsoft.  Since 2001, we have been working in identifying and reducing risk posed by applications in our enterprise.  This experience has resulted in development of processes, tools and best practices to help develop and maintain secure applications for an enterprise. We developed the Security Development Lifecycle for Line-of-Business Applications (SDL-LOB) process which defines the standards and best practices for securing LOB applications.

As part of our continued commitment towards sharing security processes, and recommendations with our customers, we are excited to announce the new addition of detailed security requirements and recommendations for LOB applications with the release of Microsoft SDL version 4.1 on MSDN. SDL-LOB provides a mainstream approach to the SDL which focuses on development of applications which support business such as accounting, human resources (HR), payroll, supply chain management and resource planning applications etc.

Couple of things around this guidance –

a)      This guidance is positioned exclusively for line-of-business applications or web applications and not for ISV/rich client and server application development.

b)      It is important to emphasize that organizations should adapt rather than adopt the SDL-LOB process.

So here it is, Security Development Lifecycle for Line-of-Business Applications. Also look out for SDL-LOB blog series on the ACE Team blog starting in June.  We’ll discuss the SDL-LOB phases and highlights.

Your comments & suggestions are very welcome.