The Connected Information Security Group

This Blog URL Has Changed – Please Update Your Readers

cisg — Thu, 16 Apr 2009 17:52:32 GMT

Things have been quite on the blog for while. There is a LOT of code being cranked out at the moment as we work towards some deadlines in the summer on various projects.

Our team name has also changed from the Connected Information Security Group (CISG) to the Microsoft IT Information Security Tools Team. This reflects an increased scope of tools that we are building and areas that we are focusing on so we have updated the blog URL. Well leave all the content as is on this blog but all new content will be posted at the new URL.

As well as news about significant work on CAT.NET and a Beta for TAM 3.0 we plan to start sharing details of the development framework CISF that we are building and a Risk Tracker application; both of which we plan to release open source under an MS-PL license this summer. CISF is a set of reusable components and code from which you can assemble your own security management applications (including gluing various security tools and technology together). It’s built in C# and on the MSFT technology stack (.NET 3.5 (WWF, WCF. ASP.NET etc)), SQL Server 2008 and Windows Server. You can think of Risk Tracker as a “Security Starter Kit” using the CISF; it’s essentially a Risk Tracking application that we have built internally for the corporate information security team which we will generalize and share with the community. You will be able to run it as is or extend it with .NET and the CISF. We plan to extend both tools on a regular basis (quarterly updates) as we improve the tools and technology for internal use.

More news in a few weeks!

You can subscribe to the new blog at http://blogs.msdn.com/securitytools

Cheers!

Mark

CAT.NET New Build – 1.1.1.8

cisg — Fri, 20 Mar 2009 22:02:47 GMT

Mainly small bug fixes and a new feature to export the findings into an Excel spreadsheet.

Download link is -

http://www.microsoft.com/downloads/details.aspx?FamilyId=0178e2ef-9da8-445e-9348-c93f24cc9f9d&displaylang=en

We recommend all users to upgrade to this version.

We have some work now starting on new features including general performance improvements, UI improvements and rules maintenance. We expect these to be complete by summer after which time we then expect to undertake some core engine updates to significantly improve the performance, scalability and vulnerability coverage.

Note: The MSDN download page says 1.0 but it is 1.1.1.8. well fix that ;-)

Getting Help for CAT.NET and Anti-XSS

cisg — Mon, 23 Feb 2009 17:42:23 GMT

We now have a discussion forum for users of CAT.NET. There is no official support for these tools but you can ask questions and we will try to help wherever we can!

CAT.NET -

http://social.msdn.microsoft.com/Forums/en-US/catnet/threads/

Anti-XSS -

http://www.codeplex.com/AntiXSS/Thread/List.aspx

MSDN Webcast: Software Security with Static Code Analysis Using CAT.NET (Level 200)

cisg — Tue, 17 Feb 2009 02:32:04 GMT

Event Overview

In this webcast, we provide an overview of what static code analysis is and typical coding errors that static analysis can and cannot detect. We also look at the recently released CAT.NET tool and how it helps with the detection of security flaws.

Presenter: Andreas Fuchsberger, Senior Software Design Engineer, Microsoft Corporation

AntiXSS Library V3.0 - Test Harness

cisg — Mon, 19 Jan 2009 13:55:15 GMT

Hi, Anil Chintala here…

In this post I wanted to talk about the new Test Harness application which was released as part of the AntiXSS V3.0 Beta and is available as a free download on MSDN with source code available for download on CodePlex. Test Harness application is created to help the users to quickly get started and validate the successful blocking of XSS issues by the Library and also to measure the enhanced performance claims of the AntiXSS V 3.0 against Microsoft .NET encoding library.

AntiXSS Test Harness is a windows console application that automates the following two categories of tests - XSS validation and performance tests. When executed, AntiXSS Test Harness displays this console menu:

Performance Test Bench uses HtmlEncode() method as a benchmark for measuring performance of the AntiXSS library - AntiXss.HtmlEncoding(…) method against the .NET - HttpUtility.HtmlEncode(…) encoding method. Input strings with a combination of safe and un-safe characters are used as payload to run the automated performance tests.

Choosing Option#1, Performance Test Bench executes performance tests that analyze such metrics as:

Input string lengths
Encoded output strings
and the total time taken for its execution.

During its run, Performance Test Bench compares the execution times of .NET's HttpUtility.HtmlEncode and AntiXss.HtmlEncode and stores in an output file containing results as displayed in this illustration:

XSS Validation Test Bench demonstrates the successful blocking of cross-site scripts. These tests use a list of XSS exploits as payload for running the automated tests. XSS exploit list are read from a text file, each payload is run through HTMLEncode() method of the library and the encoded output is stored in an output file.

When Option 2 is selected from the above console screen, Test Harness application executes the XSS validation tests and produces the following output file:

Test Harness Application provides a framework for automating the XSS validation and performance evaluation. Primary objective is to help developers and testers to quickly get started and test AntiXSS library for XSS validation and performance. With the availability of source code on CodePlex it also allows advanced users to extend the automated testing capabilities as per your specific requirements.

Thanks and more later…

Current Memory Limitations of CAT.NET

cisg — Mon, 12 Jan 2009 15:19:39 GMT

Hi, Andreas Fuchsberger here.....

It is important to understand what happens CAT.NET builds its Call Flow Super Graphs. We use a CCI object called CciControlGraph to build a Control Flow Graph for each method and each method call we find in the Common Intermediate Language (CIL) of the modules being analysed. These individual control flow graphs are collected together in a .NET List object.

Unfortunately this is where we start to encounter a shortcoming in the implementation. Even with virtual memory there are limits to how much memory a single .NET application can allocate. As reported in recent blog post a 32-bit process, such as the CAT.NET Visual Studio plug-in version can only allocate about 1200 MB, even on a 4GB RAM (32-bit) system. Moreover another shortcoming of the current implementation is that when CAT.NET runs out of memory is it exits with an unhandled OutOfMemory (OOM) exception, unfortunately this does not get reported by the Visual Studio plug-in and the plug-in just seems to hang.

There is a solution, on the same 4 GB RAM system you can more than double the amount of memory available to CAT.NET by using a 64-bit version of Vista or Windows Server and the 64-bit command line version of CAT.NET. This is why we supply the 64-bit command line version of the CAT.NET binaries: (http://www.microsoft.com/downloads/details.aspx?FamilyID=e0052bba-2d50-4214-b65b-37e5ef44f146).

For those without the option of running on a 64-bit system, one manual method for analysing larger applications that the 32-bit version of CAT.NET fails to do is to restrict the number of modules that are passed to CAT.NET. For a developer analysing their own code it is relatively easy to make sensible choices which modules should be analysed together. Typically they would choose all the their own as code that receives user input and any dependant modules, e.g. third party libraries, that process that input to produce an output, leaving all other business logic modules aside. However it must be pointed out that this method lead to some vulnerabilities being missed.

We are working on scalable long term solutions to these sorts of problems which require some relatively heavy lifting on our part. For today hopefully the above advice will at least help understand why the issue happens.

Free MSDN Webcast: Managing Cross-Site Scripting Using CAT.NET and AntiXSS (Level 200)

cisg — Sun, 04 Jan 2009 12:36:46 GMT

Language(s):
English.

Product(s):
Security.

Audience(s):
Developer.

Duration:
60 Minutes

Start Date:

Friday, January 09, 2009 12:00 PM Pacific Time (US & Canada)

Merlin: Better Specifications for CAT.NET

cisg — Fri, 02 Jan 2009 22:05:44 GMT

Guest post by Ben Livshits of Microsoft Research here....

In the last several years we have seen a proliferation of static (and sometimes runtime) analysis tools for finding web application vulnerabilities. Companies such as Fortify, Ouncelabs, Klockwork, and others have been selling tools for finding security flaws for a while now. Most focus of the OWASP top 10, a list being dominated by XSS, SQL injection, CSRF, etc.

CAT.NET is a state-of-the-art static analysis tool for .NET that we have built at Microsoft and released for free to all of our customers.

The effectiveness of these tools is generally seen as a function of false positives and false negatives. Ideally, of course, you want to find all real vulnerabilities and eliminate all false alarms. In practice, most tools have to cut corners to avoid erring too much in either direction. While much has been said about the approaches to static analysis and improving result quality, an Achilles heel of any analysis tool is the quality of the specification that comes with. Simply, the analysis engine needs to be told what to look for. If the tool doesn’t know what methods are used for sending queries to a database and are therefore crucial to SQL injection finding or if some taint sources are omitted for XSS, some potential vulnerabilities will go missing. The following is but a short list of taint sources in typical .NET APIs:

Request.Params
Request.QueryString
Request.Form
Request.Headers

Request.ServerVariables
Request.Cookies
TextBox.Text
HiddenField.Value

However, as a recent blog post discusses, this list is woefully incomplete. Even in the case of taint-style vulnerabilities, coming up with a complete specification is really hard for several reasons. First, it often requires pouring over tons of potentially relevant APIs. There are about 30 methods in .NET base class libraries alone that CAT.NET considers to be taint sources. Second, in many cases, the specification is application- or library-specific. For a large-scale app, it’s not uncommon to have a custom encoder or a sanitizer. Unless the tool knows about it, false positives will likely result.

Enter Merlin, an add-on for CAT.NET that aims to produce a better, more complete and accurate specification for finding security bugs. Merlin uses the intuition embedded within the application itself to infer a better specification.

Code Example

Consider a simple example below. Suppose we are trying to classify functions ReadData1, ReadData2, WriteData, and Cleanse as either sources, sanitizers, or sinks. Unless we know something else about these example, this is hard to do.

                1: void ProcessRequest()

               2: {

               3:    string s1 = ReadData1("name");

               4:    string s2 = ReadData2("encoding");

               5:  

               6:    string s3 = Cleanse(s1);

               7:  

               8:    WriteData("Parameter " + s1);

               9:    WriteData("Header " + s2);

              10: }

However, if you are told that ReadData1 and ReadData2 are sources and Cleanse is a sanitizer, then WriteData is likely to be is a sink. Why? Because why else would the developer sanitize tainted input on not one, but two paths. This is most obvious if you look at the propagation graph on the right.

In other words, the program itself contains valuable clues that tell us what the developer thought these functions were with respect to tainting or untaining values. Of course, the developer could still be wrong, there would be no vulnerabilities otherwise, so Merlin generally looks for preponderance of evidence before drawing any conclusions. In this case, Merlin will be able to classify WriteData as a sink with a pretty high probability. The technique that Merlin uses called statistical inference is a generalization of the intuition above.

What does Merlin give us: some results

Before we talk about the gory technical details, which are also described in great detail in our technical report, I’ll give you a glimpse of what our approach can do by citing some experimental results. What is utterly surprising is that, without requiring any initial specification at all, Merlin is able to derive the following specification shown on the right from the little program below.

       1: protected void TextChanged(object sender, EventArgs e) {

       2:     string str = Request.QueryString["name"];

       3:     string str2 = HttpUtility.HtmlEncode(str);

       4:     Response.Write(str2);

       5: }

       6: protected void ButtonClicked(object sender, EventArgs e) {

       7:     string str = Request.UrlReferrer.AbsolutePath;

       8:     string str2 = HttpUtility.UrlEncode(str);

       9:     Response.Redirect(str2);

      10: }

Here is what Merlin infers for this example program:

Sources (1):
string System.Web.HttpUtility+UrlDecoder.Getstring()
Sanitizers (8):
string System.Web.HttpUtility.HtmlEncode(string)
string System.Web.HttpUtility.UrlEncodeSpaces(string)
string System.Web.HttpServerUtility.UrlDecode(string)
string System.Web.HttpUtility.UrlEncode(string, Encoding)
string System.Web.HttpUtility.UrlEncode(string)
string System.Web.HttpServerUtility.UrlEncode(string)
string System.Web.HttpUtility.UrlDecodestringFromstringInternal...
string System.Web.HttpUtility.UrlDecode(string, Encoding)
Sinks (4):
void System.Web.HttpResponse.WriteFile(string)
void System.Web.HttpRequest.set_QuerystringText(string)
void System.IO.TextWriter.Write(string)
void System.Web.HttpResponse.Redirect(string)

This is actually a pretty good start! I highlighted a few of the methods so that you can look up their definition at MSDN. The sanitizers are in fact standard URL and HTML encoders provided by .NET. Note that even some of these are missing from the default .NET specification. You will recognize some other methods like HttpResponse.Redirect as sinks for the reflective XSS vulnerability. Similarly, HttpResponse.WriteFile exposes a command injection vulnerability.

Of course, the better your initial specification, the more Merlin can help. The second part of this blog will talk about the technical aspects of Merlin inference and also show some more results.

Technorati Tags: CAT.NET,static analysis

Security Code Review Using CAT.NET - Part 2

cisg — Mon, 22 Dec 2008 12:24:55 GMT

Hi Andreas Fuchsberger here again......

How does CAT.NET work?

As I mentioned in Part 1 here, CAT.NET is an information-flow type static analysis tool using an implementation of tainted-variable analysis.

Tainted-variable analysis is an integrity problem in which that tries to identify whether less-trusted data obtained from the user might influence other data that the system trusts. Clearly, to do this analysis, sources and sinks of possibly tainted data need to be identified. For managed code, this amounts to identifying methods that originate a tainted value and methods that use a possibly tainted value. For CAT.NET a number of XML of user editable configuration files is used to define sources and sinks. Then CAT.NET needs to find how information is stored in a variable and where it is used later in any other module of the application.

CAT.NET uses the Common Compiler Infrastructure (CCI) which is used extensively within Microsoft for building compiler-like tools.CCI is an integrated set of components that encapsulate the logic that compilers and related development tools typically have in common. CCI has many features but firstly for CAT.NET it has the ability to read the Common Intermediate Language (CIL) used to store binary code in a .NET Framework assembly directly.

Further to perform its analysis CAT.NET needs to build of a specific heap analysis called flow-insensitive points-to analysis. This analysis computes a “may point to” relation over a loaded assembly or assemblies, we’ll call this relation pointsTo, where pointsTo(o₁.f, o₂) means that the field f of the object named o₁ might refer to the object named o₂ in some execution of the program. A may-point-to relation is also computed for local variables: pointsTo(υ, o) means that the local variable υ might refer to the object named o. The relation pointsTo(υ.f, o) holds if there exists an o’ such that pointsTo(υ, o’) and pointsTo(o’.f, o).

CAT.NET uses a combination of Control Flow and Data Flow Graphs and to build the relation for every object in every module supplied to CAT.NET.

Control Flow Graphs

A control flow graph (CFG) is a representation of a program where contiguous regions of code without branches, known as basic blocks, are represented as nodes in a graph and edges between nodes indicate the possible flow of the program. A CFG shows the sequence of events as a program executes.

Data Flow Graphs

A data-flow graph (DFG) is a graph which represents operations and data dependencies and the order the operations are performed. As such any algorithm consists of a number of ordered operations. However simple DFGs are not able to represent loops or sub routine branching. Data Flow Graphs are therefore are often augmented with control-flow information and are then known as Control Data Flow Graphs (CDFG). A DFG consists of nodes and arcs, where the each node represented has an input or an output port and an arc represents a connection between and input and output port.

Data Flow Super Graphs

Defined by CAT.NET a data flow super graph is a special type of data flow graph that contains data flow information at both an intra-procedural and inter-procedural level.

CCI provides functions for building the Data Flow and Control Flow Graphs on an intra-procedural level and CAT.NET uses these to build a Data Flow Super Graph. The Data Flow Super Graph that CAT.NET builds covers all objects across methods in all modules on an inter-procedural level.

Once the Data Flow Super Graph is built, CAT.NET iterates for each of the XML that makes up the CAT.NET rules across Data Flow Super Graph to find all data flow paths between the sources and the sinks. It does this by traversing each path in the Data Flow Super Graph and colouring (i.e. assigning a constant to a traversed path) the graph according to the variables use.

Before reporting a source is linked to a sink, CAT.NET checks how the variable is transformed and filters out valid transformations. Variables that remain tainted once a complete source to sink path has been traversed are reported as a possible vulnerability in the original code including file name and line numbers.

In the next post I will explain the semantics of the XML rules and how to modify the supplied rule set.

Security Code Review Using CAT.NET - Part 1

cisg — Mon, 22 Dec 2008 12:20:19 GMT

Hi Andreas Fuchsberger here …

To coincide with the CTP release of CAT.NET and Anti-XSS, within the CSIG we have been taking a long hard look at static analysis tools for developers and Information Security professionals. Over the next series of blog posts I will explain the fundamentals of the techniques used for a code review in general and static code analysis in particular.

Today we start with an introduction into the review process and as well having a look at the techniques for automating some of the code review burden; and don't worry we will build up to the details fast.

In general we differentiate between a code review (source code) and an architectural review (software architecture). Although CAT.NET actually reads binary, actually Microsoft Intermediate Language code, it falls into the class of source code review tools.

What is a code review?

We say a code review is the process of systematically examining the source code written by one developer by someone other than the original developer, often called a reviewer. The reviewer could be a colleague or fellow developer or someone with similar developer background and skill set then we refer to this as a peer-review. Although most developers see the purpose of a code review to find coding errors the true value really lies really in the reviewer questioning assumptions by the original developer. It is often these unconsciously made assumptions that are the cause of business logic errors but also simple security errors (“Oh the user would never try to enter more than 10 digit telephone number”) .

Formal roots of the code review go as far back as 1983 when it was first mentioned in a standard, IEEE 729, although it is probably true to say that code reviews took place before using corporate or in-house rules.

According to the IEEE standard 729 “Glossary of Software Engineering Terminology”, which has since been republished as IEEE 610.12, a review is formally planned and structured analysis and evaluation process, during which the project results are presented to a team of consultant reviewers and commentated or approved by them.

The reviewer could be a fellow software developer. For inexperienced developers a code review offers a good opportunity to learn, for an experienced developer it offers the opportunity for training themselves in fast and hands-on manner.

And this unfortunately this is why the review process is so expensive both in cost of time and money. Reviewers are highly skilled people and performing a review takes time. So for some time now the Information Security research community has proposed automatic analysis techniques for identifying security vulnerabilities in code. Such techniques may employ dynamic analysis, static analysis, or both.

Dynamic analysis entails executing the program and inferring properties from its observed behaviour. Although dynamic analysis may expose bugs, it cannot ensure complete coverage of the target program as not all program paths may be actually run.

Instead, developers have adopted an approach based on sound analysis, which can provably identify all possible violations of specific security guidelines.

In contrast Static analysis does not execute the code but analyses the code itself. Simple static analysis tools analyses the actual source code, however increasingly modern static analysis tools use also or only look at the binary output of the compiler. For intermediate code output, such as Microsoft’s .NET Framework, this is in fact often preferable as the intermediate output still contains type information and

Static analysis techniques

There exist a number of static analysis techniques that can be used to automatically detect security errors in source code. These techniques cover three areas that have been associated with sources of security vulnerabilities:

Access control
Information flow
- Integrity violations
- Confidentiality violations
API conformance

Access control: Mechanisms for access control restrict access to security-sensitive resources based on a user's identity and/or membership in predefined groups. Ensuring that an access control policy enforces the required level of security can be difficult, especially for systems with many components of different trust levels with access to a multitude of restricted resources. If an access control policy does not grant sufficient permissions to users, runtime authorization failures will occur. On the other hand, if an access control policy grants users unnecessary permissions, this will result in exposing the exposing the system to security attacks.

Information flow: A secure information flow ensures that information propagates throughout the execution environment without violating two security integrity properties:

Integrity violations arise when untrusted information flows into a trusted execution environment without having been properly validated. A malicious user could compromise a system by exploiting an integrity violation.

Confidentiality violations arise when confidential information flows from a restricted execution environment to a public one without having been properly declassified. For example, a confidentiality violation arises if trusted code exposes a cryptographic private key to untrusted code.

API conformance: Applications often rely on libraries and third-party APIs to provide security-sensitive services. As an example, many applications rely heavily on cryptography libraries to protect confidentiality, prevent integrity violations, and distinguish between trusted and untrusted entities. Incorrect usage of cryptographic functions may lead to insecure storage of security-sensitive information and cause violations of integrity and confidentiality policies.

Our CAT.NET tool is based on finding integrity violations by performing a tainted data flow analysis. Future posting will take a closer look at the algorithms we used for performing our analysis.