Code Analysis Team Blog : Code Analysis

Fun with Code Analysis Rule Sets

cristend — Wed, 12 Nov 2008 22:55:00 GMT

Two weeks ago, I introduced you to Visual Studio 2010 September '08 CTP (Community Technology Preview). This CTP is available as a Virtual PC (VPC) image which can be downloaded at http://go.microsoft.com/fwlink/?LinkId=129231. The goal of the Community Technology Preview is to obtain feedback from our customers on the new scenarios we have enabled. If you haven't already downloaded the CTP, do it now and take it out for a spin.

There are many new cool features with the CTP such as an Historical Debugger and Impact Analysis of code changes, but today we are going to focus on Rule Sets for code analysis. (For an overview of all the neat stuff happening in code analysis, check out this PDC video http://mschnlnine.vo.llnwd.net/d1/pdc08/WMV-HQ/TL60.wmv ) Configuring code analysis in Visual Studio 2008 and earlier versions required evaluating each individual rule to determine whether it was important enough to turn on and fix for your project. This might be a very time-consuming process that made it difficult to make sure that the most important problems were identified and fixed. With rule sets, code analysis can now be configured much more quickly and easily with sets of rules that are targeted at specific scenarios or areas of emphasis. Let's take a look at how this works.

The following scenario comes from our CTP walkthrough. You can find the walkthrough and more details when you download the CTP as described above. Let's assume that you have the DinnerNow – Kiosk solution open and is the active project. On the Analyze menu, click Configure Code Analysis for DinnerNow.Restaurants.Kiosk. The property page for Code Analysis is displayed:

This screen tells us that Microsoft All Rules will be executed when we run code analysis, but what if we didn't want to run every Microsoft rule? What if we have a specific scenario we wish to target? We can change which rule set to run by clicking on Open which brings up:

We can select which rule set(s) by simply selecting or deselecting the appropriate box. Notice that each rule set has a brief description of the types of rules within the rule set. Further, you can open up each individual rule set and see the specific rules that make up the rules sets. You can also turn on or off individual rules within a rule set. Finally, you can adjust the type of action a rule will trigger. In the past, code analysis rules have been limited to build Warnings, but now you can elevate code analysis rules to build Errors.

As you can see, rules sets provide a powerful way to focus code analysis on the scenarios and rules that matter most to you. You can also create custom rule sets based on your specific needs. We wont go into it today, but if you download the CTP, the Walkthrough explains how to customize a rule set.

Give it a try and let us know what you think.

Cris

New Code Analysis Features in Visual Studio 2010 September '08 CTP

cristend — Thu, 30 Oct 2008 20:05:00 GMT

At this year's PDC, Microsoft released Visual Studio 2010 September '08 CTP (Community Technology Preview). This CTP is available as a Virtual PC (VPC) image which can be downloaded at http://go.microsoft.com/fwlink/?LinkId=129231. The goal of the Community Technology Preview is to obtain feedback from our customers on the new scenarios we have enabled. If you are interested in code analysis, you need to get this CTP and here is why.

The code analysis team is excited to share with you new code analysis features and rules in this CTP, including:

Rule sets - code analysis can now be configured much more quickly and easily with sets of rules that are targeted at specific scenarios or areas of emphasis
Enhanced policy support - enforce your rule set with the code analysis check-in policy
8 New Data Flow rules - find hard-to-detect flaws with these advanced dataflow rules for managed code; including certain SQL injection vulnerabilities

Included with the CTP is a walkthrough guide. Inside the walkthrough is a section dedicated to these exciting code analysis scenarios. Over the next couple of weeks we will continue to blog about rule sets, policy support, and data flow rules so be sure to come back after you get the CTP.

Now go get the CTP and starting sending us your feedback! http://go.microsoft.com/fwlink/?LinkId=129231

New for Visual Studio 2008 SP1 and FxCop 1.36 – Multi-targeting rule

David M. Kean — Mon, 25 Aug 2008 17:00:00 GMT

I've just published a post over my blog about a new feature we added for Visual Studio 2008 SP1 and FxCop 1.36.

New for Visual Studio 2008 SP1 and FxCop 1.36 – Multi-targeting rule

FAQ: How do I prevent FxCop 1.36 from firing warnings against generated code?

David M. Kean — Thu, 28 Feb 2008 21:39:29 GMT

I've upgraded from FxCop 1.35 to 1.36 and now FxCop has started to fire warnings against typed DataSets and other generated code. How do I turn this off?

The reason this is occurring is because we changed the way that FxCop analyzed generated code. Whereas previously in 1.35, FxCop would only ignore particular generated code (such as DataSets), FxCop 1.36 will now either ignore no generated code, or all generated code. The former is the default.

To change this behavior and have FxCop skip over generated code:

Using an FxCop project:

Open your FxCop project in FxCop
Choose Project -> Options -> Spelling & Analysis
Check Suppress analysis results against generated code
Click OK

Via the command-line:

Pass the /ignoregeneratedcode switch, for example:

FxCopCmd.exe /file:MyAssembly.dll /out:AnalysisResults.xml /ignoregeneratedcode

For more information on FxCop's behavior over generated code, see Correct usage of the CompilerGeneratedAttribute and the GeneratedCodeAttribute.

Tutorial on writing your own Code Analysis rule

David M. Kean — Fri, 18 Jan 2008 18:00:00 GMT

Jason Kresowaty has posted a great tutorial on writing Code Analysis/FxCop rules. He also spends time on explaining parts of the Introspection API, including the different nodes and their relationship to each other.

It's a great read, even if you've written some Code Analysis rules before. Definitely recommended.

What do you want to see in the second edition of Framework Design Guidelines?

David M. Kean — Tue, 15 Jan 2008 18:00:00 GMT

Krzysztof and Brad have announced they are working on the second edition of the awesome Framework Design Guidelines and are looking for feedback on what they should put in it.

For those that don't know, a lot of our Code Analysis rules are based on the writings in this great book, so expect to see additional rules in the future based on the new guidelines in the second edition.

To provide feedback, head over to Krzysztof's blog and post a comment.

FAQ: Which Code Analysis rules shipped in which version?

David M. Kean — Mon, 07 Jan 2008 21:04:00 GMT

In response to a lot of recent requests, we've put together a complete list of rules that shipped in the different versions of Visual Studio Code Analysis and FxCop. Attached is an Excel worksheet providing this information for Visual Studio 2005, Visual Studio 2008, FxCop 1.35 and FxCop 1.36 Beta.

One of things you'll notice as you read through the list is that we removed some rules from the later versions. There are a few reasons for this:

Noise and applicability. We use feedback from customers, SQM data (to see which rules users turn off), and input from internal teams (Windows, Office, CLR, etc) to determine the rules that are noisy without adding any perceivable value. There are also rules that are either no longer applicable or can no longer fire. for example, a rule could have been firing on a limitation of the CLR which has since been fixed in later versions.
Merged rules. Sometimes it makes sense to merge rules that fire on similar things, for example, the analysis in SecureGetObjectDataOverrides was already covered by OverrideLinkDemandsShouldBeIdenticalToBase, so these two rules were merged. Similarly, LongAcronymsShouldBePascalCased, ShortAcronymsShouldBeUppercase and IdentifiersShouldBeCasedCorrectly all fired on the casing of identifiers, and hence were merged in the later.
Analysis engine removed. In Visual Studio 2008 and FxCop 1.36 we removed one of our analysis engines. This engine was removed for a variety of reasons; it increased analysis time (although the engine encompassed less than 5% our analysis, it took up 50% of our time-to-analyze), indeterministic results (results appearing and disappearing between runs), and bugs found within the engine (and hence the rules that depended on it) required huge architectural changes. We instead decided to invest the resources that we would have spent on fixing the old engine, on a new data flow analysis engine based on Phoenix, which we will ship in a future version of Visual Studio.

There are also more differences between Visual Studio Code Analysis and FxCop than just the rules - in a future blog post I will cover these in detail.

Reporting Code Analysis Defect Counts

conorm — Wed, 02 Jan 2008 23:10:06 GMT

Code analysis defect counts are available in the cube that is part of the TFS Data Warehouse. This post shows you how to hook up Excel to show code analysis counts in a pivot table.

First off you need to connect Excel up to the data warehouse that has the data. Under the Data tab in Excel select From Other Sources, From Analysis Services.

Then you need to give the name of the server with the data warehouse. In the standalone TFS case this will be whatever machine is running TFS. Assuming you have read permissions with your Windows account you can click Next, otherwise enter a user name and password that has access to the server and click Next.

Then, select the database (usually TfsWarehouse) and select the Build table.

Click Next and then supply whatever additional information that will help you remember this data connection.

Click Finish. Then define how you want to show the data in Excel - e.g. a pivot table.

Click OK which gives you the following:

In the PivotTable Field List, Expand Build Project where you will find two fields: Static Analysis Errors and Static Analysis Warnings:

You can click the checkboxes to add these to the values for your table. In most cases Static Analysis Warnings would be the most useful since Static Analysis Errors would typically stop the build.

To complete the picture you can play with the pivot table and pivot and filter as you see fit. As a simple example you can add the Build field:

Which will give you one row per build in the pivot table.

Of course this data can be combined with other build data and charted etc. according to your needs. We are considering what reporting to provide in future versions so would love to hear your feedback on this feature and what you would like to see.

Come chat with the Code Analysis team tomorrow

David M. Kean — Tue, 04 Dec 2007 20:16:00 GMT

Join members of the Visual Studio Team System product group to discuss features available in Team Foundation Server, Team Suite, Architecture Edition, Development Edition, Database Edition, and Test Edition. In addition, discuss what's new for these editions for Visual Studio 2008.

We will be holding two sessions:

Join the chat on Wednesday, December 5th, 2007 from 10:00am - 11:00am Pacific Time. Add to Calendar | Additional Time Zones

-and-

Join the chat on Wednesday, December 5th, 2007 from 4:00pm - 5:00pm Pacific Time. Add to Calendar | Additional Time Zones

Try the Code Analysis Rule Sets Feature in the November "Rosario" November 2007 CTP

conorm — Thu, 29 Nov 2007 01:52:00 GMT

We have a great new feature that we are previewing in Microsoft® Visual Studio® Team System code name “Rosario” November 2007 CTP. This feature allows you to select pre-defined sets of rules to configure code analysis as well as define your own. To access this feature right click on a project and choose "Properties" then click on the "Code Analysis" tab.

This feature is in response to the feedback we received that the number of rules is overwhelming and difficult to configure.

For this preview release we have focused on the core concept and made minimal changes to the UI. In RTM we will have a new UI and a rich edit experience. This release is all about getting your feedback. Try out this feature on your code, use the built-in rule sets and let us know what you think. Tell us what we missed or what rules you think are there and should not be. Your real-life experience is our best guide for this feature.

You can create a custom rule set to fine tune the exact rules to use. If you think we should include your custom rule set in the out of the box sets then we want to hear from you (feedback links below). Just attach the exported file to your connect bug or mail us.

Some useful links:

Release Post from JeffBe and download instructions - here
Feedback - bugs and suggestions can be submitted via http://connect.microsoft.com/visualstudio. Make sure when you submit issues through Connect that you indicate the product version as being “Visual Studio Team System Code Name “Rosario” (November 2007 CTP)”. That will ensure it gets to the right folks for consideration.
Questions - we’ve created a special forum specifically focused on Rosario: http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=1736&SiteID=1 or you can respond to this blog post.

We look forward to hearing from you and hope you enjoy this new feature.

Maintainability Index Range and Meaning

conorm — Tue, 20 Nov 2007 19:43:00 GMT

Another question:

The maintainability index has been re-set to lie between 0 and 100. How and why was this done?

The metric originally was calculated as follows (based on the work in Carnegie Mellon University although we modified the Halstead Volume calculation a little since we don't include comments anywhere in our calculation): Maintainability Index = 171 - 5.2 * ln(Halstead Volume) - 0.23 * (Cyclomatic Complexity) - 16.2 * ln(Lines of Code)

This meant that it ranged from 171 to an unbounded negative number. We noticed that as code tended toward 0 it was clearly hard to maintain code and the difference between code at 0 and some negative value was not useful. I'll post some tech ed sample code showing very low maintainability or you can try on your own code to verify. As a result of the decreasing usefulness of the negative numbers and a desire to keep the metric as clear as possible we decided to treat all 0 or less indexes as 0 and then re-base the 171 or less range to be from 0 to 100. Thus, the formula we use is:

Maintainability Index = MAX(0,(171 - 5.2 * ln(Halstead Volume) - 0.23 * (Cyclomatic Complexity) - 16.2 * ln(Lines of Code))*100 / 171)

On top of that we decided to be conservative with the thresholds. The desire was that if the index showed red then we would be saying with a high degree of confidence that there was an issue with the code. This gave us the following thresholds (as mentioned in this blog previously):

For the thresholds we decided to break down this 0-100 range 80-20 so that we kept the noise level low and only flagged code that was really suspicious. We have:

0-9 = Red
10-19 = Yellow
20-100 = Green

Code Metrics as Check-in Policy

conorm — Fri, 16 Nov 2007 10:39:00 GMT

This is the first in a series of posts around questions I am getting asked related to code metrics.

One question that is coming up is:

Can a check-in policy be defined based on code metrics in TFS?

Well, code metrics itself does not have a check-in policy but fortunately the Maintainability Index, Class Coupling, Depth of Inheritance and Complexity metrics are all available as rules in Code Analysis in Visual Studio. This means you can use code analysis check-in policy enabled with those rules. The rules are grouped under the "Maintainability Rules" Category and are configured as follows:

Metric	Corresponding Rule	Threshold
Depth of Inheritance	CA1501 AvoidExcessiveInheritance	Warning at above 5 levels deep
Complexity	CA1502 AvoidExcessiveComplexity	Warning at above 25
Maintainability Index	CA1505 AvoidUnmaintainableCode	Warning at below 20
Class Coupling	CA1506 AvoidExcessiveClassCoupling	Warning at above 80 for class and above 30 for a method

In the policy editor this would look like the following:

This allows you to enforce that code meets these thresholds before check-in. In future we are looking in to allowing the threshold values to be changed.

FxCop 1.36 Beta Released!

David M. Kean — Wed, 10 Oct 2007 23:09:10 GMT

I'm pleased to announce that we've released a beta of the next version of FxCop. This release has a number of changes since FxCop 1.35, including, but not limited to, the following:

200+ bug fixes that reduce noise, missing analysis and rule crashes
Support for analyzing anonymous methods and lambda expressions
New option for skipping analysis over tool generated code
- Turn this on in the UI via Project -> Options -> Spelling & Analysis -> Suppress analysis results against generated code
- Turn this on in the command-line via /ignoregeneratedcode switch.
Better support for C++/CLI and the Compact Framework
Language 'friendly' API names in the UI and resolutions (ie Visual Basic syntax if running over a Visual Basic binary)
New globalization, design and usage rules
Performance improvements that cut analysis by 2x and use half as much memory
Documentation that is now available on MSDN

There are also some other underlying changes that I will talk about in some future posts.

Download it, and tell us what you think. If you have any feedback, bug reports or any questions, feel free to head over to the Code Analysis forum and ask away.

New for Visual Studio 2008 - The 'Analyze' menu

David M. Kean — Sat, 22 Sep 2007 17:00:00 GMT

While Ian Huff has already talked about the new 'Developer' menu that has been added to Visual Studio 2008, I thought I would briefly mention some of the changes to it that were made between Beta 2 and RTM.

For those that have yet to see this menu, similar to the existing Test menu, we've added a new top-level menu that will contain the Code Analysis, Code Metrics and Profiling menu items. This was done for discoverability reasons, and something we hope to leverage moving forward towards Rosario.

A quick rundown on some of the things that have changed since Beta 2:

Renamed the menu itself from the very non-specific Developer to the little bit more descriptive Analyze.
Removed the Code Analysis and Code Metrics sub menus and pulled their associated menu items up one level.
Pulled New Diff Report up one level and renamed it to Compare Performance Reports...
Renamed the Code Metrics and Code Analysis menu items:
Generate Code Metrics -> Calculate Code Metrics.
Migrate Code Analysis Policy Settings to Solution -> Code Analysis Settings for Solution.
Replace Code Analysis Settings with Code Analysis Policy Settings -> Replace with Check-in Policy
Add Code Analysis Policy Settings to Code Analysis Settings -> Merge with Check-in Policy

I probably don't have to tell that these changes, that we made to increase the usability of the menu, are great improvements, especially the renaming of probably what would have been in the running for the longest menu item in Visual Studio; Replace Code Analysis Settings with Code Analysis Policy Settings.

While a very small addition to Visual Studio 2008, we hope the new Analyze menu helps improve your workflow when working with the Code Analysis and Profiler features.

New for Visual Studio 2008 - Support for anonymous methods and lambda expressions

David M. Kean — Fri, 21 Sep 2007 17:00:00 GMT

One of my favorite new features for Code Analysis in Visual Studio 2008 is our support for analyzing anonymous methods and lambda expressions. While anonymous methods have been around in C# since Visual Studio 2005, lambda expressions are new for both C# and Visual Basic in Visual Studio 2008.

For those that are already familiar with the concept of anonymous methods (if not check out this article), lambda expressions provide a simpler, more concise syntax for writing inline methods. They also serve as the underpinnings of Linq.

Take the following code written in Visual Studio 2005 which converts a list of strings to uppercase, for example:

   internal static class StringFormatter
    {
        public static List<string> ToUpper(List<string> values, CultureInfo culture)
        {
            return values.ConvertAll(delegate(string value)
            {
                return value.ToUpper(culture);
            });
        }
    }

Using the new lambda syntax, the equivalent can be written in C# and Visual Basic like so:

        internal static class StringFormatter
    {
        public static List<string> ToUpper(List<string> values, CultureInfo culture)
        {
            return values.ConvertAll(value => value.ToUpper(culture));
        }
    }

   Friend Module StringFormatter

        Public Function ToUpper(ByVal values As List(Of String), ByVal culture As CultureInfo) As List(Of String)

            Return values.ConvertAll(Function(value) value.ToUpper(culture))

        End Function

    End Module

To understand how lambda expressions and anonymous methods are implemented underneath in IL (which is important for a tool like Code Analysis that runs over the binary and not the source), see the great in-depth posts by Raymond Chen for C# (Parts 1, 2, and 3) and Jared Parsons for Visual Basic (Parts 1, 2, 3, 4 and 5). Once you've read those posts, you'll understand why Code Analysis didn't get support for this for free.

What does this mean for Code Analysis users? Well, previously, in Visual Studio 2005, the engine would simply skip over these constructs and the user wouldn't find out about any violations, if any, contained within them. The good news is that we will now analyze them. The bad news is that any project making heavy use of anonymous methods is likely no longer Code Analysis clean on its upgrade to Visual Studio 2008. ;)

For a little more information on Code Analysis's support of anonymous methods, see the following topic: Anonymous Methods and Code Analysis (code formatting to be fixed for RTM).

Note: These is a bug in Beta 2 that prevents Code Analysis from analyzing lambdas/anonymous methods that access any locals or parameters outside of its own scope. This will be fixed in RTM of Visual Studio 2008.