Code Analysis Team Blog

New for Visual Studio 2008 SP1 and FxCop 1.36 – Multi-targeting rule

I've just published a post over my blog about a new feature we added for Visual Studio 2008 SP1 and FxCop 1.36.

New for Visual Studio 2008 SP1 and FxCop 1.36 – Multi-targeting rule

FxCop 1.36 Released!

We are pleased to announce that we've released the next version of FxCop. This release has a number of changes since FxCop 1.36 Beta 2, including, but not limited to, the following:

• Bug fixes that reduce noise, missing analysis and rule crashes.
• New portability rule: Use Only API from Targeted Framework.  This rule will fire when the target assembly taking a dependency on a new API only included in a later version or in a service pack.
• New option for new portability rule for selecting target framework for analysis.
• Set the target framework in the UI via Project -> Option -> Spelling & Analysis -> Target Framework
• Set the target framework on the command-line via the /targetframeworkversion switch. e.g: /targetframeworkversion:v2.0, /targetframeworkversion:v3.0, or /targetframeworkversion:v3.5

Compared with FxCop 1.35, FxCop 1.36 Beta 2 had changes including, but not limited to, the following:

• 200+ bug fixes that reduce noise, missing analysis and rule crashes
• Support for analyzing anonymous methods and lambda expressions
• New option for skipping analysis over tool generated code 
• Turn this on in the UI via Project -> Options -> Spelling & Analysis -> Suppress analysis results against generated code
• Turn this on on the command-line via the /ignoregeneratedcode switch. 
• Better support for C++/CLI and the Compact Framework
• Language 'friendly' API names in the UI and resolutions (ie Visual Basic syntax if running over a Visual Basic binary)
• New globalization, design and usage rules
• Performance improvements that cut analysis by 2x and use half as much memory
• Documentation that is now available on MSDN

There are also some other underlying changes that we will talk about in some future posts. 

Download it, and tell us what you think. If you have any feedback, bug reports or any questions, feel free to head over to the Code Analysis forum and ask away.

Note:  If you already have FxCop 1.36 Beta 2 installed, please uninstall it first before you install FxCop 1.36. Otherwise, you may see an error when you install FxCop 1.36 on top of FxCop 1.36 Beta 2.

If you accidentally install FxCop 1.36 without uninstalling FxCop 1.36 Beta 2, please go to Add/Remove program to uninstall FxCop 1.36, and reinstall FxCop 1.36.

David Kean's new blog

For those that are interested, I've just started a new personal blog, Dave's Box, where I will be talking about FxCop, Framework Design Guidelines, and the recently announced Managed Extensibility Framework.

I've just posted a couple of posts detailing changes we made to FxCop 1.36: 

FxCop now ships with the spell checker libraries
Reference resolution changes in Code Analysis and FxCop

You can subscribe via the one of the two following links:

 Subscribe via reader
 Subscribe via email

FxCop 1.35 now available (again) for download

We've just uploaded FxCop 1.35 to MSDN Code Gallery after we lost our home on GotDotNet - this is for those teams that either do not want to or just aren't ready to upgrade to FxCop 1.36 Beta.

For those that are already using FxCop 1.35, you'll probably notice that we flicked the switch to cause it to prompt you to download FxCop 1.36. We've love you to download it and try it out (it can be installed side-by-side without any issues with FxCop 1.35). If you hit any bugs, issues or just have some feedback about the new or existing versions of FxCop, feel free to head over to the Code Analysis forum and post away.

FAQ: How do I prevent FxCop 1.36 from firing warnings against generated code?

I've upgraded from FxCop 1.35 to 1.36 and now FxCop has started to fire warnings against typed DataSets and other generated code. How do I turn this off?

The reason this is occurring is because we changed the way that FxCop analyzed generated code. Whereas previously in 1.35, FxCop would only ignore particular generated code (such as DataSets), FxCop 1.36 will now either ignore no generated code, or all generated code. The former is the default.

To change this behavior and have FxCop skip over generated code:

Using an FxCop project:

1. Open your FxCop project in FxCop
2. Choose Project -> Options -> Spelling & Analysis
3. Check Suppress analysis results against generated code
4. Click OK

Via the command-line:

1. Pass the /ignoregeneratedcode switch, for example:
   
   FxCopCmd.exe /file:MyAssembly.dll /out:AnalysisResults.xml /ignoregeneratedcode

For more information on FxCop's behavior over generated code, see Correct usage of the CompilerGeneratedAttribute and the GeneratedCodeAttribute.

What happened to the download and documentation of FxCop 1.35?

Update: You can now download FxCop 1.35 directly from MSDN Code Gallery. 

As you many you have well noticed, GotDotNet was officially retired last month now that the last of its functionality was superceded by MSDN Code Gallery. Unfortunately, at the same time, the FxCop team page, the download for FxCop 1.35 and the rule documentation was also removed.

We were told last year about the deprecation of GotDotNet and we had hoped that by now we would have had a new team site on MSDN, however, a variety of factors, some within our control and some beyond our control, prevented that from happening in time.

In the meantime, while we sort out new home for our beloved FxCop, here are a couple of workarounds:

To download FxCop

• Upgrade to FxCop 1.36 Beta. This is one of our most stable beta's to-date; which is not surprising considering that some of the same bits also shipped in Visual Studio 2008. Go ahead, download FxCop 1.36 today - you won't be disappointed.
  
  
• Download the Windows SDK. The Windows SDK includes the FxCop 1.35 setup. You might already have it installed if you have Visual Studio 2008, otherwise, download it. Either way, once installed, FxCop 1.35 setup can be found via Start -> All Programs -> Microsoft Windows SDK [Version] -> Tools -> Install Microsoft FxCop.

 To view the documentation

• MSDN Library. Both the FxCop documentation and rule documentation are available on the MSDN Library.
  
  
• Installed Help. FxCop 1.35 comes with a CHM (pronounced 'chum') containing FxCop and rule documentation. Albeit a little out-of-date when compared to the online version, access it via Start -> All Programs -> Microsoft FxCop 1.35 -> FxCop Documentation or via Help -> Using Microsoft FxCop inside FxCop.

Tutorial on writing your own Code Analysis rule

Jason Kresowaty has posted a great tutorial on writing Code Analysis/FxCop rules. He also spends time on explaining parts of the Introspection API, including the different nodes and their relationship to each other.

It's a great read, even if you've written some Code Analysis rules before. Definitely recommended.

What do you want to see in the second edition of Framework Design Guidelines?

Krzysztof and Brad have announced they are working on the second edition of the awesome Framework Design Guidelines and are looking for feedback on what they should put in it. For those that don't know, a lot of our Code Analysis rules are based on the writings in this great book, so expect to see additional rules in the future based on the new guidelines in the second edition.  To provide feedback, head over to Krzysztof's blog and post a comment.

FAQ: Which Code Analysis rules shipped in which version?

In response to a lot of recent requests, we've put together a complete list of rules that shipped in the different versions of Visual Studio Code Analysis and FxCop. Attached is an Excel worksheet providing this information for Visual Studio 2005, Visual Studio 2008, FxCop 1.35 and FxCop 1.36 Beta.

One of things you'll notice as you read through the list is that we removed some rules from the later versions. There are a few reasons for this:

• Noise and applicability. We use feedback from customers, SQM data (to see which rules users turn off), and input from internal teams (Windows, Office, CLR, etc) to determine the rules that are noisy without adding any perceivable value. There are also rules that are either no longer applicable or can no longer fire. for example, a rule could have been firing on a limitation of the CLR which has since been fixed in later versions.
  
  
• Merged rules. Sometimes it makes sense to merge rules that fire on similar things, for example, the analysis in SecureGetObjectDataOverrides was already covered by OverrideLinkDemandsShouldBeIdenticalToBase, so these two rules were merged. Similarly, LongAcronymsShouldBePascalCased, ShortAcronymsShouldBeUppercase and IdentifiersShouldBeCasedCorrectly all fired on the casing of identifiers, and hence were merged in the later.
  
  
• Analysis engine removed. In Visual Studio 2008 and FxCop 1.36 we removed one of our analysis engines. This engine was removed for a variety of reasons; it increased analysis time (although the engine encompassed less than 5% our analysis, it took up 50% of our time-to-analyze), indeterministic results (results appearing and disappearing between runs), and bugs found within the engine (and hence the rules that depended on it) required huge architectural changes. We instead decided to invest the resources that we would have spent on fixing the old engine, on a new data flow analysis engine based on Phoenix, which we will ship in a future version of Visual Studio.

There are also more differences between Visual Studio Code Analysis and FxCop than just the rules - in a future blog post I will cover these in detail.

Reporting Code Analysis Defect Counts

Code analysis defect counts are available in the cube that is part of the TFS Data Warehouse. This post shows you how to hook up Excel to show code analysis counts in a pivot table.

First off you need to connect Excel up to the data warehouse that has the data. Under the Data tab in Excel select From Other Sources, From Analysis Services.



Then you need to give the name of the server with the data warehouse. In the standalone TFS case this will be whatever machine is running TFS. Assuming you have read permissions with your Windows account you can click Next, otherwise enter a user name and password that has access to the server and click Next.



Then, select the database (usually TfsWarehouse) and select the Build table.



Click Next and then supply whatever additional information that will help you remember this data connection.



Click Finish. Then define how you want to show the data in Excel - e.g. a pivot table.



Click OK which gives you the following:



In the PivotTable Field List, Expand Build Project where you will find two fields: Static Analysis Errors and Static Analysis Warnings:



You can click the checkboxes to add these to the values for your table. In most cases Static Analysis Warnings would be the most useful since Static Analysis Errors would typically stop the build.

To complete the picture you can play with the pivot table and pivot and filter as you see fit. As a simple example you can add the Build field:



Which will give you one row per build in the pivot table.



Of course this data can be combined with other build data and charted etc. according to your needs. We are considering what reporting to provide in future versions so would love to hear your feedback on this feature and what you would like to see.

Come chat with the Code Analysis team tomorrow

Join members of the Visual Studio Team System product group to discuss features available in Team Foundation Server, Team Suite, Architecture Edition, Development Edition, Database Edition, and Test Edition. In addition, discuss what's new for these editions for Visual Studio 2008.

We will be holding two sessions:

Join the chat on Wednesday, December 5th, 2007 from 10:00am - 11:00am Pacific Time. Add to Calendar | Additional Time Zones

           -and-

Join the chat on Wednesday, December 5th, 2007 from 4:00pm - 5:00pm Pacific Time. Add to Calendar | Additional Time Zones

Try the Code Analysis Rule Sets Feature in the November "Rosario" November 2007 CTP

We have a great new feature that we are previewing in Microsoft® Visual Studio® Team System code name “Rosario” November 2007 CTP.  This feature allows you to select pre-defined sets of rules to configure code analysis as well as define your own.  To access this feature right click on a project and choose "Properties" then click on the "Code Analysis" tab.

This feature is in response to the feedback we received that the number of rules is overwhelming and difficult to configure.

For this preview release we have focused on the core concept and made minimal changes to the UI.  In RTM we will have a new UI and a rich edit experience.  This release is all about getting your feedback.  Try out this feature on your code, use the built-in rule sets and let us know what you think.  Tell us what we missed or what rules you think are there and should not be.  Your real-life experience is our best guide for this feature. 

You can create a custom rule set to fine tune the exact rules to use.  If you think we should include your custom rule set in the out of the box sets then we want to hear from you (feedback links below).  Just attach the exported file to your connect bug or mail us.

Some useful links:

• Release Post from JeffBe and download instructions - here
• Feedback - bugs and suggestions can be submitted via http://connect.microsoft.com/visualstudio. Make sure when you submit issues through Connect that you indicate the product version as being “Visual Studio Team System Code Name “Rosario” (November 2007 CTP)”. That will ensure it gets to the right folks for consideration.
• Questions - we’ve created a special forum specifically focused on Rosario: http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=1736&SiteID=1 or you can respond to this blog post.

We look forward to hearing from you and hope you enjoy this new feature.

Positioning Code Metrics to Management

Another question that is coming up is:

How can (or should) code metrics be presented to management ?

My view is that in all cases the metrics are something that can help make decisions around focus and prioritization. Management should be made aware of the metrics and what they mean to allow them to decide where to focus resources. If management is more levels away from the code then the Maintainability index can serve as a good, simple, metric.

Sometimes you may ship code that has low maintainability or very high complexity etc. and management would make this decision based on risk, time and resources and perhaps other factors. Metrics helps them make that decision with quantitative data.

Some possible options:

1. One option is to present the metrics in terms of risk and mitigation. For example:

   1. Risk: low maintainability in some module
   2. Mitigation: the module is reviewed.
   3. Risk: High cyclomatic complexity
   4. Mitigation: the module has a high number of test cases.
2. Another option is to evaluate the metrics on legacy code. This could be to select some code that is known to have lots of issues and then run the metrics on the code to see if metrics would report any issues with the code. This could be a good way to show value to management in the metrics themselves and make them case for them as a future preventative measure.
3. Yet another option is to present a trend over time. Unfortunately the tool doesn't currently make this very easy but at each milestone of the project you can record the maintainability and other metrics for the modules. This could then be compared across time and used in future planning or to guide management in allocating time and resources to particular parts of the project that are trending badly.

I'd be very interested in hearing other approaches people have taken in presenting metrics or code analysis in general to management.

Maintainability Index Range and Meaning

Another question:

The maintainability index has been re-set to lie between 0 and 100.  How and why was this done?

The metric originally was calculated as follows (based on the work in Carnegie Mellon University although we modified the Halstead Volume calculation a little since we don't include comments anywhere in our calculation): Maintainability Index = 171 - 5.2 * ln(Halstead Volume) - 0.23 * (Cyclomatic Complexity) - 16.2 * ln(Lines of Code)

This meant that it ranged from 171 to an unbounded negative number.  We noticed that as code tended toward 0 it was clearly hard to maintain code and the difference between code at 0 and some negative value was not useful.   I'll post some tech ed sample code showing very low maintainability or you can try on your own code to verify.  As a result of the decreasing usefulness of the negative numbers and a desire to keep the metric as clear as possible we decided to treat all 0 or less indexes as 0 and then re-base the 171 or less range to be from 0 to 100. Thus, the formula we use is:

Maintainability Index = MAX(0,(171 - 5.2 * ln(Halstead Volume) - 0.23 * (Cyclomatic Complexity) - 16.2 * ln(Lines of Code))*100 / 171)

On top of that we decided to be conservative with the thresholds.  The desire was that if the index showed red then we would be saying with a high degree of confidence that there was an issue with the code.  This gave us the following thresholds (as mentioned in this blog previously):

For the thresholds we decided to break down this 0-100 range 80-20 so that we kept the noise level low and only flagged code that was really suspicious. We have:

• 0-9 = Red
• 10-19 = Yellow
• 20-100 = Green

Code Metrics as Check-in Policy

This is the first in a series of posts around questions I am getting asked related to code metrics.

One question that is coming up is:

Can a check-in policy be defined based on code metrics in TFS?

Well, code metrics itself does not have a check-in policy but fortunately the Maintainability Index, Class Coupling, Depth of Inheritance and Complexity metrics are all available as rules in Code Analysis in Visual Studio.  This means you can use code analysis check-in policy enabled with those rules.  The rules are grouped under the "Maintainability Rules" Category and are configured as follows:
 

In the policy editor this would look like the following:

This allows you to enforce that code meets these thresholds before check-in.  In future we are looking in to allowing the threshold values to be changed.