Code Analysis Team Blog

Fun with Code Analysis Rule Sets

Two weeks ago, I introduced you to Visual Studio 2010 September '08 CTP (Community Technology Preview). This CTP is available as a Virtual PC (VPC) image which can be downloaded at http://go.microsoft.com/fwlink/?LinkId=129231. The goal of the Community Technology Preview is to obtain feedback from our customers on the new scenarios we have enabled.  If you haven't already downloaded the CTP, do it now and take it out for a spin. 

There are many new cool features with the CTP such as an Historical Debugger and Impact Analysis of code changes, but today we are going to focus on Rule Sets for code analysis. (For an overview of all the neat stuff happening in code analysis, check out this PDC video http://mschnlnine.vo.llnwd.net/d1/pdc08/WMV-HQ/TL60.wmv )  Configuring code analysis in Visual Studio 2008 and earlier versions required evaluating each individual rule to determine whether it was important enough to turn on and fix for your project. This might be a very time-consuming process that made it difficult to make sure that the most important problems were identified and fixed. With rule sets, code analysis can now be configured much more quickly and easily with sets of rules that are targeted at specific scenarios or areas of emphasis. Let's take a look at how this works.

The following scenario comes from our CTP walkthrough.  You can find the walkthrough and more details when you download the CTP as described above.  Let's assume that you have the DinnerNow – Kiosk solution open and is the active project.  On the Analyze menu, click Configure Code Analysis for DinnerNow.Restaurants.Kiosk.  The property page for Code Analysis is displayed:

This screen tells us that Microsoft All Rules will be executed when we run code analysis, but what if we didn't want to run every Microsoft rule?  What if we have a specific scenario we wish to target?  We can change which rule set to run by clicking on Open which brings up:

We can select which rule set(s) by simply selecting or deselecting the appropriate box.  Notice that each rule set has a brief description of the types of rules within the rule set.  Further, you can open up each individual rule set and see the specific rules that make up the rules sets.  You can also turn on or off individual rules within a rule set.  Finally, you can adjust the type of action a rule will trigger.  In the past, code analysis rules have been limited to build Warnings, but now you can elevate code analysis rules to build Errors.

As you can see, rules sets provide a powerful way to focus code analysis on the scenarios and rules that matter most to you.  You can also create custom rule sets based on your specific needs.  We wont go into it today, but if you download the CTP, the Walkthrough explains how to customize a rule set.

Give it a try and let us know what you think.

Cris

New Code Analysis Features in Visual Studio 2010 September '08 CTP

At this year's PDC, Microsoft released Visual Studio 2010 September '08 CTP (Community Technology Preview). This CTP is available as a Virtual PC (VPC) image which can be downloaded at http://go.microsoft.com/fwlink/?LinkId=129231. The goal of the Community Technology Preview is to obtain feedback from our customers on the new scenarios we have enabled.  If you are interested in code analysis, you need to get this CTP and here is why.

The code analysis team is excited to share with you new code analysis features and rules in this CTP, including:

• Rule sets - code analysis can now be configured much more quickly and easily with sets of rules that are targeted at specific scenarios or areas of emphasis
  
  
• Enhanced policy support - enforce your rule set with the code analysis check-in policy
  
  
• 8 New Data Flow rules - find hard-to-detect flaws with these advanced dataflow rules for managed code; including certain SQL injection vulnerabilities

Included with the CTP is a walkthrough guide.  Inside the walkthrough is a section dedicated to these exciting code analysis scenarios.  Over the next couple of weeks we will continue to blog about rule sets, policy support, and data flow rules so be sure to come back after you get the CTP.

Now go get the CTP and starting sending us your feedback! http://go.microsoft.com/fwlink/?LinkId=129231

New for Visual Studio 2008 SP1 and FxCop 1.36 – Multi-targeting rule

I've just published a post over my blog about a new feature we added for Visual Studio 2008 SP1 and FxCop 1.36.

New for Visual Studio 2008 SP1 and FxCop 1.36 – Multi-targeting rule

FxCop 1.36 Released!

We are pleased to announce that we've released the next version of FxCop. This release has a number of changes since FxCop 1.36 Beta 2, including, but not limited to, the following:

• Bug fixes that reduce noise, missing analysis and rule crashes.
• New portability rule: Use Only API from Targeted Framework.  This rule will fire when the target assembly taking a dependency on a new API only included in a later version or in a service pack.
• New option for new portability rule for selecting target framework for analysis.
• Set the target framework in the UI via Project -> Option -> Spelling & Analysis -> Target Framework
• Set the target framework on the command-line via the /targetframeworkversion switch. e.g: /targetframeworkversion:v2.0, /targetframeworkversion:v3.0, or /targetframeworkversion:v3.5

Compared with FxCop 1.35, FxCop 1.36 Beta 2 had changes including, but not limited to, the following:

• 200+ bug fixes that reduce noise, missing analysis and rule crashes
• Support for analyzing anonymous methods and lambda expressions
• New option for skipping analysis over tool generated code 
• Turn this on in the UI via Project -> Options -> Spelling & Analysis -> Suppress analysis results against generated code
• Turn this on on the command-line via the /ignoregeneratedcode switch. 
• Better support for C++/CLI and the Compact Framework
• Language 'friendly' API names in the UI and resolutions (ie Visual Basic syntax if running over a Visual Basic binary)
• New globalization, design and usage rules
• Performance improvements that cut analysis by 2x and use half as much memory
• Documentation that is now available on MSDN

There are also some other underlying changes that we will talk about in some future posts. 

Download it, and tell us what you think. If you have any feedback, bug reports or any questions, feel free to head over to the Code Analysis forum and ask away.

Note:  If you already have FxCop 1.36 Beta 2 installed, please uninstall it first before you install FxCop 1.36. Otherwise, you may see an error when you install FxCop 1.36 on top of FxCop 1.36 Beta 2.

If you accidentally install FxCop 1.36 without uninstalling FxCop 1.36 Beta 2, please go to Add/Remove program to uninstall FxCop 1.36, and reinstall FxCop 1.36.

[EDIT] Note: FxCop 1.36 RTM shipped with additional tracing turned on.  This can make the FxCop output harder to read.  To turn tracing off, edit the FxCop.exe.config or FxCopCmd.exe.config files in the FxCop install directory.  Find the line that contains <add name="Trace" value="2" /> and change the "2" to "0".

David Kean's new blog

For those that are interested, I've just started a new personal blog, Dave's Box, where I will be talking about FxCop, Framework Design Guidelines, and the recently announced Managed Extensibility Framework.

I've just posted a couple of posts detailing changes we made to FxCop 1.36: 

FxCop now ships with the spell checker libraries
Reference resolution changes in Code Analysis and FxCop

You can subscribe via the one of the two following links:

 Subscribe via reader
 Subscribe via email

FxCop 1.35 now available (again) for download

We've just uploaded FxCop 1.35 to MSDN Code Gallery after we lost our home on GotDotNet - this is for those teams that either do not want to or just aren't ready to upgrade to FxCop 1.36 Beta.

For those that are already using FxCop 1.35, you'll probably notice that we flicked the switch to cause it to prompt you to download FxCop 1.36. We've love you to download it and try it out (it can be installed side-by-side without any issues with FxCop 1.35). If you hit any bugs, issues or just have some feedback about the new or existing versions of FxCop, feel free to head over to the Code Analysis forum and post away.

FAQ: How do I prevent FxCop 1.36 from firing warnings against generated code?

I've upgraded from FxCop 1.35 to 1.36 and now FxCop has started to fire warnings against typed DataSets and other generated code. How do I turn this off?

The reason this is occurring is because we changed the way that FxCop analyzed generated code. Whereas previously in 1.35, FxCop would only ignore particular generated code (such as DataSets), FxCop 1.36 will now either ignore no generated code, or all generated code. The former is the default.

To change this behavior and have FxCop skip over generated code:

Using an FxCop project:

1. Open your FxCop project in FxCop
2. Choose Project -> Options -> Spelling & Analysis
3. Check Suppress analysis results against generated code
4. Click OK

Via the command-line:

1. Pass the /ignoregeneratedcode switch, for example:
   
   FxCopCmd.exe /file:MyAssembly.dll /out:AnalysisResults.xml /ignoregeneratedcode

For more information on FxCop's behavior over generated code, see Correct usage of the CompilerGeneratedAttribute and the GeneratedCodeAttribute.

What happened to the download and documentation of FxCop 1.35?

Update: You can now download FxCop 1.35 directly from MSDN Code Gallery. 

As you many you have well noticed, GotDotNet was officially retired last month now that the last of its functionality was superceded by MSDN Code Gallery. Unfortunately, at the same time, the FxCop team page, the download for FxCop 1.35 and the rule documentation was also removed.

We were told last year about the deprecation of GotDotNet and we had hoped that by now we would have had a new team site on MSDN, however, a variety of factors, some within our control and some beyond our control, prevented that from happening in time.

In the meantime, while we sort out new home for our beloved FxCop, here are a couple of workarounds:

To download FxCop

• Upgrade to FxCop 1.36 Beta. This is one of our most stable beta's to-date; which is not surprising considering that some of the same bits also shipped in Visual Studio 2008. Go ahead, download FxCop 1.36 today - you won't be disappointed.
  
  
• Download the Windows SDK. The Windows SDK includes the FxCop 1.35 setup. You might already have it installed if you have Visual Studio 2008, otherwise, download it. Either way, once installed, FxCop 1.35 setup can be found via Start -> All Programs -> Microsoft Windows SDK [Version] -> Tools -> Install Microsoft FxCop.

 To view the documentation

• MSDN Library. Both the FxCop documentation and rule documentation are available on the MSDN Library.
  
  
• Installed Help. FxCop 1.35 comes with a CHM (pronounced 'chum') containing FxCop and rule documentation. Albeit a little out-of-date when compared to the online version, access it via Start -> All Programs -> Microsoft FxCop 1.35 -> FxCop Documentation or via Help -> Using Microsoft FxCop inside FxCop.

Tutorial on writing your own Code Analysis rule

Jason Kresowaty has posted a great tutorial on writing Code Analysis/FxCop rules. He also spends time on explaining parts of the Introspection API, including the different nodes and their relationship to each other.

It's a great read, even if you've written some Code Analysis rules before. Definitely recommended.

What do you want to see in the second edition of Framework Design Guidelines?

Krzysztof and Brad have announced they are working on the second edition of the awesome Framework Design Guidelines and are looking for feedback on what they should put in it. For those that don't know, a lot of our Code Analysis rules are based on the writings in this great book, so expect to see additional rules in the future based on the new guidelines in the second edition.  To provide feedback, head over to Krzysztof's blog and post a comment.

FAQ: Which Code Analysis rules shipped in which version?

In response to a lot of recent requests, we've put together a complete list of rules that shipped in the different versions of Visual Studio Code Analysis and FxCop. Attached is an Excel worksheet providing this information for Visual Studio 2005, Visual Studio 2008, FxCop 1.35 and FxCop 1.36 Beta.

One of things you'll notice as you read through the list is that we removed some rules from the later versions. There are a few reasons for this:

• Noise and applicability. We use feedback from customers, SQM data (to see which rules users turn off), and input from internal teams (Windows, Office, CLR, etc) to determine the rules that are noisy without adding any perceivable value. There are also rules that are either no longer applicable or can no longer fire. for example, a rule could have been firing on a limitation of the CLR which has since been fixed in later versions.
  
  
• Merged rules. Sometimes it makes sense to merge rules that fire on similar things, for example, the analysis in SecureGetObjectDataOverrides was already covered by OverrideLinkDemandsShouldBeIdenticalToBase, so these two rules were merged. Similarly, LongAcronymsShouldBePascalCased, ShortAcronymsShouldBeUppercase and IdentifiersShouldBeCasedCorrectly all fired on the casing of identifiers, and hence were merged in the later.
  
  
• Analysis engine removed. In Visual Studio 2008 and FxCop 1.36 we removed one of our analysis engines. This engine was removed for a variety of reasons; it increased analysis time (although the engine encompassed less than 5% our analysis, it took up 50% of our time-to-analyze), indeterministic results (results appearing and disappearing between runs), and bugs found within the engine (and hence the rules that depended on it) required huge architectural changes. We instead decided to invest the resources that we would have spent on fixing the old engine, on a new data flow analysis engine based on Phoenix, which we will ship in a future version of Visual Studio.

There are also more differences between Visual Studio Code Analysis and FxCop than just the rules - in a future blog post I will cover these in detail.

Reporting Code Analysis Defect Counts

Code analysis defect counts are available in the cube that is part of the TFS Data Warehouse. This post shows you how to hook up Excel to show code analysis counts in a pivot table.

First off you need to connect Excel up to the data warehouse that has the data. Under the Data tab in Excel select From Other Sources, From Analysis Services.



Then you need to give the name of the server with the data warehouse. In the standalone TFS case this will be whatever machine is running TFS. Assuming you have read permissions with your Windows account you can click Next, otherwise enter a user name and password that has access to the server and click Next.



Then, select the database (usually TfsWarehouse) and select the Build table.



Click Next and then supply whatever additional information that will help you remember this data connection.



Click Finish. Then define how you want to show the data in Excel - e.g. a pivot table.



Click OK which gives you the following:



In the PivotTable Field List, Expand Build Project where you will find two fields: Static Analysis Errors and Static Analysis Warnings:



You can click the checkboxes to add these to the values for your table. In most cases Static Analysis Warnings would be the most useful since Static Analysis Errors would typically stop the build.

To complete the picture you can play with the pivot table and pivot and filter as you see fit. As a simple example you can add the Build field:



Which will give you one row per build in the pivot table.



Of course this data can be combined with other build data and charted etc. according to your needs. We are considering what reporting to provide in future versions so would love to hear your feedback on this feature and what you would like to see.

Come chat with the Code Analysis team tomorrow

Join members of the Visual Studio Team System product group to discuss features available in Team Foundation Server, Team Suite, Architecture Edition, Development Edition, Database Edition, and Test Edition. In addition, discuss what's new for these editions for Visual Studio 2008.

We will be holding two sessions:

Join the chat on Wednesday, December 5th, 2007 from 10:00am - 11:00am Pacific Time. Add to Calendar | Additional Time Zones

           -and-

Join the chat on Wednesday, December 5th, 2007 from 4:00pm - 5:00pm Pacific Time. Add to Calendar | Additional Time Zones

Try the Code Analysis Rule Sets Feature in the November &quot;Rosario&quot; November 2007 CTP

We have a great new feature that we are previewing in Microsoft® Visual Studio® Team System code name “Rosario” November 2007 CTP.  This feature allows you to select pre-defined sets of rules to configure code analysis as well as define your own.  To access this feature right click on a project and choose "Properties" then click on the "Code Analysis" tab.

This feature is in response to the feedback we received that the number of rules is overwhelming and difficult to configure.

For this preview release we have focused on the core concept and made minimal changes to the UI.  In RTM we will have a new UI and a rich edit experience.  This release is all about getting your feedback.  Try out this feature on your code, use the built-in rule sets and let us know what you think.  Tell us what we missed or what rules you think are there and should not be.  Your real-life experience is our best guide for this feature. 

You can create a custom rule set to fine tune the exact rules to use.  If you think we should include your custom rule set in the out of the box sets then we want to hear from you (feedback links below).  Just attach the exported file to your connect bug or mail us.

Some useful links:

• Release Post from JeffBe and download instructions - here
• Feedback - bugs and suggestions can be submitted via http://connect.microsoft.com/visualstudio. Make sure when you submit issues through Connect that you indicate the product version as being “Visual Studio Team System Code Name “Rosario” (November 2007 CTP)”. That will ensure it gets to the right folks for consideration.
• Questions - we’ve created a special forum specifically focused on Rosario: http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=1736&SiteID=1 or you can respond to this blog post.

We look forward to hearing from you and hope you enjoy this new feature.

Positioning Code Metrics to Management

Another question that is coming up is:

How can (or should) code metrics be presented to management ?

My view is that in all cases the metrics are something that can help make decisions around focus and prioritization. Management should be made aware of the metrics and what they mean to allow them to decide where to focus resources. If management is more levels away from the code then the Maintainability index can serve as a good, simple, metric.

Sometimes you may ship code that has low maintainability or very high complexity etc. and management would make this decision based on risk, time and resources and perhaps other factors. Metrics helps them make that decision with quantitative data.

Some possible options:

1. One option is to present the metrics in terms of risk and mitigation. For example:

   1. Risk: low maintainability in some module
   2. Mitigation: the module is reviewed.
   3. Risk: High cyclomatic complexity
   4. Mitigation: the module has a high number of test cases.
2. Another option is to evaluate the metrics on legacy code. This could be to select some code that is known to have lots of issues and then run the metrics on the code to see if metrics would report any issues with the code. This could be a good way to show value to management in the metrics themselves and make them case for them as a future preventative measure.
3. Yet another option is to present a trend over time. Unfortunately the tool doesn't currently make this very easy but at each milestone of the project you can record the maintainability and other metrics for the modules. This could then be compared across time and used in future planning or to guide management in allocating time and resources to particular parts of the project that are trending badly.

I'd be very interested in hearing other approaches people have taken in presenting metrics or code analysis in general to management.