Code Analysis Team Blog : Code Analysis Policy

Fun with Code Analysis Rule Sets

cristend — Wed, 12 Nov 2008 22:55:00 GMT

Two weeks ago, I introduced you to Visual Studio 2010 September '08 CTP (Community Technology Preview). This CTP is available as a Virtual PC (VPC) image which can be downloaded at http://go.microsoft.com/fwlink/?LinkId=129231. The goal of the Community Technology Preview is to obtain feedback from our customers on the new scenarios we have enabled. If you haven't already downloaded the CTP, do it now and take it out for a spin.

There are many new cool features with the CTP such as an Historical Debugger and Impact Analysis of code changes, but today we are going to focus on Rule Sets for code analysis. (For an overview of all the neat stuff happening in code analysis, check out this PDC video http://mschnlnine.vo.llnwd.net/d1/pdc08/WMV-HQ/TL60.wmv ) Configuring code analysis in Visual Studio 2008 and earlier versions required evaluating each individual rule to determine whether it was important enough to turn on and fix for your project. This might be a very time-consuming process that made it difficult to make sure that the most important problems were identified and fixed. With rule sets, code analysis can now be configured much more quickly and easily with sets of rules that are targeted at specific scenarios or areas of emphasis. Let's take a look at how this works.

The following scenario comes from our CTP walkthrough. You can find the walkthrough and more details when you download the CTP as described above. Let's assume that you have the DinnerNow – Kiosk solution open and is the active project. On the Analyze menu, click Configure Code Analysis for DinnerNow.Restaurants.Kiosk. The property page for Code Analysis is displayed:

This screen tells us that Microsoft All Rules will be executed when we run code analysis, but what if we didn't want to run every Microsoft rule? What if we have a specific scenario we wish to target? We can change which rule set to run by clicking on Open which brings up:

We can select which rule set(s) by simply selecting or deselecting the appropriate box. Notice that each rule set has a brief description of the types of rules within the rule set. Further, you can open up each individual rule set and see the specific rules that make up the rules sets. You can also turn on or off individual rules within a rule set. Finally, you can adjust the type of action a rule will trigger. In the past, code analysis rules have been limited to build Warnings, but now you can elevate code analysis rules to build Errors.

As you can see, rules sets provide a powerful way to focus code analysis on the scenarios and rules that matter most to you. You can also create custom rule sets based on your specific needs. We wont go into it today, but if you download the CTP, the Walkthrough explains how to customize a rule set.

Give it a try and let us know what you think.

Cris

New Code Analysis Features in Visual Studio 2010 September '08 CTP

cristend — Thu, 30 Oct 2008 20:05:00 GMT

At this year's PDC, Microsoft released Visual Studio 2010 September '08 CTP (Community Technology Preview). This CTP is available as a Virtual PC (VPC) image which can be downloaded at http://go.microsoft.com/fwlink/?LinkId=129231. The goal of the Community Technology Preview is to obtain feedback from our customers on the new scenarios we have enabled. If you are interested in code analysis, you need to get this CTP and here is why.

The code analysis team is excited to share with you new code analysis features and rules in this CTP, including:

Rule sets - code analysis can now be configured much more quickly and easily with sets of rules that are targeted at specific scenarios or areas of emphasis
Enhanced policy support - enforce your rule set with the code analysis check-in policy
8 New Data Flow rules - find hard-to-detect flaws with these advanced dataflow rules for managed code; including certain SQL injection vulnerabilities

Included with the CTP is a walkthrough guide. Inside the walkthrough is a section dedicated to these exciting code analysis scenarios. Over the next couple of weeks we will continue to blog about rule sets, policy support, and data flow rules so be sure to come back after you get the CTP.

Now go get the CTP and starting sending us your feedback! http://go.microsoft.com/fwlink/?LinkId=129231

Come chat with the Code Analysis team tomorrow

David M. Kean — Tue, 04 Dec 2007 20:16:00 GMT

Join members of the Visual Studio Team System product group to discuss features available in Team Foundation Server, Team Suite, Architecture Edition, Development Edition, Database Edition, and Test Edition. In addition, discuss what's new for these editions for Visual Studio 2008.

We will be holding two sessions:

Join the chat on Wednesday, December 5th, 2007 from 10:00am - 11:00am Pacific Time. Add to Calendar | Additional Time Zones

-and-

Join the chat on Wednesday, December 5th, 2007 from 4:00pm - 5:00pm Pacific Time. Add to Calendar | Additional Time Zones

Code Metrics as Check-in Policy

conorm — Fri, 16 Nov 2007 10:39:00 GMT

This is the first in a series of posts around questions I am getting asked related to code metrics.

One question that is coming up is:

Can a check-in policy be defined based on code metrics in TFS?

Well, code metrics itself does not have a check-in policy but fortunately the Maintainability Index, Class Coupling, Depth of Inheritance and Complexity metrics are all available as rules in Code Analysis in Visual Studio. This means you can use code analysis check-in policy enabled with those rules. The rules are grouped under the "Maintainability Rules" Category and are configured as follows:

Metric	Corresponding Rule	Threshold
Depth of Inheritance	CA1501 AvoidExcessiveInheritance	Warning at above 5 levels deep
Complexity	CA1502 AvoidExcessiveComplexity	Warning at above 25
Maintainability Index	CA1505 AvoidUnmaintainableCode	Warning at below 20
Class Coupling	CA1506 AvoidExcessiveClassCoupling	Warning at above 80 for class and above 30 for a method

In the policy editor this would look like the following:

This allows you to enforce that code meets these thresholds before check-in. In future we are looking in to allowing the threshold values to be changed.

New for Visual Studio 2008 - The 'Analyze' menu

David M. Kean — Sat, 22 Sep 2007 17:00:00 GMT

While Ian Huff has already talked about the new 'Developer' menu that has been added to Visual Studio 2008, I thought I would briefly mention some of the changes to it that were made between Beta 2 and RTM.

For those that have yet to see this menu, similar to the existing Test menu, we've added a new top-level menu that will contain the Code Analysis, Code Metrics and Profiling menu items. This was done for discoverability reasons, and something we hope to leverage moving forward towards Rosario.

A quick rundown on some of the things that have changed since Beta 2:

Renamed the menu itself from the very non-specific Developer to the little bit more descriptive Analyze.
Removed the Code Analysis and Code Metrics sub menus and pulled their associated menu items up one level.
Pulled New Diff Report up one level and renamed it to Compare Performance Reports...
Renamed the Code Metrics and Code Analysis menu items:
Generate Code Metrics -> Calculate Code Metrics.
Migrate Code Analysis Policy Settings to Solution -> Code Analysis Settings for Solution.
Replace Code Analysis Settings with Code Analysis Policy Settings -> Replace with Check-in Policy
Add Code Analysis Policy Settings to Code Analysis Settings -> Merge with Check-in Policy

I probably don't have to tell that these changes, that we made to increase the usability of the menu, are great improvements, especially the renaming of probably what would have been in the running for the longest menu item in Visual Studio; Replace Code Analysis Settings with Code Analysis Policy Settings.

While a very small addition to Visual Studio 2008, we hope the new Analyze menu helps improve your workflow when working with the Code Analysis and Profiler features.

New for Visual Studio 2008 - Code Analysis Policy improvements

David M. Kean — Thu, 20 Sep 2007 17:00:00 GMT

One not very well known feature that lights up when connected to a Team Foundation Server, is the ability to enforce that Code Analysis be run before every check-in. Called a Code Analysis Policy, this feature allows your team to find and fix Code Analysis warnings earlier in the product cycle rather than later, where code changes are riskier and more expensive to make.

To add a Code Analysis Policy to a Team Project, see the following topic on the MSDN Library, How to: Add Check-In Policies.

In Visual Studio 2005, while a useful feature, there were a few usability issues that customers ran into time and time again:

Migrating the Code Analysis settings stored in the policy on the Team Foundation Server to the individual projects was confusing. At times, I literally sat in amazement as I watched my own colleagues new to Code Analysis Policy struggle to perform the (what should have been an easy) task of finding a menu item.
Once the settings had been migrated, users were confused as to why the settings in the individual projects didn't match exactly what was stored in the project. For example, rules turned off in the policy, were still turned on in the project.

Our team actually knew about the confusion that issue 2 would cause when Visual Studio 2005 shipped. Surprisingly as it sounds, this behavior was actually the lesser of two evils. With the time that was available in the ship cycle they knew that they had two distinct choices, either a) have the policy completely override the settings stored in the project, or b) merge the policy settings with the project settings. Because a) prevented the scenario of allowing users to turn on more rules than the policy specified (in effect being stricter than the policy), the later was chosen as the preferred behavior.

Issue 1 was somewhat of a surprise. Code Analysis Policy along with the rest of Code Analysis was a late edition to Visual Studio 2005, probably too late to get any real user feedback and perform any actionable usability studies. We didn't hear about the troubles users were having with this until after the product had already shipped.

Once Visual Studio 2008 planning began, improving both of these issues was one of our top priorities, so we set about a feature crew to fix it.

Code Analysis Policy Failure Details dialog

Clearly users were first of all struggling to actually perform the migration; they were told by the Policy Failure that their projects settings were out-of-date, but not actually how to go about updating them. Even if they somehow figured out that a menu item might do this, looking intuitively on the individual project's context menus didn't help - the menu item that performed the job, Migrate Code Analysis Policy Settings to Solution (what a mouthful!), was hidden under File -> Source Control and on the solution's context menu.

The first thing we did to solve this confusion was to add a new dialog (displayed when double-clicking on a Code Analysis policy failure) that provided extra information about the situation they were in, what projects were out-of-date and how to go about solving it.

We also chose to tell them at this time that their build date was out-of-date - something that we previously only told them after they had updated their project settings and then attempted to check-in again (breaking their workflow).

New Policy Menu Items

The way to solve the second issue was easy; provide the user a choice:

Some users preferred policy to win out and completely overwrite the local project settings, whereas, others wanted to be able to specify additional rules above and beyond what was specified in the policy (ie the current Visual Studio 2005 behavior). The new Analyze menu (which I'll talk about in a future post), now provides two menu items, Replace with Check-in Policy and Merge with Check-in Policy, for performing both of these actions, respectively.

Hopefully these changes will make using Code Analysis Policy a little more pleasant experience in Visual Studio 2008. The new dialog and menu items themselves made into Beta 2 (albeit with slightly different text and names), so install it today and tell what you think.

TIP: Scope the Code Analysis Policy to apply to specific source control paths/file types

ravkaur — Tue, 30 Jan 2007 19:00:00 GMT

On the forums, we have had posts from customers who want to customize their code analysis policy to skip certain file types such as .sql, docs etc. This is because the policy will block checkins even though code analysis doesn’t apply to those files.[ See thread: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1080300&SiteID=1 ] There is now a way to configure code analysis policy to apply to specific source control paths/file types. Check out the tool on gotdotnet that helps you to do so. http://www.gotdotnet.com/codegallery/codegallery.aspx?id=96d7da50-0d61-4230-9af9-49684ae9881e

Here's what you need to do (Part of the steps are also in the readme for the custom path policy)

a) Download the custom code analysis policy from the location specified on the link. This is a project and you will need to build it.

b) Launch Regedit (or any other registry editor of choice), and navigate to the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\8.0\TeamFoundation\SourceControl\Checkin Policies

c) Add a string value called "CustomPathPolicy" – set the value to the path where the DLL is located, such as: C:\SD_TIDM\Technical Integration\CustomPathPolicy\Extensibility\bin\Debug\CustomPathPolicy.dll

Once the registry key is set up, follow these steps to set up the custom policy.

Add code analysis policy to the team project via Team Project Settings -> Source Control -> Check in Policy
Click "Add" again and select "Custom Path Filter" item, which will invoke the Edit dialog.
From the dropdown, select 'Code Analysis'
Enter the regex path filters to which the policy should apply to, click add, repeat as needed.
Click OK to save the settings.
Disable the original code analysis policy set in step 1). If this is not done, the policy will be evaluated twice.

Once you have this set up, the code analysis policy will apply to only paths that you have specified. One thing to keep in mind is that migrating code analysis settings to solution does not respect the paths set in the custom path policy. i.e. Migrating code analysis policy settings to solution will migrate to all projects within the solution, not just the ones specified in the custom path policy.