Code Analysis Team Blog : Custom Rules

Fun with Code Analysis Rule Sets

cristend — Wed, 12 Nov 2008 22:55:00 GMT

Two weeks ago, I introduced you to Visual Studio 2010 September '08 CTP (Community Technology Preview). This CTP is available as a Virtual PC (VPC) image which can be downloaded at http://go.microsoft.com/fwlink/?LinkId=129231. The goal of the Community Technology Preview is to obtain feedback from our customers on the new scenarios we have enabled. If you haven't already downloaded the CTP, do it now and take it out for a spin.

There are many new cool features with the CTP such as an Historical Debugger and Impact Analysis of code changes, but today we are going to focus on Rule Sets for code analysis. (For an overview of all the neat stuff happening in code analysis, check out this PDC video http://mschnlnine.vo.llnwd.net/d1/pdc08/WMV-HQ/TL60.wmv ) Configuring code analysis in Visual Studio 2008 and earlier versions required evaluating each individual rule to determine whether it was important enough to turn on and fix for your project. This might be a very time-consuming process that made it difficult to make sure that the most important problems were identified and fixed. With rule sets, code analysis can now be configured much more quickly and easily with sets of rules that are targeted at specific scenarios or areas of emphasis. Let's take a look at how this works.

The following scenario comes from our CTP walkthrough. You can find the walkthrough and more details when you download the CTP as described above. Let's assume that you have the DinnerNow – Kiosk solution open and is the active project. On the Analyze menu, click Configure Code Analysis for DinnerNow.Restaurants.Kiosk. The property page for Code Analysis is displayed:

This screen tells us that Microsoft All Rules will be executed when we run code analysis, but what if we didn't want to run every Microsoft rule? What if we have a specific scenario we wish to target? We can change which rule set to run by clicking on Open which brings up:

We can select which rule set(s) by simply selecting or deselecting the appropriate box. Notice that each rule set has a brief description of the types of rules within the rule set. Further, you can open up each individual rule set and see the specific rules that make up the rules sets. You can also turn on or off individual rules within a rule set. Finally, you can adjust the type of action a rule will trigger. In the past, code analysis rules have been limited to build Warnings, but now you can elevate code analysis rules to build Errors.

As you can see, rules sets provide a powerful way to focus code analysis on the scenarios and rules that matter most to you. You can also create custom rule sets based on your specific needs. We wont go into it today, but if you download the CTP, the Walkthrough explains how to customize a rule set.

Give it a try and let us know what you think.

Cris

Tutorial on writing your own Code Analysis rule

David M. Kean — Fri, 18 Jan 2008 18:00:00 GMT

Jason Kresowaty has posted a great tutorial on writing Code Analysis/FxCop rules. He also spends time on explaining parts of the Introspection API, including the different nodes and their relationship to each other.

It's a great read, even if you've written some Code Analysis rules before. Definitely recommended.

What kind of custom rules do you want to write?

David M. Kean — Wed, 04 Jul 2007 18:00:00 GMT

Our excellent program manager, Joe Rohde, is trying to gather information on the kinds of custom rules customers want to write.

I am a program manager for our Code Analysis system. I want to know what kind of custom rules you would like to write. Ignore what you may or may not know about what's in FxCop already. If you could track anything - tell me what you want!

I don't care if it's things you know we can't do - except maybe 'find my bugs'. Tell your friends, tell anyone who has ever said: I wish this thing could...

The reason I want everything is both to help plan the rules we should just plain provide; and how to best provide an actual supported extensibility feature.

Please send what you would like to see to: JRohde@Microsoft.com

Thanks!

Joe

Provide Joe with feedback and help us plan our future extensibility.

FAQ: How do I debug a custom rule?

David M. Kean — Wed, 16 May 2007 18:00:00 GMT

While writing your custom rule, you will likely come across a need to debug it and work out why it's behaving in a particular manner.

Debugging within FxCop

Debugging a custom rule within the FxCop UI is actually quite easy. To do so, simply:

Open the FxCop project that contains your custom rule (making sure the PDB is alongside the assembly)
Open the solution that contains your custom rule and set a breakpoint in the rule your want to debug
Choose Debug -> Attach to Process
Under Available Processes, choose FxCop.exe and click Attach
In FxCop, choose Project -> Analyze

The debugger should break when it hits the breakpoint.

Debugging within Visual Studio or via command-line

However, if you want debug a rule that you've written for Visual Studio Code Analysis, or via the command-line, the procedure is not so obvious:

Open a Visual Studio Command Prompt
If debugging a custom rule written for Visual Studio, run the following:

> cd %VSINSTALLDIR%\Team Tools\Static Analysis Tools\FxCop

Otherwise, if debugging a custom rule written for FxCop, run the following:

> cd %PROGRAMFILES%\Microsoft FxCop 1.35
Run the following command, replacing [TargetAssembly] and [RuleAssembly] with the assembly you want to analyze and the assembly that contains your custom rule respectively.

> devenv /debugexe FxCopCmd.exe /file:[TargetAssembly] /rule:[RuleAssembly] /console

This will open Visual Studio in a state that is ready to debug your custom rule.
Choose File -> Open -> File and browse to and open the source file that contains your custom rule
Set a breakpoint in the rule
Choose Debug -> Start Debugging

The debugger should break when it hits the breakpoint.

The Modules window is your friend

If the breakpoint is not being hit, check to make sure that you actually have symbols for your custom rule loaded:

While debugging, choose Debug -> Windows -> Modules

To do this if you are debugging via /debugexe switch, as soon as the command window opens to start running FxCop, click PAUSE on the keyboard.
Look for your rule assembly and check under the Symbol Status column.
If this column states anything other Symbols Loaded, then right-click on the assembly and choose Load Symbols and browse manually to the PDB.

Dealing with a Code Analysis backlog?

David M. Kean — Fri, 04 May 2007 18:00:00 GMT

Nicole Calinoiu has started a (what I hope to be long) series on introducing Code Analysis (FxCop in particular) to your team. Her first couple of posts are absolute rippers, check them out:

FxCop and the big, bad backlog
FxCop backlogs: Some rules for rule activation

She's also written some custom rules content:

No rules in your FxCop rule assembly?
Rule: Justify rule violation suppression

Great stuff!

FAQ: How do I get the SourceContext for a local?

David M. Kean — Tue, 27 Mar 2007 18:00:00 GMT

I want to fire on the naming of a local, however, whenever I pass the local to the Problem constructor, the source information for the method is always used. How do I get FxCop/Code Analysis to use the source information for the local instead?

Because the declaration of a local is not associated with an executable instruction, a local does not contain any source information within the pdb. However, any usages of the local (such as an assignment) does.

The following example shows when source information is provided and not provided:

    public void Bar()
    {
        int value1;                // Does not have source information
        value1 = 10;               // Has source information
        int value2 = 20;           // Has source information

        Console.WriteLine(value1); // Has source information
        Console.WriteLine(value2); // Has source information
    }

As most locals are assigned at their time of declaration (such as in the case of value2 in the above example), a simple trick is use the source context of the first usage of the local in place of its declaration. This works in most situations.

The following example shows this (using the code from FAQ: How do I access the locals of a method in a custom rules?):

    public override ProblemCollection Check(Member member)
    {
        Method method = member as Method;
        if (method == null)
            return null;

if (method.Instructions.Length == 0)
return null;

        LocalList locals = method.Instructions[0].Value as LocalList;
        if (locals == null)
            return null;

        for (int i = 0; i < locals.Length; i++)
        {
            Local local = locals[i];

            if (someCondition)
            {   // Fire on the local

                Resolution resolution = new Resolution("Fix local {0}", local.Name.Name);

SourceContext sourceContext = FindSourceContext(local, method.Instructions);

                Problem problem = new Problem(resolution, sourceContext);
                Problems.Add(problem);
             }
        }

        return Problems;
    }

    private static SourceContext FindSourceContext(Local local, InstructionList instructions)
    {
        foreach (Instruction instruction in instructions)
        {
            if (instruction.Value == local)
            {   // Found its first usage
                return instruction.SourceContext;
            }
        }

        return new SourceContext();
    }

Note: If the local is only declared and is never assigned to, or its usage is only as the argument for an out parameter, then the source context of the method will be used.

Framework for intergrated FxCop rule testing

David M. Kean — Sun, 25 Feb 2007 19:00:00 GMT

Roy Osherove has put together a Framework for testing FxCop rules. It's an interesting approach, firing up the FxCop engine for each rule that he wants to test.

There are few things that I would like to see it support:

The ability to specify positive tests (code that rules should not fire when run over) on top of the negative (code that rules do fire when run over) tests that he supports
The ability to separate the actual test from the methods you want to run your rules over. Unfortunately, this approach will not scale when you are testing 3 difference languages (C#, Visual Basic and C++) for 200+ rules that we ship.

Unfortunately, his approach is not going to work in the next version of FxCop/Visual Studio for a couple of reasons. 1) We did some work in Orcas to reduce the visibility of API's that did not need to be public for actually writing custom rules (for example, everything in FxCopCommon.dll is now internal) and 2) We also removed the Reflection 'bridge', that is, the ability to create CCI objects from their Reflection counterparts and vice versa.

Apart from the things mentioned above, it's a great start and its got us thinking, how many users actually automate the testing of custom rules? Is this something you see as important? Would you like to see how we do this internally?

FAQ: When looking at the names of locals in a custom rule, why do I see strange names such as 'CS$1$0000' and 'VB$1$0000'?

David M. Kean — Fri, 26 Jan 2007 19:00:00 GMT

Previously we showed you how to access the locals or variables of a method. Once you started to run your rule over an assembly, you might start to notice strange locals with names that start with 'CS$' and 'VB$'. These are compiler generated locals that are outside the control of the user.

If you are writing a rule to check the names of locals, you will want to skip over these, To do this, simply pass the local to RuleUtilities.IsCompilerGenerated method and check the result.

Note: This will also skip over locals when source and line information (PDBs) is not alongside the assembly under analysis.

Free sample book chapter on Managed Code Analysis

David M. Kean — Wed, 24 Jan 2007 19:00:00 GMT

SearchVB.com is offering a free sample chapter on Managed Code Analysis from the book Professional Visual Studio Team System (co-written by fellow Team System colleague Noah Coad).

Chapter 8, Managed Code Analysis, focuses on how to use Team System's Static Code Analysis tool for C# or VB.NET code. This tool checks code for adherence to Microsoft's .NET Framework Design Guidelines. Static Code Analysis is available from both the Visual Studio IDE and from the command line, and the authors cover both instances. The chapter ends with hints for creating custom rules for the code analysis tool.

Great stuff!

FAQ: Can I create custom rules that target both Visual Studio and FxCop? [David Kean]

David M. Kean — Thu, 01 Jun 2006 08:19:00 GMT

I have users using both Visual Studio and FxCop, and I want to create custom rules that run on both. Is this possible?

Although in the future we do plan on making it easier to move custom rules between Visual Studio and FxCop, currently you must recompile each rule assembly against the version of FxCop/Visual Studio you want to target.

Typically this involves the (painful) step of manually removing, and re-adding the references to both FxCopSdk.dll and Microsoft.Cci.dll.

To improve this process, we have developed an MSBuild targets files that can be imported by your custom rules projects, which makes switching between targetting FxCop 1.35 and Visual Studio as easy as changing build configurations.

In the attached download, the same sample rules project that was made available here, has been included, however, it has been updated to import this new targets file. This project has 4 build configurations, FxCopDebug, FxCopRelease, VisualStudioDebug and VisualStudioRelease. As you can probably guess from the names, the former two target building against FxCop, while the later two target building against Visual Studio. You can use this project as a starting base, or follow the below steps to import this targets file in your own projects:

Copy Microsoft.CodeAnalysis.CustomRules.targets (attached below) to the same folder that contains your Visual Studio solution
Open your custom rules project and remove any references to FxCopSdk and Microsoft.Cci
Right-click on your project and choose Unload Project
Right-click on your project and choose Edit [ProjectName]
Add the following bolded line at the location indicated (at the bottom of the file):
[...]
<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
<Import Project="..\Microsoft.CodeAnalysis.CustomRules.targets" />
</Project>
By default, this looks for the targets file one directory above your project file (which typically contains your solution).
At the top of the project file, after the first <PropertyGroup></PropertyGroup> element, but before the second, add the following bolded lines at the location indicated:
[...]
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'FxCopDebug|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>full</DebugType>
    <Optimize>false</Optimize>
    <OutputPath>bin\FxCopDebug\</OutputPath>
    <DefineConstants>DEBUG;TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <RunCodeAnalysis>true</RunCodeAnalysis>
    <CodeAnalysisRules></CodeAnalysisRules>
    <CodeAnalysisBuildTarget>FxCop</CodeAnalysisBuildTarget>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'VisualStudioDebug|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>full</DebugType>
    <Optimize>false</Optimize>
    <OutputPath>bin\VisualStudioDebug\</OutputPath>
    <DefineConstants>DEBUG;TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <RunCodeAnalysis>true</RunCodeAnalysis>
    <CodeAnalysisRules></CodeAnalysisRules>
    <CodeAnalysisBuildTarget>VisualStudio</CodeAnalysisBuildTarget>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
Right-click on the project and choose Reload Project, answering Yes to both questions that are asked
Right-click on the solution and choose Configuration Manager
In the Active solution configuration dropdown, choose New
In the New Solution Configuration dialog, enter FxCopDebug as the name and click OK
In the Active solution configuration dropdown, choose New
In the New Solution Configuration dialog, enter VisualStudioDebug as the name and click OK

Once you have done the above, switching the active build configuration between FxCopDebug and VisualStudioDebug will now also change the references to FxCopSdk and Microsoft.Cci to point to the associated version.

If you have any issues with the Microsoft.CodeAnalysis.CustomRules.targets file, or with the custom rules themselves, don't hesitate to visit the FxCop forum.

FAQ: How do I access the locals of a method in a custom rule? [Michael Fanning, David Kean]

David M. Kean — Fri, 07 Apr 2006 18:00:00 GMT

Note: The custom rules API is likely to change in the next version of FxCop/Visual Studio.

Currently, the locals of a method are stored in a pseudo-instruction in the method's InstructionList. We plan to change this in a future version of the tool and the locals will directly hang off the method.

To access them, do the following:

public override ProblemCollection Check(Member member)
{
Method method = member as Method;

if (method == null)
return null;

if (method.Instructions.Length == 0)
return null;

LocalList locals = method.Instructions[0].Value as LocalList;

if (locals == null)
return null;

   for (int i = 0; i < locals.Length; i++)
   {
      Local local = locals[i];

// Do something
}

return base.Problems;
}

In FxCop 1.35, the LocalList is foreachable.

FAQ: How do I integrate custom rules with Visual Studio? [David Kean]

David M. Kean — Sun, 12 Mar 2006 08:16:00 GMT

Update: To have rules target both Visual Studio and FxCop, see the following entry: FAQ: Can I create custom rules that target both Visual Studio and FxCop?

Although not officially supported or documented, the Managed Code Analysis (FxCop) feature available with Visual Studio Team System comes with a rich API for writing custom rules.

There are a number of samples and articles on the web that already cover using this API and its usage with the externally available FxCop, so instead of covering the same information, this entry will instead give a quick rundown on integrating existing custom rules with Visual Studio.

Before beginning, download the attached zipped project (at the bottom of the post) and extract the contents. This project contains a single custom rule called PrivateFieldsShouldHaveCorrectPrefix that performs a simple check to make sure that private class fields (both instance and static) are prefixed with an underscore. The following code shows the type of field names it fires violations against (and those it doesn’t):

public class Class1
{
    private int _Field;
    private int _field;
    private int Field;   // Violates PrivateFieldsShouldHaveCorrectPrefix
    private int field;   // Violates PrivateFieldsShouldHaveCorrectPrefix
    private int m_field; // Violates PrivateFieldsShouldHaveCorrectPrefix
}

As the source has been provided, you can change PrivateFieldsShouldHaveCorrectPrefix to follow your own naming rules or add additional rules to the library to enforce your own company’s guidelines.

To register the rule library with Visual Studio, the built assembly (in this case, Microsoft.FxCop.Rules.Samples.dll) should be placed in the Code Analysis Rules folder, which by default is located at C:\Program Files\Visual Studio 8\Team Tools\Static Analysis Tools\FxCop\Rules. Once the library has been registered, all opened projects (new and existing), will have any rules that are contained in the assembly enabled by default. You can now simply run Code Analysis over a project to analyze it with your new rules.

Before attempting this there are a couple of things you should be aware of:

You must have either Visual Studio Team Edition for Software Developers or Visual Studio Team Suite, as only these editions include the Code Analysis tools.
Rules or their base classes must extend BaseIntrospectionRule.
The rule description XML file (provided in the sample download) should be an embedded resource and end with ‘Rules.xml’ (case sensitive).
Rule libraries need to be compiled in .NET 2.0 and against the versions of Microsoft.Cci.dll and FxCopSdk.dll that ship with Visual Studio. These are located in C:\Program Files\Visual Studio 8\Static Analysis Tools\FxCop by default.
The SDK is currently undergoing a massive change, so any rules you write now for Visual Studio 2005, will not work in future versions, however, it should be relatively painless to convert them over.

Please note that this is not supported by Microsoft so please don’t call Product Support Services if you have any troubles, however, feel free to post any issues you encounter on the FxCop forum.

Simplified FxCop Rules SDK [Michael Fanning]

mimurray — Fri, 25 Jun 2004 23:40:00 GMT

As mentioned previously here and elsewhere, we're planning to improve and simplify several aspects of FxCop's extensibility model. This is the first of a series of threads about those improvements, in order to solicit feedback and give custom rules developers a heads up on the coming changes. Here's the new simplified IIntrospectionRule interface:

public interface IIntrospectionRule : IManagedCodeRule
{
  Problem[] Check(string namespaceName, TypeNodeList types);
  Problem[] Check(Module theModule);
  Problem[] Check(Resource resource);
  Problem[] Check(TypeNode type);
  Problem[] Check(Member member);
}

This rather drastic simplification has reduced surface area for custom rules and increased FxCop performance without limiting the kinds of checks rules developers can right. Do you need an Assembly rule? Implement a Module rule and check the argument to be sure it's an assembly node. Do you need a Property rule? Implement a member rule and insure the argument is a Property.

Note that parameter checks have been eliminated. Instead, implement a Member rule and use a new helper method, RuleUtilities.GetParameters(members) to determine whether there's a parameter list to analyze (this helper returns parameters for both properties and methods). Problem class constructors will now accept a parameter argument if you want to associate a message with a specific parameter.

All comments, questions, and feedback are welcome

FxCop 1.30 and creating custom rules using Introspection Engine [Michael Murray]

mimurray — Thu, 20 May 2004 06:56:00 GMT

As John MacKenzie pointed out on the GotDotNet FxCop message board, John Robbins has a great article in the June issue of MSDN magazine on FxCop 1.30 and custom rules using the new Introspection engine. Check it out at http://msdn.microsoft.com/msdnmag/issues/04/06/Bugslayer/default.aspx .

Writing Custom Rules in FxCop & FxCop Visual Studio Integration [Jeffrey van Gogh]

mimurray — Sat, 10 Jan 2004 04:42:00 GMT

Two common questions we often get are:

Can I write my own rules for FxCop and customize existing rules?
Can I integrate FxCop into visual studio and can I jump to the location in my source of the item that FxCop flags as an error?

That's why I want to give some more information on both:

FxCop supports writing custom rules. Custom Rules can be written by writing a class in any.NET language. This class has to implement one of the interfaces FxCop provides via its SDK. After compiling the class into an assembly, FxCop can load the custom rule by pointing it to that assembly. Samples and Documentation on how to do this can be found on our webpage:

the SDK Documentation: http://www.gotdotnet.com/team/fxcop/docs/gotdotnetstyle.aspx?url=UsingFxCopSdk.htm

the Samples: http://www.gotdotnet.com/team/fxcop/docs/SDKSamples.zip

FxCop doesn’t really support editing the default rules right now. We are making some rules configurable for our next release, e.g. you can change settings of the spelling rules and you can change on which members the Naming rules run (by default they run only on members visible outside of the assembly).

FxCop integrates into VS.NET as an external tool using the Commandline version of FxCop. On details how to do this, see http://www.gotdotnet.com/team/fxcop/docs/UsingFxCopCmd.htm#_Visual_Studio_Integration

FxCop is able to find sourcecode location for an error message if both following conditions are met:

The compiler emitted a .pdb file for the assembly (/debug+ switch for C# Compiler, enabled by default in VS)
The error message occurs in an executable piece of code inside the assembly (assembly, namespace, type, field & attribute error messages are not in executable code and the compiler therefore doesn’t generate source lookup information)

In the upcoming release of FxCop, with our new Metadata reader, we will be able to do a much better job for jumping to source code. Currently FxCop jumps to the first line of a method in which an FxCop issue is found. In the next version of FxCop we will jump to the exact line of code that has an issue.