Code Analysis Team Blog : Code Metrics

Come chat with the Code Analysis team tomorrow

David M. Kean — Tue, 04 Dec 2007 20:16:00 GMT

Join members of the Visual Studio Team System product group to discuss features available in Team Foundation Server, Team Suite, Architecture Edition, Development Edition, Database Edition, and Test Edition. In addition, discuss what's new for these editions for Visual Studio 2008.

We will be holding two sessions:

Join the chat on Wednesday, December 5th, 2007 from 10:00am - 11:00am Pacific Time. Add to Calendar | Additional Time Zones

-and-

Join the chat on Wednesday, December 5th, 2007 from 4:00pm - 5:00pm Pacific Time. Add to Calendar | Additional Time Zones

Positioning Code Metrics to Management

conorm — Thu, 29 Nov 2007 01:08:08 GMT

Another question that is coming up is:

How can (or should) code metrics be presented to management ?

My view is that in all cases the metrics are something that can help make decisions around focus and prioritization. Management should be made aware of the metrics and what they mean to allow them to decide where to focus resources. If management is more levels away from the code then the Maintainability index can serve as a good, simple, metric.

Sometimes you may ship code that has low maintainability or very high complexity etc. and management would make this decision based on risk, time and resources and perhaps other factors. Metrics helps them make that decision with quantitative data.

Some possible options:

One option is to present the metrics in terms of risk and mitigation. For example:
1. Risk: low maintainability in some module
2. Mitigation: the module is reviewed.
3. Risk: High cyclomatic complexity
4. Mitigation: the module has a high number of test cases.
Another option is to evaluate the metrics on legacy code. This could be to select some code that is known to have lots of issues and then run the metrics on the code to see if metrics would report any issues with the code. This could be a good way to show value to management in the metrics themselves and make them case for them as a future preventative measure.
Yet another option is to present a trend over time. Unfortunately the tool doesn't currently make this very easy but at each milestone of the project you can record the maintainability and other metrics for the modules. This could then be compared across time and used in future planning or to guide management in allocating time and resources to particular parts of the project that are trending badly.

I'd be very interested in hearing other approaches people have taken in presenting metrics or code analysis in general to management.

Maintainability Index Range and Meaning

conorm — Tue, 20 Nov 2007 19:43:00 GMT

Another question:

The maintainability index has been re-set to lie between 0 and 100. How and why was this done?

The metric originally was calculated as follows (based on the work in Carnegie Mellon University although we modified the Halstead Volume calculation a little since we don't include comments anywhere in our calculation): Maintainability Index = 171 - 5.2 * ln(Halstead Volume) - 0.23 * (Cyclomatic Complexity) - 16.2 * ln(Lines of Code)

This meant that it ranged from 171 to an unbounded negative number. We noticed that as code tended toward 0 it was clearly hard to maintain code and the difference between code at 0 and some negative value was not useful. I'll post some tech ed sample code showing very low maintainability or you can try on your own code to verify. As a result of the decreasing usefulness of the negative numbers and a desire to keep the metric as clear as possible we decided to treat all 0 or less indexes as 0 and then re-base the 171 or less range to be from 0 to 100. Thus, the formula we use is:

Maintainability Index = MAX(0,(171 - 5.2 * ln(Halstead Volume) - 0.23 * (Cyclomatic Complexity) - 16.2 * ln(Lines of Code))*100 / 171)

On top of that we decided to be conservative with the thresholds. The desire was that if the index showed red then we would be saying with a high degree of confidence that there was an issue with the code. This gave us the following thresholds (as mentioned in this blog previously):

For the thresholds we decided to break down this 0-100 range 80-20 so that we kept the noise level low and only flagged code that was really suspicious. We have:

0-9 = Red
10-19 = Yellow
20-100 = Green

Code Metrics as Check-in Policy

conorm — Fri, 16 Nov 2007 10:39:00 GMT

This is the first in a series of posts around questions I am getting asked related to code metrics.

One question that is coming up is:

Can a check-in policy be defined based on code metrics in TFS?

Well, code metrics itself does not have a check-in policy but fortunately the Maintainability Index, Class Coupling, Depth of Inheritance and Complexity metrics are all available as rules in Code Analysis in Visual Studio. This means you can use code analysis check-in policy enabled with those rules. The rules are grouped under the "Maintainability Rules" Category and are configured as follows:

Metric	Corresponding Rule	Threshold
Depth of Inheritance	CA1501 AvoidExcessiveInheritance	Warning at above 5 levels deep
Complexity	CA1502 AvoidExcessiveComplexity	Warning at above 25
Maintainability Index	CA1505 AvoidUnmaintainableCode	Warning at below 20
Class Coupling	CA1506 AvoidExcessiveClassCoupling	Warning at above 80 for class and above 30 for a method

In the policy editor this would look like the following:

This allows you to enforce that code meets these thresholds before check-in. In future we are looking in to allowing the threshold values to be changed.

Code Metrics Customization

conorm — Fri, 16 Nov 2007 02:54:22 GMT

A number of people have asked:

Is there a way to customize the existing code metrics or add additional ones?

The simple answer at this time is No. There are no extension points or customizations available at this time for code metrics. This feature was added late in the cycle for VS 2008 and we were out of time to design and add the required hooks. We will use this release to gather feedback about the sorts of extensions you want.

To that end please feel free to respond to this post with information about any customizations or additional metrics that would be of interest to you. This will help us a lot in our future planning.

New for Visual Studio 2008 - Code Metrics

David M. Kean — Wed, 03 Oct 2007 17:00:00 GMT

While we've already briefly spoken about Code Metrics when we announced it, I thought I would discuss it in a little more depth, in particular the metrics it provides.

As mentioned previously, Code Metrics is a new tool window that helps users find and act upon complex and unmaintainable areas within an application.

The following shows the results of running Code Metrics over a fictional business application:

As you can see from above, for Visual Studio 2008, we're providing five metrics. These are detailed below.

Class Coupling

At each level, this indicates the total number of dependencies that the item has on other types. This number excludes primitive and built-in types such as Int32, String and Object. The higher this number, the more likely changes in other types will ripple though this item. A lower value at the type level can indicate candidates for possible reuse.

The following shows how coupling is calculated:

For example, as you can see above, Account is coupled to two other types, Address and Order, whereas Country is not dependent on any other type.

Depth of Inheritance

At the type level, depth of inheritance indicates the number of types that are above the type in the inheritance tree. For example, a type that derives directly from Object would have a depth of inheritance of 1. At the namespace and project level, this indicates the highest depth of inheritance of all the types contained within it. This number does not take into consideration the depth of any implemented interfaces. Deep inheritance trees can indicate an over-engineering of a problem and can increase the complexity of testing and maintaining an application.

The following shows how depth is calculated:

For example, in the above inheritance hierarchy, ListControl and Label have a depth of inheritance of 3, whereas Component has a depth of inheritance of 1.

Cyclomatic Complexity

At each level, this measures the total number of individual paths through the code. This is basically calculated by counting the number of decision points (such as if blocks, switch cases, and do, while, foreach and for loops) and adding 1. This number is also a good indication on the number of unit tests it will take to achieve full line coverage. Lower is typically better.

The following shows how complexity is calculated:

Lines of Code

At each level, this is a measure of the total number of executable lines of code. This excludes white space, comments, braces and the declarations of members, types and namespaces themselves. Lower is typically better.

The following shows how the lines are calculated:

Maintainability Index

At the member and type level, this is an index from 0 to 100 indicating the overall maintainability of the member or type. At the namespace and assembly level, this is an average of the maintainability index of all types contained within it. This index is based on several other metrics, including Halstead Volume (which factors in the number and use of operands and operators), Cyclomatic Complexity and Lines of Code. A low number indicates code that is complex and hard to maintain.

The Maintainability Index column also includes a icon that gives a quick indication as to the overall maintainability and complexity of a particular item. The following table shows the range at which an icon is shown:

Icon	Level	Range
	High Maintainability	Between 20 and 100 inclusive
	Moderate Maintainability	Between 10 and 19 inclusive
	Low Maintainability	Between 0 and 9 inclusive

These icons allow you to see at a glance any trouble spots that you should start to focusing on or filling bugs against.

Conclusion

Now that you know what these metrics measure, and how they are calculated, the next thing you'll want to know is what you should do when you start to see some red in your application. In a future post, I will cover why should should care about each metric, and how to go about fixing some of the common issues that cause particular metrics to high (or low in the case of Maintainability Index).

New for Visual Studio 2008 - The 'Analyze' menu

David M. Kean — Sat, 22 Sep 2007 17:00:00 GMT

While Ian Huff has already talked about the new 'Developer' menu that has been added to Visual Studio 2008, I thought I would briefly mention some of the changes to it that were made between Beta 2 and RTM.

For those that have yet to see this menu, similar to the existing Test menu, we've added a new top-level menu that will contain the Code Analysis, Code Metrics and Profiling menu items. This was done for discoverability reasons, and something we hope to leverage moving forward towards Rosario.

A quick rundown on some of the things that have changed since Beta 2:

Renamed the menu itself from the very non-specific Developer to the little bit more descriptive Analyze.
Removed the Code Analysis and Code Metrics sub menus and pulled their associated menu items up one level.
Pulled New Diff Report up one level and renamed it to Compare Performance Reports...
Renamed the Code Metrics and Code Analysis menu items:
Generate Code Metrics -> Calculate Code Metrics.
Migrate Code Analysis Policy Settings to Solution -> Code Analysis Settings for Solution.
Replace Code Analysis Settings with Code Analysis Policy Settings -> Replace with Check-in Policy
Add Code Analysis Policy Settings to Code Analysis Settings -> Merge with Check-in Policy

I probably don't have to tell that these changes, that we made to increase the usability of the menu, are great improvements, especially the renaming of probably what would have been in the running for the longest menu item in Visual Studio; Replace Code Analysis Settings with Code Analysis Policy Settings.

While a very small addition to Visual Studio 2008, we hope the new Analyze menu helps improve your workflow when working with the Code Analysis and Profiler features.

Code Analysis (Team System) Chat Roundup

David M. Kean — Wed, 02 May 2007 18:00:00 GMT

We had two great chats last Friday. Thank-you to all that attended, I had a lot of fun.

The following is a list of Code Analysis questions that were asked:

Code Metrics

DavidKean_MS (Moderator):
Q: JeffBe: I'm very interested in the code metrics, are these integrated into build reports though? We'd like to be able to track code metrics over time.
A: This is something that we've received a lot requests for. Unfortunately, for Orcas we won't be integrating this into TeamBuild or any sort of reports. However, this is definitely something that we want to do in a future release.

DavidKean_MS (Moderator):
Q: Where can i find more information about CyclomaticComplexity feature in Orcas ?
A: Besides what we've already published on our blog (http://blogs.msdn.com/fxcop/), there's not much information around at the moment. This will change in Beta 2, we also do plan on doing more in-depth posts on the blog in the coming months.

DavidKean_MS (Moderator):
Q: I'll ask the audience. What would be your preference on something that we should focus on in Code Metrics? 1) TeamBuild integration 2) Extensibility (the ability to add your own metrics) 3) Policy Enforcement of Metrics (being able to prevent check-ins of code that doesn't pass a certain threshold of maintainability. 4) More extensive metrics

DavidKean_MS (Moderator):
Q: Extensibility, enforcement, more extensive metrics (in that order)
A: Thanks Peter. You want to plug in your own metrics? Do you think we're missing some?

DavidKean_MS (Moderator):
Q: metrics. Complexity is a bit subjective--that's something that would be best suited to customization. I'd be a big proponent of something that measures degree of Law of Demeter violations...
A: Yes we've heard that. This is one of the reasons we really want to wait to see what customization customers really want. ie Do they simply want to tweak the numbers that make up the magic 'Maintainability Index', or do they want to completely replace the metric altogether.

Code Analysis

DavidKean_MS (Moderator):
Q: Did you have to do anything special with CodeAnalysis in Orcas to support multiple versions of the .NET Framework?
A: The great thing about Orcas is that we actually get this for free. The new MSBuild mult-targeting means that you simply select the Framework you want to build against and Code Analysis 'just works'.

DavidKean_MS (Moderator):
Q: Any idea if nStatic-like features will be added to code analysis?
A: While I haven't yet played around with nStatic (I don't believe it has been released), we are looking at doing a lot more advanced analysis in a future version of Visual Studio.

DavidKean_MS (Moderator):
Q: Any added Code Analysis rules for C# 3.0 language extensions?
A: Unfortunately no. Similar to the introduction of the .NET 1.0, it’s still not very clear on the gotchas and mistakes that people make when writing Linq. With regards to the other features, such as Extension methods, Automatic Properties, etc what would you like to see?

DavidKean_MS (Moderator):
Q: The Static Analysis results are published to the data warehouse, will we ever see them in the build reports? Or is there a way to customise the build reports to include them?
A: Grant, I'm not an expert on our integration with Team Build, but I'll certainly pass this request onto them.

DavidKean_MS (Moderator):
Q: I guess I'm out of luck on var (doesn't make it into the assembly, per se). I know many people watching C# 3. demos are really put off by wholesale abilities of their developers to use features like extension methods.
A: It's pretty likely that you won't see any rules from us 'stating' don't use Extension methods. ;) However, this wouldn't be too hard to write a simple rule to do this. We are thinking about releasing rule samples when Orcas releases, I'll make sure this gets chucked in.

DavidKean_MS (Moderator):
Q: to specifically answer your question "...missing some". Not really, other than possibly new C# 3.0 stuff. Most of the rules would be very specific to either an in-house class library or the way one shop mandates coding/design.
A: As you know, a lot of the C# 3.0/VB 9 improvements are simply syntactic sugar over IL. So it actually makes it hard to write rules that target them. What I actually meant though, do you think we're missing some metrics?

Custom Rules

DavidKean_MS (Moderator):
Q: David: Question for you, in addition to FxCop do you think a tool similar to StyleCop will be available in the future? Although API level code analysis is great there are certain rules we'd like to enforce on non-public code constructs.
A: As you well know, FxCop/Managed Code Analysis currently runs over an assembly and not actually the source code. To change this would be a significant change. However, in saying that, this is one of many features we may consider doing in a future release, however, if we do end up doing it, it’s likely to be more a long term feature, rather than short term. For the next couple of versions at least, you will still see binary analysis.

DavidKean_MS (Moderator):
Q: I think the greatest potential of TFS/CodeAnalysis is in customization. Given it's something that makes it to the assembly level, for shops to be able to define their custom rules and enforce them at check-in would be a huge plus.
A: Peter we hear ya loud and clear. This is definitely something that we want to focus on in the future. The question is, do we focus on making rules more customizable or do we focus on allow teams to write their own rules? What does everyone think?

DavidKean_MS (Moderator):
Q: RE: customizable/rule-creation. I think rule creation would tend to be more accepted. Most of the shops I deal with accept most of the out-of-the-box rules and I can't think of many that could be improved by customization (bar metrics...).
A: What about naming/casing guidelines? Do you see many shops wanting to customize these or are they happy following the Design Guidelines?

DavidKean_MS (Moderator):
Q. We do having naming standards that are a subset of the Design Guideline, i.e. we've made some rules more restrictive so we would want to customise these.
A: Good to know. Will bring this to the product team.

FxCop

DavidKean_MS (Moderator):
Q: Is a release of FxCop planned to coincide with the RTM of Orcas? What classes of code analysis with TFS include that FxCop won't?
A: It is likely that the next version of FxCop will be released sometime around the release of Orcas. Things that are included in Visual Studio that FxCop does not have are (but not limited to); Code Analysis Policy (the ability to enforce that code passes particular rules on check-in), Maintainability and Reliability rules, Code Metrics, out-of-the-box support for MSBuild (so warnings can be displayed in the Error List) and a few other features.

DavidKean_MS (Moderator):
Q: Are there any updates coming to FxCop 1.35? It's been a long time since release, and there are several bugs.
A: Although I can't get give an exact date, we do plan on releasing an update to FxCop sometime in the next quarter. What bugs are you hitting?

DavidKean_MS (Moderator):
Q: The fxcop bugs are generally "rule exceptions". We have worked around some of them (disabling rules), but this is not ideal. We have 100s of projects and it is hard to manage.
A: By 'rule exceptions' do you mean crashing rules? Can you make sure that we know about these over on the Code Analysis forum (http://forums.microsoft.com/msdn/ShowForum.aspx?ForumID=98)? We have fixed a lot of these since 1.35, however, I want to make sure we fixed the ones you've encountered.

Correct usage of the CompilerGeneratedAttribute and the GeneratedCodeAttribute

David M. Kean — Fri, 27 Apr 2007 18:00:00 GMT

Both Code Analysis, FxCop and Code Metrics make extensive use of CompilerGeneratedAttribute and GeneratedCodeAttribute to distinguish between user-written code and tool and compiler generated code.

The following describes this behavior:

Code Analysis in Visual Studio 2005 and FxCop 1.35

- Compiler Generated. Does not raise warnings against compiler generated code. Uses an algorithm (mainly based on the name) to determine if code is compiler generated.
- Tool Generated. Raises warnings against nearly all tool generated code. Uses an algorithm to turn off particular rules against code marked with GeneratedCodeAttribute and generated by specific code generators.

Code Analysis in Visual Studio 2008 and FxCop 1.36

- Compiler Generated. Does not raise any warnings against code marked with the CompilerGeneratedAttribute. Also users an algorithm to determine if other code is compiler generated.
- Tool Generated. Does not raise warnings against code marked with the GeneratedCodeAttribute if Suppress results from generated code is turned on (the default).

Code Metrics in Visual Studio 2008

- Compiler Generated. Does not display or generate metrics for code marked with the CompilerGeneratedAttribute. Also users an algorithm to determine if other code is compiler generated.
- Tool Generated. Does not display or generate metrics for code marked with the GeneratedCodeAttribute.

Unfortunately, there are many cases of incorrect usage of these attributes both internally and externally of Microsoft, and this blog post is an attempt to make things a little clearer in the correct application of these attributes.

CompilerGenerateAttribute

This attribute is for compiler use only and indicates that a particular code element is compiler generated. This should never be used in source code whatsoever. In fact, some users believe that usage of it should be a compilation error. I tend to agree.

For example, the following is an incorrect usage of this attribute (this is not compiler generated code):

/// <summary>
/// A strongly-typed resource class, for looking up localized strings, etc.
/// </summary>
// This class was auto-generated by the StronglyTypedResourceBuilder
// class via a tool like ResGen or Visual Studio.
// To add or remove a member, edit your .ResX file then rerun ResGen
// with the /str option, or rebuild your VS project.
[DebuggerNonUserCodeAttribute()]
[CompilerGeneratedAttribute()]
internal class Resources
{
    private ResourceManager resourceMan;
    private CultureInfo resourceCulture;

    [SuppressMessageAttribute("Microsoft.Performance", "CA1811:AvoidUncalledPrivateCode")]
    internal Resources()
    {
    }

    /// <summary>
    /// Returns the cached ResourceManager instance used by this class.
    /// </summary>
    [EditorBrowsableAttribute(EditorBrowsableState.Advanced)]
    internal static ResourceManager ResourceManager
    {
        get
        {
            if ((resourceMan == null))
            {
                ResourceManager temp = new ResourceManager("WindowsApplication1.Properties.Resources", typeof(Resources).Assembly);
                resourceMan = temp;
            }

            return resourceMan;
        }
    }

    /// <summary>
    /// Overrides the current thread's CurrentUICulture property for all
    /// resource lookups using this strongly typed resource class.
    /// </summary>
    [EditorBrowsableAttribute(EditorBrowsableState.Advanced)]
    internal static CultureInfo Culture
    {
        get { return resourceCulture; }
        set { resourceCulture = value; }
    }
}

GeneratedCodeAttribute

This attribute is for use by custom tools that generate code. It should only be applied to code that is re-generated over and over and should not be used by templates that the user is expected to modify. Nor should it be applied at the type level if the type being generated is a partial class. In this situation, it should be applied only against the individual members that are contained within the generated part of the type.

For example, the following shows the incorrect usage of this attribute (it is being applied at the type level to a partial class):

[GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "9.0.0.0")]
internal sealed partial class Settings : ApplicationSettingsBase
{
    private static Settings defaultInstance = ((Settings)(ApplicationSettingsBase.Synchronized(new Settings())));

    public static Settings Default
    {
        get { return defaultInstance; }
    }

    [UserScopedSettingAttribute()]
    [DebuggerNonUserCodeAttribute()]
    [DefaultSettingValueAttribute("")]
    public string MySetting
    {
        get { return ((string)(this["MySetting"])); }
        set { this["MySetting"] = value; }
    }
}

The following shows the correct usage of this attribute:

internal sealed partial class Settings : ApplicationSettingsBase
{
    [GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "9.0.0.0")]
    private static Settings defaultInstance = ((Settings)(ApplicationSettingsBase.Synchronized(new Settings())));

    [GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "9.0.0.0")]
    public static Settings Default
    {
        get { return defaultInstance; }
    }

    [GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "9.0.0.0")]
    [UserScopedSettingAttribute()]
    [DebuggerNonUserCodeAttribute()]
    [DefaultSettingValueAttribute("")]
    public string MySetting
    {
        get { return ((string)(this["MySetting"])); }
        set { this["MySetting"] = value; }
    }
}

Unfortunately, due to the sheer number of teams within Microsoft that release tool generators and templates (there are hundreds within Visual Studio itself), it becomes impossible to track down all usages of these.

Therefore, if you encounter any incorrect usages of these attributes, free feel to head over to Microsoft Connect and tell us about it.

Announcing Visual Studio Code Metrics!

David M. Kean — Wed, 28 Feb 2007 19:00:00 GMT

Announcing the new Code Metrics feature for Visual Studio ‘Orcas’! Available in Visual Studio Team Developer and Team Suite, this new feature allows users to generate code metrics for projects and solutions and displays the results in the Code Metrics Results tool window:

As you can see above, this feature currently calculates five different metrics; Maintainability Index, Cyclomatic Complexity, Depth of Inheritance, Class Coupling, and Lines of Code. All metrics are averaged at the type, namespace, and assembly levels with the exception of Class Coupling. The Class Coupling metric displays the total number of distinct types referenced at the method and type levels rather than the total number of type references. This is why above, the ProgramLibrary type’s class coupling is 83 instead of 103.

Users will are able to sort the results in the window by column. For example, above the results are sorted by the Maintainability Index column. Note that the proper hierarchy is maintained after sorting.

Users can also filter the results from a particular metric for values between a specified minimum and maximum value. As you can see in below the results of filtering out any results with a Maintainability Index greater than twenty are displayed. (The Library project is still displayed because it contains the code elements within the filter.

The Maintainability Index column includes a maintainability indicator light that changes based on the maintainability of the code. Green indicates good maintainability, yellow indicates moderate maintainability, and red indicates poor maintainability. The thresholds for the maintainability indicator light and the scale of the Maintainability Index column are still being finalized. Look for more information on this in future blog posts.

Users can also export these results into Excel where they can perform their own calculations and transformations.

Finally, to generate code metrics, simply do the following:

In Solution Explorer, right-click on your solution/project and choose Generate Code Metrics

Download the March CTP (either as a Virtual PC image or a self-extracting install) and tell us what you think! What do you like? What needs improvement? And if we were to add support for additional metrics in the future which ones are you most interested in seeing?

Todd King
Visual Studio Code Analysis Team