Today I attended an interesting talk on Performance Signatures by Rico Mariani. According to his experience, a high proportion of performance related problems are due to:

  1. Bad choice of algorithm
  2. Dependencies we take on other frameworks or platforms.

He talked about the second problem in detail. He proposed a mechanism of using performance signatures to evaluate performance of external dependencies we take. Performance signatures are a qualitative measure that can help us to understand the performance characteristics of a method eg. synchronization, memory allocation complexity etc.

He gave an interesting analogy from the architectural world. A structural engineer can evaluate materials on the basis of their tensile strength to build a robust structure. In the same way, a software engineer can use performance signatures to chose the dependencies he can afford to take.

Rico also mentioned how they have done the groundwork to enable this (through static analysis) and create performance signature for the .NET framework. They are expected to be published on MSDN in due time. A link to the cost metrics is here.

It would be great if these could be integrated into the VS IDE using Intellisense to use different colors - red, yellow, green to indicate the cost involved for a certain method. This will help people understand and detect performance issues early in the cycle. Another cool idea, as suggested by Krzysztof Cwalina, is to enable this using FxCop.

One might ask why does one need performance signatures? I think performance signatures will be most relevant when you are working against a library that has a high level of abstraction. Take for example - the .NET framework XmlDocument class. I have seen it being used all too often in cases, when it was not necessarily required. The same XML operation could be done without loading the complete DOM tree of the document into memory.

In general, .NET framework offers a high level of abstraction to make the life of a developer simple by creating very easy-to-use friendly APIs. But like any abstraction, it is bound to be leaky (See Joel's original post on leaky abstractions). Developers become care-free, thinking that the framework will do everything to manage memory and do the heavy-lifting, and they do not need to care about it. They do not give so much thought to dependencies they are taking during the design and development phase. Only after 6 months of investment in the project when the end-to-end scenario is working, they realize that the performance just sucks and the software is quite unusable. In this case, performance signatures can help get a better understanding of the performance implications of taking a dependency on a particular method.