IntelliTrace Info : Profiler

Collecting performance counter information with the Visual Studio Team System profiler

ianhu — Wed, 17 Oct 2007 20:21:00 GMT

Collecting performance counter information with the Visual Studio Team System profiler

Introduction

If you have ever done performance work on Windows systems you are probably already familiar with the PerfMon tool. This tool gives you an overview of your system performance and can be an invaluable tool in the early (and sometimes the late) stages of a performance investigation. If you look at either of my two PerfMon links you can see that PerfMon is usually used to pick out what the performance bottleneck is when examining an application. By “performance bottleneck” I mean picking out the resource that is being saturated by your program (CPU, network, disk). PerfMon does this by tracking some subset of performance counters that monitor different aspects of system performace. This is important as you often need to use different techniques and tools for investigating different bottlenecks. For example, using the Visual Studio profiler it’s usually best to use sampling mode to investigate CPU issues and instrumentation mode to investigate memory and disk issues.

With the Visual Studio Profiler we wanted to give customers an easy and integrated way to collect this performance counter information and view it alongside their performance data. This was especially important to us as with this information we could help customers to analyze specific trouble areas of their program or to choose the correct profiling modes based on their performance bottleneck.

Turning on performance counter collection

To access this performance counter collection start out by going to the session properties of your performance session. In the session properties you will want to select the new “Windows Counters” options, shown in the screenshot below.

On this page you can check the box in the top of the screen labeled “Collect Windows Counters” to add windows performance counter collection to your next profiling run. Below that checkbox you see a box to input a value for how often the counters are collected labeled “Collection interval (msecs).” But before we dig into the collection interval it is important to understand just how these counters are collected. With our profiler we have the concept of “marks” that show up in the data collection stream with a comment associated with them. Unlike all other profiling data these marks are not aggregated and show up in chronological order in the “Marks” view of the performance report. These marks are usually added by annotating your code with the data collection APIs (native API here). When we tell the profiler to collect Windows performance counters they get collected every time one of these marks gets hit. The collection interval control allows you to automatically insert marks in your program while running at some specific time interval. That way you can collect performance counter information easily without all the hassle of adding marks to your program manually.

In addition to setting the collection interval you also need to control what performance counters are being collected. In the lower section of the property page are all the different performance counters that you can selected to be collected. We surface all the same counters as PerfMon, so if you have a favorite counter that you like to track you can be sure that we’ll have it. By default we’ve included basic processor, memory and disk usage counters. We picked these as they give a good basic overview as to how your application is taxing the system.

Viewing performance counter data

So to demonstrate how these marks and associated counter values are actually surfaced in the report file we need to run the profiler and collect our data. For this example I’m using the PeopleTrax sample application and running it in sampling mode with the default set of performance counters and the default 500 ms marking interval. For my profiling scenario I’m just going to wait for the application to initialize, hit “get people” button to load the records from the database and then shut down the application. After the application is shut down the performance report will load automatically and then I will jump to the “Marks” view.

In the view above you can see the automatically collected marks. And for each mark you can see the memory, disk and processor usage at that point in time. Now this data can already tell us much about our program. For example we can see that we started out with a brief processor spike between marks three and five closely followed by a memory usage spike between marks five and eight. But how can we actually tie these values into all the rest of the performance data that we collected? After all these are timeline values while the rest of the data in the report is aggregate data from the whole run. Luckily we’ve provided a new filtering feature to help use the performance counter data to guide your performance investigations.

Filtering performance data from the marks view

Looking further into the marks data there was another big memory spike later in my program between marks 129 and 133. But how can we tell what was actually happening in the program during that time? To start the investigation ctrl-click marks 129 and 133 in the marks view so that they are both selected. Then right-click on one of them and select the “Add Filter on Marks” command. When you do this a new filter dialog will appear docked on the top of your performance report.

The filter control has a lot of depth to it, but for now you can just understand that is it currently saying “only show the data collected between mark 129 and mark 133.” To actually apply the filter to the performance report click the “execute filter” button in the toolbar (it’s the button with the green play icon). You will see a progress bar for the reanalysis that is being performed and then the report will pop back up. Only now all the aggregate data for the report is only the data that we captured between those two marks! With just a quick glance at the summary page below you can tell exactly what our application was doing and why we were seeing the memory usage spike at that point (big blue circle added by me for emphasis ;-) ). However it is important to keep in mind that you will need to clear the filter items and rerun the empty filter or to close and reopen the report to get back to seeing all of your performance data.

In addition to filtering on marks the filter grid can so some other very neat stuff such as filtering on threads, by timestamps or by time percentage (to do something like “show only the first 10%” of my run). I don’t have room to cover all the other filtering types here though, so I’ll have to come back and hit them in a later article.

Conclusion

Collecting performance counters is a very cool new feature that we’ve added to the profiler for Orcas. Even before we get into how cool it is to be able to filter down your profiling results based on performance counters it’s nice to just be able to collect and store those values side by side with your profiling data. And when you throw in the filtering features it quickly becomes an invaluable tool for performance analysis.

Pinpoint a performance issue using hotpath in Visual Studio 2008

ianhu — Fri, 14 Sep 2007 20:34:00 GMT

Analyzing a performance report

When you first open a performance report in Visual Studio 90% of the time the question on your mind is “where is my program spending most of its time?” In Visual Studio 2005 the best way to find this was to move to the calltree view and from there to follow the execution path with high inclusive functions down the tree. If you are not familiar with the calltree view, this view places all the execution paths of your program into one big tree structure. In this tree you can see both inclusive and exclusive values for either samples (in sampling mode) or time (in instrumentation mode). In a nutshell, inclusive means “functions that caused work” so any inclusive time or samples for a function can be attributed to that function or any of its sub-functions. On the other hand, exclusive values can be attributed only to that function and do not include any values from sub-functions.

So with the above explanation the way to find out where your program is spending most of its time is to drill down the tree and whenever you have a choice between children of a function pick the child with the highest inclusive value (meaning that it is doing more work below it then the other children). Eventually you will reach some functions with high exclusive values, which are the functions that are actually doing the most work in your program. At that point you can examine those functions and the path taken to them and start working to fix your performance issue.

Hotpath

The problem with the above scenario is that, quite frankly, it’s a pain in the butt to have to always drill down on the tree to find these hot functions and hot paths. Especially with sampling reports the calltree can be many levels deep and it can take a while to get to the functions of interest. So for Visual Studio 2008 we’ve added a small, but very useful feature called “Hotpath” to automate this scenario. Now when you open a performance report and move to the calltree view you will see a small flame icon located in the toolbar. When you push this button it will expand the hot path from the root following the rules that we mentioned above (following the high inclusive numbers when choosing between children). In the example seen below I’ve run a profiling session on a managed application that calculates and displays a Mandelbrot set. When running Hotpath on the report it jumps me down the tree directly to the DrawMandel and SetPixel functions that are performing the bulk of the work for this scenario. The Hotpath will stop whenever it reaches a function with high exclusive values or when it hits two children with similar inclusive values. Also, it can be run from any point in the tree (just right click on the function) so you don’t have to always run it from the root function.

As you can see, the hotpath feature can save you a lot of time in performance analysis. Already folks who use the profiler internally here at Microsoft have become big fans of this feature. In fact, there has been spontaneous applause at meetings when we demoed the feature for the first time! As of right now it’s the closest that we have to a “show me my performance issue button.” And it’s one of the many ways that we are trying to help users to better understand and take action on their performance information in Visual Studio 2008.

The New Developer Menu in Visual Studio Team System

ianhu — Tue, 07 Aug 2007 19:29:00 GMT

The new Developer menu in Visual Studio Team System

In my recent series of blog posts I’ve been covering some of the cool new features that the Visual Studio Profiler team has been adding to the code profiler that is included with Visual Studio Team System for Developers (and Team Suite edition). With all the features that have required hours and hours of work from us we’ve also made a small change that we think will have a big impact on how customers find the profiler. This change was creation of the new top-level “Developer” menu and moving the profiler and code analysis commands to this new menu.

It was scary for us to find out how many customers were unaware that a powerful profiler shipped with VSTS. Some were even purchasing competing profilers without first checking out the one that they already had! So to make the profiler more discoverable we moved it out from where it was buried in the Tools->Performance Tools menu to the new Developer menu. Now the name of this Developer menu might change, but there will assuredly be a top level menu with both your profiling and code analysis tools available in it.

The Visual Studio Profiler Data Collection Control Part 1: Excluding Application Startup Time

ianhu — Tue, 17 Jul 2007 20:44:00 GMT

The Visual Studio Profiler Data Collection Control Part 1: Excluding Application Startup Time

Even if you are familiar with Visual Studio Team System you may not already know that a powerful profiler is included with both the Developer and Team Suite editions of that product. If you are not familiar with the profiler you can check out some of my earlier blog entries or TechNotes. In my new series of blogs posts, starting with my first post about the comparison documents, I’m coving all the cool new features added to the profiler that customers will finally be able to see in the upcoming Orcas Beta releases.

The Data Collection Control

In this article I’ll be focusing in on the data collection control. This control allows users several new ways to control and annotate your profiling files from the IDE while the profiler is running. Previously we have offered most of this control already, but only from the VSPerfCmd command line tool. The Data Collection Control lets you do things like pause and resume profiling, name marks and annotate the VSP file by adding marks to it. To show how to use the basic functions of the Data Collection Control I’ll walk through a few quick scenarios to demo these features. This TechNote assumes that you are familiar with the basics of using the profiler, if not check out the TechNotes here for a good intro.

Eliminate Startup Time from Analysis

A highly common customer scenario that we are addressing with the data collection control is the customer who wants to collect steady-state data on their process and who does not care about how their time is being spent in startup. By controlling when you are collecting performance data the data collection control makes it a breeze to exclude startup information from being collected. For my example, I’m using the PeopleTrax sample application. When you first start up PeopleTrax the application take a while to initialize, then you can click the “Get People” button to populate a list of customer names. For the purpose of this demonstration I want to just collect data on the actual “Get People” operation and not collect any data during the application startup or shutdown.

First off, make sure that you have a performance session set up in the Performance Explorer that is targeting your application. Now normally we would use the green launch button on the Performance Explorer to start our profiling session. But in our case this would not work, as we would begin by collecting data, and we don’t want to record any of our startup time. So instead of clicking launch, we’re going to click the dropdown control on the launch button and select “Launch with profiling paused.” This will launch the application with the profiler attached to the process, but will the actual collection of profiling data paused.

Now after we have launched the PeopleTrax application will pop up, and back in the IDE we will see the new data collection control docked with the Performance Explorer (the data collection control should be on top of the Performance Explorer when you are currently profiling).

Along the top toolbar of the data collection control you will see several icons, two of which are of particular interest to our current scenario; these being the pause profiling (second from the left) and resume profiling (third from the left) icons that control profiling state. Since we started out with profiling paused the profiler is not currently collecting data and thus the pause profiling button is grayed out. After the PeopleTrax application has initialized and popped up on-screen we have now passed the startup portion of the application and we want to collect data on the “GetPeople” operation.

To do this, we will enable profiling data collection from the data control. Just click the “Resume Profiling” button. Now the pause profiling button will be active and the infobar on the data collection control will inform you that profiling is active. Right after activating profiling move over to the PeopleTrax application and click “Get People” to populate the customer list from the database. After the listview is populated click the pause profiling button to stop collecting data and then shutdown the PeopleTrax application to generate your report file.

Now the generated report will have only the data collected after you pressed the resume profiling button and before you pressed the pause profiling button, excluding startup and shutdown time. You also could exclude this time by annotating your code to start and stop profiling, but the data collection controls makes this a quick and simple operation from the IDE, without having to muck about with command line tools.

In my next blog entry I’ll look at how we can use the data collection control and filter queries to view only specific parts of performance reports after collecting a large data file.

Comparing performance reports with the Visual Studio Team System Profiler

ianhu — Tue, 03 Apr 2007 20:54:00 GMT

Comparing performance reports with the Visual Studio Team System Profiler

With the recent release of the first Beta for Visual Studio Team System (codename Orcas) customers will get their first chance to see all the great new features that we are adding to the product. For the profiler in particular we’ve added some very cool new features that I’m really happy to finally be able to reveal publically. Remember that the profiler is only included with VSTS: for Developers and VSTS: Team Suite, so make sure that you have one of those SKUs if you are following along.

Comparison Reports from the IDE

The first feature that I want to cover is the new comparison document that we’ve introduced to help users compare profiling data from two different profiling runs. Say that you have just checked in a possible fix for a performance issue and you want to compare a new performance report to one that existed before the change to see if your change really helped. Our new comparison features make these types of questions easy to answer. Comparing two files of performance data is a very common scenario for customers, especially when dealing with regression testing, so this was a priority feature for us in this release. After all, performance data when taken in isolation, without goals to hit or old values to compare with, can be pretty hard to work with. Our goal with this new comparison work is to help customers to make better use of their performance data to achieve the performance results that they desire for their applications.

I would be remiss if I did not take a quick second here to dive a little deeper into the importance of setting performance goals for your applications. Too often developers end up in the situation of closing in on product release and realizing “my app is just way too slow.” Now, it is perfectly understandable that developers want to save performance optimization for the end of the product cycle; after all if the underlying structure is going to change greatly why waste too much time early on trying to tweak things to run as fast as possible? But the real issue with the situation above is in the generalness of the “my app is just way too slow” part. What exactly do you mean by “too slow?” What parts of the app are too slow? What type of performance do customers expect from your app? How long does it take for other similar products to do the same task? For our new comparison features to be really useful you will need to take some time before and during development to get at least basic answers to some of the questions above.

So now that you know to set performance goals for major functionality of your application how can you use the new comparison functionality of the profiler to meet those goals? First off, let’s say that you have just submitted some major code changes to your product and you want to see how those changes have either improved or degraded the performance of your app. Start off by moving to the new “Developer” top level menu in VS. This menu (Figure 1.1) is new, so expect that the names and locations of the items in it may change by the time that RTM rolls around. From this menu select the “New Comparison Report” option to start the process of creating a new comparison file.

(Figure 1.1 – The new VSTS developer menu)

Next, the UI will pop up a dialog that will ask you to select two performance data files to compare. Note that in this screen you can select any combination of VSP and VSPS files provided that both files are either both sampling mode files or both instrumentation mode files. If you have used the VSTS profiler before you will know that VSP files are the report files created by the profiler, but you may not be familiar with VSPS files. I will cover VSPS files in more depth in a following article, but in a nutshell they are VSP files that have been saved after performance analysis, resulting in a much smaller file that can be opened much quicker than a full VSP file. However, after saving as a VSPS file you lose some of the advanced analysis options that you had with a VSP file. For this example, I’ll just be comparing two profiling runs of a simple console application. This application concats a bunch of strings (a common performance issue) and since VSTS code analysis flags this as a possible issue I’ve converted it to use the StringBuilder class instead. The second analysis file is from the performance run that uses StringBuilder and I want to compare it to the earlier run to see if I have made any performance gains.

If you select one sampling mode performance file and one instrumentation mode performance file then you will get an error popup about incomparable file types. Sampling and instrumentation work in totally different ways and with completely different column sets, so we don’t want to allow comparisons between the two types of files.

After selecting the two files to compare and clicking “ok” a new document window will open in the IDE and you will see progress bars for the analysis of both of the performance files. After the loading finished you will see a comparison document like the one below open in the IDE (Figure 1.2).

(Figure 1.2 – The comparison report view)

So there is a heck of a lot of information in that comparison report, so let me see if I can help to break it down. In the top left “Comparison Files” groupbox we see the filenames of the two report files that we are comparing (Concat.vsp and StringBuilder.vsp). In the “Comparison Options” groupbox we specify what values to use for comparison and what magnitude of change we require to actually show items in the comparison report. The “Table” value in this group specifies the type of objects that we are comparing. The default value is to compare functions but you can switch this to compare other values like modules or IPs. The value in the “Column” combo box specifies what report column to use to compare between the baseline report and the new report. The column values available for comparison will change based on the comparison table that you have selected. By default, we are comparing the “Exclusive Samples %” value for the functions table. If you are not familiar with the various performance columns (you can get more info on performance report columns here) “Exclusive Samples %” is the number of total samples that were taken in the specific function and not in any subfunction. Finally we have the “Threshold” value. This value specifies the minimum difference that there must be between new and baseline values for the function to actually be listed in the comparison report. For example, let’s change the threshold value to 1.0 and click the apply button to apply the changes. Now the comparison report has been trimmed to just show functions that changed by one percent of the overall samples (figure 1.3).

(Figure 1.3 – The trimmed comparison document)

Now that we have trimmed our report to a more manageable size let’s take a look at the body of the comparison report. In the “Comparison Column” we see the names of the functions that are being compared between reports. Next we see columns that show the value of “Exclusive Samples %” in both the baseline report, in the comparison report and the delta between the two (comparison value – baseline value). Note that now that we have changed our threshold value to 1.0 only functions with an absolute delta value greater than 1.0 show up in the report. If we were to change threshold to 0.0 we would see every function in both reports, even if the function did not change value at all. Functions that were only present in the baseline report or only present in the comparison report also show up in this report as long as their value (and thus their delta value) is greater than the threshold value.

So what does the comparison report tell us about our code change from using String.Concat to using StringBuilder.Append when constructing our strings? Well it appears that our overall change to system performance was basically a wash, we used to spend 4.4% of our time in the AppendItemsConcat function and now we spend 4.62% of our time in the AppendItemsStringBuilder function. At a lower level we spend less time in the String.wstrcpy function and more time in memcopy, this is to be expected based on how String.Concat and StringBuilder handle combining strings. Also, if we see something in the comparison report that we want to investigate further we can right click on that function and choose “show matching row in baseline file” or “show matching row in new file.” These options will open up (this open should be very quick since we already have it analyzed the file for the comparison) the matching performance report so that you can dig deeper into the calltree for that specific function. You can see the result below of right-clicking the “show matching row in new file” on the String.wstrcpy function. The new performance file has been opened and the String.wstrcpy function has been highlighted in the function view (fig 1.4).

(fig 1.4)

From the investigation above we can see some differences between the Concat implementation and the Stringbuilder implementation but we really don’t see any measurable performance gains. Perhaps instead of saving CPU cycles we actually ended up saving time and space in allocations. If we want to take a look at the different allocation patterns for the two scenarios we can create two memory profiling reports and compare them. Note that memory allocation reports are a separate type of report so you will not be able to compare allocation reports to non-allocation reports. In the example shown below we can see the difference in managed allocations between the String.Concat and the StringBuilder scenarios (Figure 1.5).

(Figure 1.5 – A managed memory comparison report)

In this comparison I’m looking at the “Types” table and checking the difference in bytes allocated for that specific type (Exclusive Bytes). As we can see, for the cost of allocating one StringBuilder and a Char[] I’ve cut down the amount of string bytes allocated in half! Not too shabby. Outside of specific performance investigations the new comparison reports can be a really handy learning tool for when you want to compare a few different approaches to the same scenario.

Comparison Reports from the Command Line

So now that I’ve show you how to work with these nifty comparison reports from the IDE you may wonder how you would integrate this functionality into something like a script driven regression prevention system. We’ve foreseen this need (and we know that not everyone loves the VS IDE like I do) so we added on comparison report functionality to the command line performance report tool VSPerfReport. VSPerfReport can be found in the Team Tools\Performance Tools directory under the main visual studio directory in Program Files.

If you run -? on VSPerfReport.exe you will see the new comparison (called diffing in the command line tool) in the “file diffing” section.

---- File Diffing ----

/diff Activate diffing mode for comparing two vsp files (summary options will be ignored in diff mode)

/diffthreshold:[value] Below this value of difference the differ will regard two values as the same. Also, new data with values under this threshold will not be shown.

/difftable:[tablename] Use this specific table to perform diffing. The default is the functions table.

/diffcolumn:[columnname] Use this specific column to perform diffing. The default is the exclusive samples percent column.

/querydifftables List the valid diff tables and columns for the two vsp files provided.

To perform a simple comparison using all the default values just type something like the following:

Vsperfreport /diff Concat.vsp StringBuilder.vsp

This will compare the two reports using the “Functions” table and the “Exclusive Samples Percent” column with a default threshold level of 0.1. This command will generate a CSV (comma separated value) file that contains the same data that we would see in the IDE comparison report.

Location, Function Name, Old ExclSamplesPercent, New ExclSamplesPercent, Delta

Both,"System.String.wstrcpy(char*,char*,int32)",10.35549,6.076618,-4.278869

In this text report you will see the location of the function (Both, OrigOnly, NewOnly) as well as the name, old value, new value and delta. Remember that by default this has a threshold value set, so if you don’t see all the functions that you expect some of them may be under the threshold value.

Just like in the IDE reports you can tweak the table and column used for the comparison. To see all of your options just run /querydifftables with the /diff command to see the output below (customized for report type of course).

Available Tables and Columns for Diff:

Table: Function

Column: InclSamples -- (Inclusive Samples)

Column: ExclSamples -- (Exclusive Samples)

Column: InclSamplesPercent -- (Inclusive Samples %)

Column: ExclSamplesPercent -- (Exclusive Samples %)

Table: Module

Column: InclSamples -- (Inclusive Samples)

Column: ExclSamples -- (Exclusive Samples)

Column: InclSamplesPercent -- (Inclusive Samples %)

Column: ExclSamplesPercent -- (Exclusive Samples %)

Table: Line

Column: ExclSamples -- (Exclusive Samples)

Column: ExclSamplesPercent -- (Exclusive Samples %)

Table: IP

Column: ExclSamples -- (Exclusive Samples)

Column: ExclSamplesPercent -- (Exclusive Samples %)

Then just use the /difftable and /diffcolumn switches with the diff command to change the table or the column being used. Below I’ve listed an example of doing a comparison of Inclusive Samples on a per module instead of a per function basis.

vsperfreport /diff /difftable:Module /diffcolumn:InclSamples Concat.vsp StringBuilder.vsp

Also, you have the /diffthreshold switch that can be used to set which items will actually get included in the comparison report. From here you should be able to figure out how to generate the same types of reports from the command line as you could from the IDE.

Hopefully this article has given you a good intro to using the new performance report comparison tools in Visual Studio Team System codename Orcas. Before I go I’ll leave you with one last cool feature that we’ve added for Orcas. Say you wanted to show a co-worker the results of your performance investigations into String.Concat and StringBuilder. Just highlight those rows, hit copy and then paste into an Outlook e-mail (fig 1.6).

(Figure 1.6 – Comparison data pasted into an e-mail)

Check out that fancy HTML autopasting! Much nicer then having to look over the unformatted text.

Two years in the software performance world

ianhu — Sat, 07 Oct 2006 00:05:00 GMT

This past summer marked my second year (plus one summer internship) of working at Microsoft and on the Visual Studio Team System profiler. As a fresh out of college hire I had no experience working with large applications or working with application performance in particular. For me, as for many other college students I would assume, performance only meant that the application didn’t lag visibly when presenting it to a professor. If an application did have performance issues my education did give me some clues about where to look to improve application performance but improving performance was mostly a trial and error process of tweaking the high CPU time functions to try and get them running faster. Working at Microsoft has exposed me to so many brilliant performance engineers that sometimes I still feel like I’m in that same college stage of not really knowing what I’m dealing with when it comes to performance. Yet when I look back over my past two plus years here I realize that I’ve learned quite a bit when it comes to performance engineering. So I figured that I would take this chance to share some of the most important things that I’ve learned about performance in my time here at Microsoft. Note that these are more general guidelines that I’ve learned about performance as opposed to specific X is better than Y recommendations.

1. Improving application performance should be a planned process as opposed to something that you do at the last minute in response to a major crisis

One of the trickiest things about performance is detailing exactly what “good” or “acceptable” performance is from an application. To really engineer application performance you need to have some goals about how fast various operations in your program need to finish. Now setting these goals can be very hard, especially if you are working in a relatively new area. Still try to get some numbers down about how you expect / need your program to perform. If you are having trouble getting these numbers try looking at some competitor’s products or products that perform similar tasks to yours. If you are working on a video conversion utility to convert DVD video to Zune sized video and a similar program already exists you better make darn sure that your program takes about the same amount of time or less. Also, remember that these numbers can be changed as you flesh out your design and are able to get some initial performance numbers from prototypes and initial builds. The important thing about these numbers is that early in your development you are thinking about performance issues and about how your application’s performance will affect the experience of your end user.

2. Don’t go crazy trying to optimize low level performance with the profiler before you understand the high level operation of your program and have examined the basic algorithms that you are using

So the profiler is a very powerful tool and it can be indispensible for optimizing the performance of your applications. But you have to know the best times to actually unleash this tool. If you spend a ton of time early in your development cycle using the profiler to optimize some order n squared algorithm you might not be making good usage of your optimizing time. Perhaps you should take a step back and examine if you really need to use that order n squared algorithm and if you could replace it with something order n log n. Remember that the profiler is not a magic performance wand, you need to understand your program, your algorithms and what you are trying to accomplish to use it properly.

3. Know your framework, domain and language

A lot of time in school you will hear your professors tell you (often heard when students complain about having to work with a specific language) that if you learn good programming skills you should be able to pick up different languages and be effective in them very quickly. Now there may be a lot of truth that if you learn how to be a good programmer you can work well in different languages; but when it comes to performance optimizations you really do need to know the language, domain and framework that you are currently dealing with. For example when working with a .NET datatable you need to call BeginLoadData before you start loading in your initial datarows so that you can turn off things like table constraints until all of the initial data is loaded and you call EndLoadData. But if you don’t know this Datatable will happily allow you to add items one at a time and your performance will be terrible. And if you look use the profiler you can see that it is taking a long time to load your initial data but you may not have any clue how to fix it. Really knowing a framework takes a lot of time getting familiar its syntax and all of the little nooks and crannies with it. Performance optimizations can be really hard until you’ve put in enough time to at least be proficient with the framework, domain and language you are working with; general programming and performance knowledge won’t be able to bail you out here.

4. Be willing to ask experts for help in specific areas

In the previous section I talked about really needing to know the specific domain and framework that you are working with before you can make the best use of profiler performance data. But how is it possible to keep up with all the different programming domains and frameworks even internally at a company like Microsoft? The simple answer is that you really can’t, sometimes you just need to isolate a performance issue and then take the results to someone who knows that domain to help you to analyze you results. Say that in the example above you are adding rows to a dataset without calling BeginLoadData and your startup time is scary slow. At this point, if you don’t know much about working with datasets ditch the profiler and seek out someone with expertise in this area. Note that this doesn’t mean you need to know someone personally to help you out. This can be as simple as searching MSDN or live.com for “initialize .NET Datatable” or something of that type. The more important point is to not go crazy staring at the profile trying to squeeze blood out of a stone. Go ask someone who knows better than you or start doing some research of your own online. Just make sure that you’ve done the footwork first and can narrow down the problem for whoever you ask. Being able to use the profiler can help you ask questions like “My application startup time is about five seconds longer then I would like as I seem to be taking to long time to load my initial data into my Datatable, can you help me out with this?” as opposed to the perpetual horror “my app is way too slow d00d!!111!!1fix teh .NET framework l4zy M$ devs!!!!1111!!!"

~Ian

Depends.exe and working with assembly dependencies

ianhu — Tue, 18 Jul 2006 02:52:00 GMT

So in my last blog article I talked about using the fuslogvw.exe tool to figure out .NET assembly binding issues. In this article I’m going to look at another tool that I find myself using daily in the debugging world, depends.exe. In a nutshell, depends looks at your binary files and creates a tree of all their dependencies along with some other useful information. Unlike fuslogvw.exe this works with both .NET and native applications. You can get depends from the official site and it also ships with many different Microsoft development products so you may already have it present on your system. You’ll want to use depends when one of your programs fails to load with some sort of system error or initialization message. I make it a habit to check any program that fails on startup with depends first so that I don’t waste any time trying to debug what is just a missing assembly or a versioning mismatch.

At a basic usage level depends is super simple to operate, just load depends and do a file open on the binary file that you are having issues with. When the file opens up you should see something similar to what I’m showing below.

This example I just did using by using notepad.exe. In the top-left pane you can see the tree structure of all the dependent assemblies of notepad. In the middle pane you can see more detailed information about all of the items in the dependency tree. And in the bottom you can see any errors that popped up during analysis. As you can see, some errors are benign as I currently have a possible error in notepad.exe but it runs just fine. Another benign example is that MSJAVA.dll often shows up as missing in the dependency tree but the application will run just fine without it.

So we’ve seen what depends looks like when a binary with all its binaries is correctly added. So what types of errors and mistakes can depends help you to find? Here is a quick list of the ways that I’ve used depends or seen depends used:

1. As a quick way to check the architecture of a binary

2. CRT side by side issues for msvcr80.dll

3. Discovering that a driver was linking in user mode code (a big no-no)

4. Checking the timestamps of dependent files

5. Checking to see what version of a dependent file is being loaded

So if you are running into initialization or dependency issue with your native or managed applications give depends a try to see what you turn up.

Fuslogvw.exe and diagnosing .NET assembly binding issues

ianhu — Thu, 13 Jul 2006 01:55:00 GMT

So let’s say that you are working with a large managed project and for some reason, one of your binary files just refuses to load. Perhaps you get some esoteric error message and perhaps not. If so then you may very well be in the world of managed assembly binding issues.

For me, Microsoft was my first real programming job after college, so I was not used to having to debug assembly binding issues. But now that I work here I realize how much “debugging” time is really spent getting all of your assemblies to have the correct signing, versioning and to play nice together as opposed to actually stepping through code and fixing issues. As I’ve had to learn all of this in a work environment I thought that I would share some of the small tools and tricks that I’ve learned for diagnosing these issues. Please note that I’m still quite the rookie at these matters so if you are already a kung-fu managed developer then I’m probably not saying anything here that you don’t already know.

One of the first tools that I turn to in these situations is fuslogvw.exe (great name, really rolls off the tongue easily ;-)). This tool keeps track of all managed assembly binding attempts that happen on your system. However, like its name, fuslogvw.exe is rather esoteric and its output can be a little hard to interpret. Hopefully this article will help to get you started with fuslogvw and understanding what it is telling you.

When you first open fuslogvw you will see the screen below. In the main section is where you will see specific binding events, but only after you have turned on the proper settings.

Click on the settings menu to bring up the settings for fuslogvw. As long as my application does not load too many dlls I like to turn on “Log all binds to disk” so that I can see all of the binds that are happening and not just the failures. Also, I sometimes get funny issue when not using “Enable custom log path” so I usually turn that on and point it at a log directory (this keeps it from using the IE cache for logging). For this example I’m going to purposefully delete one of my dependent dlls, select “Log bind failures to disk” and then try to run my application. When it crashes I will then move back to fuslogvw and click the refresh button. Now I will see an entry in the screen about the dll that failed to load.

By double clicking on the entry I can get a text file of information about the file that failed to load. I’ve included the text file from my example below (some names omitted to protect the innocent). From the log we can see that we tried to look up the assembly in the GAC then we tried a few different extensions and locations to try to locate the file. Also, if the file was found but had incorrect signing we would see that here at this stage of the game. Hopefully this has helped to introduce you to fuslogvw and invaluable tool for diagnosing binding issues with managed .NET applications.

*** Assembly Binder Log Entry (7/12/2006 @ 3:27:59 PM) ***

The operation failed.

Bind result: hr = 0x80070002. The system cannot find the file specified.

Assembly manager loaded from: C:\WINDOWS\Microsoft.NET\Framework\v2.0.60425\mscorwks.dll

Running under executable C:\Program Files\Microsoft Visual Studio 8\Team Tools\Performance Tools\XXXXviewer.exe

--- A detailed error log follows.

=== Pre-bind state information ===

LOG: User = REDMOND\XXXX

LOG: DisplayName = XXXXvisualization, Version=8.0.0.0, Culture=neutral, PublicKeyToken=XXXXXXXXXXXX

(Fully-specified)

LOG: Appbase = file:///C:/Program Files/Microsoft Visual Studio 8/Team Tools/Performance Tools/

LOG: Initial PrivatePath = NULL

LOG: Dynamic Base = NULL

LOG: Cache Base = NULL

LOG: AppName = XXXXviewer.exe

Calling assembly : XXXXviewer, Version=8.0.0.0, Culture=neutral, PublicKeyToken=XXXXXXXXXXXX

===

LOG: This bind starts in default load context.

LOG: No application configuration file found.

LOG: Using machine configuration file from C:\WINDOWS\Microsoft.NET\Framework\v2.0.60425\config\machine.config.

LOG: Post-policy reference: XXXXvisualization, Version=8.0.0.0, Culture=neutral, PublicKeyToken=XXXXXXXXXX

LOG: GAC Lookup was unsuccessful.

LOG: Attempting download of new URL file:///C:/Program Files/Microsoft Visual Studio 8/Team Tools/Performance Tools/XXXXvisualization.DLL.

LOG: Attempting download of new URL file:///C:/Program Files/Microsoft Visual Studio 8/Team Tools/Performance Tools/XXXXvisualization/XXXXvisualization.DLL.

LOG: Attempting download of new URL file:///C:/Program Files/Microsoft Visual Studio 8/Team Tools/Performance Tools/XXXXvisualization.EXE.

LOG: Attempting download of new URL file:///C:/Program Files/Microsoft Visual Studio 8/Team Tools/Performance Tools/XXXXvisualization/XXXXvisualization.EXE.

LOG: All probing URLs attempted and failed.

Profiling IO Bound applications with Visual Studio Team System – Part One

ianhu — Wed, 21 Jun 2006 19:19:00 GMT

It just so happens (and I know that I’ve mentioned it here before) that the Visual Studio Team System Profiler has both sampling and instrumentation modes available for you to use. However, in most of my samples so far I’ve been using the sampling mode of the profiler to profile some small applications. And for almost all CPU bound applications the sampling mode is the mode to use as it collects much smaller data files as compared to instrumentation profiling. However, for diagnosing performance problems that are not CPU bound then you may only be able to track down the performance issue by using the instrumentation mode of the profiler. In this blog entry we will look at using the instrumentation mode of the profiler to track down an IO bound issue in a small sample application.

But before we get too deep into using the profiler to diagnose an IO Bound application we may want to better define exactly what “IO Bound” and “CPU Bound” mean.

· IO Bound – An IO bound application is an application where its speed is dependant on waiting for IO operations to complete. In other words, you are asking for data and moving it around more then you are actually computing things

· CPU Bound – A CPU bound application is an application where its speed is dependant on waiting for the CPU to process data. In this case you have the data, but you have to wait to perform calculations on it

There are other types of binding (such as memory bound) but we’re not going to look at those in this specific blog entry. In general, IO bound issues are caused by things such as moving data from your hard drive into memory in some inefficient fashion. This is opposed to things that are CPU bound where your time is spent waiting for the CPU to finish processing tasks.

In general we recommend that users start out by using the sampling mode of the profiler as it is lighter weight and collects smaller data files. One exception to this rule is IO bound applications. For IO bound issues sampling on CPU cycles (take a snapshot of program execution every X cycles) is not a good was to investigate. Since program execution is blocked on waiting for IO operations to complete you will end up seeing very high sample numbers in wait functions.

On the other hand, instrumentation mode will help to point you at the exact function that is taking all of the time as opposed to the wait function. It will help to keep you from being mislead by the time that is spent in those wait functions. Right now I’m working at cooking up some examples of IO Bound and Memory Bound applications. So as soon as those are up I’ll post some walkthroughs of profiling them with the Visual Studio Team System Profiler.

Using the new Connect community feedback system for Visual Studio Team System

ianhu — Wed, 14 Jun 2006 20:45:00 GMT

For my team I’m the point man on tracking all of the customer feedback bugs that we get submitted from external users. This is one of the best parts of my job as I really enjoy interacting with customers and getting to keep my finger on the pulse of what issues are bugging our users the most. Having a deeply involved community of users that will call out any flaws in your software is not just a perk; it’s a necessity to ship high quality software in this day and age.

Previously I’ve blogged about the ladybug system that customers use to submit feedback issues to us. While Ladybug worked ok for us it had some issues with how it interfaced with customers and with our work item database. So for our Orcas release we’ve switched over to a new Connect system to use for reporting customer issues. Connect is a nice system and we hope that it will be easy for customers to use and easy for them to get issues filed against the correct team (this was a big issue with the ladybug system).

Say that you are working with the profiler in Visual Studio Team System for Developers when you think up a suggestion for a new visualization that you’d like to be able to use to view performance data. What are the steps that you would take to get a new suggestion filed in Connect? Note that many of these steps will only need to be done the first time that you open a Connect issue.

1. Navigate to the main Connect site

2. In the left hand navigation menu select Available Connections

3. “Visual Studio and .NET Framework” should be in the available connection list, click on it

4. Agree to the terms of service on the next page

5. Fill out the registration information on the next page

6. After registering, read the introduction on the welcome page and click the “Feedback” link to get to the actual feedback page

Now that you are registered you can just bookmark the feedback page so that you can get to it easily when you have more suggestions to add or issue to report. So how do we go about opening our new suggestion from the feedback page?

1. When submitting a new issue or suggestion the place that you will want to start is the big “Submit Feedback” button at the top of this page.

2. The first step in the submission process will force you to search to see if your issue or suggestion has already been files.

3. Please, before you submit your feedback use the search box with a few different keywords to see if your particular suggestion or issue is already registered with Connect. If it is already registered then please just add a vote for it as opposed to opening up a new feedback issue. That way the developers here at Microsoft can spend less time resolving issues as duplicates and more time actually fixing customer feedback issues ;-)

4. So you’ve done a few searches and you don’t see anything like the issue or suggestion that you want to raise. To proceed just click the “Submit Feedback” button located at the bottom of the search results.

5. Next, pick if you are submitting a bug or a suggestion

6. Now we are finally ready to submit our bug or suggestion. On the submission page please try to fill out as much information as you possibly can (attaching files or screenshots is nice as well). We try hard to work on all the issues that come in, but the better that you explain your suggestion or scenario the easier it is for us to figure out what you want, repro the issue and get back to you with a fix or a workaround. Putting non-specific statements like “Actual Results: It doesn’t work” and “Expected Results: It should work” don’t really help us at all. Try to be specific about exactly what you are seeing, about what is wrong about what you are seeing and about what would be the ideal fix for the issue. Remember that the more specific detailed information you can give us the better the chance of us being able to understand your issue and get it fixed.

7. Also, put some thought into what Category and Subcategory you will place your issue into. Like with giving us more detailed information placing the bug in the right spot will help us to not have to bounce the bug around in our work item tracking software (go TFS!) and we will be able to get back to you quicker.

8. After submitting your bug you will be able to track it from the “My Participation” tab on the main connect website. Not only will Microsoft employees be able to give you feedback and ask you questions but other Connect customers will be able to vote on the issue or suggestion and help you with possible solutions.

So that’s my brief little introduction to using the Connect system. Remember that the more you participate with Connect the better future releases of Visual Studio will be. So if you use Visual Studio you should also be using the Connect system to help get the features and fixes that you want.

Visual Studio Team System 2005 Code Coverage Tool FAQ

ianhu — Wed, 19 Apr 2006 21:33:00 GMT

This FAQ will collect the most common questions that are asked about the code coverage tool that is included with Visual Studio Team System for Developers (and Team Suite edition).

Q: Why am I seeing some uncovered blocks in my program when I’m sure that my test should be exercising 100% of my code?

A: The simple answer is that while you may be sure that you are testing 100% of the visible lines of code there are a few small cases where there are underlying IL blocks that are not being tested. It’s important to know that (in all the cases that we’ve found so far) the code coverage tool is not lying to you; there are code coverage blocks that were not covered by the test. Usually, this is cause by some extra lines of IL that the CLR adds in under the covers.

A few of the times that we have already seen these problems are with foreach loops and with switch statements that check six or more string values. If you see any more examples of this behavior please report them on the forum.

Q: I’m seeing partially covered lines in a statement with logical operators (||, &&, ect) what does a partially covered line mean?

A: The code coverage tool operates on blocks as opposed to lines of code. For example, say that you are testing the following statement:

if(A || B)

If you run a test with code coverage with the statement A evaluating to true you will see the above line as a partially covered line. Since A was true the B block was not evaluated so it was not covered by code coverage. Since A was covered but B was not this shows up as a partially covered line. To get full coverage you will have to modify you test to make sure that the B statement is also covered.

Q: I already have some unit tests for my project. How to I collect code coverage information from them?

A: Navigate to the code coverage tab in your current test run config. In that page, select all of the binaries that you want to collect code coverage from. Note that for any signed binaries you will need to click the re-sign button to resign them after they are instrumented for code coverage. Then just run your tests as normal. After a test run just right click on the test run results and select “Code Coverage Results” to view the code coverage data. For a more detailed look at this process take a look at the following TechNote:

http://msdn.microsoft.com/vstudio/teamsystem/reference/technotes/code_coverage/basics_ide.aspx

Q: How do I merge the results from multiple code coverage runs?

A: In the code coverage results window is a small button with a green plus sign placed between two white boxes. When you click this button you will see the results from the last group of test runs. To join code coverage information, just select all the runs you want to join and click ok to create a set of merged results.

Q: How can I collect and manipulate code coverage data programmatically?

A: We know that many users will be doing code coverage in some sort of custom build scenario and won’t be able to use the excellent IDE integration, so we provided tools to collect and manipulate code coverage data programmatically. This topic is a little too broad for a FAQ entry and is best covered by Joc’s excellent article on the topic, linked below:

http://blogs.msdn.com/ms_joc/articles/406608.aspx

Q: Can I collect code coverage data from web tests?

A: At this time we do not support collecting code coverage information from web tests. You can collect code coverage information from unit tests and other test types for which you have symbols and where you can select what binaries to instrument (via the test run config file).

Q: What are the basic steps for using the command line tools to collect code coverage data?

A: You will need to use vsinstr and vsperfcmd tools that are found in the Team Tools/Performance Tools directory under your Visual Studio installation directory. For the binary that you are going to test run the following vsinstr command line to instrument the binary for code coverage:

>vsinstr –coverage MyAssembly.exe

Then you need to start up the monitor to collect the data. You do this with the vsperfcmd tool:

>start vsperfmon –coverage –output:mytestrun.coverage

Now run your tests or exercise the code that you want to collect code coverage data for. When your program finishes, you will have a .coverage file that you can open in the IDE for analysis.

Advanced instrumenting with the VSTS profiler 101: /start

ianhu — Tue, 07 Feb 2006 21:59:00 GMT

When you are working with the Visual Studio profiler in instrumentation mode you often want to be able to limit the amount of data that you are seeing. I posted an earlier TechNote (from an Angry Richard blog article) that described (in 40,000 foot terms) some of the various functions in the Microsoft.VisualStudio.Profiler namespace that you can use to limit data captured during instrumentation profiling. This blog series on advanced instrumentation will look at some of the more interesting ways to use the instrumentation API (in MS.VS.profiler) and the command line instrumentation options to help prune your trace data.

As I’ve shown in my earlier TechNote, it can be very easy to trim your analysis to just a specific function by using the Microsoft.VisualStudio.Profiler.DataCollection classes. But what if I’ve already built my binary and I don’t want to have to re-compile it? Luckily we have command line instrumentation options for just this type of scenario. To just instrument the function A(), we’ll use the /start option for the VSInstr tool. The /start option will allow us to instrument a specific function with a start / stop profiling pair at its entry and exit points. But before we can use /start we need to find the name of the function that we are interested in. To see the function names in an assembly in a format that VSInstr accepts just use the /dumpfuncs command on vsinstr.

>VSInstr /dumpfuncs ConsoleApplication1.exe

Microsoft (R) VSInstr Post-Link Instrumentation 8.0.60117 x86 (Debug)

Functions contained in C:\Test\ManInstr\ConsoleApplication1.exe:

NOTE: No instrumentation will be performed

ConsoleApplication1.Program::.ctor

ConsoleApplication1.Program::DontSeeMe

ConsoleApplication1.Program::FirstFunction

ConsoleApplication1.Program::Main

ConsoleApplication1.Program::SecondFunction

ConsoleApplication1.Program::SeeMe

For this example I wanted to just instrument the function FirstFunction. This function calls SeeMe a bunch of times, while Main and SecondFunction both call DontSeeMe. So in our results we will know if we are getting correct information if we only see FirstFunction and calls to SeeMe. Also, I need to specify the control level of the start profiling and stop profiling commands that I will be instrumenting into the code. Since I’m running this application all by itself I’m setting them up to be global commands with the /control option. Now we will actually instrument ConsoleApplication1 with the /start option (if you are wondering, the output path command switch that I used just places the instrumented binary in a separate folder):

>VSInstr /outputpath:C:\Test\ManInstr\instr /control:global /start:inside,ConsoleApplication1.Program::FirstFunction ConsoleApplication1.exe

Now we need to start up our monitor and actually run our profiling scenario on the instrumented ConsoleApplication1.exe. Also, after we start up the monitor we will disable global profiling. This is because we automatically start up profiling when we start profiling so we would be collecting data from the beginning of the run up until the start / stop profiling pair instrumented into FirstFunction. Since we don’t want to collect all the information up until FirstFunction is called, we need to shut off profiling globally before we run our scenario.

>VSPerfClrEnv /traceon //enable trace profiling of managed apps

>VSPerfCmd /start:trace /output:FirstOnly.vsp
>VSPerfCmd /globaloff
>ConsoleApplication1.exe
>VSPerfCmd /shutdown

You now have a vsp file that you can open up in the IDE for analysis or analyze using the VSPerfReport command line tool. And this vsp only contains trace data captured inside FirstFunction or one of its subfunctions. Next entry I’ll take a look at using advanced instrumentation options to skip the first 20 times a function is executed in analysis.

Code coverage and the foreach loop

ianhu — Thu, 12 Jan 2006 20:21:00 GMT

In our support forums for Visual Studio Team System I’ve seen a large number of questions from customers who have seen inconsistencies from the code coverage tool when covering sections of code that containing a foreach loop. What customers are seeing is that they are getting partially covered blocks when they would expect all of the blocks to be totally covered.

What makes this situation so difficult is the fact that there *are* some blocks that are uncovered in specific foreach loop conditions. Under the surface a foreach loop is more complicated then a for loop or a while loop. For a foreach loop the CLR creates some code that may not be exercised by the loop under specific conditions.

The long and short of this is that when you are testing code coverage with foreach loops, you may see some uncovered or partially covered blocks where you would not expect them. Don’t worry, the coverage tool is not lying to you; there really are uncovered blocks in your code. Unfortunately there is no real workaround for this, as the blocks simply are not being covered and will be reported as such by the code coverage tool. I’m going to research this issue a little more and I’ll report back here if I can find out more about what situations with the foreach loop cause these uncovered blocks.

Performance Profiling Parse vs. TryParse vs. ConvertTo

ianhu — Tue, 20 Dec 2005 02:55:00 GMT

When programming in .Net you are presented with several different ways to extract a numerical value (I’m using an Int32 for my example) from a string. Recently I was looking at the differences between Parse, TryParse and ConvertTo. So I figured that I would use the new performance profiler included with Visual Studio Team System 2005 to figure out the performance differences between these three functions.

The first of these functions, Parse, is one that should be familiar to any .Net developer. This function will take a string and attempt to extract an integer out of it and then return the integer. If it runs into something that it can’t parse then it throws a FormatException or if the number is too large an OverflowException. Also, it can throw an ArgumentException if you pass it a null value.

TryParse is a new addition to the new .Net 2.0 framework that addresses some issues with the original Parse function. The main difference is that exception handling is very slow, so if TryParse is unable to parse the string it does not throw an exception like Parse does. Instead, it returns a Boolean indicating if it was able to successfully parse a number. So you have to pass into TryParse both the string to be parsed and an Int32 out parameter to fill in. We will use the profiler to examine the speed difference between TryParse and Parse in both cases where the string can be correctly parsed and in cases where the string cannot be correctly parsed.

The Convert class contains a series of functions to convert one base class into another. I believe that Convert.ToInt32(string) just checks for a null string (if the string is null it returns zero unlike the Parse) then just calls Int32.Parse(string). I’ll use the profiler to confirm this and to see if using Convert as opposed to Parse has any real effect on performance.

For a CPU bound application like this I’ll be using the sampling mode of the profiler. Sampling mode takes a snapshot of program state in some periodic cycle (I’ll be using clock cycles as my metric) and has a much lower overhead then instrumentation mode for a test like this. The three main test functions that I will be using are as follows:

private List<Int32> TestParse(String[] strings)

{

Int32 intValue;

List<Int32> intList = new List<int>();

for (int i = 0; i < 5000000; i++)

{

intList.Clear();

foreach (String str in strings)

{

try

{

intValue = Int32.Parse(str);

intList.Add(intValue);

}

catch (System.ArgumentException)

{ }

catch (System.FormatException)

{ }

catch (System.OverflowException)

{ }

}

return intList;

}

private List<Int32> TestTryParse(String[] strings)

{

Int32 intValue;

List<Int32> intList = new List<int>();

Boolean ret;

for (int i = 0; i < 5000000; i++)

{

intList.Clear();

foreach (String str in strings)

{

ret = Int32.TryParse(str, out intValue);

if (ret)

{

intList.Add(intValue);

}

return intList;

}

private List<Int32> TestConvert(String[] strings)

{

Int32 intValue;

List<Int32> intList = new List<int>();

for (int i = 0; i < 5000000; i++)

{

intList.Clear();

foreach (String str in strings)

{

try

{

intValue = Convert.ToInt32(str);

intList.Add(intValue);

}

catch (System.FormatException)

{ }

catch (System.OverflowException)

{ }

}

return intList;

}

To test the performance when good strings are passed in I used the following set of strings: { "123", "4567", "7890", "1", "1231280", "10" }. Show below is calltree view with the Main program function set as the root and with just the nodes below it expanded. Looking at this view we can see that about equal time was spent in all three of the conversion functions.

On the next screen, I’ve expanded some of the callstacks below the TestConvert, TestParse and TestTryParse functions. As you might have guessed, the callstacks down here are very similar, especially for TestConvert and TestParse. The only real difference between those two is that TestConvert is making a call toThread.get_CurrentCulture to check for a null string before it calls down to Number.ParseInt32. TestTryParse has the same basic structure as TestParse, except it calls Number.TryParseInt32 instead of Number.ParseInt32. But all three functions end up at the same System.Number.ParseNumber where they all spend about the same amount of time (Convert spends a little less but I do not think this is enough to be statistically significant).

So if all of these functions take about the same time with good data, how about if we make them parse bad data? The second set of data that I will use will have some oddballs like invalid characters, nulls, overflows and empty strings along with a couple of valid strings to parse. Here is the string set: { "12345", null, "123", "1324dfs", "51235", String.Empty, "43", "4123412341234123412341234123412341234123" }. I did have to reduce the number of iterations in each loop down from 5000000 to 50000 (that’s two zeros removed if you don’t want to count) in order to get approximately the same number of samples. Shown below you can see the huge difference between TryParse, Parse and ConvertTo when you are using bad data.

So while TryParse is only taking .5% of execution time Parse is taking 18% while Convert is taking 14%. The difference is, as we guessed, in the exception handling code. That is why Convert was faster then Parse as Convert handles the null string in the bad data set without throwing an exception. Shown below you can see where all the execution time is going to inside of the TestParse function. Most of the time is spent in the constructor for ArgumentNullException (in red) and in looking up the resource string for the exception (in blue).

So hopefully this little walkthrough has convinced you that if you plan on parsing a large number of strings you should use the new TryParse function instead of the old Parse or ConvertTo functions. Exceptions are meant to be “exceptions to the rule” not something that you want to be happening often as it can kill your program’s performance. So if you expect a lot of strings that you cannot parse make sure to use TryParse.

Visual Studio Team System TechNotes

ianhu — Tue, 06 Dec 2005 03:22:00 GMT

Looking for some good information on the Visual Studio Team System profiler or just on Team System topics in general? The kind folks at MSDN have opened up a new section of TechNotes to aggregate lots of quick-hitting articles on various Team System components. TechNotes are a little less polished and shorter then a typical MSDN article but are a fair bit spiffier then a typical blog post. I’ve adopted fifteen blog posts from my own blog, a few from other team members and written an original TechNote or two for the site.

Currently if you look under the development section all the TechNotes there (1200-1214) were written or adapted by me and have to do with either the profiler or with the code coverage tool. If you already follow many of the profiler blogs then many of the entries will be familiar to you, but hopefully this site will give you a better one-stop shop for your profiler and Team System info.

http://msdn.microsoft.com/vstudio/teamsystem/reference/technotes/default.aspx

~Ian