Everything you want to know about Visual Studio ALM and Farming
Brian Harry is a Microsoft Technical Fellow working as the Product Unit Manager for Team Foundation Server. Learn more about Brian.
More videos »
I got some data on some of the changes the languages team has made from Scott Wiltamuth (PUM of the languages team).
The first change is in the message dispatching priority that we do in keystroke processing. As a user is typing code that involves statement completion UI, we have a choice to make about the relative priority of (a) processing the keystrokes and (b) displaying and rendering the statement completion window. The VS 2008 behavior was to delay the processing of IntelliSense presenters until the WM_KEYDOWN queue is empty. As we transitioned the editor to WPF, the relative priorities changed. Changing them back to something closer to the VS 2008 behavior made a huge difference. The new code is show below, with the change highlighted (in case you can’t see the highlight, it’s “DispatcherPriority.Background”):
public void QueueSpaceReservationStackRefresh() { if (Interlocked.CompareExchange(ref _queuedSpaceReservationStackRefresh, 1, 0) == 0) { this.Dispatcher.BeginInvoke(DispatcherPriority.Background, new DispatcherOperationCallback(delegate { Interlocked.Exchange(ref _queuedSpaceReservationStackRefresh, 0); if (!_isClosed) { _spaceReservationStack.Refresh(); } return null; }), null); } }
public void QueueSpaceReservationStackRefresh()
{
if (Interlocked.CompareExchange(ref _queuedSpaceReservationStackRefresh, 1, 0) == 0)
this.Dispatcher.BeginInvoke(DispatcherPriority.Background, new DispatcherOperationCallback(delegate
Interlocked.Exchange(ref _queuedSpaceReservationStackRefresh, 0);
if (!_isClosed)
_spaceReservationStack.Refresh();
}
return null;
}), null);
This one change generated huge wins on some of the typing tests that we use – between 39% and 55% from one build to the next!
The second change was to re-use rather than re-create the statement completion window, so that when you type o.Foo.Bar, we reuse the statement completion window that displayed “Foo” for the statement completion of “Bar”.
Combined, these performance wins will make a big difference, especially for fast typists who have slow graphics hardware or are using virtualization. Here are the % changes for some of our C# typing scenarios:
PU
Scenario
B2Rel.21006 (Run 22042)
vs_langs.21201 (Run 23531)
% Change
VSL C#
Typing - Add classes at end of Large File
71,961.75
47,747.25
33.65%
Typing - After Top Level Edit
117,091.50
54,787.00
53.21%
Typing - In a Deeply Nested Object Initializer
90,684.00
30,404.75
66.47%
Typing - In a Large Method
118,181.50
55,767.25
52.81%
Typing - Large Collection Initializers
170,896.75
83,959.00
50.87%
Typing - Query Expressions
117,562.25
32,993.50
71.94%
Brian
Really cool. Thanks for sharing.
Good work!
Positive percentages for a negative change are confusing. It is more illuminating to say that, for the last item, the performance is 3.74x (or 374%) of the old version, rather that a 72% change. It really is a 72% decrease, but that is also confusing.
Don't sell yourself short. The perf went up almost 400%, don't show it as 72%!
http://www.zedshaw.com/essays/programmer_stats.html
Are those averages/medians, cherry picked single runs or what? And if so, how about distribution/deviation?
If skim through the above link you'll gain understanding why presenting such figures amounts to not very much and is borderline "get the facts" level of marketing. And why it's very important to always present more data than just the average/median when dealing with multiple measurements.
I believe those are averages over several runs. You are correct that a proper statistical analysis is required for rigorous understanding of something. We do look at standard deviations and statistical distribution, but it wasn't my intent here to provide that level of detail.
In the end almost all performance measurement is, at best, a rough indicator. Forgetting the math behind it, there are way too many other variable - hardware, OS versions, drivers, load on the system, size and complexity of project and more.
My intent with these posts is to show before and after to show that we are making progress. In the end, if you really want to know how it performs, get a build and try it. Then you can see how it works in your environment, on your apps.
I look forward to the release of VS2010 - it will bring us one tiny step closer to the functionality of Jetbrain's Java IDE, which has been streets ahead for years now. As much as I'm enjoying my foray into the .NET world, I do miss the IDE functionality that I've become used to.
PingBack from http://techrrr.blogspot.com/2010/03/last-comment-on-editorintellisense.html
IntelliSense improvements are not useless. I use http://searchizz.com for free downloading of all updates.
- it will bring us one tiny step closer to the functionality http://www.legendarydevils.com of Jetbrain's Java IDE, which has been streets ahead for years now. As much as I'm enjoying my foray into the .NET world, I do miss the IDE functionality that I've become used to.