Rico Mariani's Performance Tidbits : performance

Visual Studio 2010 Performance Part 3: A Technical Update

ricom — Wed, 19 Aug 2009 20:14:00 GMT

If I have any readers left out there, they might be interested to know what it is I’m up to this month. It’s a very important something. It’s... my vacation! Ok, seriously, I am in fact on vacation and I’ll be out for the next few weeks mostly because I’ve been working really hard on VS performance for many months now and I’m kinda tired. But, you still deserve a brief note from this tired camper, which I’ve been meaning to write for a while but something always came up – sorry about that :)

So, from my vacation, here’s a quick note.

This is, more than usual, one of those postings where I wish I could be giving you a lot more detail but you’re just going to have to wait for that next beta for things to be fully revealed. But I can say a few things, especially about technical issues, without getting into trouble. Really, if you want to know about what features need(ed) work, the easiest way to guess is to look at what people were complaining about. It turns out they weren’t imagining things – go figure.

Technically, we found all sorts of fun issues. For instance, the product is a lot more multi-threaded than previous versions and in some cases synchronization between threads was less than great. In some cases code designed to defer background work to keep the foreground responsive was actually slowing things down; in two cases I actually took out Sleeps, no joke! In some cases threads where synchronizing when they did not need to do so, destroying effective parallelism. In other cases latent parallelism designed into the system wasn’t exploited at all, and rounding things out there were cases where there should have been parallelism in the design and it was totally absent. That’s pretty much the full menu of problems you could have in that space.

In the past, the IDE was just “flirting” with becoming a managed application, for some scenarios you could actually avoid the managed code entirely. In Visual Studio 2010 you need only look at the profiles and observe how much time we spend in clr.dll to see that at this point Visual Studio is dominantly managed. That brings a lot of benefits but also potential problems. In many cases our first cut at communication between the managed and unmanaged pieces was pretty much terrible – all kinds of redundant copying of data, predominantly text, resulted in a lot of wasted CPU cycles and memory. Many internal interfaces were redesigned to better cooperate between layers. COM interop cases sometimes made this even trickier with CCW/RCW objects being created destroyed far too aggressively. In some cases moving a nicer chunk of the work to the managed side helped a lot – kind of like writing stored procs for your database – good procedures mean fewer round trips and less data to move.

Recently, we started taking a hard look at overall consumption and a lot of improvements are coming out of that work. Memory is of course the main issue and there are three main sources of bad consumption that are getting targeted:

Managed images, especially ngen’d images. It’s just super, super, simple to take a dependency on a whole DLL in the managed world when all you need is a tiny slice of it. This has the lovely property that others then build on those DLLs creating a great big tree of unneededness. The good news is that examination can often lead to good refactoring opportunities which can save everyone a lot of memory. This is for sure the #1 issue.
Unshared, unmanaged Heaps. There are many low level pieces that manage their own memory and they inevitably do it with CreateHeap – joy. Of course they make their own nice giant heap ready to go and then put about 22 bytes in it. Multiply that by all the components that think they need their own private heap and it gets unpleasant awfully quick. Thankfully, this, too, is somewhat addressable. Interestingly managed memory use hasn't been a big problem other than it's fair share of leaks (see next)
Memory leaks, pretty much universally, can be a problem in a product the size of VS. No technology is immune but I sure like finding the managed ones better than the unmanaged – they literally can’t hide. Other things that look like leaks but aren’t really – it’s important to use AddMemoryPressure/RemoveMemoryPressure to give the GC a chance to help you with your handle objects. VS has a lot of those.

I think you’re going to be pleased because we’ve been hitting most of the complaints squarely on the head. From my chair, the product feels much, much, better.

That’s it for now, I can’t wait to get another build into your hands!

Visual Studio: Why is there no 64 bit version? (yet)

ricom — Thu, 11 Jun 2009 06:34:00 GMT

Disclaimer: This is yet another of my trademarked "approximately correct" discussions

From time to time customers or partners ask me about our plans to create a 64 bit version of Visual Studio. When is it coming? Why aren’t we making it a priority? Haven’t we noticed that 64 bit PC’s are very popular? Things like that. We just had an internal discussion about “the 64 bit issue” and so I thought I would elaborate a bit on that discussion for the blog-o-sphere.

So why not 64 bit right away?

Well, there are several concerns with such an endeavor.

First, from a performance perspective the pointers get larger, so data structures get larger, and the processor cache stays the same size. That basically results in a raw speed hit (your mileage may vary). So you start in a hole and you have to dig yourself out of that hole by using the extra memory above 4G to your advantage. In Visual Studio this can happen in some large solutions but I think a preferable thing to do is to just use less memory in the first place. Many of VS’s algorithms are amenable to this. Here’s an old article that discusses the performance issues at some length: http://blogs.msdn.com/joshwil/archive/2006/07/18/670090.aspx

Secondly, from a cost perspective, probably the shortest path to porting Visual Studio to 64 bit is to port most of it to managed code incrementally and then port the rest. The cost of a full port of that much native code is going to be quite high and of course all known extensions would break and we’d basically have to create a 64 bit ecosystem pretty much like you do for drivers. Ouch.

[Clarification 6/11/09: The issue is this: If all you wanted to do was move the code to 64 bit then yes the shortest path is to do a direct port. But that’s never the case. In practice porting has an opportunity cost, it competes with other desires. So what happens is more like this: you get teams that have C++ code written for 32 bits and they say “I want to write feature X, if I port to managed I can do feature X plus other things more easily, that seems like a good investment” so they go to managed code for other reasons. But now they also have a path to 64 bit. What’s happening in practice is that more and more of the Visual Studio is becoming managed for reasons unrelated to bitness. Hence a sort of net-hybrid porting strategy over time.]

So, all things considered, my feeling is that the best place to run VS for this generation is in the 32 bit emulation mode of a 64 bit operating system; this doubles your available address space without taking the data-space hit and it gives you extra benefits associated with that 64 bit OS. More on those benefits later.

Having said that, I know there are customers that would benefit from a 64 bit version but I actually think that amount of effort would be better spent in reducing the memory footprint of the IDE’s existing structures rather than doing a port. There are many tradeoffs here and the opportunity cost of the port is high.

Is it expensive because the code is old and of poor quality?

It’s not really about the quality of the code – a lot of it is only a few releases old – as it is the amount of code involved. Visual Studio is huge and most of its packages wouldn’t benefit from 64 bit addressing but nearly all of it would benefit from using more lazy algorithms – the tendency to load too much about the current solution is a general problem which results in slowness even when there is enough memory to do the necessary work. Adding more memory to facilitate doing even more work that we shouldn’t be doing in the first place tends to incent the wrong behavior. I want to load less, not more.

Doesn’t being a 64 bit application save you all kinds of page faults and so forth?

A 64 bit address space for the process isn’t going to help you with page faults except in maybe indirect ways, and it will definitely hurt you in direct ways because your data is bigger. In contrast a 64 bit operating system could help you a lot! If you’re running as a 32 bit app on a 64 bit OS then you get all of the 4G address space and all of that could be backed by physical memory (if you have the RAM) even without you using 64 bit pointers yourself. You’ll see potentially huge improvements related to the size of the disk cache (not in your address space) and the fact that your working set won’t need to be eroded in favor of other processes as much. Transient components and data (like C++ compilers and their big .pch files) stay cached in physical memory, but not in your address space. 32 bit processes accrue all these benefits just as surely as 64 bit ones.

In fact, the only direct benefit you get from having more address space for your process is that you can allocate more total memory, but if we’re talking about scenarios that already fit in 4G then making the pointers bigger could cause them to not fit and certainly will make them take more memory, never less. If you don’t have abundant memory that growth might make you page, and even if you do have the memory it will certainly make you miss the cache more often. Remember the cache size does not grow in 64 bit mode but your data structures do. Where you might get savings is if the bigger address space allowed you to have less fragmentation and more sharing. But Vista+ auto-relocates images efficiently anyway for other reasons so this is less of a win. You might also get benefits if the 64 bit instruction set is especially good for your application (e.g. if you do a ton of 64 bit math)

So, the only way you’re going to see serious benefits is if you have scenarios that simply will not fit into 4G at all. But, in Visual Studio anyway, when we don’t fit into 4G of memory I have never once found myself thinking “wow, System X needs more address space” I always think “wow, System X needs to go on a diet.”

Your mileage may vary and you can of course imagine certain VS packages (such as a hypothetical data analytics debugging system) that might require staggering amounts of memory but those should be handled as special cases. And it is possible for us to do a hybrid plan with including some 64 bit slave processes.

I do think we might seem less cool because we’re 32 bit only but I think the right way to fight that battle is with good information, and a great product.

Then why did Office make the decision to go 64 bit?

This section is entirely recreational speculation because I didn’t ask them (though frankly I should). But I think I can guess why. Maybe a kind reader can tell me how wrong I am :)

First, some of the hardest porting issues aren’t about getting the code to run properly but are about making sure that the file formats the new code generates remain compatible with previous (and future) versions of those formats. Remember, the ported code now thinks it has 64 bit offsets in some data structures. That compatibility could be expensive to achieve because these things find their way into subtle places – potentially any binary file format could have pointer-size issues. However, Office already did a pass on all its file formats to standardize them on compressed XML, so they cannot possibly have embedded pointers anymore. That’s a nice cost saver on the road to 64 bit products.

Secondly, on the benefit side, there are customers out there that would love to load enormous datasets into Excel or Access and process them interactively. Now in Visual Studio I can look you in the face and say “even if your solution has more than 4G of files I shouldn’t have to load it all for you to build and refactor it” but that’s a much harder argument to make for say Excel.

In Visual Studio if you needed to do a new feature like debugging of a giant analytics system that used a lot of memory I would say “make that analytics debugging package 64 bit, the rest can stay the way they are” but porting say half of Excel to 64 bits isn’t exactly practical.

So the Office folks have different motivations and costs and therefore came to different conclusions -- the above are just my personal uninformed guesses as to why that might be the case.

One thing is for sure though: I definitely think that the benefits of the 64 bit operating system are huge for everyone. Even if it was nothing more than using all that extra memory as a giant disk cache, just that can be fabulous, and you get a lot more than that!

Visual Studio 2010 Performance Part 2: Text Editor

ricom — Sat, 06 Jun 2009 06:04:00 GMT

Part 1 of this series talked about the startup problems we face. In Part 2, I want to talk about the editor.

Many people have reported that editing with the new editor is slower. I’ve experienced the same thing myself so I certainly do not want to accuse people of hallucinating but I thought it might be interesting to understand why that might happen, especially since this new editor is supposed to be better than the old.

Why is this editor better and how is it better?

I’m glad you asked :)

It’s actually better in pretty much every editing way. The data structures associated with this new editor do not require complex locking algorithms for access, it delivers logical “micro-snapshots” of the editor buffer which do not change – it is a copy of the buffer at an instant in time. This is a fantastic situation if you are, for instance, a background compiler. Previously to get consistent snapshots the entire buffer had to be copied, potentially on every keystroke!

And if that’s not enough, there’s a lot more. Another big tax on the editor is region management. Regions are used to track everything from bookmarks to breakpoints – and more. Those squiggles you see, those are all regions and there can be thousands of them. That is far beyond what the original editor design intended and those algorithms were starting to degenerate to quadratic behavior. You can see that yourself by noting that simply scrolling around at the end of large files in the old editor tended to get slower.

Those are just two areas, but generally where the old editor had quadratic performance we were able to improve things to linear and in some places we were able to do even less work by being more careful about processing only visible segments of the text buffer. All goodness.

So why is the thing slower if it’s so much better?

Well let me elaborate a little more, but before I answer that directly, first let me tell you what things are almost certainly not the problem.

Not the problem #1: WPF

With the exception of some cases where we found that remoting WPF primitives over terminal server connections was slow, generally the WPF system is more than capable of keeping up with the editor. If you are seeing sluggish typing WPF is almost certainly not to blame.

Not the problem #2: Editor Primitive Operations.

The basic insert primitives are blazing fast, as low as a few microseconds even in very large files. If you’re seeing a problem, the chances that the text buffer is letting you down are slim to none.

Ok, so, if it’s not those things, what is it? There’s two good bets.

Compatibility Shims

The new editor is managed, and it has its own brand new managed interface. You can call it directly. However, there is lots of existing code that knows how to talk to the OLD editor and the old editor had a COM interface with its own particular API. We didn’t want to port all of the editor related code to the new editor’s managed interfaces – for several reasons but perhaps the most important is that there’s just so much of it. So we created a compatibility layer – shims – that allow old callers to use the new editor. That’s also important for our partners out there. That’s all fine and well but of course that means you’re going to be forced to use COM-interop where there was none before and of course those shims have to emulate the old editors behavior even when that wasn’t exactly the fastest choice in the west.

So sometimes improving editor responsiveness has meant converting more code to the direct interface and thereby avoiding the shims. The trick here is to make sure you convert the code that needs converting rather than going crazy trying to convert all of it in one go.

OK but who are all these users? What are we talking about here?

That brings us to the second point.

Event Listeners

Another reason why the editor can get bogged down is that various clients can register for notifications of interesting events. This is actually very popular (understatement) and important things like the language’s intellisense services want to be notified of various things going on in the buffer. Now it turns out that the actual editing primitives are so fast that the bulk of the cost of doing edits is actually in these various event processors. Sometimes because they are using shims, sometimes just because they are being over-aggressive in subscribing to things. Sometimes because they have their own personal problems that are only incidentally related to editing.

One way that you can see the difference in the root cause of editing yourself is to try your editing operations in a file named ‘foo.txt’ rather than ‘foo.cs’ or ‘foo.vb’ or whatever the case may be. This will disable many of the event listeners and give you a truer feel for what the editor itself is doing. Even that isn’t perfect because of course there are still listeners for bookmarks, and other kinds of things that are applicable to even plain text files.

What are we doing about it?

Well I think I’ve already alluded to it. Make better use of the underlying editor primitives. Move more things off the shims where the cost is high. Reduce the cost and number of listeners generally. And of course make sure that our text and region management primitives stay nice and fast.

What can you do?

If you see the editor behaving dumbly, enter bugs. They really help!

Visual Studio 2010 Performance Part 1: Startup

ricom — Sun, 31 May 2009 10:46:23 GMT

I want to start by thanking everyone that has commented on the Beta (by posting their thoughts here or elsewhere) for doing so. Please keep those comments coming! They have a great impact on the senior leadership here and they are excellent rallying points for all the teams. They really do make a difference!

Jason has come right out and said he’s tapped me to work basically full-time on performance work for Visual Studio until we’re done; that’s understating the situation if that’s possible. I think I had already tapped myself before he tapped me and at this point I’m a bit of a lightning rod for performance issues in the product. I think that’s a good thing. :)

Just like I did back when I was writing about performance problems for the CLR I’m going to talk about things as candidly as possible. Including the bad parts. While I do that I want you to keep in mind that I write the truth as I see it and I often have to write what I like to called “approximately correct” articles, mostly in the interest of brevity; being more than approximately correct causes the word count to skyrocket. I won’t keep reminding you that I’m doing this but it’s been a while so I thought maybe a refresher would be good.

Lastly, I try to use affirmative language about what I’m doing or what I think the teams are doing but I generally avoid making promises even when any idiot could see that we would be colossally stupid to do other than what I am discussing or perhaps what one of my readers has suggested. I avoid it because there are many teams involved, with many choices at hand, and many opportunities for things to go badly despite the best laid plans. So I just try not to go there. I’m sorry about that.

OK enough disclaimers already.

This is, what I hope is the first in a series of articles about performance related happenings in Visual Studio. I hope you find them helpful and interesting. I’m going to start at the start – and that’s Visual Studio startup.

I’ve actually talked about Startup several times over the last few years. That’s because I want people to understand that of all the things that are going to get better, startup is not one of them. Yup, that’s right; I gave up at the start. No kidding. No pretending. It’s going to start slower. Sorry.

But wait; don’t give up on me yet!

We decided to make some investments in better display technology in this release and for the future; that meant loading WPF at startup. We decided to have a superior extensibility model for our future and that meant MEF at startup. And in general we went from having no managed code at all at startup to having a LOT of managed code at startup. That stuff isn’t free, and nobody knows it better than me. [I rhymed]

In some cases I’ve seen as much as 75% of our time in current startup scenarios actually in clr.dll – incredible isn’t it? In VS2008 that was 0%! But even though that’s technically true, it’s also a bit of a lie…

In VS2008 you could start the IDE without bringing in the managed stack and you could avoid allocating say the GC heaps and whatnot but it’s an illusion. For the most part, if you did anything real you would find that you had paid that cost very shortly after startup and almost certainly before you were “Ready to Work.” That experience is the one that I wanted teams to focus on in this release: don’t prioritize start to empty, prioritize start to ready.

Now in start to ready we are in much better shape. There were many, what shall I call them, hmm, known weaknesses? There were known weaknesses in the startup path from initial boot, to opening your solution, to opening an editor. I’ve talked about many of these in the past. Consumption in the project system and language systems was high. Editor related algorithms were costly, quadratic complexity or worse in some important cases. That means there is opportunity there. So we keep fighting to create an overall experience that you’ll find better, one that will make you more productive, even though we’re giving ground on some parts of the experience.

Now if you think we’re going to open a small foo.txt file as fast as say notepad… well… I’m sorry, we’re not going to do that. We never had a chance. Other editors are making other kinds of tradeoffs and are much better positioned to give a great experience in that case. But if you want to open say a medium sized solution now we’ve got a fighting chance to offer real advantages.

Now maybe that sounds great, or maybe it doesn’t, but in any case the trouble with focusing on end-to-end productivity is that even if you do make the “on average” experience better you still run the risk of having some painful parts be worse than they used to be and those parts just totally ruin the experience for customers. We’re trying to be mindful of that. Take for instance our friend the Add References Dialog, I don’t think anyone is thinking about how “on average the product is faster” when they are waiting for that thing to come up. I’m sure not.

So this article is supposed to be about Startup, what are we doing to make it better? Well, there are some very specific things that we think are going to help. These ones represent maybe 25% of the total cost of startup (to empty) in the beta build.

We use WPF toolbars and the toolbar layout code in the beta is naïve. That needs to get better (I like what I see so far)
WPF does drawing in the background with a render thread; that thread sometimes has to synchronize with the main UI thread. Some of those look like they can be removed.
The way we create the toolbars is more complicated than it needs to be. We can get the exact same result using fewer UIElements with some style and code changes
We have a hot list of methods in the CLR that are important in our startup, we’re hoping some of those can be improved, and also that those improvements aren’t unique to Visual Studio
A classic thing that happens in every Visual Studio release is that we go through a period where we have bunches of code (often VS packages) that are getting loaded at startup that to not need to get loaded at startup. We’re making a list and checking it twice.

The best thing about most of these changes is that they actually help many scenarios (e.g. almost every time we change modes) not just startup.

These things will help Start to Empty to be as good as it can be. I’m going to talk about Start to Ready more in a later posting.

Visual Studio 10 -- Your Performance Feedback plus Beta 1

ricom — Wed, 20 May 2009 23:14:00 GMT

At Microsoft you can't say you're excited about anything you have to say that you're "super excited". I don't know why that's just the way it is. So, I'm happy to say that I'm super excited about the release of Beta 1 and I encourage you all to visit Jason's blog (again) to see screen-shots and get the download info. And more importantly to actually try it out and give us feedback. Your feedback so far has made a big difference and it will continue to do so, if only because it gives me ammunition with which to beat up -- err, I mean, educate -- executives here in DevDiv :)

Right now we're at a stage where it's crucial to lock in the gains we have and make sure everything we do is only making things better. That means detecting any performance regressions builds, day to day, is paramount. Now I have some general advice about that, I've been trying to teach for year that consumption metrics are very important and often a better way to track performance even though they aren't directly what your customers care about -- this old article discussed that a bit with regard to memory use. For regression detection I think it's even more important to focus on consumption rather than elapsed time. I was trying to explain this yesterday and I came up with this pithy quote:

"Trying to control your performance by measuring only elapsed time is kind of like trying to control your weight by measuring how much time you spend eating."

It's -- dare I say it -- super-important to use a variety of metrics and watch your overall consumption because too many extraneous, impossible to control, variables affect elapsed time run-to-run.

Now moving on to some feedback, I first wanted to thank everyone that wrote in response to my last posting and encourage you to keep giving feedback. There were a few common themes that I wanted to discuss in detail.

Add/Remove References Dialog

I've personally made a pitch to make this better and I expect to see it getting attention. At the very least I want the default tab changed to be a faster one. This dialog is universally despised and got more negative comments than any other feature.

Unit Tests, especially batch runs

That team is looking at larger run cases and trying to optimize the amount of data gathered versus the run speed. Hope to see some nice gains there.

General Typing speed (C#, VB, C++)

I've been looking at that scenario personally and with a lot of help from our languages team. There are many factors that affect speed, not the least of which is the language service that provides intellisense but there are other important factors like the raw speed of the new editor, the amount of outlining, etc.. We've made big improvements here since the PDC but we're still not happy with it so we're pushing for even more gains before we ship.

"Cat" commented that "I'd hazard that partial class support was added to C# just because of this problem." I can assure you that that was not the case. Partial classes were added for several reasons but I would say the main one was to allow better separation of designer generated code vs. human generated code.

Silverlight and WPF tooling

Some things are already better because WPF is loaded first but other things have suffered because there were many changes in that tool suite and it's still in need of performance work. I think you'll find Beta1 is better than the PDC but I expect you'll still find some areas wanting. Typing speed in XAML files was something that we were especially wanting to look at.

Help

I really wish we could be showing you more of this in the beta it was so close to being ready but the internal version just wasn't quite there. We've totally overhauled how it works, the core storage is a much simpler standards-based solution, the reader is much simplified, the first load should be vastly faster, installing new help and getting updates will be vastly faster. There's a bit of information on April's blog and in particular you can try out the lo-band version of online help (which I think is a lot closer to what our final look will be). I've been very involved in this effort and I'm especially looking forward to getting the new help technology into more people's hands.

Copy Local

Patrick brought up an opportunity to improve Copy Local based on his experience. I'm hoping we can move that solution to use NTFS hard links when they are available to provide that functionality without moving so many bits. I noticed the same activity in some of our test cases -- the problem is of course not unique to NUnit.

WPF Optimizations

Personally I'm focused on WPF changes that will help Visual Studio -- like for instance improvements in how Toolbars are arranged, and general synchronization between the UI thread and the rendering thread. Some of this work translates to gains that are helpful universally, some not so much. But I am encouraging the WPF team to read this posting and that should help too.

Solution/Project management

Lots of work is happening in this space. Customers sometimes have huge solutions and they can't afford to wait for us to load everything before they are allowed to start working. I'm actively driving improvements in that area. I wish I could promise more but, I'm not especially merciful, I can promise that :)

Specific Bugs

There were several bug reports already in the feedback, I will do my best to get the right people to look at this stuff. A lot of times all I can do is shrug -- it's a big product, I know a lot of it but, even after 20 years, I don't know it all.

I'm sorry if I didn't respond to your particular comment, I did read every word but this posting is already too long.

Visual Studio -- working on performance

ricom — Thu, 14 May 2009 20:46:00 GMT

Jason has a new posting on the progress of Visual Studio and I wanted to chime in myself. Some people have been wondering what I’ve been up to… I think you’ll be happy to hear that about 2 months ago I put down a bunch of my long term planning responsibilities so that I could work on Visual Studio performance directly. Woo hoo!

So performance work: what kind of work? Well the most important thing I’ve been doing is helping people in VS to understand where the problems are, and how to make directed improvements that get locked in. I’ve been doing lots of training, I’ve been creating custom analysis tools for studying VS performance problems, and I’ve also been yelling at a lot of people and just generally making all kinds of friends in my division. :)

During these last weeks we’ve made a lot of progress, I’m sure you’re going to feel the product is a lot snappier than the builds we provided at the PDC. But of course I’m never satisfied – there are even more wins coming later. I worked with many different teams to help us with our startup, with UI transitions, with memory usage, with threading issues – especially with how WPF and our main thread synchronize. This very morning I’m busy reviewing goals for every major group in Visual Studio and we’re working hard to create a great experience. I’m especially happy with the changes that will benefit many applications (like some of the ones that are finding their way into WPF, or interop)

So, especially for the next big push before release, performance will be a very high priority for Visual Studio – I’m going to be very pleasantly busy.

What does this all mean to you? My blog is now a great place to give your feedback, especially on the Beta when it comes out. Tell me what is hurting you the most, many people will be watching, including me, and we still have time to get attention on key problems. I'd love to hear about it.

Performance Advice, Southern Style

ricom — Sat, 29 Nov 2008 00:29:00 GMT

While I was at PDC2008 I was introduced to Keith and Woody -- pretty soon there was a microphone in front of me and we were doing a podcast. Now I already liked these guys but when they used a picture of me from about 1998 I really liked them a lot more. I wish I still looked like that :).

And the interview was pretty fun too. ;)

Keith, Woody, thanks for having me.

You guys can check it out here:

http://deepfriedbytes.com/podcast/episode-21-talking-performance-with-performance-preacher-rico-mariani/

Performance by Design at PDC2008!

ricom — Fri, 12 Sep 2008 22:48:58 GMT

I'm happy to announce that I will be at PDC2008 (and I will work the booths until I can't speak) but in particular I'll be kicking off this precon event with my longtime colleagues Mark, Joe, and Vance.

Come on out, I'd love to meet some of my readers!

Performance by Design: Using the .NET Framework

Presenters: Mark Friedman, Joe Hellerstein, Rico Mariani, Vance Morrison

The.NET Framework offers developers a dizzying array of design choices for building applications, whether they are client or server-side applications. These choices can greatly impact the performance and scalability of your application. This session highlights best practices in application architecture, testing, tools, and techniques for building responsive and highly scalable applications using the .NET Framework. More than just practical advice, this session focuses on proven techniques for building high quality applications. You will also learn what measurement data is available from the CLR and the OS and will understand how to leverage this data to optimize your application’s performance.

Is Performance a "functional" requirement?

ricom — Wed, 27 Aug 2008 19:57:00 GMT

We had an internal thread on this yesterday, maybe I was a little too glib but here is what I had to say about the topic [edited so it can be read standalone]

The original question, “Is performance a functional requirement” is highly unexciting for me. However the revised question, “What is the best way to capture performance requirements” is profoundly interesting. It is precisely the (anti-)correlations between attainable functionality and level of responsiveness that sometimes makes for the very best engineering work. How do you capture all this? Sometimes pervasively. I would argue that pervasively is in fact the only answer that works in non-trivial cases. And this is not limited to performance as the quality metric of interest – it applies to any quality that is not going to be trivially achieved.

The nomenclature to me is uninteresting except to the extent that it assists designers and implementers in capturing and understanding the various kinds of requirements – i.e. I know where to put them, I know where to find them, I know that I got them all, because I am, literally, on a first name basis with all the requirement types.

I don’t want to trivialize that aspect, but, beyond that, meh, whatever… functional schmuntional, I don’t care what you call it, just do it :)

Linq Compiled Queries Q & A

ricom — Mon, 25 Aug 2008 22:21:00 GMT

I did a series of postings on Linq Compiled Queries last year, I recently got some questions on those postings that I thought would be of general interest.

Q1:

Why use the 'new' keyword in this snippet?

var q = from o in nw.Orders
select new {o.everything …};

If you did just :

var q = from o in nw.Orders
select o;

You're getting editable orders. Linq then has to track them in case you change them and want to submit the changes. If you use new effectively you're making a copy of the orders that is not going to be change tracked. That's faster for read only cases. The other thing you can do is mark the query context as read-only and then you get the same effect. When I wrote that test case, that feature wasn't available yet so I used new to simulate it.

Q2:

What do you mean when you say that linq will 'Create custom methods that bind the data perfectly' ?

Whenever you use linq to sql to read data from a database it has to do two important things for you. The first is convert your Linq query into SQL. The second is to make a method that takes the stream of data that comes back from the database and converts it into the managed objects you required. That's the data-binding step. Linq creates the necessary methods automatically, and it makes the perfect code for doing this.

Q3:

How did Linq to SQL beat your ADO.Net code for insert times. Shouldn't a tie be the best possible result?

The SQL I used in my test case was pretty much the standard simplest SQL you would use for such a job. The automatically generated SQL from Linq was better than what I wrote by hand because they had parameterized the insert statements which I never bothered to do. Had I changed my SQL to what they created it would have been a tie. This is kind of like when the C++ compiler finds a machine code pattern that is better than what you would have written doing it by hand because it did something you don't usually bother doing with hand tuned machine code. But you *could* replace what you wrote with what the compiler generated.

Q4:

What are the downsides to precompiled queries?

There is no penalty to precompiling (see Quiz #13). The only way you might lose performance is if you precompile a zillion queries and then hardly use them at all -- you'd be wasting a lot of memory for no good reason.

But measure :)

Just another day in the perf lab

ricom — Thu, 19 Jun 2008 00:55:00 GMT

Even though I've been doing general architecture work on Visual Studio for nearly a year now, my friends in DDPERF are still plugging away on performance problems and finding some interesting results.

This most recent thread is very interesting because it shows yet another example of how the consequences of hardware changes can be subtle and very hard to predict.

Just today I was working with Cameron McColl again-- this time we were trying to understand why a particular benchmark was sometimes mysteriously slower than normal for no apparent reason. To our delight we found (well, mostly Cameron, I was the "consultant" <grin>) that the problem was in how the timing was being triggered and so the bulk of the variability seems to be measurement error and not actual test variability. But to our chagrin, the slower time seems to more accurately reflect reality. Well at least now we know.

How are these things related?

They remind us all that it's very important to track down anomalies in your reported results because otherwise you have little understanding of why you are making things faster, what works and what doesn't.

In the words of Alastor "Mad Eye" Moody: "Constant Vigilance!"

P.S. If you're looking for the further adventures of the devdiv perf team, you could do worse than subscribe to their blog.

Shutdown Is No Time For Spring Cleaning

ricom — Mon, 12 May 2008 22:56:00 GMT

I think my current performance pet peeve is shutdown. Assorted flavors of it, they all seem to have the same kind of problem. Sometimes we're stuck with it... but maybe we shouldn't be?

This is one time when our basic training, which normally I love so much, tends to let us down. We were all taught to clean up our own messes -- programming wise that means freeing your resources after you are done with them. But this backfires in the shutdown case.

Many times I watch as I hit the [X] close button on some application and my poor computer starts to swap as the application goes about paging in vast amounts of its code and then dutifully walking all its data structures-- more paging -- and giving them back to the operating system. My reaction to this in a word: ACK!!!!

When your application is ordered to shutdown the last thing you should do is enumerate every piece of memory you have ever allocated and systematically give them back to the operating system. Your program has a death sentence, and soon your resources are going back to the operating system whether you like it or not: what you must do is look at the minimum possible amount of memory necessary to get to a nice safe stable state and then exit as quickly as possible. Abandoning your memory like this gives the operating system the best chance to get your process unloaded while swapping in the least amount of memory and causing the least impact to the systems disk and memory caches.

In short, shutdown is no time for spring cleaning.

And why all this cleaning anyway? Many people report that they have all these important resources that need flushing and so forth. They couldn't possibly get to a safe state without considerable work but usually that in itself is symptomatic of assorted problems. Any application that has important data to manage almost certainly needs to be tolerant of power-failure and if that's the case when the user makes important edits they likely should be automatically saved to a durable location. In fact at any given moment, probably only a few seconds worth of data should be at risk. If your application has been idle for any length of time it should be fully saved -- and even if the user hasn't chosen to save their work it's still effectively saved somewhere so that you could restore in the current "dirty" state.

So if you have to do all this work to be resilient to power failures, then take advantage of that logic to simplify your shutdown paths. Your users will thank you.

Computer Measurement Group offers past papers to the public!

ricom — Tue, 12 Feb 2008 20:44:17 GMT

As you know I presented my performance signatures paper at the Computer Measurement Group's (CMG) conference a little over a year ago and I am happy to announce that CMG is now offering previous proceedings from 2005 and older to the public. I think this is a fabulous move by them because there is some first rate content here that just wasn't seeing nearly the light that it deserved to see.

I hope to provide more specific recommendations when I have had the chance to look at the site myself in more detail and of course

If you're looking for a way to grow as a performance professional this could be a fabulous resource for you. I thoroughly enjoyed the event and the presentations when I went.

See http://www.cmg.org/proceedings/

Performance Quiz #13 -- Linq to SQL compiled query cost -- solution

ricom — Mon, 14 Jan 2008 20:51:00 GMT

Well is there really a "solution" at all in general? This particular case I think I constrained enough that you can claim an answer but does it generalize? Let's look at what I got first, the raw results are pretty easy to understand.

The experiment I conducted was to run a fixed number of queries (5000 in this case) but to break them up so that the compiled query was reused a decreasing amount. The first run is the "best" 1 batch of 5000 selects all using the compiled query. Then 2 batches of 2500, and so on down to 5000 batches of 1. As a control I also run the uncompiled case at each step expecting of course that it makes no difference. Note the output indicates we selected a total of 25000 rows of data -- that is 5 per select as expected. Here are the raw results:

Testing 1 batches of 5000 selects
5000 selects uncompiled 9200.0ms 25000 records total 543.48 selects/sec
5000 selects compiled 5401.0ms 25000 records total 925.75 selects/sec

Testing 2 batches of 2500 selects
5000 selects uncompiled 9181.0ms 25000 records total 544.60 selects/sec
5000 selects compiled 5402.0ms 25000 records total 925.58 selects/sec

Testing 5 batches of 1000 selects
5000 selects uncompiled 9169.0ms 25000 records total 545.32 selects/sec
5000 selects compiled 5432.0ms 25000 records total 920.47 selects/sec

Testing 100 batches of 50 selects
5000 selects uncompiled 9184.0ms 25000 records total 544.43 selects/sec
5000 selects compiled 5511.0ms 25000 records total 907.28 selects/sec

Testing 1000 batches of 5 selects
5000 selects uncompiled 9166.0ms 25000 records total 545.49 selects/sec
5000 selects compiled 6526.0ms 25000 records total 766.17 selects/sec

Testing 2500 batches of 2 selects
5000 selects uncompiled 9165.0ms 25000 records total 545.55 selects/sec
5000 selects compiled 7892.0ms 25000 records total 633.55 selects/sec

Testing 5000 batches of 1 selects
5000 selects uncompiled 9157.0ms 25000 records total 546.03 selects/sec
5000 selects compiled 10825.0ms 25000 records total 461.89 selects/sec

And there you have it. Even at 2 uses the compiled query still wins but at 1 use it loses. In fact, the magic number for this particular query is about 1.5 average uses to break even. But why? And how might it change?

Well, as has been observed in the comments, Linq query compilation isn't like regular expression compilation. In fact compiling the query doesn't do anything that isn't going to have to happen anyway. In fact, actually creating the compiled query with Query.Compile hardly does anything at all, it's all deferred until the query is run just as it would have been had the query not been compiled. So what is the overhead? Why is it slower at all? And what's the point of it?

Well the main purpose of that compiled query object is to have an object, of the correct type, that also has the correct lifetime. The compiled query can live across DataContexts, in fact it could potentially live for the entire life of your program. And since it has no shared state in it, it's thread-safe and so forth. It exists to:

1) Give the Linq to SQL system a place to store the results of analyzing the query (i.e. the actual SQL plus the delegate that will be used to extract data from the result set)

2) Allow the user to specify the "variable parts" of the query. The most common case isn't that the query is exactly the same from run to run, usually it's "nearly" the same... That is it's the same except that perhaps the search string is different in the where clause, or the ID being fetched is different. The shape is the same. Creating a delegate with parameters allows you to specify which things are fixed and which things are variable.

Now there was some debate about how to make compiled queries durable, automatically caching them was considered, but this was something I was strongly against. Largely because of the object lifetime issues it would cause. First, you would have to do complicated matching of a created query against something that was already in the cache -- something I'd like to avoid. Secondly you have to decide where to store the cache, if you associate it with the DataContext then you get much less query re-use because you only get a benefit if you run the same query twice in the same data context. To get the most benefit you want to be able to re-use the query across DataContexts. But then, do you make the cache global? If you do you have threading issues accessing it, and you have the terrible problem that you don't know when is a good time to discard items from the cache. Ultimately this was my strongest point, at the Linq data level we do not know enough about the query patterns to choose a good caching policy, and, as I've written many times before, when it comes to caching good policy is crucial. In fact, analogously, we had to make changes in the regular expression caching system back in Whidbey precisely because we were seeing cases where our caching assumptions were resulting in catastrophically bad performance (Mid Life Crisis due to retained compiled regular expressions in our cache) -- I didn't want to make that mistake again.

So that's roughly how we end up at our final design. Any Linq to SQL user can choose how much or how little caching is done. They control the lifetime, they can choose an easy mechanism (e.g. stuff it in a static variable forever) or a complicated recycling method depending on their needs. Usually the simple choice is adequate. And they can easily choose which queries to compile and which to just run in the usual manner.

Let's get back to the overhead of compiled queries. Besides the one-time cost of creating the delegate there is also an little extra delegate indirection on each run of the query plus the more complicated thing we have to do: since the compiled query can span DataContexts we have to make sure that the DataContext we are being given in any particular execution of a compiled query is compatible with the DataContext that was provided when the query was compiled the first time.

Other than that the code path is basically the same, which means you come out ahead pretty quickly. This test case was, as usual, designed to magnify the typical overheads so we can observe them. The result set is a small number of rows, it is always the same rows, the database is local, and the query itself is a simple one. All the usual costs of doing a query have been minimized. In the wild you would expect the query to be more complicated, the database to be remote, the actual data returned to be larger and not always the same data. This of course both reduces the benefit of compilation in the first place but also, as a consolation prize, reduces the marginal overhead.

In short, if you expect to reuse the query at all, there is no performance related reason not to compile it.

Performance Quiz #13 -- Linq to SQL compiled queries cost

ricom — Fri, 11 Jan 2008 23:08:00 GMT

I've written a few articles about Linq now and you know I was a big fan of compiled queries in Linq but what do they cost? Or more specifically, how many times to you have to use a compiled query in order for the cost of compilation to pay for itself? With regular expressions for instance it's usually a mistake to compile a regular expression if you only intend to match it against a fairly small amount of text.

Lets do a specific experiment to get an idea. Using the ubiquitous Northwinds database and getting the same data over and over to control for the the cost of the database accesses (and magnify any Linq overheads) we run this query:

var q = (from o in nw.Orders
            select new {
                OrderID = o.OrderID,
                CustomerID = o.CustomerID,
                EmployeeID = o.EmployeeID,
                ShippedDate = o.ShippedDate
           }).Take(5);

and compare it against:

var fq = CompiledQuery.Compile
(
    (Northwinds nw) =>
            (from o in nw.Orders
            select new
                   {
                       OrderID = o.OrderID,
                       CustomerID = o.CustomerID,
                       EmployeeID = o.EmployeeID,
                       ShippedDate = o.ShippedDate
                   }).Take(5)
);

So now the quiz: How many times to I have to use the compiled version of the query in order for it to be cheaper to compile than it would have been to just use the original query directly?