Rico Mariani's Performance Tidbits

Shutdown Is No Time For Spring Cleaning

I think my current performance pet peeve is shutdown.  Assorted flavors of it, they all seem to have the same kind of problem.  Sometimes we're stuck with it... but maybe we shouldn't be?

This is one time when our basic training, which normally I love so much, tends to let us down.  We were all taught to clean up our own messes -- programming wise that means freeing your resources after you are done with them.  But this backfires in the shutdown case.

Many times I watch as I hit the [X] close button on some application and my poor computer starts to swap as the application goes about paging in vast amounts of its code and then dutifully walking all its data structures-- more paging -- and giving them back to the operating system.  My reaction to this in a word:  ACK!!!!

When your application is ordered to shutdown the last thing you should do is enumerate every piece of memory you have ever allocated and systematically give them back to the operating system.  Your program has a death sentence, and soon your resources are going back to the operating system whether you like it or not:  what you must do  is look at the minimum possible amount of memory necessary to get to a nice safe stable state and then exit as quickly as possible.  Abandoning your memory like this gives the operating system the best chance to get your process unloaded while swapping in the least amount of memory and causing the least impact to the systems disk and memory caches. 

In short, shutdown is no time for spring cleaning.

And why all this cleaning anyway?  Many people report that they have all these important resources that need flushing and so forth.  They couldn't possibly get to a safe state without considerable work but usually that in itself is symptomatic of assorted problems.  Any application that has important data to manage almost certainly needs to be tolerant of power-failure and if that's the case when the user makes important edits they likely should be automatically saved to a durable location.  In fact at any given moment, probably only a few seconds worth of data should be at risk.  If your application has been idle for any length of time it should be fully saved -- and even if the user hasn't chosen to save their work it's still effectively saved somewhere so that you could restore in the current "dirty" state.

So if you have to do all this work to be resilient to power failures, then take advantage of that logic to simplify your shutdown paths.  Your users will thank you.

Cycles in Computer Science, or Am I Ancient?

It's been a strange couple of weeks.  No, really.

It all started when a friend of mine, let's call him "Desi", posted a question asking about what he should do in his last few months of college.  A reasonable enough question and I guess I'm as qualified as anyone to give him some advice because it was a question about computer science courses he could take rather than general career guidance.  Well maybe I'd be qualified to give advice on that too but anyway... let's at least try to stay on topic.

So far so good.  But here's the kicker, he was considering taking a course in C++ but was dubious about its value because it was "ANCIENT" [emphasis in original].

Eeep.

But... but... it can't be ancient.  I mean, wasn't it just last week I was reading some OOPSLA notes on C++ -- no wait that was 20 years ago.  Could it be true?

But wait it gets better.

Then I went to this other talk where people were talking about the importance of C++ and how many companies [many of which are important MS customers] have great investments in C++; they have highly valueable and important codebases that their company's future is based on.  Microsoft of course has huge investments in C++ that we expect to endure for some time.

Wow ok so all that is true but I had this feeling of déjà vu.  Hadn't I heard a talk just like this one while I was in college?  I think I had... it was about COBOL.  We were too cool for COBOL back then, or perhaps too unenlightened is more accurate.

Let's see, if we take 1960 as the birth of COBOL (1959-61 seems to be the range of initial activity) that would mean that in 1985 it was 25 years old.  Let's say that my opinion of COBOL in 1985 was not especially high.  But wait that same year could arguably be called the birthday of C++ because the first commercial C++ compiler became available then.  So let's see -- it's now 2008.... so C++ is 23 years old.  The same age as COBOL when I started college.

The fact that I can even offer this perspective is already making me feel old :)

But, aside from nostalgia and pointing out what a nub I was in college, what's my point?  I always have a point right.  And I assure you it isn't slamming folks still working on COBOL because high quality COBOL implementations have continued to show the viability/future of that programming language for my entire career.

My point is this:  In many ways the same kinds of problems face our industry today as always.  Migration of existing codebases to the latest technology while preserving their value isn't a new problem, it's an old one.  And I use the word migration loosely because often it's not really some kind of conversion or retirement of code but rather more like a meeting of worlds.

When I was in college one of the big buzzwords we heard about was 4th generation languages, and sometimes even (ooo, aaah) 5th generation languages.  I don't know if I would really score the industry very high in terms of actual *language* evolution in the last 25 years (though I'll be the first to give Anders kudos for enabling your average VB programmer to use functional programming constructs without even having to know it).  Where I will score us highly is in runtime evolution.

So programming languages are somewhat similar, at least broadly speaking, to what they were say 25 years ago.  What's changed is the environments we like to run in.  GUI environments drove event-based programming which caused a need to express those notions.  Object oriented programming fit that need well and had other benefits and so many embraced it, or borrowed notions from it (tell me, what's the difference between a MessageProc (with its associated switch statement) and an object with vtable?  Mostly syntatic sugar.  But the programming model is fundamentally different than say a console application.

Now its happening again, and the real need facing C++ programmers is somewhat the same as what faced COBOL programmers say 25 years ago.  It's not that the language is out of joint -- it isn't.  I mean, ok maybe you like or don't like COBOL syntax but that doesn't doom a language and surely C++ syntax is not the zenith of wonderfulness.  But that isn't what's holding C++ programmers back.  The biggest problem, at least in my opinion, is one of accessing new/modern runtime features that may have a different programming environment from the context of an existing environment.  Hybrid applications are the norm.

Most programming languages you can name have natural environments that they like to run in and natural kinds of application notions they can express well. 

And now, GUI environments are changing; more declarative models like the WPF/XAML programming model are becoming popular.  Multi-threaded needs are becoming more mainstream.  Then there are all the rich client models like AJAX.  And many others.  Many of these have complicated resource management problems that make you want to reach for languages that have automatic management of resources (like the .NET family and Java).

The truly successful environments of the future may have more to do with their ability to mesh together many different kinds of assets than defining some new uber language and uber runtime.  I'd love to do this nowish because, 25 years from now, I bet there will be something newer/better and I don't want to have to migrate an industry again.  :)

Sara Ford goes retro

In honor of the leap year, she's posted a couple of articles I wrote back around 1993 describing things you could do in VC++ 2.0.  Oddly enough most of them are still applicable.

 Cheers Sara

Computer Measurement Group offers past papers to the public!

As you know I presented my performance signatures paper at the Computer Measurement Group's (CMG) conference a little over a year ago and I am happy to announce that CMG is now offering previous proceedings from 2005 and older to the public.  I think this is a fabulous move by them because there is some first rate content here that just wasn't seeing nearly the light that it deserved to see.

I hope to provide more specific recommendations when I have had the chance to look at the site myself in more detail and of course

If you're looking for a way to grow as a performance professional this could be a fabulous resource for you.  I thoroughly enjoyed the event and the presentations when I went.

See http://www.cmg.org/proceedings/

Performance Quiz #13 -- Linq to SQL compiled query cost -- solution

Well is there really a "solution" at all in general?  This particular case I think I constrained enough that you can claim an answer but does it generalize?  Let's look at what I got first, the raw results are pretty easy to understand.

The experiment I conducted was to run a fixed number of queries (5000 in this case) but to break them up so that the compiled query was reused a decreasing amount.  The first run is the "best" 1 batch of 5000 selects all using the compiled query.  Then 2 batches of 2500, and so on down to 5000 batches of 1.  As a control I also run the uncompiled case at each step expecting of course that it makes no difference.  Note the output indicates we selected a total of 25000 rows of data -- that is 5 per select as expected.  Here are the raw results:

Testing 1 batches of 5000 selects
5000 selects uncompiled 9200.0ms 25000 records total 543.48 selects/sec
5000 selects compiled 5401.0ms 25000 records total 925.75 selects/sec

Testing 2 batches of 2500 selects
5000 selects uncompiled 9181.0ms 25000 records total 544.60 selects/sec
5000 selects compiled 5402.0ms 25000 records total 925.58 selects/sec

Testing 5 batches of 1000 selects
5000 selects uncompiled 9169.0ms 25000 records total 545.32 selects/sec
5000 selects compiled 5432.0ms 25000 records total 920.47 selects/sec

Testing 100 batches of 50 selects
5000 selects uncompiled 9184.0ms 25000 records total 544.43 selects/sec
5000 selects compiled 5511.0ms 25000 records total 907.28 selects/sec

Testing 1000 batches of 5 selects
5000 selects uncompiled 9166.0ms 25000 records total 545.49 selects/sec
5000 selects compiled 6526.0ms 25000 records total 766.17 selects/sec

Testing 2500 batches of 2 selects
5000 selects uncompiled 9165.0ms 25000 records total 545.55 selects/sec
5000 selects compiled 7892.0ms 25000 records total 633.55 selects/sec

Testing 5000 batches of 1 selects
5000 selects uncompiled 9157.0ms 25000 records total 546.03 selects/sec
5000 selects compiled 10825.0ms 25000 records total 461.89 selects/sec

And there you have it.  Even at 2 uses the compiled query still wins but at 1 use it loses.  In fact, the magic number for this particular query is about 1.5 average uses to break even.  But why?  And how might it change?

Well, as has been observed in the comments, Linq query compilation isn't like regular expression compilation.  In fact compiling the query doesn't do anything that isn't going to have to happen anyway.  In fact, actually creating the compiled query with Query.Compile hardly does anything at all, it's all deferred until the query is run just as it would have been had the query not been compiled.  So what is the overhead?  Why is it slower at all?  And what's the point of it?

Well the main purpose of that compiled query object is to have an object, of the correct type, that also has the correct lifetime.  The compiled query can live across DataContexts, in fact it could potentially live for the entire life of your program.  And since it has no shared state in it, it's thread-safe and so forth.  It exists to:

1) Give the Linq to SQL system a place to store the results of analyzing the query (i.e. the actual SQL plus the delegate that will be used to extract data from the result set)

2) Allow the user to specify the "variable parts" of the query.  The most common case isn't that the query is exactly the same from run to run, usually it's "nearly" the same... That is it's the same except that perhaps the search string is different in the where clause, or the ID being fetched is different.  The shape is the same.  Creating a delegate with parameters allows you to specify which things are fixed and which things are variable.

Now there was some debate about how to make compiled queries durable, automatically caching them was considered, but this was something I was strongly against.  Largely because of the object lifetime issues it would cause.  First, you would have to do complicated matching of a created query against something that was already in the cache -- something I'd like to avoid.  Secondly you have to decide where to store the cache, if you associate it with the DataContext then you get much less query re-use because you only get a benefit if you run the same query twice in the same data context.  To get the most benefit you want to be able to re-use the query across DataContexts.  But then, do you make the cache global?  If you do you have threading issues accessing it, and you have the terrible problem that you don't know when is a good time to discard items from the cache.  Ultimately this was my strongest point, at the Linq data level we do not know enough about the query patterns to choose a good caching policy, and, as I've written many times before, when it comes to caching good policy is crucial.  In fact, analogously, we had to make changes in the regular expression caching system back in Whidbey precisely because we were seeing cases where our caching assumptions were resulting in catastrophically bad performance (Mid Life Crisis due to retained compiled regular expressions in our cache) --  I didn't want to make that mistake again.

So that's roughly how we end up at our final design.  Any Linq to SQL user can choose how much or how little caching is done.  They control the lifetime, they can choose an easy mechanism (e.g. stuff it in a static variable forever) or a complicated recycling method depending on their needs.  Usually the simple choice is adequate.  And they can easily choose which queries to compile and which to just run in the usual manner.

Let's get back to the overhead of compiled queries.  Besides the one-time cost of creating the delegate there is also an little extra delegate indirection on each run of the query plus the more complicated thing we have to do: since the compiled query can span DataContexts we have to make sure that the DataContext we are being given in any particular execution of a compiled query is compatible with the DataContext that was provided when the query was compiled the first time.

Other than that the code path is basically the same, which means you come out ahead pretty quickly.  This test case was, as usual, designed to magnify the typical overheads so we can observe them.  The result set is a small number of rows, it is always the same rows, the database is local, and the query itself is a simple one.  All the usual costs of doing a query have been minimized.  In the wild you would expect the query to be more complicated, the database to be remote, the actual data returned to be larger and not always the same data.  This of course both reduces the benefit of compilation in the first place but also, as a consolation prize, reduces the marginal overhead.

In short, if you expect to reuse the query at all, there is no performance related reason not to compile it. 

Performance Quiz #13 -- Linq to SQL compiled queries cost

I've written a few articles about Linq now and you know I was a big fan of compiled queries in Linq but what do they cost?  Or more specifically, how many times to you have to use a compiled query in order for the cost of compilation to pay for itself?  With regular expressions for instance it's usually a mistake to compile a regular expression if you only intend to match it against a fairly small amount of text.

Lets do a specific experiment to get an idea.  Using the ubiquitous Northwinds database and getting the same data over and over to control for the the cost of the database accesses (and magnify any Linq overheads) we run this query:

var q = (from o in nw.Orders
            select new {
                OrderID = o.OrderID,
                CustomerID = o.CustomerID,
                EmployeeID = o.EmployeeID,
                ShippedDate = o.ShippedDate
           }).Take(5);

and compare it against:

var fq = CompiledQuery.Compile
(
    (Northwinds nw) =>
            (from o in nw.Orders
            select new
                   {
                       OrderID = o.OrderID,
                       CustomerID = o.CustomerID,
                       EmployeeID = o.EmployeeID,
                       ShippedDate = o.ShippedDate
                   }).Take(5)
);

So now the quiz:  How many times to I have to use the compiled version of the query in order for it to be cheaper to compile than it would have been to just use the original query directly?

Rico's Instrumentation Aphorisms

A few months ago, Mary Gray of the Management Practices Team came to talk to me about good practices for creating performance counters and doing measurements generally.  She interviewed me on the topic for about an hour and was madly scribbling notes the whole time while I talked a mile a minute.  What's below is a slightly edited version what she took away from the interview.  I thought it was interesting enough that you guys might like to see it so here it is.

Mary, thank you for allowing me to share.

Adding instrumentation in the form of events and performance counters to your software is one of the most important things you can do to make your component or application more manageable by IT personnel, more supportable by CSS, more easily tuned and debugged by developers and testers.

The OS already has performance counters you can use for such resources as CPU, disk, memory, and network resources. These are the primary resources that you will need to track for most software. You don't need to add a lot of performance counters or events to your software for raw resources; the trick is to correlate what your software thinks it is doing with the operating system resource impact of those operations.

Judiciously added instrumentation allows you to more easily pinpoint the states that lead to poor performance or failure. Well designed events inform monitoring software and IT admins about whether the software is operating normally, in a degraded state, or has failed completely. Good tracing events in conjunction with perf counters related to the work of the software allow diagnosis and tracking of trends. Events targeted to the administrator can identify what work was being done for which user context when a failure occurs.

Rico's Instrumentation Aphorisms

Instrumentation aphorism #1: Attribute the cost, don't describe it.

To attribute costs, the important word is "correlation". You want to correlate what your software thinks it is doing to what the operating system knows about resource usage. You can use (e.g.) ETW tracing events to mark the beginning and end of "jobs" or transactions in your software's work life.

What is a “transaction” in the runtime life of your software?  Is it a mouse click event?  A business transaction of some kind?  An HTML page delivered to the user?  A database query performed?  Whatever it is, look at your critical resources and consider the cost per unit of work.  For example, consider CPU cycles per transaction, network bytes per transaction, disk i/o’s per transaction, etc.  

Tracing events, to be useful, need to be associated with the higher level transactions of the software rather than associated with the life of single objects. You can have too many events and events at too low a level or marking time intervals that are too short to be useful. This use of events and perf counters just creates overwhelming noise and does not allow you to see trends easily.

This correlation between the work of the software and resources should also be used in administrator events marking changes of state, not just tracing events. Administrators are running the software for a reason and have every interest in knowing why (e.g.) MOM 2005 is reporting a degraded state for it - why the system is slowing down or why the software is banging away at the disks continuously. These events, as opposed to tracing events should provide actionable advice.

Instrumentation aphorism #2: Account for consumption.

To account for consumption, you will want to calculate rates rather than just measure occurrences. Look at the resource costs per unit-of-work of work. What is your software accomplishing to justify its consumption of CPU, memory, disk, network , or other resources? Expressing resource costs in a per-unit-of work fashion will help you to see which costs are reasonable and which are problems. You want to be able to trace or to inform adminstrators what resources are being used.

The operating system already gives you a variety of performance counters that measure CPU consumption, disk I/Os, memory usage, and network activity. These are the primary measuring sticks you need to compare to what your software is doing. The performance counters you add are most useful when they calculate the rate of work accomplished.

You can generate tracing events that tell you the rate of work, what the user context is. The combination of events that mark the start and end of transactions with rate counters allows developers and CSS people to pinpoint the resource that is being pinched and wrecking performance or starting a death spiral to failure.

If you are considering a design which sequesters a chunk of memory which your software managed, you may want to think twice about it. The OS already tracks memory resources. If you manage your own memory, then you have to duplicate the operating system plumbing to be able to diagnose performance problems and failures. The programming and maintenance costs for this may outweigh the hoped-for design benefits. 

Linq to SQL Compiled Queries are thread-safe

JD Conley wrote an interesting article showing the benefits you can get by using compiled queries on his blog.  He wrote me today saying:

"I enjoyed your postings on linq to sql performance and compiled query optimization. I recently published a blog about a real world situation that benefited from a compiled query. However, it got me thinking. Are compiled queries thread safe? Can I really create a static instance of a query and use it from multiple threads simultaneously (like in an ASP.NET application)?

My gut tells me no, but all the examples I've seen assume that they are in fact thread safe. Any ideas?"

A: Absolutely you can use it like that.  That's like the main scenario it was designed for :)

And by the way JD, nice job on your analysis -- textbook approach: cast a broad net, find the key resource, drill down. Don't make too many assumptions before you look at data and don't get detailed data until you know what to look at. 

^5

Visual Studio 2008 and .NET 3.5 ship today

Well you can read all about it on Soma's blog -- today is the day!

I'm really proud of the work we did in this release and I'm especially proud of that performance work -- great results in many areas with Linq and VB taking top honors from me on really getting the perf out.  But those are just my personal favorites -- what could I say in just a few words about a product that's so expansive? There's just so much to see! 

Get it, explore, and enjoy!

Performance Threat Models

I've been meaning to post this for ages and somehow I kept forgetting.

J.D. and I have long thought that many of the techniques used to do a security threat model are actually directly applicable to doing performance analysis as well.  The idea of threats and mitigations is quite general but more importantly a direct analysis of the architecture is invaluable and its something you can do very early in the lifecycle of a product.  Think of it as "testing" the architecture while it's still just a diagram.

A while ago J.D. produced this analysis which I think you might find useful: http://blogs.msdn.com/jmeier/archive/2007/08/28/performance-threats.aspx

The idea of testing the architecture is something I want to do a lot more of in the next version of Visual Studio (but more on that another time)

File Open Performance -- Beware of 'Extensions'

Here is a little interchange I had a few days ago; "Nick From Chicago" graciously allowed me to share it nearly verbatim.  I'd like to claim that I was psychic in diagnosing this but as you can see in the thread it isn't the first time I've run into this problem.

Read on :)

From: Nick From Chicago
To: Rico Mariani
Subject: VB.NET App Eating RAM for Breakfast

Mike Stall accused you of being a "Performance super-guru" - so I thought you might be able to answer this one.

I usually run my (VB.NET 2005 SP1 WinXP 32-bit) app through visual studio debugger. The hosting process takes up a grand total of 22k RAM.

So far, so good.

But when I run the NGEN'ed release version, it asks for 120 MB RAM just for hello...

And when I hit this line in the code:

If TheOpenFileDialog.ShowDialog() <> Windows.Forms.DialogResult.OK Then

My RAM usage skyrockets to 170MB RAM, as soon as the Open File Dialog box appears.

I tried to profile it...and by the time the CLR profiler loads the app, I'm up to over 1GB virtual memory. The profiler claims the app is using only a few KB or so of memory. The highest number I got was about 16MB for the Kernel (unmanaged code).

Am I missing something? I somehow don't think my app should need this much RAM.

Nick

From: Rico Mariani
To: Nick From Chicago
Subject: VB.NET App Eating RAM for Breakfast

When running the CLR profiler you might get better results if you set it to record only allocations and not all call stacks there is a little option for that.

I suspect that the bulk of the memory usage is coming from the file open dialog itself though which is not managed. That dialog is actually a "mini" version of the explorer and it can include explorer extensions and so forth right into your process. Sometimes when memory skyrockets like that the culprit is some kind of explorer extension that you may not even know you have.

When my friends have bizarre memory usages on their systems I generally start by uninstalling shell extensions and other things like that. Some of them are not friendly at all.

The technique on this posting:

http://blogs.msdn.com/ricom/archive/2004/12/10/279612.aspx

Could be helpful in accounting for all your memory usage no matter where it came from.

Bests,

-Rico

From: Nick From Chicago
To: Rico Mariani
Subject: VB.NET App Eating RAM for Breakfast

Wow! I spent about 2 hours un-installing two years worth of accumulated junk; what do you know, I'm down to <20K RAM usage for my app!

Thanks for all of your great advice, I hope never to become one of the people writing apps that cause these kinds of problems!

All the best,

Nick

Some things to visit

My sometimes cohort J.D. Meier in Patterns and Practices has been busy:

My other cohorts in the Technical Community Network have also been busy

Even my old has-been Behind The Code video is still popular :)

On a sadder/personal note, this week I lost my little furry friend Francesco, a 16-year old smoke Persian with the sweetest disposition you could imagine.  I'll miss him.

Programs to Run

I haven't posted anything "fun" for a while.  Here's another of my little parody songs.

Based on Sheryl Crow's "Soak Up The Sun"

Programs to Run

My friend the program lead
Can't even code in VB
I can't afford his bloat
So I'm stuck here fixing his tree

I don't have profile runs
I don't have diddly squat
I just wish they had a goal
It's not like I ask a lot

CHORUS:

I've got programs to run
Gotta tell everyone
To speed them up

I'm gonna tell them that:

I've got no-one to blame
for every function that's lame
I'll speed it up

I've got a crummy test
It don't help near enough
To find the things it takes
To fix these these bugs they're so tough

Every time I turn around
The space goes up, the speed is down
Surely if they only knew
They wouldn't act the way they do?
I think I'm going crazy too!

CHORUS

Don't have no testing suite
But I'm still the king of speed
You've got to ship worldwide, but baby
I'm the one with goals to meet

Every time I turn around
The space goes up, the speed is down
Surely if they only knew
They wouldn't act the way they do?
I think I'm going crazy too!

CHORUS

I've got programs to run
Got my profiler set
So I can rock on

Gamefest 2007: The Costs of Managed Code: The Avoidable and the Unavoidable

A few weeks ago I spoke at Gamefest 2007 where I delivered this talk:

The Costs of Managed Code: The Avoidable and the Unavoidable

This talk is for those who want to understand the inescapable performance consequences of the managed programming method: the things you cannot avoid and the things you can. The presentation explains those few characteristics of managed code, such as array bounds checking, application domain isolation, and write barriers, that profoundly affect the code generation at a primitive level. Comparing and contrasting the consequences for the .NET Compact Framework and the classic .NET runtime, the talk explains the reasons for these overheads, the benefits they provide, and what practices minimize the associated costs. Additionally, we discuss some commonly occurring costs, such as boxing, that aren't inherent to all managed code, and we offer some tips for minimizing those costs. Speaker: Rico Mariani

The talk was based on an older article that I wrote on Qualitative Code Differences in Managed Code which was itself based on an internal paper written by my colleague Vance Morrison (thank you again Vance).

You can see the talk at the XNA Content Creators Site and I encourage you to look at the other talks too as this whole series was very popular and many of the concepts are broadly useful.

Performance Improvements in Visual Studio 2008

It seems I'm doing a lot of referral postings this month and I've got a couple more for you.

This is the first one, near and dear to my heart because I worked with Cameron on some of these (and he didn't mention me at all, *sniff*) especially the performance problems in the cases with "Venus" (the ASP.NET project system), and one of my teammates, Hazim Shafi, worked with VB on many of the other issues too.  I'm very happy with what we managed to do, although there's always more :)

Anyway, without further ado, here's Cameron McColl talking about performance improvements VB made in VS2008.