Rico Mariani's Performance Tidbits : ramblings

I'm not 30 yet, I'm only 0x2d :)

ricom — Tue, 08 Sep 2009 02:35:00 GMT

A friend of mine pointed out that, in hex, I'm under 30 :)

How wonderful!

Visual Studio: Why is there no 64 bit version? (yet)

ricom — Thu, 11 Jun 2009 06:34:00 GMT

Disclaimer: This is yet another of my trademarked "approximately correct" discussions

From time to time customers or partners ask me about our plans to create a 64 bit version of Visual Studio. When is it coming? Why aren’t we making it a priority? Haven’t we noticed that 64 bit PC’s are very popular? Things like that. We just had an internal discussion about “the 64 bit issue” and so I thought I would elaborate a bit on that discussion for the blog-o-sphere.

So why not 64 bit right away?

Well, there are several concerns with such an endeavor.

First, from a performance perspective the pointers get larger, so data structures get larger, and the processor cache stays the same size. That basically results in a raw speed hit (your mileage may vary). So you start in a hole and you have to dig yourself out of that hole by using the extra memory above 4G to your advantage. In Visual Studio this can happen in some large solutions but I think a preferable thing to do is to just use less memory in the first place. Many of VS’s algorithms are amenable to this. Here’s an old article that discusses the performance issues at some length: http://blogs.msdn.com/joshwil/archive/2006/07/18/670090.aspx

Secondly, from a cost perspective, probably the shortest path to porting Visual Studio to 64 bit is to port most of it to managed code incrementally and then port the rest. The cost of a full port of that much native code is going to be quite high and of course all known extensions would break and we’d basically have to create a 64 bit ecosystem pretty much like you do for drivers. Ouch.

[Clarification 6/11/09: The issue is this: If all you wanted to do was move the code to 64 bit then yes the shortest path is to do a direct port. But that’s never the case. In practice porting has an opportunity cost, it competes with other desires. So what happens is more like this: you get teams that have C++ code written for 32 bits and they say “I want to write feature X, if I port to managed I can do feature X plus other things more easily, that seems like a good investment” so they go to managed code for other reasons. But now they also have a path to 64 bit. What’s happening in practice is that more and more of the Visual Studio is becoming managed for reasons unrelated to bitness. Hence a sort of net-hybrid porting strategy over time.]

So, all things considered, my feeling is that the best place to run VS for this generation is in the 32 bit emulation mode of a 64 bit operating system; this doubles your available address space without taking the data-space hit and it gives you extra benefits associated with that 64 bit OS. More on those benefits later.

Having said that, I know there are customers that would benefit from a 64 bit version but I actually think that amount of effort would be better spent in reducing the memory footprint of the IDE’s existing structures rather than doing a port. There are many tradeoffs here and the opportunity cost of the port is high.

Is it expensive because the code is old and of poor quality?

It’s not really about the quality of the code – a lot of it is only a few releases old – as it is the amount of code involved. Visual Studio is huge and most of its packages wouldn’t benefit from 64 bit addressing but nearly all of it would benefit from using more lazy algorithms – the tendency to load too much about the current solution is a general problem which results in slowness even when there is enough memory to do the necessary work. Adding more memory to facilitate doing even more work that we shouldn’t be doing in the first place tends to incent the wrong behavior. I want to load less, not more.

Doesn’t being a 64 bit application save you all kinds of page faults and so forth?

A 64 bit address space for the process isn’t going to help you with page faults except in maybe indirect ways, and it will definitely hurt you in direct ways because your data is bigger. In contrast a 64 bit operating system could help you a lot! If you’re running as a 32 bit app on a 64 bit OS then you get all of the 4G address space and all of that could be backed by physical memory (if you have the RAM) even without you using 64 bit pointers yourself. You’ll see potentially huge improvements related to the size of the disk cache (not in your address space) and the fact that your working set won’t need to be eroded in favor of other processes as much. Transient components and data (like C++ compilers and their big .pch files) stay cached in physical memory, but not in your address space. 32 bit processes accrue all these benefits just as surely as 64 bit ones.

In fact, the only direct benefit you get from having more address space for your process is that you can allocate more total memory, but if we’re talking about scenarios that already fit in 4G then making the pointers bigger could cause them to not fit and certainly will make them take more memory, never less. If you don’t have abundant memory that growth might make you page, and even if you do have the memory it will certainly make you miss the cache more often. Remember the cache size does not grow in 64 bit mode but your data structures do. Where you might get savings is if the bigger address space allowed you to have less fragmentation and more sharing. But Vista+ auto-relocates images efficiently anyway for other reasons so this is less of a win. You might also get benefits if the 64 bit instruction set is especially good for your application (e.g. if you do a ton of 64 bit math)

So, the only way you’re going to see serious benefits is if you have scenarios that simply will not fit into 4G at all. But, in Visual Studio anyway, when we don’t fit into 4G of memory I have never once found myself thinking “wow, System X needs more address space” I always think “wow, System X needs to go on a diet.”

Your mileage may vary and you can of course imagine certain VS packages (such as a hypothetical data analytics debugging system) that might require staggering amounts of memory but those should be handled as special cases. And it is possible for us to do a hybrid plan with including some 64 bit slave processes.

I do think we might seem less cool because we’re 32 bit only but I think the right way to fight that battle is with good information, and a great product.

Then why did Office make the decision to go 64 bit?

This section is entirely recreational speculation because I didn’t ask them (though frankly I should). But I think I can guess why. Maybe a kind reader can tell me how wrong I am :)

First, some of the hardest porting issues aren’t about getting the code to run properly but are about making sure that the file formats the new code generates remain compatible with previous (and future) versions of those formats. Remember, the ported code now thinks it has 64 bit offsets in some data structures. That compatibility could be expensive to achieve because these things find their way into subtle places – potentially any binary file format could have pointer-size issues. However, Office already did a pass on all its file formats to standardize them on compressed XML, so they cannot possibly have embedded pointers anymore. That’s a nice cost saver on the road to 64 bit products.

Secondly, on the benefit side, there are customers out there that would love to load enormous datasets into Excel or Access and process them interactively. Now in Visual Studio I can look you in the face and say “even if your solution has more than 4G of files I shouldn’t have to load it all for you to build and refactor it” but that’s a much harder argument to make for say Excel.

In Visual Studio if you needed to do a new feature like debugging of a giant analytics system that used a lot of memory I would say “make that analytics debugging package 64 bit, the rest can stay the way they are” but porting say half of Excel to 64 bits isn’t exactly practical.

So the Office folks have different motivations and costs and therefore came to different conclusions -- the above are just my personal uninformed guesses as to why that might be the case.

One thing is for sure though: I definitely think that the benefits of the 64 bit operating system are huge for everyone. Even if it was nothing more than using all that extra memory as a giant disk cache, just that can be fabulous, and you get a lot more than that!

Is Performance a "functional" requirement?

ricom — Wed, 27 Aug 2008 19:57:00 GMT

We had an internal thread on this yesterday, maybe I was a little too glib but here is what I had to say about the topic [edited so it can be read standalone]

The original question, “Is performance a functional requirement” is highly unexciting for me. However the revised question, “What is the best way to capture performance requirements” is profoundly interesting. It is precisely the (anti-)correlations between attainable functionality and level of responsiveness that sometimes makes for the very best engineering work. How do you capture all this? Sometimes pervasively. I would argue that pervasively is in fact the only answer that works in non-trivial cases. And this is not limited to performance as the quality metric of interest – it applies to any quality that is not going to be trivially achieved.

The nomenclature to me is uninteresting except to the extent that it assists designers and implementers in capturing and understanding the various kinds of requirements – i.e. I know where to put them, I know where to find them, I know that I got them all, because I am, literally, on a first name basis with all the requirement types.

I don’t want to trivialize that aspect, but, beyond that, meh, whatever… functional schmuntional, I don’t care what you call it, just do it :)

Celebrating Twenty with a Pint

ricom — Fri, 15 Aug 2008 21:42:00 GMT

No it isn't what you think. :)

Today is my 20th anniversary at Microsoft.

Wow.

20 years of doing anything is pretty amazing and 20 of this definitely counts as amazing, to me anyway. It's been a lot of fun.

I got to celebrate in a very special way. You see, I have Hereditary Hemochromatosis -- an Iron Overload condition -- and I've been having therapy for it for many years. It's really very pleasant therapy, I go to the blood center every few weeks and they take a pint of blood, it keeps the iron under control. I even get cookies and juice!

Until today they had to throw that blood away because I was a therapeutic donor. Today was special. Many years ago studies had shown that the blood of hemochromatic donors was safe to use and the FDA recently approved my local blood center to join others in accepting donations from people with my condition.

So, although they've been taking my blood for 15 years (sometimes as often as once per week!) today was the first time I *gave* blood that could be used by others.

I thought it was a great way to celebrate, I hope lots of you join me.

My Last Words to Bill

ricom — Mon, 30 Jun 2008 20:57:10 GMT

We had a little internal yearbook thing you could sign for Bill last week. This is what I wrote:

Dear Bill,

In not too many weeks now I’ll be celebrating my 20th anniversary at Microsoft. I think I owe you some thanks for these 20 years, and some from before.

In fall of 1979 I got my first real access to a computer. It was a Commodore PET and it was running Microsoft BASIC. For me, and many others like me, that exposure caused a radical change in our life trajectories.

By Christmas I was learning 6502 assembler and those MOS tech handbooks were not exactly rich in examples. If you wanted to see *real* code you had to disassemble/understand the ROMs. So I guess what I’m saying is that, at the tender age of 15, I was ripping off your intellectual property. Sorry about that.

I did manage to get pretty good at 6502 assembler and I like to think some of that code was yours, so I tell my friends I got my first low level programming lessons from Bill Gates. Of course you didn’t know it, but it was nonetheless successful long-distance education through the magic of software.

Eight years, one diploma, and one degree later, I landed in Redmond. That was 1988. Since then, I’ve had many chances to meet, learn from, and work with some great people inside and outside of Microsoft – even Melinda for a time – and in turn affect the lives of others.

Thank you for the education, the opportunities, and the inspiration.

-Rico

Cycles in Computer Science, or Am I Ancient?

ricom — Mon, 05 May 2008 20:11:32 GMT

It's been a strange couple of weeks. No, really.

It all started when a friend of mine, let's call him "Desi", posted a question asking about what he should do in his last few months of college. A reasonable enough question and I guess I'm as qualified as anyone to give him some advice because it was a question about computer science courses he could take rather than general career guidance. Well maybe I'd be qualified to give advice on that too but anyway... let's at least try to stay on topic.

So far so good. But here's the kicker, he was considering taking a course in C++ but was dubious about its value because it was "ANCIENT" [emphasis in original].

Eeep.

But... but... it can't be ancient. I mean, wasn't it just last week I was reading some OOPSLA notes on C++ -- no wait that was 20 years ago. Could it be true?

But wait it gets better.

Then I went to this other talk where people were talking about the importance of C++ and how many companies [many of which are important MS customers] have great investments in C++; they have highly valueable and important codebases that their company's future is based on. Microsoft of course has huge investments in C++ that we expect to endure for some time.

Wow ok so all that is true but I had this feeling of déjà vu. Hadn't I heard a talk just like this one while I was in college? I think I had... it was about COBOL. We were too cool for COBOL back then, or perhaps too unenlightened is more accurate.

Let's see, if we take 1960 as the birth of COBOL (1959-61 seems to be the range of initial activity) that would mean that in 1985 it was 25 years old. Let's say that my opinion of COBOL in 1985 was not especially high. But wait that same year could arguably be called the birthday of C++ because the first commercial C++ compiler became available then. So let's see -- it's now 2008.... so C++ is 23 years old. The same age as COBOL when I started college.

The fact that I can even offer this perspective is already making me feel old :)

But, aside from nostalgia and pointing out what a nub I was in college, what's my point? I always have a point right. And I assure you it isn't slamming folks still working on COBOL because high quality COBOL implementations have continued to show the viability/future of that programming language for my entire career.

My point is this: In many ways the same kinds of problems face our industry today as always. Migration of existing codebases to the latest technology while preserving their value isn't a new problem, it's an old one. And I use the word migration loosely because often it's not really some kind of conversion or retirement of code but rather more like a meeting of worlds.

When I was in college one of the big buzzwords we heard about was 4th generation languages, and sometimes even (ooo, aaah) 5th generation languages. I don't know if I would really score the industry very high in terms of actual *language* evolution in the last 25 years (though I'll be the first to give Anders kudos for enabling your average VB programmer to use functional programming constructs without even having to know it). Where I will score us highly is in runtime evolution.

So programming languages are somewhat similar, at least broadly speaking, to what they were say 25 years ago. What's changed is the environments we like to run in. GUI environments drove event-based programming which caused a need to express those notions. Object oriented programming fit that need well and had other benefits and so many embraced it, or borrowed notions from it (tell me, what's the difference between a MessageProc (with its associated switch statement) and an object with vtable? Mostly syntatic sugar. But the programming model is fundamentally different than say a console application.

Now its happening again, and the real need facing C++ programmers is somewhat the same as what faced COBOL programmers say 25 years ago. It's not that the language is out of joint -- it isn't. I mean, ok maybe you like or don't like COBOL syntax but that doesn't doom a language and surely C++ syntax is not the zenith of wonderfulness. But that isn't what's holding C++ programmers back. The biggest problem, at least in my opinion, is one of accessing new/modern runtime features that may have a different programming environment from the context of an existing environment. Hybrid applications are the norm.

Most programming languages you can name have natural environments that they like to run in and natural kinds of application notions they can express well.

And now, GUI environments are changing; more declarative models like the WPF/XAML programming model are becoming popular. Multi-threaded needs are becoming more mainstream. Then there are all the rich client models like AJAX. And many others. Many of these have complicated resource management problems that make you want to reach for languages that have automatic management of resources (like the .NET family and Java).

The truly successful environments of the future may have more to do with their ability to mesh together many different kinds of assets than defining some new uber language and uber runtime. I'd love to do this nowish because, 25 years from now, I bet there will be something newer/better and I don't want to have to migrate an industry again. :)

Sara Ford goes retro

ricom — Fri, 29 Feb 2008 16:51:00 GMT

In honor of the leap year, she's posted a couple of articles I wrote back around 1993 describing things you could do in VC++ 2.0. Oddly enough most of them are still applicable.

Cheers Sara

Programs to Run

ricom — Sun, 21 Oct 2007 04:05:00 GMT

I haven't posted anything "fun" for a while. Here's another of my little parody songs.

Based on Sheryl Crow's "Soak Up The Sun"

Programs to Run

My friend the program lead
Can't even code in VB
I can't afford his bloat
So I'm stuck here fixing his tree

I don't have profile runs
I don't have diddly squat
I just wish they had a goal
It's not like I ask a lot

CHORUS:

I've got programs to run
Gotta tell everyone
To speed them up

I'm gonna tell them that:

I've got no-one to blame
for every function that's lame
I'll speed it up

I've got a crummy test
It don't help near enough
To find the things it takes
To fix these these bugs they're so tough

Every time I turn around
The space goes up, the speed is down
Surely if they only knew
They wouldn't act the way they do?
I think I'm going crazy too!

CHORUS

Don't have no testing suite
But I'm still the king of speed
You've got to ship worldwide, but baby
I'm the one with goals to meet

Every time I turn around
The space goes up, the speed is down
Surely if they only knew
They wouldn't act the way they do?
I think I'm going crazy too!

CHORUS

I've got programs to run
Got my profiler set
So I can rock on

News About Me

ricom — Thu, 12 Jul 2007 23:23:00 GMT

From time to time things get shuffled around here and I was involved in a recent shuffle that has changed my job yet again. I’ve been working directly on performance in one capacity or another for nearly 5 years now but a few weeks ago I was asked to take on a new, broader, job. That job is Chief Architect of Visual Studio. I could hardly say no.

So, I hope you’ll all be pleased and you won’t worry that I will suddenly forget everything I knew about performance or stop driving performance excellence at Microsoft but I will have a new way to do that.

Naturally with Beta 2 of Orcas coming “soon” the results of my old job are what you’re likely to see in terms of Visual Studio performance (and other things) for a while, but hopefully that will change – in a good way – in the months and years to come.

Wish me luck. It’s a huge job. Easily the biggest of my career.

Partly sunny, chance of showers, bring an umbrella

ricom — Thu, 14 Jun 2007 00:23:34 GMT

Everyone knows that my 10 rules of performance are measure, measure, measure... etc. 10 times :)

OK, well, if you didn't know, now you do.

But does that mean that every time you tell someone about performance problems and/or risks that you should be doing so in quantitative terms?

I think not. Although that got me a little friendly scolding today -- "What's up with this commentary with no numbers Rico?"

I'll tell you what's up: In a performance report, like any form of communication, it's important to know your audience and give them data in a form that is both useful and consumable. What does that mean? Well it means that if someone asks you what the weather is going to be like this afternoon probably giving them a description of location of local isotherms isn't what they had in mind. It's a lot more likely that they just want to know if they need to bring their umbrella.

Now you don't want to be redendering your opinion on the weather, or anything else for that matter, without reasonable data to base it on but that does not mean that you need to present every bit of the data or even any of it for any particular audience.

One of the things that I often get asked to do is to provide a gut-level feel as to where we are likely to run into trouble, and why. This helps to provide some coarse steering and also to set the level of urgency. This is an important function and it's not one that needs to be encumbered with vast amounts of data. In fact, vast amounts of data can do the opposite -- make the conclusions less approachable, less likely to be read, and less likely to be acted upon.

It's important that when asked you can back up your reasoning with hard numbers but it's not always important, or even a good idea, to encumber executive summaries with a lot of data. And it's certainly a good idea to keep your executives duly summarized.

Everything I ever needed to know about programming I learned in Gamergarten

ricom — Fri, 06 Apr 2007 19:50:00 GMT

Don't block the user from moving her ship while she fires her laser at the space aliens
Don't redraw more of the screen than you have to when the aliens move
Make sure the sounds start right away when the buttons are pushed
It's better to not have a bonus ship go across the screen if its presence makes the game laggy

I love XNA. Really I do. Not because I think writing games is the single most important activity on the planet -- although maybe it is -- but because I think it's one of the most educational things a budding programmer can do.

Game development is a great exercise because, even with fancy graphics hardware and super-computer like CPU resources available, it's so constrained. You always want to get the most out of that hardware because that's what will make your game great. Real considerations like frame-rate, lag, available textures, are in your face and directly impact the users experience in comparatively easy to understand ways.

Everyone should write a (little) game at least once in their life and I wish more Computer Science students did just that.

Some people have to spend an entire career unlearning callously taught maxims like "You should just let the [compiler|hardware] guys do the optimization", "Don't worry about performance, the [.NET|Java] runtime takes care of that for you."

If you're in school now, trust me... It ain't so.

Perhaps more than anything else I could possibly teach a CS student it would be not a datastructure, or a programming language but a basic understanding of the costs of low and medium level operations common in programming today. Maybe that's why I'm so motivated to talk about performance signatures.

Good Advice Scorned

ricom — Fri, 19 Jan 2007 21:04:47 GMT

I thought it might be interesting to start a discussion about what is possibly the number one frustration of the performance architect: What do you do when they won't listen to you?

I'm sure you've all faced this, you lay out the situation, you show them the numbers and your team still won't take the actions needed to avoid the forecasted problems. Then what?

Well, you could put on your best Vincent Price voice and read them this poem:

Once, twice, three times warned,
And each time good advice you've scorned.
See the perf disaster loom,
Now you face your project's doom!

That's certainly a dramatic attention getter, but is it likely to help? Err, probably not -- unless you're trying to get yourself a whole lot of time away from it all.

So what do you do? Well, one reason I'm making this posting is I'd like to hear what sorts of things you do. But let me tell you what I do:

My job is to help my team make better decisions, it doesn't end the moment they make one that I don't think is the best, if anything that makes my job all the more important going forward.
If we've decided to take a hard road then it's all the more important to understand what the likely outcomes are going to be and start planning for them. For instance:

Can we clearly articulate the costs we've decided to pay (performance or otherwise) so we all know what they're going to be.
Can we make afforances for the costs so that those costs are easier to bear?
Can we mitigate the costs at all?
Can we prioritize the problems/costs we're likely to face so that we can address the ones that matter the most if not all of them?

Your team may have settled on a plan other than the one you endorse but perhaps you can find other ways to gain some of the benefits of your original plan.

The long and the short of it is that we all sink together. Some of the choices won't go the way I would like -- that's both expected and healthy -- that just makes the adventure more fun.

Five Things About Me

ricom — Fri, 12 Jan 2007 23:38:51 GMT

I guess it's fate: I was blog tagged by both Jeff Beehler and J.D. Meier within hours of one another. So here are my five things you probably didn't know about me (but some of you do).

I got my first professional programming job (I was 15 years old) at Comspec in Toronto where, among other things, I did 6502 assembly language programming for a variety of devices. Some fun ones were the Microshare printer and drive sharing device -- this was before LANs -- and an interface for a scale so sensitive you could use it to measure the mass transfered from pencil to paper writing your name.
I studied at the University of Waterloo where among other things I was a president of the Computer Science Club (and it was not even a total geekfest those years). I also worked with the Symbolic Computation Group which was loads of fun. The last thing I did at UW was to rewrite the Maple pretty-printer so it did better formatting and labelling. I expect their current one still has bits of my code in it.
While at UW I was a co-op student and I did work-terms (internships) at The Ontario Science Centre. It turns out that, IMHO, "OSC" is one of the most fun places to work in the entire universe. You just don't know what might happen next -- I even got to meet Sophia Loren when she visited though she probably doesn't remember me. :)
While I was at OSC we did a show on Chaos and there was great interest in Fractals, and in particular The Mandelbrot Set. I was studying Complex Analysis at the time and I used what I knew to come up with a faster way to render the Mandelbrot set. I shared the technique and a partial proof of correctness for the underlying theory with Rollo Silver who was then publishing Amygdala -- which sent subcribers 35mm slides of cool mandelbrot renders -- Rollo finished the proof and published it. Meanwhile A.K. Dewdney also got a demonstration and in 1988 he mentioned the technique in Scientific American's Computer Recreations column -- he called it "Mariani's Algorithm" but it's more popularly known as the Mariani/Silver Algorithm. That's my claim to fame :)
In 1988 I joined Microsoft and within a few days I met Steve Ballmer. The way I met Steve was that I crashed a party he was throwing for his summer interns -- he was running the Systems division at the time. More specifically, I sat down next to someone who I didn't know and, being the gregarious guy I am, proceeded to tell him I wasn't even supposed to be here but my friends made me come and I just started a few days ago and hi my name is Rico. He said, "Hi, I'm Steve." Ya. That Steve. Fortunately for me he saw the humor in it. But don't try this at home kiddies :)

I'm not tagging anyone else because otherwise this whole pyramid tag thing will self implode but feel free to write your five things if you haven't already and blame me for motivating you.

My 18th anniversary at MS

ricom — Tue, 15 Aug 2006 17:18:00 GMT

Today marks 18 years for me at Microsoft. In a little twist of fate, I'll be presenting at our Gamefest conference which is kind of fitting because I got my start programming games for the PET 2001 in Microsoft BASIC (before it was Visual) and 6502 assembly language about 8 years before that. It turns out writing games is very educational :)

Today I'll be talking about writing real-time programs in managed code. Hope to see you today, if not, I'll be posting a detailed summary of the talk (more than just slides) when I'm back from vacation next week.

Rico's thought for today

ricom — Fri, 28 Jul 2006 20:00:00 GMT

I've said this before but I like it so much I thought I'd post it for posterity :)

"Never use the advice of some loudmouth perf expert like me when you could do a nice solid measurement instead."

Maybe I'll get it as a tattoo :)