Rico Mariani's Performance Tidbits

Coding in Marble

ricom — Fri, 03 Feb 2012 03:26:00 GMT

I wish I could remember where I first read it because perhaps it deserves attribution. But many years ago I read about the two world views of physicists and they resonated with me. One world view is that prescibed by things like General Relativity and Maxwell's Equations. These have, in some sense a great mathematical beauty to them. This is the "the universe is made out of marble" viewpoint. Nice clean fields. Very solid. Then there's the other perspective, this is the perspective you have in systems like Quantum Electrodynamics. There are lots of particles flying all over the place and probabilities. Lots of statistics. The universe is a messy organic place. It might have been tempting to take those people and lock them away or something but darn it if they didn't keep getting great results. So Marble and Wood. The highly regular vs. messier put powerful. I soon started using this analogy in computer science and that's what I'm writing about today.

One place this comes up is when programming with promises, or really any notification system even ad hoc ones. In the case of a promise the situation is that you have some work that needs to be done "later" after some asynchronous operation has completed: let's say that you are waiting for some data and when it arrives you're going to validate and extract some values from it, it doesn't much matter. The natural way to do this with a promise would look something like this:

p.then( () => {do your work} )

or in javascript you might write

p.then( function() { do your work } )

In fact you can keep combining these things, if you have more work that has to be done after the first batch asynchronously completes you might end up writing something like this:

p.then( () => {do your work} ).then( () => {do more work}).then( () => {even more work});

Now the next thing that's going to happen is you're going to put this in a loop for your various 'p' read operations that need to happen and presto magico your application is done. And you've fallen into the wood trap.

The reason that I try to avoid this pattern is that what's happened now is that on every iteration you're creating delegate objects and new promise objects that connect the various stages for each operation. But the thing is they're all the same! Despite the fact that all the objects are being handled identically (typical) we get no savings, we are using the most general dispatch mechanism to dispatch the very same code thereby creating far more garbage than is needed. Of course each one of these promise chains encourages the next guy to keep doing the same thing. You could even add stages that merge, select, batch, whatever you need, more stages, more wood.

The hallmark of the wood pattern is that the same chain of dependencies/computations is re-represented in data repeatedly -- achieving no savings for its sameness. It is the most flexible choice, each datum could be handled seperately in a unique fashion but they probably are not.

How can you make it better?

Well of course a different pattern alltogther might be the right choice, something that looks more like this:

datasource.handler += () => { do your work }

or maybe

w1 = new {worker stage one object }
w2 = new {stage 2 handler};

datasource.handler += w1;
w1.stage2 += w2;

Now this isn't as flexible but it is much more economical. With one fell swoop we're saying that they are all going to be handled symmetrically and we do not have allocation cost per datum.

Even if that kind of refactoring isn't possible just this simple thing might be very helpful.

var d1 = () => {do your work};
var d2 = () => {do more work};
var d3 = () => {even more work};

p.then(d1).then(d2).then(d3); // this in your loop

This second form avoids creating a whole new delegate/closure in each loop iteration so in some sense it's at least partly marblized.

Now of course this kind of transform isn't universally possible but if you start by saying "what are the rules for all my data" and then encoding that then you tend to end up in a place where you have low-to-zero overhead costs for each item and you just do the work. If you don't you can easily end up in a place where there most flexible pattern is being used in a very regular way, at great cost.

I've often admonished people to use the simplest programming technique that will do the job to get the best performance. Specifically, don't use virtual methods if non-virtual will do. Don't use interfaces if virtuals will do. Don't use delegates if interfaces will do. This discussion has in some ways been a rehash of that point. But then I always found that grounding a concept in an example is helpful so hopefully you all found it worth the read.

P.S. I'm putting the "wood" programming technique in a bad light here but it has its place. And that doesn't mean I'm down on "wood" physics. I happen to think QED is every bit as cool as cool as relativity :)

Ngen or not? The rules haven't changed very much since 2004

ricom — Tue, 31 Jan 2012 20:53:00 GMT

I still get questions that amount to "should I ngen my <something>" from time to time and the best answer I can give is still "it depends." I wrote this article many years ago, and I'd say it's still pretty accurate: http://blogs.msdn.com/b/ricom/archive/2004/10/18/244242.aspx

Essentially the situation is this: if you ngen your IL then of course the jit will not have to run but you will have to do more I/O because the compiled code is bigger than the IL. If that I/O is likely to be cached because either:

The code is going to be loaded "a goodly amount of time" after ice cold startup and it's commonly used so superfetch is likely to fetch it, or
The code is in a library shared by many programs and so only one of those programs at most will have to actually read it at full cost

then it's likely that ngen will help you overall.

If that's not the case then ngen isn't likely to help you. But really you need to measure for yourself.

Remember also that the code generation for ngen'd binaries is going to tend to optimize for maximum sharability which may come at a (typically small) cost in raw speed so that's a factor as well.

The most common framework DLLs, like mscorlib, system.dll and friends, tend to be get the most benefits from ngen. Single-use application libraries and executables tend to get the least benefit.

It's really hard to say more than that with any kind of precision.

Improving .NET Application Performance and Scalability

ricom — Wed, 11 Jan 2012 22:41:21 GMT

This series can still be found here:

http://msdn.microsoft.com/en-us/library/ff649152.aspx

While some of the content is stale since it refers specifically to .NET 2.0 I think all of the conceptual content broadly applicable (even beyond managed code systems to software systems generally.)

If you're looking for it some time in the future you can find it again easily with bing. I doubt that they'll get rid of it entirely for a very long time.

Performance and Design Guidelines for Data Access Layers

ricom — Tue, 10 Jan 2012 21:47:00 GMT

Many problems you will face are actually the building data access layer, sometimes thinly disguised, sometimes in your face; it’s one of the broad patterns that you see in computer science – as the cliché says: it keeps rearing its ugly head.

Despite this, the same sorts of mistakes tend to be made in the design of such systems so I’d like to offer a bit of hard-won advice on how to approach a data access problem. Mostly this is going to be in the form of patterns/anti-patterns but nonetheless I hope it will be useful.

As always, in the interest of not writing a book, this advice is only approximately correct.

The main thing that you should remember is that access to the data will take two general shapes. In database parlance you might say some of the work will be OLTP-ish (online transaction processing) and some of the work will be OLAP-ish (online analytical processing). But simply, there’s how you update pieces of your data and how your read chunks of it. And they have different needs.

At present it seems to me that people feel a strong temptation to put an OO interface around the data and expose that to customers. This can be ok as part of the solution if you avoid some pitfalls, so I suggest you follow this advice:

1. Consider the unit of work carefully

There are likely to be several typical types of updates. Make sure that you fetch enough data so that the typical cases do one batch of reads for necessary data, modify the data locally, and then write back that data in a batch. If you read too much data you incur needless transfer costs, if you read too little data then you make too many round trips to do the whole job.

You may have noticed that I began with a model where you fetch some data, change it locally, and write it back. This is a fairly obvious thing to do given that you are going to want to do the write-back in probably a single transaction but it’s important to do this even if you aren’t working in a transacted system. Consider an alternative: if you were to provide some kind of proxy to the data to each client and then RPC each property change back to the server you are in a world of hurt. Now the number of round trips is very high and furthermore it’s impossible to write correct code because two people could be changing the very same object at the same time in partial/uncoordinated ways.

This may seem like a silly thing to do but if the authoritative store isn’t a database it’s all too common for people to forget that the database rules exist for a reason and they probably apply to any kind of store at all. Even if you’re using (e.g.) the registry or some other repository you still want to think about unit-of-work and make it so the each normal kind of update is a single operation.

Whatever you do don’t create an API where each field read/write is remoted to get the value. Besides the performance disaster this creates it’s impossible to understand what will happen if you several people are doing something like Provider.SomeValue += 1;

2. Consider your locking strategy

Implicit in the discussion above is some notion of accepting or rejecting writes because the data has changed out from under you. This is a normal situation and making it clear that it can and does happen and should be handled makes everyone’s life simpler. This is another reason why an API like Provider.SomeValue = 1 to do the writes is a disaster. How does it report failure? And if it failed, how much failed?

You can choose an optimistic locking strategy or something else but you’ll need one. A sure sign that you have it right is that the failure mode is obvious, and the recovery is equally obvious.

I once had a conversation with Jim Gray where I told him how ironic it was to me that the only reason transactions could ever succeed at all in a hot system was that they had the option of failing. Delicious irony that.

Remember, even data from a proxy isn’t really live. It’s an illusion. The moment you say “var x = provider.X;” your ‘x’ is already potentially stale by the time it’s assigned. Potentially stale data is the norm, it’s just a question of how stale and how do you recover. That means some kind of isolation and locking choice is mandatory.

3. Don’t forget the queries

Even if you did everything else completely correctly you’ve still only built half the system if all you can do is read and modify entities. The OLAP part of your system will want to do bulk reads like “find me all the photos for this week”. When doing these types of accesses it is vital to take advantage of their read aspect. Do not create transactable objects just bring back the necessary data in a raw form. Simple arrays of primitives are the best choice; they have the smallest overheads. Do not require multiple round-trips to get commonly required data or the fixed cost of the round trip will end up dwarfing the actual work you're trying to do.

These queries are supposed to represent a snapshot in time according to whatever isolation model your data has (which comes back to the requirements created by your use cases and your unit of work). If you force people to use your object interface to read raw data you will suffer horrible performance and you will likely have logically inconsistent views of your data. Don’t do that.

One of the reasons that systems like Linq to SQL were as successful as they were (from various perspectives I think) is that they obeyed these general guidelines:

you can get a small amount of data or a large amount of data
you can get objects or just data
you can write back data chunks in units of your choice
the failure mode for read/writes is clear, easy to deal with, and in-your-face (yes, reads can fail, too)

Other data layers, while less general no doubt, would do well to follow the same set of rules.

Performance Guidelines for Properties

ricom — Mon, 19 Dec 2011 20:27:00 GMT

I can’t say I’ve asked the framework guidelines folks about this but I’m fairly sure there would be a lot of agreement from the guidelines gurus; so in the spirit of approximately correct advice I give you Rico’s Guidelines for Performant Properties.

I should start by saying I wish more people just used fields instead of committing these terrible sins with their properties. If we had the notion of a type-safe, read-only field the world would be a better place. Alas…

So, you’re using a property, the most important thing to remember is that it will seem very much like a field in all ways. It looks like a field and feels like a field, people will expect it to perform like a field. So with that in mind:

Accessing the property should not allocate any memory; people are going to use this thing in loops walking over whatever data structure you are presenting, they expect that this data is sitting around and being accessed.
Accessing the property should not “synchronize” – if there is any locking to be done it should be done at a higher level than a single field, by the time you’ve acquired whatever object has the property on it, it should already be safe to read it.
Accessing the property should not do any I/O, especially not any network I/O, again see above, by the time you’re reading the property the object offering it should have done whatever I/O was needed.
Accessing the property should not be an operation with complexity greater than O(1) – that means no loops. At all. You could haggle me as high as O(lg(N)) if we’re talking about an property that is an “indexer” but no higher.
Accessing the property should not have side-effects (i.e. it's strictly a read operation, it changes nothing)

What you’re left with is you can use your object state and the argument (if an indexer) to do a constant-time lookup, or log-time at worst, in an already existing data structure and immediately return the result. That’s it.

Why?

Pit of Success is the only reason you need. In my opinion, patterns more complicated than the above are doomed to fail. If you allow say network access, an RPC or the like, on each property fetch, then you promote things like field-at-a-time access to remote data. Not only is this astonishing to users of the API it is incredibly inefficient. If you follow the rules above you soon realize that you must allow some kind of query to get the data you need and then you can use properties all you like to access that data. That is a much better pattern, it leads to success.

Remember that if you allow things like I/O or synchronization you can easily get property access times that are measured in milliseconds, this simply will not do. A typical interactive scenario might have a budget of say 100ms for prompt response and it might require access to several dozen properties just to paint the screen properly. This has several issues:

If property access is say 1 millisecond then accessing 10 properties (not at all crazy) would be 10% of your overall budget – that’s nuts!
The interactive code must be able to use your properties with confidence, that means they cannot block and should not be subject to large variability, else you get UI that is non-responsive for extended periods of time

Frequent access to properties is very common and therefore must be astonishingly cheap if only to keep a handle on the Joules your program consumes – an increasingly important metric for modern platforms.

From a time perspective, I like to see properties that have access time measured in nanoseconds. Let’s say 10^-8s. I can reasonably see systems with tricky indexing that might go has high as 10^-7 or even 10^-6. When your time is getting to be 10^-5 or larger, you really need to start thinking about your design. That’s too much work to put inside a thing that looks like a field access. Give your users a fighting chance to understand where their costs are and let them query for a batch of results, probably asynchronously and then access them quickly when they arrive. That’s going to be a successful pattern.

It goes without saying that under these conditions, the flexibility to version/replace/subtype virtual-call properties is actually illusory. If you were to try to do something new and costly, or even just very different in a subtype you would likely cause all manner of problems for callers. So while you do get to change things a somewhat, you cannot and must not, use that flexibility to access new and different subsystems or the like inside of a method that is supposed to be as small as a getter. You are necessarily very constrained.

When I consider all of this I arrive squarely at the conclusion that more fields and fewer properties would be better. Don’t hide real work inside a property and don’t use them as a synchronization tool. Once you decide that you need a fetch/read pattern you soon find that that reading thing can be very simple/fast indeed and that perhaps fields were just fine after all.

Something to think about anyway.

Going to Internet Explorer

ricom — Sat, 12 Nov 2011 17:21:00 GMT

Many of you have commented that I've been kind of quiet lately. This was largely a reflection of my job, I was working on an internal team that I could not write much about for the last couple of years. I had a great experience there but that trek is now coming to an end. In a few days I'll be working on Internet Explorer and back in the thick of the public facing products. IE is a very exciting place to be and I'm sure it will be loaded with issues I can write about that will be of general interest without giving away product plans and so forth.

I hope to be writing a lot more in the coming months.

Thank you for your support over the years :)

Measure!

ricom — Tue, 20 Sep 2011 19:41:12 GMT

I know I haven't posted for a while; I'm hoping that will change soon. However in the mean time my daughter forwarded me this link because she thought it was cute.

The thing is, she didn't realize it should be my theme song.

http://www.youtube.com/watch?v=MMsQYjYlBEo

It's rule #1

http://blogs.msdn.com/b/ricom/archive/2003/12/02/40779.aspx

Now even Sesame Street agrees! It must be right!

Less Loosely Coupled Than Meets The Eye

ricom — Tue, 28 Sep 2010 04:35:00 GMT

I don’t know that it is possible to write anything like a unitary software system in a way that is truly loosely coupled. It’s not that you can’t make boxes and lines that are cleanly separated in all sorts of pretty ways, though that’s hard enough. The problem is that even if you manage to do that, what you end up with is actually still pretty tightly coupled.

What do I mean?

Well, let me use Visual Studio as an example. It’s made up of all kinds of extensions which communicate through formal interfaces. Putting aside the warts and just looking at how it was intended to work it seems pretty good. You can make and replace pieces independently, there is a story for how now features light up; it’s all pretty good. Yes, it could be better but let’s for the moment idealize what is there. But is it loosely coupled, really?

Well the answer is heck no.

These systems all share resources in fundamental ways that actually tie them together pretty tightly. It starts with the address space, each extension can be victimized by the others, and it goes on. In fact every important resource on the machine is subject to direct interference between nominally isolated extensions. Is this avoidable?

I don’t think it is actually. I think unitary software systems are fundamentally tightly coupled when you look at them through a performance lens. And that is why I’m so often telling people that they need to look at their overall system and see if, when it’s all put together, it has any hope of working the way you want. Two subsystems that both (loosely) use 2/3 of the L2 cache are going to use 4/3 of a cache… that’s not good. There may be no lines between them in the architecture diagram but they are going to destroy each others ability to work.

So, does that about wrap it up for agile then? Should we just waterfall our way to victory?

Hardly.

Everything you’ve ever read about requirements changing and not over-designing systems that are loosely specified (much less loosely coupled) still applies, but just because you’re going agile and you want to keep your flexibility is no reason to not think about what it’s all going to look like when you put it all together. Some kind of blend is needed.

If you do run into problems that loose architecture is going to help you. But waiting until you’re done and counting on your agility to save you at the finish line isn’t so smart either. Agile development doesn’t prohibit you from planning out the parts that need planning and understanding your key constraints. In fact I think it encourages this – knowing your constraints keeps you honest and lets you make adjustments smartly as you go.

Performance is never loosely coupled. Don’t be fooled by your diagram, the profiler doesn’t know where your little boxes and lines are. Trust me :)

Debugging Multi-threaded Applications: Some Tidbits

ricom — Fri, 04 Jun 2010 20:06:00 GMT

I was lamenting that we haven't really done terribly much to make multi-threaded debugging easier in say the last decade and I was fortunate enough to be able to have a conversation with Brain Crawford about it. Brian is a long-time friend an colleague, we worked on the VC++ debugger years ago, he's the lead architect for all things debugger these days. Anyway, he gave me this succinct information about VS2010 and was kind enough to allow me to share it with you all.

In his own words Brian writes:

While I agree that there is more we could do to make multi-threaded debugging even better, VS 2010 has a number of features that help with multi-threaded debugging for both native and managed code:

Stepping has affinity with the thread being stepped. Whatever thread is active (has the yellow IP marker next to it in the Threads window) will be stepped and no other threads will complete the step started on that thread. However, by default, breakpoints will fire on any thread, so you can step one thread and hit a breakpoint on another.
Any time you started execution (step or go) on a thread and execution stops on another thread (e.g. you hit a breakpoint or exception), there will be a small blue marker overlaid on the yellow IP arrow shown in source code to indicate a thread switch occurred.
Per thread breakpoints can be created by setting a breakpoint and then right clicking on the breakpoint glyph and choosing Filter. In the Filter dialog, you can enter an expression like “ThreadId=5” to restrict the breakpoint to a particular thread.
The threads window in VS 2010 has a number of features to help manage multiple threads:
- Threads can be flagged manually so it’s easy to find the subset of threads you’re currently working with.
- Threads can be flagged according to just my code or by particular modules.
- Call stacks for each thread can be viewed by hovering over the Location cell or the call stack can viewed inline by expanding the Location cell (so multiple call stacks can be viewed simultaneously).
- Threads can be grouped according to different criteria, such as process, name, category, flag, etc.
- Threads, including the call stacks for each thread, can be searched using the search box in the window. For example, if you’re looking for all threads with Foo on the stack, enable Search Call Stack, and type Foo in the Search box.

For multiple threads running the same/similar workload, the new Parallel Stacks view is very useful. While primarily designed for use with the parallel libraries and runtime, it can also be useful for traditional multi-threaded debugging. It consolidates threads with common stacks into a “cactus stack” view. I don’t know the kinds of scenarios you’re debugging, but it might be worth checking out. Here’s some additional info on this view: http://msdn.microsoft.com/en-us/library/dd998398.aspx

Thank you Brian, and please get a blog soon :)

A few words about Micro-Benchmarks

ricom — Tue, 27 Apr 2010 00:57:00 GMT

It’s been a long time since I included my “this discussion is only approximately correct” disclaimer so I’ll just preface it here. In the interest of space and clarity, this discussion is only approximately correct. OK, now we can move on…

I love micro-benchmarks.

Really.

I rely on micro-benchmarks to help me understand what is going on in my system on a day to day or even build to build basis. Understanding which micro-benchmarks are really vital is essential to keeping your performance under control. But you have to be careful not to do more than this.

One thing you should never do is use your micro-benchmarks to drive prioritization of your performance problems. Remember, a micro-benchmark’s mission in life is to magnify some aspect of your programs performance. Because of this anyone can readily generate a micro-benchmark that makes any given issue seem like a major problem. Do not be fooled! Micro-benchmarks in-and-of-themselves tell you almost nothing about what is important. What they help you do is manage and track the important things once you have discovered them with representative workloads.

Once you have established which micro-benchmarks are key performance drivers in your system, you can use this to help you make progress on the large problems more quickly, but you must be careful to re-validate. Seemingly positive-looking changes in the micro-benchmark can easily be negative under your real workload because the micro-benchmark is often much less resource constrained. For instance, you could make a change that doubles your metric at the expense of addition L2 cache usage. That might be a good thing but it could also be a disaster, the collateral damage you would do to the rest of the program could easily erase all those gains.

So, should you use these quick running scenarios? Yes, but certainly not exclusively. And not as the primary planning tool, but as a planning aid.

My History of Visual Studio (Epilog)

ricom — Wed, 14 Apr 2010 19:19:00 GMT

Visual Studio 2010 Launched on Monday. Wow! It’s HUGE. A major round of congratulations are in order for everyone involved, not just on the Visual Studio team but also on the Frameworks team and the supporting teams and of course the customers whose feedback was so vital to the success of the product.

I’ve already written a lot about VS2010 previously in the series and I don’t want to go over all that stuff again. In my last history posting, back when Beta 2 came out, I covered all the major things that we wanted to get into this version of Visual Studio from an architectural perspective. You could look down that list and get a feel for whether or not the release was a success from that perspective. I think key architectural features like a full WPF surface, MEF extensibility, extension management generally, and of course multi-targeting were delivered. General purpose multi-targeting is a huge thing because it means we don’t have to do a VS SKU release for every new target platform that comes out – that was a critical thing for our own sanity. And WPF has certainly (finally?) proved itself as a viable platform for a flagship application. We set out on a multi-release remodeling plan and I think we delivered an excellent first-installment.

You might be wondering why I was so quiet during the home stretch. I’m afraid my perspective on the VS RC and RTM milestones was an outsider's. The Beta 2 push was my last major hands-on involvement with the product; probably the simplest diagnosis is that I burned myself out at that point. In any case I took a vacation after that Beta with the goal of returning to find a new challenge. I’d been in the Developer Division about 7 years, for the second time, and I like to move around about that often, so it was time. I’d love to take some credit for fixing the daunting memory leaks that were left in Beta 2 but I’m afraid all credit for that goes to my capable colleagues to whom I am forever grateful for taking this particular release across the finish line.

Some things were lost on the way in the name of getting the necessary stability and basically locking things down. It’s kind of a sad thing when that happens but it’s a necessary thing – they’ll just have to wait for the service release. Nearly ready to go things that just needed a little more bake time are a great thing to have in your pocket at that point, but there are a few especially delightful things that I wanted to mention, sort of unexpectedly delightful, for me anyway.

We started with an ambitious goal to make a major dent in Visual Studio’s setup experience. A lot of what we wanted to do ended up having to be pared back and I feared we would end up making little progress but in the final analysis I think the setup experience is substantially better. Seeing tweets on how people had successfully installed in less than 30 minutes was very nice. More than I hoped for in the darkest hours. My own experiences during the beta were the best of any VS release I’ve ever installed – and that’s a lot of releases.

I feared the historical debugging feature, IntelliTrace, was going to be too bulky to use all the time but that’s not proving to be the case at all. I’m just getting used to having it available all the time and it’s proving to be much more of a game-changer than I ever imagined. I wrote more about the history of that feature previously so I won’t go on other than to give some kudos to those that worked hard to make that feature reality.

I didn’t think TFS was going to be so universally important but the ease with which TFS Basic can be brought online was just staggering. In just a few minutes I configured my own home TFS installation – it’s great to have that kind of quality ALM system so easily available to everyone.

The rate at which new extensions are appearing in the gallery is incredible. I’m so glad to see this hub creating energy for all our customers, but there was one extension in particular that made my day. Dmitry Kolomiets produced, I think, the first Solution Load Manager extension. Why do I care so much about that particular extension? Well there’s a story there: from very early on in the history of VS2010 I was espousing a philosophy that we could not universally load every project in a solution and achieve our memory goals – many solutions are just too large and it’s entirely normal to not need every project. But there is no reasonable way to manage solutions. Unfortunately there was just not enough time to make this feature a reality – but there was enough time to add the necessary APIs so that a 3rd party could make us load less and manage what was loaded. With that support they could build their own UI to drive the experience. It was hoped that the performance benefits from such a system would motivate the creation of a variety of managers with different policies – potentially tied in to the great refactoring engines engines already out there (e.g. CodeRush and Resharper). But it was just a hope. Now there is one!

I could go on for ages. The diagramming support and of course our uses of it. Integrated Silverlight support, ASP.NET 4 with MVC2 (read about it on Scott’s Blog). Language improvements in C#, VB, and C++. Did I mention Windows Azure support as well? Sharepoint support? F#? It’s huge.

This was only “My” History of Visual of Visual Studio. For me this is the "epilog" but for everyone else, VS2010 out the door, it’s time to begin your own History.

Congratulations and Enjoy!

Variability in Benchmarks

ricom — Wed, 07 Apr 2010 22:43:00 GMT

I’m not especially a great fan of micro-benchmarks, they’re handy as a compliment to the larger tests but I often find that they lead to confusion because they are interpreted as reflecting reality when nothing could be farther from the truth. The fact is that micro-benchmarks are intended to magnify some particular phenomenon to allow it to be studied. You can expect that normal things like cache pressure, even memory consumption generally, will be hidden – often on purpose for a particular test or tests. Nonetheless many people, including me, feel more comfortable when there is a battery of micro-benchmarks to back up the “real” tests because micro-benchmarks are much more specific and therefore more actionable.

However, benchmarks in general have serious stability issues. Sometimes even run-to-run stability over the same test bits is hard to achieve. I have some words of caution regarding the variability of results in micro-benchmarks, and benchmarks generally.

Even if you do your best to address the largest sources of internal variability in a benchmark by techniques like controlling GCs, adequate warm-up, synchronizing participating processes, making sensible affinity choices, and whatnot, you are still left with significant sources of what I will call external variability.

The symptom of external variability is that you’ll get a nice tight results from your benchmark (a good benchmark might have a Coefficient of Variation of well below 1%) but the observed result is much slower (or, rarely, faster) than a typical run. The underlying cause varies, some typical ones are:

Scheduler variability – the scheduler makes an unfortunate choice and sticks with it

The entire run is much slower than normal

Uncontrolled background activity external to the test

Part of the run is slower, maybe several consecutive test cases in a suite

Interference from attached devices

Incoming network packets or other sources of interrupts

These problems are often not the dominant sources of variability initially but as the benchmark is improved, controlling for other factors, they become dominant.

Statistically the underlying issue is this: repeated benchmark measurements in a run are not independent: a phenomenon affecting a particular iteration is highly likely to also affect the next iteration. A closer statistical model might be independent measurements punctuated by a Poisson Process that produces long lasting disruptions. "Waiter, there is a fish in my benchmark."

My experience with this sort of thing, and I expect the battle-scarred will agree, is that reducing external variability is an ongoing war. There isn’t really a cure. But it does pay to not have hair-trigger reflex on regression warnings, and to screen for false positives in your results.

My Annual Personal Posting

ricom — Tue, 06 Apr 2010 06:26:00 GMT

I try to keep the content on this blog strictly professional (although sometimes it's like an editorial but at least it's topical) About once a year I break down and write something that's basically just some personal thing I felt like sharing. This is my link for this year.

A couple of years ago my family became involved in animal rescue (mainly cats and dogs). It all started when my wife asked me if we could get a third dog and I told her that was a bad idea -- "too many" I said. But I thought fostering would be a good thing. Ha! I had no idea I was saying "no" to three and now sometimes we have nine in the house!

Still it was the best stupid thing I ever said.

One of the organizations our family is associated with is Furbaby Rescue here in Washington. If you are a pet person, or think you might like to be one some day, I invite you to watch this.

http://www.youtube.com/watch?v=YExxNEpEdJY

Warning: it's quite sad as are many of rescue stories. But I hope this one has a happy ending when all is said and done.

Why you really want to avoid catching and rethrowing exceptions

ricom — Mon, 05 Apr 2010 19:18:00 GMT

I like processes that simply drop-dead fail when they have an unrecoverable fault. Trying to continue is often dangerous and unlikely to actually help anyone. This policy is all fine and well but in all cases it is vital that the “death” stack of a process be the true problem, not a generic error handler. Those kinds of handlers ruin the lives of developers who are trying to analyze error logs, and effectively make services like Watson useless. I can’t stress enough how critical it is to NOT mess with the failure stack.

I’ve seen many of these in the wild, including of course Microsoft code, and it means that instead of getting a nice actionable stack in the log (which would have been a Watson report in the field) you have to go back to the code and try to repro. In the case below I stared at the source for a while trying to figure out what was likely wrong and it turned out the true problem was quite distant from the likely candidates.

2010/04/05 11:02:14 AM BEGIN Peek_Range TEST
2010/04/05 11:02:14 AM ---- Exception in [StackTest] thrown type=[TestCodeException], eip=[00000004`4716C93E], message=[Index (1) is out-of-range [0..1]!]

That was the failure right there, and that stack was the one I needed! Nothing after that should have happened... But... execution continues until finally....

2010/04/05 11:02:15 AM Test completed exceptionally [Peek_Range]: Index (1) is out-of-range [0..1]!

The stack reported at this point is useless… the real problem was millions of cycles ago. I'm not even including the stack from the log because it's just stupid.

The real stack can be extracted with a repro (at great pain)

And it ends up looking something like this (cleaned up)

StackTest!TestUtilities::ParamCheckIndex<int32>
StackTest!CollectionTestUtils::TestUtilities_SimpleList<int>::RangeCheckIndex
StackTest!CollectionTestUtils::TestUtilities_SimpleList<int>::get_Item

The actual problem is in the test code! The only useful information from the original log was basically the test that I should run. Thank goodness it’s a 100% repro.

VSX Keynote posted

ricom — Wed, 09 Dec 2009 01:55:00 GMT

Just a quick plug, if you missed the VSX conference you can see it all online on Channel 9. Some great talks including my keynote speech on VS Futures.

VS2010 Beta2 performance and other issues

ricom — Wed, 28 Oct 2009 20:08:00 GMT

Just a few words of encouragement today: I can't emphasize enough how valueable your feedback is to us at this point; no matter how hard we try we simply cannot duplicate the diversity that is the real world. We need to know what's really working out there -- what works for us isn't enough. Whether it's performance, stability, or any other issue, your feedback will help us to "get it right" for the release.

I've seen some super-positive comments about product performance and I've seen some negative ones. Your comments will help us to figure out which areas we've missed, what combinations aren't working, what hardware are we using poorly, or any other goofy thing we might have done.

You can use http://connect.microsoft.com/ or you can provide comments here. We're even watching twitter and the blog-o-sphere generally but those are less reliable.

Brian Harry is also reaching out http://blogs.msdn.com/bharry/ and I know all the other executive bloggers have their ears on -- and they'll get help. Connect bugs are the best of all but don't be shy, we'll take the feedback however we can get it.

Thank you!

Inside Visual Studio Beta 2 - Performance and Reliability

ricom — Tue, 20 Oct 2009 19:13:00 GMT

After my keynote speech yesterday at the Development Tools Ecosystem Summit Charles Torre caught up with me and we made this video. Hot off the presses: Rico Mariani: Inside Visual Studio Beta 2 - Performance and Reliability

It's super-exciting :)

If you missed the keynote you can get virtually all the content by watching this video and reading the last entry in my History of Visual Studio series My History of Visual Studio (Part 10, final).

My History of Visual Studio (Part 10, final)

ricom — Mon, 19 Oct 2009 21:00:00 GMT

[All the other Parts: History of Visual Studio]

[Visit the Microsoft Visual Studio 2010 and .NET Framework 4 Beta web site for the latest info]

Visual Studio 2008 Winds Down, Visual Studio 2010 Begins

Just as things were starting to wind down on VS2008, I got a new assignment. I was going to be Chief Architect of Visual Studio. Wow. So here this series takes a bit of a turn because “my” history is quite a bit different at this point – I actually stopped working on the current product at all, and much as I would have liked to write about what I was doing back then, you can imagine there wasn’t a whole lot of enthusiasm about starting blog discussions on what we should do in “Dev10” (the term “Hawaii” never stuck and was soon abandoned) while we were trying to launch VS2008.

Remodeling

From a technical perspective, the idea of VS2010 was to begin a remodeling process that would span several versions. Making key investments that would help long-standing problems and at the same time offer some immediately compelling features as well. I gave a lot of thought to what enabling steps we would have to take to make our experience be something great in say three versions. It was hard to see exactly what that future might be, so I tried not to think about specific features at all. I had seen all sorts of “flashware” concepts for things we might do and I tried to think about what key problems we would have to solve to implement things of that nature. Using all that, and what I already knew, I came up with a list of fairly low-level considerations for a “VS of the Future” that I thought we should work on, and why.

I think the actual document I created is pretty dry, it’s full of observations about the size of processor caches, the number of cores, the memory wall, the locality of our data structures, blah, blah, blah. It goes on like this for pages, but I think I can summarize it into a couple of major themes.

Concurrency

The first major theme is concurrency, and the need to increase our ability to use it successfully. You might think that with all of those threads Visual Studio spawns that there is a great deal of parallelism but, for the most part, it’s only superficial (there are exceptions), what happens more often than not is that those threads run something like relay race with only one thread having the baton at any given moment. One key reason for this is that due to the single-threaded history of our code-base, and indeed many Windows applications, a lot of the objects are tied to the STA that is the main thread.

To create opportunities for concurrency you have to start to disentangle the important data structures from the STA and give them better access models that are friendly to concurrency. A great example of this is the new editor in VS2010. It allows you to access editor buffers via immutable micro-snapshots. Previously, if you wanted to do a something like say background analysis on a live text buffer, you had to do crazy things like copying the entire editor buffer on every keystroke. Concepts familiar to database programmers, like isolation, need to be present in the key structures of any concurrent sytem.

The STA concerns are a bit like a plague, they can cause a massive “entanglement” of data processing and presentation. Transitions into or between STAs can result in unexpected message pumping, leading to bizarre re-entrancies which in turn result in astonishing bugs that are very hard to find and fix.

Additional entanglements occur between processes trying to create parent/child relationships between their windows where the single pump effectively fuses together certain STAs – a necessary step given typical application assumptions but a total bug nightmare and a parallelism disabler.

Ultimately the most important thing is to make it as easy as possible to move work off of the UI thread. I think it’s vitally important that there is a single thread responsible for the UI – it’s the sanest way to preserve the order of operations the user initiated – the causality from their perspective if you will – but as their requests become non-trivial you need the flexibility to do them asynchronously.

There are two major kinds of background activity that come up in an IDE; it is easiest to illustrate these two needs by example:

Search for “foo”
- This can be done in a totally asynchronous fashion with different “threads” proceeding in parallel against different editor buffers or other searchable contexts. All the foos light up as they are discovered for instance.
- Foos that are not visible need not be processed, so this sort of parallelism has the option to be “lazy”
Rename “foo” to “bar”
- We can still do this asynchronously but now there is an expectation that we will eventually do it all and of course we must do all the foos not just the visible ones
- Generally this requires progress, and a transaction
- It can fail, or be cancelled, these are normal things

Both of these represent two cases where we can use a “coarse” parallelism model (e.g. one file at a time, independently). Of course we also would like to do fine-grained parallelism in places and in fact searching could be amenable to this as well (e.g. search odd lines with one thread even lines with another). Fine-grained parallelism has the opportunity to keep data locality good because you can access the data in one stream rather than access disparate data streams all at once, at the expense of more complex scheduling. Locality is good for performance, more threads are good, sounds good but you need some fundamentals to make any of this happen:

Limited memory usage in the face of many threads running
Ability to safely access the data in slices that are durable and limited in span
Isolation from changes that happen during the operations
No STA requirements in the data access and a asynchronous way to deliver updates to the UI

Putting the text editor on a plan that can satisfy these needs is itself a big step forward but on a long term basis it would be nice to do many things on this plan.

New UI Model

Again, I went on at great length about this in my original memo but I’ll treat this topic even more briefly than concurrency. The long and the short of it is that your average developer’s machine has the equivalent of a few 1990s supercomputers for a GPU (we could haggle over how many) and it’s important to be able to get good mileage out of that unit. It’s hard to imagine achieving those kinds of successes at reasonable cost using the good old WM_PAINT + GDI model – a high quality retained mode graphics system is basically a must.

In addition to a making it easier to use the GPU, there is also a tremendous desire to do good separation between the UI and the data elements of your design. Not just because this is architecturally sensible and clean but, as I discussed in the previous section, because to do otherwise significantly hinders your ability to achieve a good degree of parallelism. The retained mode graphics model actively encourages badly needed separation and you all know I love the Pit of Success.

Ultimately that leads to the choice of WPF for presentation in Visual Studio because the long term choices are to either use that or roll our own. However WPF and Silverlight evolve in future frameworks, and you’d have to be crazy to think they won’t, we’re better off using WPF notions than relying on the past, or worse, on something of our own making.

What came of all this?

I’ve deliberately avoided going into feature discussions so far, just as I did in my original memo, but ultimately we had to choose particular product areas where we would take these notions and invest in moving them into a good direction. That’s the point of the “remodel” after all.

These are some of the major initiatives that got an investment; they had impact throughout the entire product.

Multi-targeting

A new framework, and in particular a new CLR, essentially mandated that we do this work. We had been able to do this more cheaply in previous releases because all the target frameworks shared a CLR (the CLR had been on v2.0 even though the framework moved on to 3.0 and 3.5). This base level of compatibility allowed a simpler “onion-skin” kind of model to deliver an effective solution in Orcas – that wasn’t going to work in Dev10. In Dev10 we had to be able to target CLR 2.0 and CLR 4.0 and we had to do it in such a way that any given solution might use the related frameworks in any combination. We certainly couldn’t assume that the mscorlib that the IDE had loaded was the one that the user wanted to target.

Then there’s the matter of tools. There are different compilers for different frameworks and we can’t assume that, for instance, the C# compiler for version 4.0 is the right tool for the job when building a project designed to run on version 3.5. And even if we were super careful with the compilers, what about all the other tools that create outputs, like say resources? Those assets might be in a format that is serialized with the expectation that a particular framework will load it. So no one tool could do the job.

And let’s not forget the native tools, they have all these problems too, though I’ve been using managed examples, you can search and replace the runtime names in my discussion with native tools and libraries and you get basically all the same phenomena.

So in VS2010, based on the project target, we have to select the right libraries and tools to create all your outputs and we have to be able to vary these on a project by project basis. That sounds good.

That’s just the start.

The build chain is in some ways the least of our problems, what about all those designers? What about Intellisense? As you change focus from from file to file, depending on the framework you’re targeting in that file we need to show you Intellisense for just the methods you can actually use in that context. Likewise designers should include only choices that are reasonable for the current target, so for instance, the items in the Winforms designer toolbox should not include controls that are available in the version of the framework the IDE happens to be using, but rather the ones that are available on the desired target, and of course those are not likely to be the same.

Add to this mix the desire to target platform subsets like the “client profile” of framework 4.0 and you’ve got quite challenge on your hands.

I remember a very compelling breakdown that the teams used to explain the levels of success to management (and me)

Good: Designers and Intellisense prevent you from making mistakes; you see only things that you can really use for your target.

Not Good: Designers didn’t warn you, but at least you got a compile time error that was helpful and can be corrected.

Bad: There was no compile time error but when you ran the code you could readily see that it had some target specific aspect that shouldn’t be there and you could correct the code to avoid the problem.

Ugly: As above, but there is no reasonable diagnostic information and/or no reasonable way to fix it.

Obviously we want all aspects to be “Good” and if that can’t be achieved then as few as possible cases in each successively lower category. I think you’ll find that as of Beta 2 we’ve been very successful.

New Editor

Without going into a full discussion of the problems with the “old” editor it is worth mentioning these basic facts:

The old system was blocking innovation because of limitations in how it does the drawing – adornments are painful to add
It was “tricky” to use in non-trivial cases, often requiring extreme interventions such as wholesale copying of buffers to achieve stable state, the locking mechanism is prone to mistakes
When combined with its undo-manager it sometimes (always?) highly wasteful in terms of memory
It has assorted algorithms that do not scale well, generally those to do with region tracking – increasingly critical to editor consumers

Many customer performance complaints boil down to lack of scalability in the current editor structure. These issues in turn boil down to non-linear memory usage, and non-linear algorithms, where sub-linear choices are possible, and necessary, to create the proper experience. We get real scalability complaints from customers on an ongoing basis and these problems can’t be solved by “tuning.”

To address these problems we had already built a new editor platform, including visualization and text management services. The design had proven scalability and extensibility and had already shipped (in Blend). It was a very real technology. However, there was a great deal of code in Visual Studio that depended on old editor behaviors.

To simplify adoption of the new editor we created “shim” services that supported the most popular of the old interfaces allowing existing clients to continue to use those interfaces. Since most editor consumers actually use only a subset of the whole API we were able to substantially mitigate the cost of conversion across Visual Studio – shims weren’t a universal solution, but they helped.

Now I say that like it was some easy thing but wow it’s a ton of work even at the “reduced” cost. And the number of shims seemed to keep growing, and growing, and… it took a while to get this under control.

There were other problems. The new editor wasn’t identical to the old, despite being more functional overall it was missing some things – like column select – that would need to be added (it’s in Beta 2). Then there are the complicated editor interactions in cases like those presented by the Web Forms designer – the presence of markup intermixed with a language (like VB or C#) makes Intellisense very tricky. The language service needs to see the file without the markup to provide coloring and Intellisense but of course the user edits the composite file. Getting this to work on the new editor was quite a trick.

Having gone through all this I think I can say that the new editor has a very bright future; it has unprecedented extensibility – everything from text replacement, to object insertion, to HUD–like overlays. In this version we’ve barely begun to tap its capabilities.

New Shell

It would be a mistake to think of the new shell as just a redo of the old in WPF, it’s quite a lot more than that. There is a whole new toolbar system, a whole new menu system, both based on WPF. The docking controls, easily taken for granted, those are all new. There are the bits that allow document windows to completely float for better multi-monitor support. There are all the decorative bits, the window tabs, the output window tabs. It’s starting to add up but these things are only one aspect of the Shell, its UI.

Behind the scenes, there to enable pieces like the new editor that I discussed above, and the project system that will be next, are the Shell Services. You might see screenshots of the new Extension Manager and the Visual Studio Gallery, they are in some ways the face of much of the underlying work: a new deployment model for extensions centered around the VSIX packaging format and the MEF extension model.

But those dialogs really are just the face, to make the systems work there are many invisible things, like the MEF catalog and services that manage extension state. In fact, if you were to visit the Shell Services link I provided above, you could go down that entire list and you would find very few of the major systems that were not affected in this rewrite. From the Start Page, to Intellisense, most services have had significant work. MEF itself lays out a roadmap for future levels of extensibility in the product, simultaneously more comprehensive, easier to use, and in more areas.

That last notion is perhaps the most important one, I’ve said this many times in various speeches but it bears repeating: It’s hard to look at Visual Studio, a product that is substantially ALL extensions, and say “it needs more extensibility” – but it does. The original Visual Studio shell provided excellent extensibility of the core services – VS itself is the proof of that – but what is lacking is general extensibility of the extensions themselves. These extra levels are what we sought to add in Editor and the Common Project System, and hopefully they will pervade the product over time.

“Common” Project System

I’ve alluded to this system several times so far in the document; it’s also an important step on a long-term plan.

Anyone who has created a project type for Visual Studio knows that it can be a serious pain to try to keep up with the Joneses. The flagship languages add new features (ClickOnce is the canonical example) and since there is no central place to put project features, they end up having to be re-implemented in the code for many different project types. The ability to create project “flavors” (described a little bit here) helps alleviate this somewhat but certainly does not provide complete relief.

So, at the start of the cycle we find ourselves in a situation that is something like this:

We’d like to create project system that would provided the needed extensibility so that we could unify many project types
We think it would be nuts to create this thing and then try to change all the projects to that plan in one release
We have the C++ project system which needs to move to MSBuild anyway to be more like the others and gain MSBuild's incremental build and scale benefits

So the solution seems obvious – we’ll start building “CPS” and we’ll do C++ first, that way we have a test case, we don’t have to change the world all at once, and the C++ team will no longer have to be in the business of managing all its own project stuff. The reason that I quoted “Common” is that, at least for now, it’s only the C++ project system – though it has ambitions to be a lot more.

How hard can that be? It’s only an all new project system with virtually all of its attributes encoded cascaded XML files, general purpose editing UI, with pervasive MEF extension points, and enough scale to handle the largest C++ solutions we have. Why it’s triviality itself J

Other Highlights

When I wrote about “Dolphin” I gave a litany of changes, it was an impressive list, but I think I can say with candor that I gave a fairly complete list of the major new pieces of work in that release. I was able to do that because the team was a lot smaller and the whole release could fit inside one person’s head. I’m not even going to pretend that I can write about everything that’s in VS2010 – So far I’ve stayed very close to my own team and the things that were most dear to me – this is only “my” history after all, the complete history would require input from thousands. So even though I can’t possibly be complete, there are a few of my favorite new things that I want to talk about.

The New C++ Code Model

The VS2010 C++ Code Model is essentially a complete redo of the old technology. Now I’m a funny guy to be writing about this because probably more than any other one person I’m responsible for creation of the old technology. All those .bsc files, that was my baby, we came up with the current system when we were working on Minimal Rebuild (previously discussed) and several of my friends had the joy of maintaining that code in the years after I left the C++ team (thanks, you know who you are, and sorry!). The .ncb code was the “no compile browser” and the original flavor of that was created by another fellow who I supervised, no name dropping, this was another case of “give the new guy this really hard project, I’m sure he’ll do fine”. Gee I find myself apologizing a lot in this history.

Anyway suffice to say they were fine in say 1993 but I think it’s possible to do a lot better in 2009 and nobody is happier to see my old code leave the building than me. The new system offers improvements almost across the board, especially in reliability and performance.

Historical Debugging

You know I couldn’t do another entry without talking about debugger technology. Historical Debugging is a feature that’s been incubating in the minds of debugger people probably for as long as they’ve been debugging.

I was the debugger lead in the early 90s and I used to explain the utility of debuggers and debugging tools in this way: Imagine a program with a bug, it has been running along, everything is fine, everything is going wonderful, the flow of execution arrives at a point we’ll call Albuquerque, where it turns right. Now as every Bugs Bunny fan knows, the correct thing to do at Albuquerque is to turn left. The program’s decision to go right has led it down an incorrect path and sometime later we will observe a problem.

Now if we’re very lucky “sometime later” will be very soon, like for instance it might be that we just de-referenced a null pointer and we’re going to take an exception about 2 nanoseconds after the mistake. That’s an easy bug to fix. On the other hand it could be that “turning right” was more subtle – maybe we corrupted a data structure in a minor way and it might be days before we can see an observable effect – that kind of bug is a nightmare.

Finding “Albuquerque” is what I call The Fundamental Problem of Debugging. The debugger provides you with tools (e.g. breakpoints) that allow you to stop execution while things were still good and slowly approach the point where things first went wrong. The debugger likewise provides you with tools to examine the state afterwards, hoping to find evidence of what went wrong in the recent past that will help you to see the origin. The callstack window is a great example of looking at the past to try to understand what might have already gone wrong.

To find the problem, you might start after the failure and try to look back, finding a previously unobserved symptom and moving closer to the original problem or you might start before the failure and try to move forward slowly, hopefully not overshooting the problem by too much. Or you might do a combination of these things. You might add assertions or diagnostic output to help you to discover sooner that things went wrong, and give you a view of the past. It’s all about finding Albuquerque.

Historical debugging directly addresses the Fundamental Problem by giving you the ability to look at a lot more of the past. And it’s far superior to the limited ideas we had back in the 90s for accomplishing the result. I know a couple of people who have had this idea in their head for over fifteen years – it’s great to see it out there for programmers to enjoy. I think you’ll love it, it’s like a drug! J

Help

The Help system was another case where we saw very early on that it should get a remodel. I think the energy around that effort began with a very well written memo from a friend of mine. Frankly the sad truth is that it was pretty hard to find people who had anything nice to say about the Help system in late 2008. I think many users had given up and were just using their favorite online search to find what they needed, which, as a doubly tragic statistical truth, was probably not our search.

The Help team didn’t have an architect so I loaned myself out to them and together we made what I think are some pretty cool plans that have borne fruit. The new system is much lighter weight and is standards based. Taking a cue from MS Office the content files are basically XHTML in a .ZIP file with a custom extension MSHC. This is an archivist’s dream, but it also opens up the Help authoring space tremendously – with no documentation at all you can now just look at one of the archives and pretty much figure out how to get your content anywhere in the system.

For offline viewing, the customer help viewer is gone, replaced with a mini localhost web-server that delivers the help to the browser of your choice from the archives. We tried various indexing strategies for full-text and other searches but ultimately settled on a custom indexer that I wrote in a few days (well, to be fair they did a lot of work on it after I finished my parts, but I wrote the core). The indexer is actually very similar in operation to my old bscmake program, the C++ program database generator – those techniques fit Help content well, and it produces MSHI files.

Importantly, you don’t have to create our index format to publish Help for your topics; the MSHC file has all the information in the XHTML. We ship pre-built MSHI files to save installation time because of the size of our corpus, but you don’t have to. Additionally, the MSHI files can actually be merged as much or as little as desired for even more performance. But don’t merge MSHI’s from different locales because there is no usable universal sort order that looks decent to customers.

A 3^rd party help viewer need not use MSHI at all, you could easily build your own index.

The content itself is delivered in a much lighter weight format, the big tree control is gone. In fact most of the styling for the pages is itself stored in the content as a topic and it can all be updated like any other content. You can have a lot of help on your system, or just a minimal set of your favorite books, your choice.

Overall I’m very happy with how this effort went. We replaced aging indexing technology with something new and much more precise. We removed the viewer application entirely. We drastically simplified the interface between the IDE and help (basically it’s a URL at this point). And it’s faster!

The Benefits of Dogfooding

It’s fairly well known that we try to dogfood all our own products. In this particular release I think there were some nice significant benefits from this practice. I’ll highlight just a few of the ones that I think are the most important here.

.NET Framework 4.0

It’s hard to imagine not dogfooding this in the release given its importance but there were several unexpected benefits for customers. First, the general character of the release was profoundly affected by Visual Studio’s early adoption, and our need to create designer assets for many different framework versions. The effects were felt in everything from binding policy to metadata format. As an architect I was very pleased by these results because in many cases we were able to find and prevent problems on the whiteboard – where corrections are a lot a cheaper.

A secondary benefit came about because Visual Studio 2010 had to implement so many existing COM interfaces in managed code – resulting in a large number of COM Callable Wrapper (CCW) objects. Likewise much new managed code needed to call existing interfaces that were not being converted, hence significant use of Runtime Callable Wrapper (RCW) objects. That’s all well and good, these technologies had existed for years, however, fairly innocuous seeming choices, like “when should RCW’s be cleaned up” were having profound consequences. The additional CLR cleanup code present in past releases introduced unexpected reentrancy that was not present in the strictly native implementation. Importantly Visual Studio gave us fairly easy and important test cases to work with and ultimately resulted in significant improvements in COM interoperability – these kinds of experiences make the Framework better for everyone.

Windows Presentation Foundation

The WPF team had many headaches to deal with thanks to having VS for a customer but I’d like to think WPF also got more than a few benefits. Some of the corrections we asked for were fairly minor – like the toolbar panel’s layout code which was fairly weak, and we have a lot of toolbars. But in other cases there were significant weakness.

I’d say that if there was a theme to our WPF problems it was this: that the complexity of Visual Studio’s use of hybrid traditional Win32 and WPF elements was a new challenge. But ultimately this resulted in a lot of goodness. It’s possible to do good focus transitions now between WPF elements and Win32 elements, something that would make a person’s head hurt. And, perhaps even more importantly, different combinations of hosting WPF in Win32 UI and Win32 UI in WPF were resulting in extra drawing delays. These paths were exercised by the many window combinations present in Visual Studio and the abundance of big display transitions (like switching to debug mode and back). Generally speaking the hosting services got a lot of exercise and were significantly improved as a consequence.

I’ll talk a little bit more about WPF below specifically as it relates to Beta 2.

Team Foundation Server

Of all the technologies that we beat on, I think TFS probably got the beating worse than any other system. For sheer volume of users, size and number of shelf-sets, number of branches, merges, automated builds, reports, to say nothing of bugs, issue tracking, and basically every other TFS feature our entire division put a level of load on our TFS systems that makes me think that anyone with any load that is even remotely normal is going to have no trouble at all. Sometimes it was painful for us, like dogfooding is (and is supposed to be) but it’s going to be that much better for you.

From Beta 1 to Beta 2

I’ve saved the best news for the last section of the last installment. Beta 1 of VS2010 represented some really great progress towards our goals but there were a lot of issues, I’m happy to report that we’ve been hard at work on the most serious ones and I think we’ve got great results for you. I’ve listed just a few of the improvements that I think are most important below:

Beta 1 got many complaints due to “fuzzy text”, in response to this WPF adopted Direct Write to give a much better overall experience. Those changes accrue to everyone (and yes you’ll get lovely text in VS!)
Performance in Remote Desktop situations is much improved
Startup time is significantly reduced in virtually all cases
Raw typing speed is vastly improved, I know that for instance, C# raw typing speed has never been faster
The new editor includes column select, as well as dozens of other underlying improvements that help region management, outlining and so forth
Debugger performance, with and without historical debugging, is much improved, stepping is quite snappy
The Visual Basic project system can now emit an executable while it is compiling a different one, those should yield the best build times ever for VB solutions
Assorted shell changes in important panels, in data caching, in MEF catalog construction, and in style application will help virtual all cases but notably transitions between modes
The new offline Help system is ready to you to use
Find in Files uses a parallel search algorithm, and last, but not least…
The hated Add References dialog begins on a sensible tab and populates the full list of references asynchronously should you ask for them

There are so many improvements I can’t possibly list them all. I lost count of the number of performance issues we addressed, there were probably over 100 performance analysis reports counting only my efforts on the most critical scenarios and I was hardly alone in this!

Generally, looking at the full battery of our performance tests we find that VS2010 outperforms VS2008 in the majority of cases, and, if you weigh the scenarios by importance, VS2010 looks even better than that. Our remaining performance battles are around reducing overall memory consumption, and I’m sure that will continue to be a focus area between now and the final release as we fight for every byte.

Internally, folks are very surprised and pleased by how far we have come in this Beta. I hope you are equally pleased.

To everyone that got this far, thanks so much for reading My History of Visual Studio, and I look forward to your feedback on either the Beta or the series.

[See The Documentary on Channel 9!]

[All the other Parts: History of Visual Studio]

[Visit the Microsoft Visual Studio 2010 and .NET Framework 4 Beta web site for the latest info]

My History of Visual Studio (Part 9)

ricom — Sat, 17 Oct 2009 00:35:00 GMT

[All the other Parts: History of Visual Studio]

In the last posting I talked about the “Whidbey” release, VS2005, but I feel like I left out two really important aspects so I’d like to start this part by rewinding a bit for those two topics.

I had mentioned in Part 7 that some of the UML support in the “Everett” time-frame was starting to foreshadow the release of VSTS. Well, in Whidbey there is no foreshadowing disclaimer necessary – VSTS arrives and I think at that point it’s clear that Microsoft is going to try to have a real offering in the ALM space. I think it’s fair to say that the venerable VSS was not going to be the product to carry us into the 21^st century.

I don’t want to understate the difficulty of bringing just TFS to market, and that’s only one part of the equation, but even giving those hard-working folks a well-deserved nod, I felt like the first VSTS was just a taste of ALM at this time – whetting the appetite as it were. But even so, with so many shops wanting to formalize their practices in some way – even lightweight practices like Scrum – this kind of support was welcome relief. I think those years included a much greater level of attention to the process of creating software than at any other time in my experience.

Just 4 years after its initial 2005 release, VSTS is a huge part of what we deliver – and in 2002 it didn’t even exist.

I started my discussion about the Whidbey release with an observation about how big of an influence Windows Vista was on our tools. I think I should have ended with my single biggest regret about that release because it’s highly related. Despite its huge influence, our support for Windows Vista was lousy in the initial release, like really lousy. It wasn’t until we delivered SP1 and especially the Vista support for SP1 pack that we got to a reasonable level of support. I could make a bunch of excuses but they all sound pathetic to me so I’ll just stick with “It’s my #1 regret.”

OK so much for the downer part of the article; let’s move on to Visual Studio 2008 “Orcas.”

The thing that I most remember about “Orcas” is “MQ” – the Quality Milestone. This is where we spent the equivalent of an entire coding just taking care of nagging issues that otherwise seem to not get handled. I’ll name-drop again since I just linked her video, I’d have to say it was Carol more than anyone else that was keeping us honest about the “debt” we had accumulated in our product and popularizing that meme in our division. Debt is a great way to think about trade-offs: every time you make a choice that isn’t right in the long term, any short-cut, accumulates some debt. Any bug you choose to defer, that’s debt. Some debt you should write-off, that fix just isn’t happening, some you should address, but always you should be aware that you’ll have to deal with it sooner or later, and it may as well be sooner.

That mind-set, which was the focus of MQ was pervasive for the whole release, and I think it shows. From a stability and performance perspective VS2008 was universally better than 2005 – a very hard thing to achieve in any major release with significant additions.

I think the “star” of VS2008 was Linq – my favorite flavor was “Linq to SQL” which I’ve written about at some length before. Perhaps the most amazing thing about Linq is that its introduction into both C# and VB allowed mainstream programmers to start using functional concepts in a natural way, often without even realizing it! But a close second are two other amazing things:

1) Linq variants (Linq to XXX) started appearing like crazy – the notation was incredibly useful

2) You could actually get great performance out of this kind of layer without inflicting craziness on users (see my article again for the Linq to SQL discussion)

In the same way that the needs of high quality designers drove advances like partial classes (critical for code separation) Linq variants and their needs, drove the type inference, extension methods, and expression tree features, including concise lambda notation, necessary for its success into the mainstream languages. This brings great collateral benefits because those exact same notions are helpful in so many other scenarios – I use them pervasively in my WPF code for event handling for instance.

As usual this all comes at a cost, it’s not just language notation: it’s also the debugging support, profiling, garbage collection, FXCOP, and so on. After all, just because a method is dynamically generated, has no name, and no .pdb file information, doesn’t mean you don’t want to analyze it like the others. And of course the ability to generate expression trees in addition to IL represents another significant undertaking for the compilers. All of this is fundamentally necessary for Linq to succeed but it’s kind of invisible – it’s the stuff you expect to just work.

“Orcas” also included the “Cider” designer for WPF. Remember that during this period there had been several runtime releases including 3.0 and it shipped alongside 3.5 of the framework. Well by the WPF was a first class application target and it needed designer support and the richness of WPF resulted in perhaps one of the most complicated designers ever created. It’s doubtful that a successful designer could have been created if not for the lessons from “Sparkle” aka Expression Blend, and language advances like partial classes to ease things along.

The abundance of framework subsets led to the need for multi-targeting – that is the ability of the tool to target any one of several framework versions or to have solutions with projects targeting arbitrary combinations. This was tricky enough in Orcas but at least the frameworks all shared a common core – simplifying things significantly. That grace would be absent in the follow-on release VS2010.

Other application models were popping up as well – Silverlight had made its debut, taking WPF concepts to the web, and Windows Workflow (WF) also delivered a release and a designer. If it wasn’t already de facto reality, the general theme that “every application model needs a suitable design experience” was certainly cemented during this period.

Debugging technology takes itself to new amazing places like XSLT debugging (XSLT makes my head hurt even more than APL) and gains the ability to debug into the BCL and dynamically download the source as needed to do the job. Very slick, but by now you know I’m partial to debugging technology J

And on the native code front, MFC 9 makes its appearance – adding support for new Vista concepts, and representing the first significant update to MFC in a very long time.

But for me the biggest memories are just working on performance across the board, from MQ to the finish. In the framework, in the tools, we made big dents in many scenarios and overall, even with lots of new features, Orcas felt snappier than Whidbey which was a great accomplishment. I loved using it!

I just hit 1200 words so I think it’s time to stop, having skipped vast regions and barely mentioned VSTS, I’ll pick up again in a bit.

Thank you for all the feedback and comments, they’re much appreciated.

[See The Documentary on Channel 9!]

My History of Visual Studio (Part 8)

ricom — Thu, 15 Oct 2009 02:32:00 GMT

[All the other Parts: History of Visual Studio]

I can’t really talk about what was going on in the IDE without covering what was happening in the runtime because their fates are so intertwined, so even though it’s off topic a little bit, allow me to cover some details from Framework 2.0.

“Longhorn”, which became Windows Vista, was probably the single greatest influence on the Developer Division during the years when .NET Framework 2.0 and Visual Studio 2005 (collectively “Whidbey”) were being developed. I think I could write several books on those years as a study in being too successful for your own good.

As I wrote earlier there was a certain mania in managed code adoption during that time and, though ultimately some of those efforts had to be scrapped, many were a positive influence on the tool chain.

Anecdote: It’s not really important to the story but just for fun I can’t resist telling you about my first day on the job when I joined the CLR performance team. I had been working for the last 7 years in the MSN area, with a great emphasis on server workloads, data architectures, and so forth. So I was a bit surprised when on day one I was told I was going to be working on our client performance problems – that’s a Whisky Tango Foxtrot moment. Pays to be flexible I guess J

Client workloads were a lot harder on the .NET Framework at that time. We did have an ngen story but it needed a lot of work. Putting as much code as possible into readily shared DLLs is fundamentally necessary to getting reasonable memory usage on the client – much less of an issue on dedicated servers for instance. Contrariwise, code sharing is even more essential on a Terminal Server with potentially hundreds of users running client applications.

Sharing also vitally important to flagship applications, like Visual Studio, that are trying to get their code loaded as quickly as possible using technologies like Superfetch. It’s a pretty simple chain of events, jitted code can’t be shared, unshared code uses more memory, use too much memory and you’re dead. Sharing is good. We had to do more of it. It was doubly important because major new Framework elements were being developed while this was all going on: things like Windows Presentation Foundation (WPF) and Windows Communication Foundation (WCF) to name a few.

The Base Class Library (BCL) was gaining support for Generics (starring my favorite Nullable<T>) and XML use was exploding in all parts of the stack. I used to joke that if angle brackets <> had never been invented I would have no performance issues to work on. If only it were true.

In the universe of technologies requiring tool support we really should add at least two more. There were yet more improvements to ASP.NET, there were compelling 64 bit architectures (x64 and ia64), and, probably most difficult of all the considerations, there was SQL Server 2005 “Yukon” which introduced SQL CLR allowing users to write stored procedures in managed code.

Again, I could write whole books on what was going on in the runtime components, but I’m trying to stay focused on Visual Studio for this series so I think that’s probably enough landscape setting, as it is with all those changes it’s easy to see why Whidbey took nearly three years.

For starters, there were whole new project and deployment types needed to support the new ASP.NET with its own, new, developer server; and likewise for the SQL scenarios. And while I’m on the topic of project and deployment this is also where “ClickOnce” makes its debut.

ClickOnce had to permeate our project systems but its introduction also highlighted an inherent weakness: despite the extensibility offered in the project space there was (and indeed is) no one central place ClickOnce deployment could be added so that all project types would benefit. This is a classic example of why ongoing refactoring/remodeling of architecture is so important.

Personal note: my first direct interaction with the Visual Studio team during this time period was working on the performance of the Create New Project dialog – something which can never quite go fast enough to make us happy.

Ok so the project system needed work. What else? Well, anytime you add pretty much anything you can expect the debugger to be affected, this was no exception. Generics – check, needs debugging support, SQL CLR – check needs debugging support, it was a long list. But there was even more than just that going on: I have written earlier about the challenges of what we call “interop debugging” (where you debug both managed and unmanaged code at the same time) – in this release both the debugger and runtime teams put considerable effort into making interop debugging more reliable. That meant taking a very hard look at all the communications logic, the locking model, the safe-stopping points vs. “skid” points as I called them. It was a huge endeavor, but as a result interop debugging got noticeably better in VS2005.

All this hard work actually had an unexpected extra bonus. We also introduced a managed code profiler in this timeframe and as it turns out, trying to stop managed code as it runs so that you can walk the stack the way profilers like to do tends to have all the same kinds of problems as trying to stop the code in a debugging context. As a result, much of the hard work that went into getting the debugger working resulted in a more effective profiler solution.

I should come back to SQL 2005 support – it isn’t enough to just add project and debugging features but ADO.NET 2.0 was making its appearance and that requires suitable designers. Visual Studio perhaps earns its name more so than its antecedent Visual Basic because it’s Visual in many different ways – just visual forms no longer suffices. New SQL engine means data designers are a must.

As if all of that wasn’t enough, there were two brand new pieces of technology that appeared during this time as well. These were part of our Office programming story: VSTA and VSTO. In a few words, VSTA is tooling for ISV’s to provide scripting features to their users, allowing their users to write managed language programs that automating their product. The primary user is clearly Office but in principle anyone can participate – you get a complete authoring experience with it. VSTO, in contrast, allows you to write office extensions, intended to be written as native code against a COM API, using managed languages instead. That poor sentence hardly does justice to the difficulty of achieving this result, and I’m glossing over the fact that there were other VSTO solutions as far back as 2003 but this is where I remember the spotlight first shining on that technology with any kind of luminosity.

But, even with all this good work, there was one thing that stands out in my memory as being more important than all the others.

Edit and Continue was back – and there was great rejoicing.

[See The Documentary on Channel 9!]

My History of Visual Studio (Part 7)

ricom — Wed, 14 Oct 2009 02:43:00 GMT

[All the other Parts: History of Visual Studio]

[I know I promised to talk about “Whidbey” in this installment but I realized I needed a bridge to get there or else I’d totally skip over “Everett” – so this is that bridge.]

In MSN the arrival of the .NET Framework and Visual Studio .NET was like a breath of fresh air. My team was working on (among other things) a COM based object model for a content management system. I once estimated that fully half the code was associated with the implementation of IUnknown in direct or aggregated cases (let’s hear it for punkOuter) or else tricky ref count management – to say nothing of the similar code that was in every client.

Using the old code as a reference, just one developer was able to re-implement the entirety of the interfaces – save one class which we chose to wrap with COM interop instead – in about 3 months. While it’s true that I have a high opinion of her (you know who you are) I think this was a great example of the .NET Framework shining in its strong areas.

The resulting libraries integrated seamlessly with ASP.NET (naturally) and were maybe ¼ the size with superior performance. One should never discount the cost of all those interlocked increments in COM objects and all the small allocations and long term fragmentation.

Fragmentation, or rather lack of it, was another key benefit we got from the .NET stack in that time frame. Until then it had been the bane of our existence, with 72 hour tests frequently showing performance degrading over time – we found that we got 5x the speed of regular ASP and stable performance over time.

We had other systems, formerly written in Perl that we were again able to rewrite in C# in a fraction of the time. A key benefit was the strong UNICODE support that .NET offered – we just weren’t getting what we needed out of the Perl implementations we had, to say nothing of the threading benefits. Debugging it was a dream compared to the old world.

In less than 6 months we had an all new data pump and all new managed content management system both scalable and more reliable than either of the previous systems. Life was good.

So I decided I should give up on MSN and go work on the CLR; I don’t think they ever forgave me. :)

By the time I joined the CLR, fall of 2002, their work on “Everett” (aka Visual Studio .NET and .NET Framework 1.1) was nearly done, it didn’t release until the spring but things were winding down there and so I started working on “Whidbey” almost right away, but I don’t want to just gloss over Everett because some important stuff happened.

C++ is a moving target, I guess all non-dead languages share that property but perhaps C++ is a lot more robust in this regard than many other modern languages, as a result there is almost always standards-catch-up work a compiler team should be doing. In this case it was around partial templates – this stuff makes my brain hurt – but I guess it should hurt: C++ templates are Turing Complete. Frankly I was amazed by what the C++ team had managed to accomplish in the time they had, the needs of managed C++ and IJW (It Just Works) managed support.

But there was a lot more in “Everett” despite the fact that it was supposed to be a minor update. For instance, Everett introduced support for managed code on mobile devices. I can’t believe I’m saying that like you just push the “go mobile” button on .NET and out pops .NET CF and all the compilers, debuggers, emulators, and deployment tools just like that. I think they made it look too easy.

Everett also introduced the Enterprise Architect version with the first UML support in our systems, maybe foreshadowing VSTS. Designers were definitely here to stay.

But truthfully I can’t say very much about “Everett” because I was desperately trying to learn about the runtime and getting ready for “Whidbey”. Meanwhile “managed code mania” was at its zenith at Microsoft; sometimes it seemed like every product was trying to find a way to incorporate the .NET Framework into their plans.

[See The Documentary on Channel 9!]

My History of Visual Studio (Part 6)

ricom — Tue, 13 Oct 2009 02:42:00 GMT

[All the other Parts: History of Visual Studio]

The years 1998 to 2002 were very busy ones in the Developer Division. I’ve previously written about “Dolphin” and I tried to give a sense of exactly how much changed during the Dolphin release and the sheer volume of work it required, but I think this period dwarfs that one in a number of ways.

Visual C++ 2.0 “Dolphin” was, by definition, “just” a C++ product. Putting aside the fact that it was designed to allow extensions for multiple languages, the scope of the system was limited to only one part of the Developer Division. In contrast, Visual Studio .NET, which spans the four years we’re talking about, and maybe a little more if you count the foundational parts that were already in VS6, was an effort that required the engagement of the entire division. Arguably it was the first time we ever attempted such a thing in the developer space.

Just thinking about it from the perspective of the number of people involved is telling – VC2 was the work of 100ish people. VS.NET required more like 20 times that, for more than twice as long.

What kinds of things happened? Well for starters the entire managed stack of what we’d call the .NET Framework had to be invented. Not completely from scratch but pretty close. That’s not just the runtime and the framework but also:

all the supporting file structures for IL and executable files, and inspectors for the same
a world class garbage collector, in two flavors
several variations of JIT compilers, as well as precompiled (ngen)
several new major language compilers (C#, VB.Net, and managed C++)
a complete “partial trust” model allowing for hosts that want a sandbox solution
debugging and profiling APIs for these components used heterogeneously – bringing us back to the world of soft-mode debugging the runtime
rich interop choices for both direct calls and COM, into and out of the framework, as well assorted serialization and object remoting strategies
the entire ASP.NET stack for IIS
new application models: Winforms and Webforms for this environment including a flagship design experience for both of these
solutions that contain projects of all these types in arbitrary combinations
an IDE that would handle of the these new experiences in addition to allowing unification of every major feature of all the other IDEs put together
an IDE with an extension model that would allow 3rd parties to do their own languages/environments, just like ours
lots more that I don’t even have room for

It’s quite an eye-opening experience to consider that all of these things were required to succeed and you can sort of understand why it took so long – some of the things on that list really can’t be started until others of them are substantially done, but of course the more complex later projects are likely to highlight problems in the earlier stages. With so many people involved just the communication overhead could be daunting, and the tasks above are hardly easy in the first place.

Well I’d like this to not read quite so much like a marketing bullet list so allow me to recall a few stories about all this from my own perspective – remember I was still in MSN here.

When I first heard about the next version of COM+, which became the .NET Framework, it was in the context of a pitch from the developer division to my MSN group asking that maybe we should consider moving some properties to ASP.NET to give feedback. There was a lot of information to disseminate and many things were still preliminary but I remember that there was one thing that I fundamentally did not “get” about the whole thing. It was because they were still calling it COM+ at that time and I had assumed that they were trying to come up with a new COM+ framework that was backwards compatible with the old COM+ stuff like what we used in the DTC. When they told me that they were trying to do a compacting memory scheme in that world I thought they were completely nuts, at minimum you’d have to have proxy objects for everything so that you could move the real objects without anybody knowing. We were maybe 45 minutes into the meeting before I realized the magnitude of what they were proposing – they wanted an entirely new object model with an entirely different memory management strategy in all new languages. Wow. Just, wow.

The next demo hit close to home. It was a managed client for a web service. They just pointed the tool at a web service and instantly had Intellisense over of it using VB.NET. It had some rough spots but definitely another wow. The next was a quick demo where they imported a COM component, the kind we used on our web pages for ad delivery and so forth, and then started calling that with Intellisense support sweet-as-you-please using nothing but the component’s TLB.

I was so excited I got myself an early drop of the thing and started writing benchmarks. I wrote applications that did the kind of string manipulations that our web sites usual did and sent them feedback based on my results. A lot of them made a difference. I had a sabbatical coming up and I ended up spending most of my six weeks reading the base documentation for what they had built, partly to provide feedback, but even more because I thought it was going to be really important to learn it. It would turn into my next job two years later.

What about some of those other items on the list? There’s some pretty meaty stuff there; I’d like to pick off a few and talk about them a little bit, and I’m so fond of debuggers; let’s start there.

It turns out that debugging managed code can be pretty tricky. I alluded to the fact that it’s a soft-mode debugger (like VC1). I say this because it has the key property of soft-mode debuggers which is that the debuggee isn’t really stopped when you stop it. Both the VC1 debugger and the .NET managed debugger share this but they accomplish it totally differently and for different reasons. In managed debugging “your” code really does stop normally, but there is a debugger helper thread in the debuggee that provides access to key structures and otherwise relays important information back to the debugger, so technically the debuggee isn’t completely stopped.

So far so good, but if that were the extent of the situation, then this wouldn’t be a very interesting discussion – maybe you could call that “nearly hard” mode or something – the real situation is a lot more complicated. One complexity is the fact that when if the user tries to stop the debuggee because it’s say in an infinite loop, it’s possible that the debuggee is in the middle of some runtime call and not directly executing the code the user wrote. It could be in the middle of a garbage collection for instance. If that happens you don’t really want to stop the program right away do you? If you did, you’d find that your universe looks wrong – some of the objects have been moved some have not, some pointers may still need to be corrected. In short, the world is not in a good state and there are lots of these temporarily-bad states that could be visible to a debugger. Ouch. So this is another typical soft-mode problem, when you try to stop, you kind of have to skid, get the debuggee somewhere sensible, then stop (or pretend to stop).

If that wasn’t bad enough, the managed debugging has to work in a hybrid program that’s partly managed and partly unmanaged, maybe with some threads having different combinations. So if you try to stop one thread that’s unmanaged you should be able to do the usual hard-mode thing but if you then try to inspect a different thread that is managed well that could then cause you problems unless you’re treating each thread as it needs to be treated for the kind of code its running at the time its stopped… Oh my…

Add to this fun the fact that you often have to actually run managed code to do normal debugger things like evaluate properties and you find that our poor debugger folks had a few things on their minds while they were integrating all of this. It’s so easy to assume those call stacks and parameter values are easy J

Let’s pick a couple more of the more technically interesting problems from that bulleted list I started with. One very interesting one is the Winforms designer. Now this particular designer is interesting (perhaps unique at the time) because it needs to provide a full-fidelity visualization of the form you are authoring, including, for instance, controls on that form you have written yourself in the very same project. These are what I like to call the “first party” controls. It’s easy to show that in general the only way you can provide that kind of fidelity is to actually run the code in the designer. So now we have to take the code you are writing, compile it on the sneak, load it within the IDE itself (using the very same framework of course) and then you can see your whole form, panels and all, just as it will appear in your application. Wrap it with handles and rulers and so forth to allow direct manipulation and you have yourself one very slick designer! Not easy to get that right.

What about all that new IDE integration and extensibility? Well to make that all happen you have to painstakingly go over all of the extensibility features that are in each of the existing shells, formalize them with clean COM contracts – usually after conducting personal interviews to find out what the existing informal contracts “really mean” – and then fit those into an all new framework of interfaces and components that is itself extensible by 3rd parties. Naturally this is a totally thankless job and it’s as likely as not that everyone involved will say you did it wrong no matter how careful you are. It’s kind of like tax-assessment: you know you have it perfectly fair when everyone hates it equally. But in the end it was super-successful; there are literally dozens (if not hundreds) of language extensions available for Visual Studio – you really could do this even if you didn’t work in Redmond!

I could keep writing about how technically impressive Visual Studio .NET was, maybe I could even win a debate with the thesis “Visual Studio .NET was the most technically impressive release ever” but the fact of the matter is a lot of people didn’t like it; history isn’t all sunshine and roses after all. I think I’d be remiss if I didn’t talk about at least some of the sore points so I’m going to hit one squarely on the head.

A lot of VB programmers did not want VB.NET at all and liked VS.NET just as much (i.e. they didn’t like it at all).

Why? Well the answer to that question is probably a whole book right there but let me boldly make some guesses.

First, VB.NET was not the language they wanted. The runtime changes presented a challenge, there was Winforms to learn for instance, but I think those might have been more acceptable if the language itself had been more VB-ish. Traditionally there had always been a compiled version of BASIC and an interpreted version at the same time. VB.NET decidedly had that compiled-language feel and that didn’t sit well with those that wanted an interpreted feel. I think a language that was more like “Iron Basic” (e.g. like our IronPython language but Basic instead of Python, still targeting .NET) would have been well received. It scripts like a dream, it has direct access to .NET objects, you can run it in a little immediate window if you like, and change anything at all about your program on the fly. I suspect we would have loved to deliver such a language but during that time we simply didn’t yet know how to do so.

Second, VS.NET was not the IDE they wanted. They were used to something smaller, tailor-made for VB that had genetically evolved for VB users, and this wasn’t it. In the initial version of the integrated shell, Edit and Continue wasn’t working for the .NET languages leading to the astonishing first-time situation that the C++ system had edit and continue and the RAD programming languages did not!

I think the net of all this was that there were divided loyalties among that generation of VS users – I think that still persists actually. It was impossible to not acknowledge VS.NET as a great technical achievement but it was also impossible to say that every customer was pleased with the direction.

Whatever else you say though, the 2002 offering became the new foundation for tools innovation at Microsoft and it began a new era in IDE development here. One in which a key distinguishing factor was the presence of rich graphical designers for virtually every development task. It was in this release that the notion that it was enough to simply throw up a text editor and call it good became insufficient. Arguably, even now, Visual Studio’s bevy of designers are a key aspect of its success.

Not long after Visual Studio .NET I returned to the Developer Divison, I’ll pick up the story at the “Whidbey” release in the next part.

[See The Documentary on Channel 9!]

My History of Visual Studio (Part 5)

ricom — Sat, 10 Oct 2009 18:50:00 GMT

[All the other Parts: History of Visual Studio]

We’re still on the part of the story where I was off working in MSN which means all I can give you are the first- and second-hand stories that I’ve heard over the years. So I guess that makes them second- and third- hand to you. This would be a great time to watch the Channel Nine Documentary for more authoritative information J

In the last part I wrote a little bit about what was happening in the VC++ IDE – “edit and continue” for C++ programmers – pretty amazing stuff.

At about this time the Internet happened. I mention that like it’s in passing but, even looking at the phenomenon super-narrowly, just from the perspective of how it affected our programming tools it had some pretty big effects.

Of course probably the most major one was the creation of Visual Interdev to help people create great web applications. You know that shell is going to be central to the story in a way nobody expected at the time (ok, maybe the VID guys expected it, but nobody else J) but also script and dynamic languages made a huge resurgence. Perl, Python – remember web sites in Perl? And of course jscript, at first as a client-side language in the browser, but then more generally, and its twin in our world: vbscript.

But for the web server needs and the browser needs, we might not have driven these languages within Microsoft as hard as we did. They helped our tools grow up in unexpected ways, like for instance, I think in VC5 you could finally call the editor “grown up” – it had been originally fairly underpowered as programmer’s editors go – but in VC5 full macro support was added (not just little recordings) using the vbscript engine. It still needed polish but you could now no longer argue that it was lacking in raw potential/power. Like other MS applications the macro language encompassed not just the editor but virtually all aspects of the product. I remember making a command line debugger interface in an editor window with macros as a example for a friend.

Speaking of BASIC, during all this time, our friends in VB weren’t exactly idle either. In the VB5 IDE, Intellisense makes its appearance.

Can you imagine programming without Intellisense? Well I guess you probably all can because many of you did without it in the past, but certainly a good amount of cursing happens in my office when I find myself without it. It isn’t just that it saves you typing – it fundamentally changes the way you work. For many applications Intellisense makes API documentation obsolete because it used to be that the #1 use for that stuff was to get method signatures, it is easily the #1 way people get “help” – blowing away the F1 key for usage. But it’s more important than that… it goes as far as freeing you to use better naming techniques, especially in frameworks. Maybe Dennis Ritchie would not have left off the ‘e’ in ‘creat()’ if he’d had Intellisense. And what about what it did for Office? The Visual Basic for Applications (VBA) engine got Intellisense too and that meant Office programmers got much easier automation scripting.

At any rate, Intellisense was a total game-changer and was soon in the entire Visual Suite. And by soon I mean what you might call Visual Studio 6.0 – by an amazing coincidence the language version numbers had synchronized or been forced to synchronize (VJ and VID).

Now VS6, and more importantly its parts, VB6 and VC6 are widely considered to be the best we ever made. Not universally but certainly there are factions that think we should have stopped right there, or stayed on that track. I think this is true because they represent the last of that evolutionary line, for both languages. The IDE that would become Visual Studio became available in its first incarnation at that time, as I have alluded to in the past, and it was Visual Interdev. In some ways, from a consolidation perspective it looked like we had taken a step backwards -- there were now four major shells rather than three. The respective teams polished their products to a bright shine and Intellisense bloomed like daisies across the suite.

For those of you that just lost me on shell count, I went to four with the addition of the VID shell (used by VJ++), add VB and VC++ and that makes three. What is this fourth shell you speak of Rico?

The problem is that in this entire history I’ve been completely silent on another suite member – Visual FoxPro which also had its own shell. Now there’s a reason for this – and that is that I know painfully little about that product other than when I did a study of database technologies for Sidewalk in 1995 I was stunned by how many advanced features were already in the FoxPro product. Who remembers “Rushmore” – I remember getting some pretty slick demos! Little known fact: it was the FoxPro team that taught me the basics of database programming -- thank you, you know who you are :)

But, with apologies, I must go on, paying only very little homage to that shell. It would be great if someone else posted a My History that included more FoxPro stuff. Or any other My Histories for that matter…

I feel like I should mention Office a little bit more at this point. I’ve hardly mentioned Office but it’s important to note that Office, and the needs of Office Programmability, were often key forces in language and tools design, to say nothing of the needs of the Office programmers themselves – which frequently pushed our tools to the breaking-point. IMHO, probably more than any other single force, it was Office that was driving BASIC – from as early on as the Embedded Basic which then went on to power VB, and then later VBA, Office was a huge influence. I think specifically it was Office that drove how application automation had to work and that in turn drove OLE Automation generally and that in turn drove BASIC. Those of you who have ever had to program with VARIANTs a lot can probably thank or curse this dependency cycle.

One other big thing to mention and then I’ll wrap up for this installment. Another notable change in this time period was the maturation of the data access layers. I think this, too, was partially internet-needs driven but also I think the technologies were just ripe. OLEDB was born during these years and with it the BASIC access mechanism – ADO. These things seem pretty mundane on the surface but they offered much richer data access than ever before and inspired the first of increasingly popular and powerful datagrid mechanisms for both display and editing of data. When compared to the clumsy DDX/DDV type things we had been doing it was a breath of fresh air for data programming and forms over data. All of this would go on to influence many different types of designers and data management services, but I think the spark for that was here, in say 1997.

Visual Studio 6.0 – synchronized across the versions and starting to show some (distant) signs of consolidation between its various parts appeared in 1998. It was a great release, as I wrote above, it’s still the favorite release of many. After that, the developer division “went dark” for about 4 years, they were working on what would become “Visual Studio .NET”.

[See The Documentary on Channel 9!]

My History of Visual Studio (Part 4)

ricom — Fri, 09 Oct 2009 00:04:00 GMT

[All the other Parts: History of Visual Studio]

I didn’t really intend to write one of these per day but here it is day 4 and we’re still going strong. I’d like to take a moment to thank the many people who commented on this series either here, or on their own blogs, or elsewhere. I’m watching J

I started the series by saying that this is only “My History” and I’d really love to see your histories, however much you’d care to write, even if it’s just a fond memory or a memorable aggravation. Many of us went through the same programmer-generation-defining experiences, so it’s fun to share them. For instance, I was really surprised just how many people I knew spent far too much time exploring the Mandelbrot Set. Why we all felt compelled to generate almost exactly the same pictures on our own computers, often with our own custom math library, is something I may never understand J

OK back to My History of Visual Studio, I want to go back a little bit again like I did last time and cover something that I skipped that I wish I hadn’t, especially because there’s a fun story that goes with this one.

As it happens, at that time, and for several years, the big show was the Software Development Conference, usually the spring one. One of the popular attractions was competition between the major tools vendors at that time where they were all given the same challenge to produce from “scratch” (using just whatever came with their tools, including samples) some kind of modest applet that you could build in about half an hour while the audience watched. Usually there were several “levels” and you’d get points for each level. One year it was a bitmap viewer.

After one such conference, where we had competed (and won) with Dolphin, we got to thinking that this was actually not such an exotic situation to find yourself in – needing to cobble together some code from pieces you had lying around – and we wanted to drive code reuse as theme in Olympus so we decided we should have something like Code Gallery. This could very well be the birth moment of what we’d call Code Snippets today. We had no idea how it should be built exactly but we imagined you should be able to point it at pretty much any chunk of code, make a blob, make some parts of it be variables, and then you should be able to put it elsewhere in the same or a different project. We gave it to The New Guy (you know who you are J).

I can’t believe how mean we were – “Here, take this vague uncooked idea that touches most of the important systems and go build it. Let us know how you do.”

I think he did ok though.

The funny thing is, we figured that this would give us an unfair advantage over our competitors at the next SD conference but we also figured that having a long list of features we were banned from using (remember everyone is watching so it’s easily enforceable) was kind of a badge of honor and it really was a cool feature.

Having re-read the above I feel it’s important to remind my readers that a preposition is something you should never end a sentence with.

We’re going to move forward in time now, but we’ve come to a part of the story where I didn’t have first hand knowledge. In 1995 I changed jobs, and for 7 years I worked in MSN land. I was busy developing the technology for Sidewalk.com but I still had a lot of friends telling me what was going on and some of it was pretty amazing. I think I can best describe this period with an allusion. This is historical fiction but it kind of hits the key points in a fun way.

A long time ago on a campus far, far away…

SHELL WARS!

Episode 5.0:
The Great Landgrab

It has been many years since the Visual Tools first came on the scene and now the many factions are vying to be “on top” as customers clamor for a Grand Unification. All the contenders are widely regarded as Evil by the other contenders and the whole thing looks to an outsider like it belongs in a Dilbert cartoon.

The VB Consortium claims that they have a crucial edge because their shell provides the most immediate feedback, their excellent design time experience is a must for any successful Unification. But the Compiled Language Bloc, led by the notorious C++ team, claims they already have a shell that is hosting multiple languages and that the VB shell is ill-equipped to handle things like “The Mixed Mode Nightmare.”

Meanwhile, unbeknownst to these factions, another, newer, shell has been born. This Child Shell was the host of a little known system called “Blackbird”, but the Internet was coming, and all those web-developers would need tooling too. And, perhaps, Interdev could restore balance…

</scrolling text>

OK I’ve had my fun. But there are some key truths in there. There were various competitive shells and we needed to choose one. But of course choosing one was really quite impossible because what we really needed was to cherry pick features from each of them and then consolidate the result. A critical choice here is “which shell do you start with?” and that is largely what the fuss was all about.

Visual Interdev was literally in its infancy when this was going on and I think it’s fair to say that it, ultimately, became the basis shell which integrated the others. There are several reasons for this, mostly technical (there may be non-technical ones too but I don’t know them so they’ll have to be part of someone else’s history).

One reason was that there was a hope of having a public extensibility story that made sense with that codebase. The VB shell had very limited extensibility, and the VC++ shell, while it had tons of it (proven by the number of languages it was able to host) had a serious flaw: when we tried to let 3rd parties extend it they completely failed because the contract between the shell and the packages was both tight and complicated. There was an interesting design tension there, well there are many, but let me give you just a taste of one.

VC++ did not use COM for its extensions, in fact it would have been anathema for us to do so. Our philosophy was that we needed to be able to debug COM when it wasn’t working, not rely on it to boot. But here’s a flip side – if you don’t use a registry-based activation model like COM has, then you suffer two key problems: #1 your contracts don’t have clean separation so 3rd parties will go crazy trying to use them, and #2 you’ll find you have to load all of your extensions just to know what they do – something you ultimately can’t afford.

In the end the VC++ shell failed the test because while it was designed to be extensible from the start, it was designed to be extensible by us, and not by 3rd parties. It was designed for a smallish number of extension packages which would always be loaded at startup, and that was something we couldn’t afford – Visual Studio is actually in the business of not loading extensions while giving the illusion that they are loaded, including cute tricks like enabling/disabling menu items of extensions that are not loaded on their behalf. I haven’t name dropped to this point but I can’t not mention the guy who is most responsible for solving these problems, and I’m about to link you to his video so, you can give Doug Hodges a word of thanks if you ever meet him (which I highly recommend).

I think it’s fair to say though, that at the time, not everyone agreed this was the right choice and, at the time, I would have not agreed myself. But in retrospect, I would have been wrong. But I was working on Sidewalk anyway so nobody asked me J

Now, getting everyone onto a new shell doesn’t happen overnight and as I mentioned, the ultimate winner of this contest was still just nascent at this time. So it would be a while before we’d see the changes that were in the works. The Visual Studio brand was alive at this point but it wasn’t much more than some lashed together boxes.

There were other pretty cool things going on at this as well.

I have to mention Visual J++ at this point because I think it was just too darn important to the future of Visual Studio and the .NET Framework to omit. Remember we had been thinking about simpler language models, and simpler programming models, for years, at virtually every offsite. But how do we move our users to it? Why would they want to change languages? What might motivate that? How do you sell the benefits? And then along came Java.

I’m going to keep the discussion focused on interesting technical things that affected the history of VS, the rest of the checkered history of our relationship with Java I have no first-hand, or even second-hand knowledge of anyway.

Java is a managed language, already that drives a bunch of choices, and, additionally while it doesn’t require a JIT compiler that’s pretty much the best/only way to get world class performance out of a JVM. You might think, so what? Whatever. Managed schmanaged, it compiles to native and runs, what’s the big deal?

Well it’s true that running managed code does look a lot like regular running native code but the similarity is only superficial from the tool’s perspective. Let me talk about it from the perspective I know best – debugging – to highlight those differences. Keeping in mind that the reason I’m mentioning this now is because I think Visual J++ was the system that forced us to think about these things and get solutions in place as early as we did. .NET would have many of the same problems and so the experience would be helpful, even if the code and people were all different.

OK one problem with managed memory is that there’s a garbage collector, and good garbage collectors compact and that means objects move around. OK so what, so objects move around; the contents of some pointer variables might change or something right? That happens all the time while debugging anyway, it's called “running”, how is this a new thing?

Well for instance, if you’re debugging and you tell the debugger “stop anytime the contents of this global variable changes” then what it likes to do is say “ok the address of that global variable is 0x12345678 I’ll just use a hardware register to tell the CPU to interrupt me if the contents of that memory are altered and I’m done” Oops, that doesn’t work so good anymore, the contents might be altered by garbage collector at pretty much any time without actually logically changing (the global just points to the same object which is now somewhere else) and worse still, the global itself might move so that lovely fixed address you used to have 0x12345678 could itself change at any time. Joy. We’ll have to handle all of that.

OK, so, data moves, great. What about JIT compilation, does that do anything important? It sure does. In regular code, or even code that’s been compiled in the normal way to p-code (we supported that remember?), the address of code doesn’t change from run to run. It might be relocated as a chunk but within any given DLL the offsets remain the same. That means one set of symbolic information is all you need to find any given method, or to determine which method contains the current instruction pointer. When you JIT, that doesn’t work; instead you have to remember where you put every given method from run to run, and you can’t assume the method exists as native code just because you loaded the IL for it, it might be not jitted yet – so that means that if a user asks for a breakpoint you have to wait for it be jitted then insert it…

All this jitting and data motion makes an already complicated task, like coming up with a calllstack that shows method names and parameter values, a lot more challenging. But it had to be done and those challenges were met. I think VJ++ drove a lot of that.

I think the need to interrogate the runtime state to find object information, and to get notification of things like garbage collection, is most likely the reason why they chose a soft-mode debugging strategy and why ultimately the .NET framework got the same strategy. Personally, I would have gone hard-mode all the way but it wasn’t my call, and arm-chair quarterbacks don’t have to deal with the Blitz so I should just probably shut up about that. Likewise that JVM provided great experience building a garbage collector.

I think I’ll change gears a bit for the end of this posting. While all this was going on: managed future, shell consolidation, etc. etc. The individual language teams continued to produce very cool things. We started using Visual Basic’s new ability to create compiled executables to make web pages written in interwoven BASIC, that looked exactly like .asp files, but that we preprocessed into VB projects and compiled into ISAPI DLLS. And you could debug them with a native debugger and get seamless cross language debugging! Very cool.

And, in the C++ space, another micracle was occurring. I have almost no idea how they pulled this off, even though I did get a few peeks at the secret sauce, those guys took incremental linking and compiling one step further and delivered general purpose Edit and Continue for C++, maybe for the first time ever in any production system. My 20 minute test failures on web servers running C++ had suddenly had dreamy debugging.

I was lucky enough to be one of few that could call the guy that wrote the code and say to him: “DO YOU HAVE ANY IDEA HOW MUCH TIME YOU JUST SAVED ME! I LOVE YOU!!”

It was that good. Like “Thunder.”

[See The Documentary on Channel 9!]

My History of Visual Studio (Part 3)

ricom — Thu, 08 Oct 2009 02:01:00 GMT

[All the other Parts: History of Visual Studio]

I was going to go forward again in this installment but I got some requests to talk about some older things again before I did that. You might be getting bored of the distant past so I’ll try to keep that part short.

There were some other key pieces of technology floating around back in the early 90s: One of them was this thing called Codeview for Windows (CVW). This thing was substantially written by my racquetball partner (no name dropping, but I haven’t forgotten) and it was the “hard mode” debugger for windows. Remember how I said you couldn’t debug Win16 the normal way because you can’t stop it? Well you can if you have a debugger that has a whole alternate UI based on “COW” that runs on its own monochrome character-mode-only monitor (usually a Hercules Graphics Card, remember those?)

You thought multi-mon was a new feature but no, in this flavor it’s very, very, old. :)

This baby could give you true symbolic debugging for all kinds of Windows programs for which you had source code and debug info, but if you wanted even lower level stuff you could always hook up a serial cable to a dumb terminal and use wdeb386. Wdeb386 is your buddy!

CVW was the main debugger that we used to debug “Sequoia”, which I alluded to but didn’t go into the details of earlier. “Sequoia” (that’s a lot of vowels, huh) was a before-its-time IDE that we worked on around the time of PWB 2.0. Ultimately it was cancelled because it was too ahead of its time I think, but, as I’m fond of telling my wife, being ahead of your time is just one of the more creative ways of being fundamentally wrong. That quip didn’t make me feel any better then, either.

Anywho, Sequoia was interesting for lots of reasons but one of the most interesting things about it was that it's commanding was entirely based on the same BASIC engine that drives VB. Everything, everywhere, was a basic command in disguise so you could record anything. It had a flexible editor that used a piece-table structure for space and had cool font support, coloring, and outlining. It had a source code debugger with the beginnings of a soft-mode solution, and it had graphical build environment, and graphical program visualizations. It was pretty slick. Of course it was seriously incomplete and it had a tight dependence on some technologies (especially that BASIC) that were not going to be available on the 32 bit OS anytime soon and we needed those to ship. We also needed all hands on deck for VC++ 1.0 “Caviar” and so Sequoia got scrapped.

A lot of the Sequoia ideas came back later in different versions of the IDE, some of them are just happening now in VS2010 but I guess that’s par for the course in cancelled projects. Perhaps, when viewed as a research project, it wasn’t so dumb. It wasn’t that many people and it wasn’t for that long, and it was quite an education. Unlike QCW, it was written in C++, with its own framework, so it provided a great test case for the C++ compiler (C7) and it influenced both COM and MFC.

I planted a Giant Sequoia in my backyard in its honor (I called it Sequoia 0.1) and, true to its name, it’s quite big after 15 years.

Another funny story about Sequoia is the series of codename changes as we cut features before finally cancelling it altogether, it went from “Sequoia”, to “Bonsai”, to “Bonfire” – One of the best things about that project was how nice the specifications were, I still have some. Thanks for those – you know who you are Calibri; mso-hansi-theme-font: minor-latin; mso-char-type: symbol; mso-symbol-font-family: Wingdings;">J

Speaking of codenames, did you know that the various milestones of C7 had interesting codenames? There were far too many of them of course because that was a hard release to get under control: John, Paul, George, and Ringo were 4 of them; We were still in need of Beatles after that so we added Stu and then “The Long and Winding Road” (though I bet the memos all said LWR) and finally shipped with “Let It Be.”

Ok let’s get back to the more recent past.

After “Dolphin” we had achieved a pretty significant miracle I think. The shell actually worked as intended and we shipped other languages that lit up the splash screen (and more) – multi-language project creation, browsing, and debugging all in the same core bits. Basically all of the compiled languages could participate but we still had a lot of ambition.

The next release began as “Olympus” Visual C++ 3.0 – but we soon renumbered it to 4.0 to align it with MFC so that we didn’t have to say VC++ 1.0 with MFC 2.0, then VC++ 2.0 with MFC 3.0… This time it was going to be VC++ 4.0 with MFC 4.0. In retrospect this was highly stupid, especially because MFC ended up locking down on version 4.2 when backwards compatibility became a more valuable feature than anything else we could add – and so it stayed for a very long time.

I handed off responsibility for the debugging components in Olympus and with a few close friends we put together some very cool improvements for the tool chain. Incremental Linking, Incremental Compilation, and Minimal Rebuild (I was mostly involved with the latter). We had done something like this before in the original 1988 C# – it had all of those features – so we had some idea how we would want to do them in the mainstream tools, and actually ship them this time.

In case you don’t know what those things are let me quickly describe them: Incremental Linking is where you make a modest change to some small fraction of your source files, creating a few new .obj files, and instead of totally relinking the resulting executable you “simply” patch it in place. Sounds not so hard, the thing is already logically organized and the linker knows where it put stuff, you just leave a little room for additions in the .exe and away you go right? Err, not so much no, what are you going to do about all the debug information, that has to be patched too, new offsets, new line numbers, etc. What about the browser info? You could easily spend as much time on other collateral pieces figuring out what to patch as you would have just redoing the whole thing. Some cleverness is required.

Incremental Compilation is where you make a minor changing in one or more source files and rather than regenerating the entire .obj file you “simply” recompile only those portions (usually methods) that have changed and leave the other object code in place, thereby giving the back end much less work and saving you parsing time. This is especially tricky because it has the same “what about the auxiliary files?” problem as incremental linking and you already have PCH helping you to do this fast, so the .obj files need to be largish to come out ahead.

And last, but not least, Minimal Rebuild. This is motivated by the kinds of edits that Class Wizard made. This guy notices that you’ve changed a popular .h file, and you’ve changed a single .cpp file (usually adding a method in both the code and header file) and although many files depend on the .h file only one file actually depends on the specific change you made, usually the one you modified. The others could be skipped. This has great potential to save compilation time and paid off big in larger projects but it too has complications associated with the auxiliary files – the debug information has to be patched in files you didn’t touch and so forth. We actually started with a version of this feature that had Class Wizard telling us what files had been changed for Class Wizard edits and then generalized it so that we could determine what was different by looking at the debug information for the previous version of a build and then the new version of a build and from there conclude what we could skip.

The slick thing about Minimal Rebuild is that it turns out that keeping the full dependency tree for a large project around is just too expensive. We kept a set of approximately correct dependencies with a Bloom Filter data structure and by tolerating a small false positive rate we were able to realize the bulk of the savings. Using the existing debugger information to do the change analysis was probably the most clever bit of all.

While all of this was going on, an equal effort was going into the compiler back-end and they were wanting to unify their system on a tuple representation so that they could use more of the same optimizations in more places and generally get better code quality. Such is the way of compilers, huge efforts to squeeze out maybe another 5%. 10% improvements are a thing of dreams.

Olympus was getting quite large and it now used enough memory that it was affecting the compiler’s ability to get its job done. If there isn’t enough RAM (we’re talking about say 4M here) to hold the IDE and the PCH in the disk cache then performance suffers. I went on a working set jihad and got the IDE’s working set (while building) under 64k (it was 13 pages). Back then you did really insane stuff to squeeze out every page – I even turned off the carat flashing in the output window when I found it caused extra code to run during the build!

Other things were going on in the industry but one thing was really on our minds often, mentioned at many offsite retreats – C++ programming is too hard. We really could use a different language/system that would make this a lot easier. Delphi was being fairly successful. Visual Basic was still strong. But now it’s 1995… The web is coming, and with it, Java.

[See The Documentary on Channel 9!]