Rico Mariani's Performance Tidbits

Some nice performance features in Visual Studio 11

2012-03-21T22:17:02Z

This sort of solution managment brings welcome relief.

Amazing Grace Hopper on Nanoseconds

2012-03-02T19:15:42Z

Just found this: http://highscalability.com/blog/2012/3/1/grace-hopper-to-programmers-mind-your-nanoseconds.html

Worth it just to hear Grace.

I think I'll get some microseconds to hang around people's necks. Maybe I'll be kind and use fiber instead of copper :)

Software Performance for Metro Style Applications

2012-03-01T18:56:00Z

With the Windows Consumer Preview out the door, I thought it would be interesting to write something about creating great performing Windows applications. I hope to have a lot more to say about this in the future but I think really the most important things I have to say are more inspirational rather than informational at this point. The key point is that things are quite a bit different than they were, and how you think about creating an application with great performance is perhaps different than it ever has been.

OK that’s a mouthful, what am I talking about?

Well one thing is the operating system itself is a lot more frugal. Improvements in Win7 and continuing improvements in Win8 mean you get more of your system resources available for your applications. Great news! And what are those applications going to do with those resources? Why use them of course! Hopefully wisely, to create great experiences when they are active and economically fade to nothing when they are not. As we’ll see, resource usage is more temporal in Win8 and so management is essential.

Back when I worked on the Whidbey CLR we thought a modest forms-based application ought to be able to start up in something like 50ms and use something roughly like 5MB of working set.

In many ways we considered the working set number to be too big. We addressed that at least in part by making as much of that memory shared as possible – good for amortizing the cost over many processes. Big fractions of the private memory came from things like the GC heap’s initial allocation, and pre-committed stack pages plus other fixed costs but much of it had to do the raw cost of getting a CLR application going. Still that’s where we were: 50ms and maybe 5MB, fighting to go down from there.

Fast forward to 2012 and things look a lot different. A metro style application might spend say 9 frames @60 Hz fading in a splash screen (call it 150ms) and then another 9 fading it out. And then maybe as many doing some cool slide-in type effect as the title comes in from one side and the data from the other. These kinds of effects are part of the Metro look and feel. Now at that point we haven’t actually done anything per se and we’ve spent maybe 450ms just on animations.

Memory wise, to do those transitions we probably have spent private memory to the tune of 3 full screen surfaces. Easily 4MB each (at 1366x768) with probably 2 live at any given moment and that’s just the memory for the video. Clearly the world is a different place. And already the temporal nature of resource usage is showing clearly, those costs are short term.

So what does it mean?

This metro style application experience, despite the fact that it takes longer to start up, is actually much more pleasing to use. It responded instantly to our command and fit nicely into the expected design. I think that’s the most important observation: in a metro style application we do get some more latitude to use video memory especially for effects but the price we pay is that we have even less latitude for stalls. People expect a smooth responsive system at all stages. No video glitches, no jerkiness, no synchronous pauses to get resources, no long wait-cursors. The expectation has been set from the first frame that your experience will be a lovely one and applications that don’t deliver will look like old klunkers.

Let’s also consider where this extra memory came from: video memory is in many ways the cheapest in terms of time-to-get-ready. In previous frameworks we were dealing with mostly code that had to be brought in and working set was a good proxy to startup time because working set tends to translate directly to disk IO. Clean video surfaces don’t have to be paged in so that (nearly) 1:1 relationship between working set and startup time is ancient history. Which isn’t to say you can now load all the code you like: you can’t, especially not all at once because page faults turn into glitches. Like all your other resources you’ll want to get them loaded asynchronously and have a good UI experience while they are coming in. That means don’t load your whole program at startup, don’t initialize everything you might ever need, and create suitable transitions and incremental experiences for data and program parts arriving. In a great application all of that is part of the overall experience.

This is why I think it’s even harder on the programmer to get all these things right and why it’s even more important than ever for the frameworks to give you a Pit of Success experience when you use the normal templates and the normal services to get your job done. If you aren’t careful then you’ll destroy the intended experience and even “a little bit broken” will look terrible in contrast to the expectation set by the design.

Compounding all of this is our desire to run metro style applications on lower end hardware. Some of these systems are equipped with decent GPUs (though not the greatest) but their CPUs offer a raw compute speed that might be say 1/6th (on a per core basis) of a high end desktop. If you want your application to run well in that environment you’ll need to think carefully about how much work you do in one transition and how you use asynchronous patterns to keep you application feeling crisp even with CPU resources that are less abundant. Careful testing and measuring on these lower end systems will be essential because your normal development machine could delude you with speed you don’t have.

And as if that’s not enough, almost universally power consumption is a huge concern. In unplugged scenarios, whether it’s laptop or tablet every joule consumed matters. In some cases a core feature of the tablet is the excellent battery life and hence the lower power CPU and so forth. As an application or framework author you must consider these things very carefully. The system will want to suspend your application when it isn’t running – how will this affect the way you store your state? You will want to have lovely animations while your application is active because that’s what a great application does but you’ll want to be able to turn those off when you aren’t visible. Background activity generally should be looked at with great suspicion because it will drain power. What background work must happen? What could be reduced or suspended entirely? Great choices will mean superior battery life which could be a huge aspect of your success.

Despite these challenges I’m optimistic. I think the Metro design is a great one because has a lovely look while at the same time it is not so terribly hard to create that you must spend all resources to make it. The Metro look and feel is largely composed of things that GPUs do comparatively well. That’s good news for everyone and it’s a good basis to create your applications. Asynchronous design patterns are possible in all the languages WinRT supports and especially in Javascript, which is likely to be the most popular choice by volume. Asynchronous patterns are already normal for most javascript libraries.

Bottom line: what constitutes a great performing windows application has changed in the metro world, but great performance is still important, perhaps more important than ever.

As I said above, I hope to write more about these concerns in the coming weeks but for now I'd like to refer you to some articles that have already been written on MSDN. It's not everything by a longshot but its something. While this particular work discusses the problems in the context of javascript, most of the advice is actually broadly applicable no matter what framework you use. Check it out the guidance here.

Coding in Marble (Part 2)

2012-02-20T22:13:00Z

I thought I'd follow up on my last technical post with a few extra details about the Marble pattern, despite the name of the article I mostly talked about the Wood pattern. I guess perhaps this is timely because the use of Promises to represent asynchronous operations is increasingly popular, but these notions are not really limited to context pattern, though promises do make for good examples.

So, what are the essential properties?

Wood:

objects tend to be smaller and more abundant (model the nodes not the tree)
dependencies and dataflow tends to be represented on a per object basis
dependencies frequently make long chains (such as a series of promises lashed to one I/O)

each object is participating in its own dependency chain but it they are really all the same

Marble:

objects tend to be bigger (model the tree not the nodes)
dependencies and dataflow tend to flow through these big objects, the dependencies are not dependent on the size of the problem but on the necessary transforms

the dataflow that SQL uses for running a query is an example of this, complexity of plan is determined by joins/merges not by data size

dependencies still can make long chains but

the nature of the depenencies is expressed exactly once no matter how much data there is

Again, using promises as an example (and I am not down on promises at all, they work great, but you don't have switch to the wood pattern universally just because you have async i/o)

Wood:

create many async requests for data
attach additional promises to these requests for processing
end each promise chain with some operation to do whatever needs doing at the end (such as inserting stuff into a DOM)

Marble:

create many async results for data
give the promises to a broker which will handle all of them (and maybe created them in the first place as needed)
provide any number of handlers to the broker which will run those handlers on each datum as the promises resolve

In the marble pattern I can still write nice anonymous delegates if I choose, or I can provide a class or other handler, whatever is necessary. However I don't create one handler instance for every datum. I created one or more broker objects that handle the various stages of processing and coordinate some or all of the post steps.

From a memory perspective Marble is much more economical and of course it doesn't have the terrible property that I'm making lots of long lived objects. In the wood pattern everything ends up being long lived because everything lives at least as long as the I/O request is outstanding.

Measuring IE Performance

2012-02-17T16:25:00Z

The Building Windows 8 blog has an interesting article on how we measure IE perf.

Coding in Marble

2012-02-03T03:26:00Z

I wish I could remember where I first read it because perhaps it deserves attribution. But many years ago I read about the two world views of physicists and they resonated with me. One world view is that prescibed by things like General Relativity and Maxwell's Equations. These have, in some sense a great mathematical beauty to them. This is the "the universe is made out of marble" viewpoint. Nice clean fields. Very solid. Then there's the other perspective, this is the perspective you have in systems like Quantum Electrodynamics. There are lots of particles flying all over the place and probabilities. Lots of statistics. The universe is a messy organic place. It might have been tempting to take those people and lock them away or something but darn it if they didn't keep getting great results. So Marble and Wood. The highly regular vs. messier put powerful. I soon started using this analogy in computer science and that's what I'm writing about today.

One place this comes up is when programming with promises, or really any notification system even ad hoc ones. In the case of a promise the situation is that you have some work that needs to be done "later" after some asynchronous operation has completed: let's say that you are waiting for some data and when it arrives you're going to validate and extract some values from it, it doesn't much matter. The natural way to do this with a promise would look something like this:

p.then( () => {do your work} )

or in javascript you might write

p.then( function() { do your work } )

In fact you can keep combining these things, if you have more work that has to be done after the first batch asynchronously completes you might end up writing something like this:

p.then( () => {do your work} ).then( () => {do more work}).then( () => {even more work});

Now the next thing that's going to happen is you're going to put this in a loop for your various 'p' read operations that need to happen and presto magico your application is done. And you've fallen into the wood trap.

The reason that I try to avoid this pattern is that what's happened now is that on every iteration you're creating delegate objects and new promise objects that connect the various stages for each operation. But the thing is they're all the same! Despite the fact that all the objects are being handled identically (typical) we get no savings, we are using the most general dispatch mechanism to dispatch the very same code thereby creating far more garbage than is needed. Of course each one of these promise chains encourages the next guy to keep doing the same thing. You could even add stages that merge, select, batch, whatever you need, more stages, more wood.

The hallmark of the wood pattern is that the same chain of dependencies/computations is re-represented in data repeatedly -- achieving no savings for its sameness. It is the most flexible choice, each datum could be handled seperately in a unique fashion but they probably are not.

How can you make it better?

Well of course a different pattern alltogther might be the right choice, something that looks more like this:

datasource.handler += () => { do your work }

or maybe

w1 = new {worker stage one object }
w2 = new {stage 2 handler};

datasource.handler += w1;
w1.stage2 += w2;

Now this isn't as flexible but it is much more economical. With one fell swoop we're saying that they are all going to be handled symmetrically and we do not have allocation cost per datum.

Even if that kind of refactoring isn't possible just this simple thing might be very helpful.

var d1 = () => {do your work};
var d2 = () => {do more work};
var d3 = () => {even more work};

p.then(d1).then(d2).then(d3); // this in your loop

This second form avoids creating a whole new delegate/closure in each loop iteration so in some sense it's at least partly marblized.

Now of course this kind of transform isn't universally possible but if you start by saying "what are the rules for all my data" and then encoding that then you tend to end up in a place where you have low-to-zero overhead costs for each item and you just do the work. If you don't you can easily end up in a place where there most flexible pattern is being used in a very regular way, at great cost.

I've often admonished people to use the simplest programming technique that will do the job to get the best performance. Specifically, don't use virtual methods if non-virtual will do. Don't use interfaces if virtuals will do. Don't use delegates if interfaces will do. This discussion has in some ways been a rehash of that point. But then I always found that grounding a concept in an example is helpful so hopefully you all found it worth the read.

P.S. I'm putting the "wood" programming technique in a bad light here but it has its place. And that doesn't mean I'm down on "wood" physics. I happen to think QED is every bit as cool as cool as relativity :)

Ngen or not? The rules haven't changed very much since 2004

2012-01-31T20:53:00Z

I still get questions that amount to "should I ngen my <something>" from time to time and the best answer I can give is still "it depends." I wrote this article many years ago, and I'd say it's still pretty accurate: http://blogs.msdn.com/b/ricom/archive/2004/10/18/244242.aspx

Essentially the situation is this: if you ngen your IL then of course the jit will not have to run but you will have to do more I/O because the compiled code is bigger than the IL. If that I/O is likely to be cached because either:

The code is going to be loaded "a goodly amount of time" after ice cold startup and it's commonly used so superfetch is likely to fetch it, or
The code is in a library shared by many programs and so only one of those programs at most will have to actually read it at full cost

then it's likely that ngen will help you overall.

If that's not the case then ngen isn't likely to help you. But really you need to measure for yourself.

Remember also that the code generation for ngen'd binaries is going to tend to optimize for maximum sharability which may come at a (typically small) cost in raw speed so that's a factor as well.

The most common framework DLLs, like mscorlib, system.dll and friends, tend to be get the most benefits from ngen. Single-use application libraries and executables tend to get the least benefit.

It's really hard to say more than that with any kind of precision.

Improving .NET Application Performance and Scalability

2012-01-11T22:41:21Z

This series can still be found here:

http://msdn.microsoft.com/en-us/library/ff649152.aspx

While some of the content is stale since it refers specifically to .NET 2.0 I think all of the conceptual content broadly applicable (even beyond managed code systems to software systems generally.)

If you're looking for it some time in the future you can find it again easily with bing. I doubt that they'll get rid of it entirely for a very long time.

Performance and Design Guidelines for Data Access Layers

2012-01-10T21:47:00Z

Many problems you will face are actually the building data access layer, sometimes thinly disguised, sometimes in your face; it’s one of the broad patterns that you see in computer science – as the cliché says: it keeps rearing its ugly head.

Despite this, the same sorts of mistakes tend to be made in the design of such systems so I’d like to offer a bit of hard-won advice on how to approach a data access problem. Mostly this is going to be in the form of patterns/anti-patterns but nonetheless I hope it will be useful.

As always, in the interest of not writing a book, this advice is only approximately correct.

The main thing that you should remember is that access to the data will take two general shapes. In database parlance you might say some of the work will be OLTP-ish (online transaction processing) and some of the work will be OLAP-ish (online analytical processing). But simply, there’s how you update pieces of your data and how your read chunks of it. And they have different needs.

At present it seems to me that people feel a strong temptation to put an OO interface around the data and expose that to customers. This can be ok as part of the solution if you avoid some pitfalls, so I suggest you follow this advice:

1. Consider the unit of work carefully

There are likely to be several typical types of updates. Make sure that you fetch enough data so that the typical cases do one batch of reads for necessary data, modify the data locally, and then write back that data in a batch. If you read too much data you incur needless transfer costs, if you read too little data then you make too many round trips to do the whole job.

You may have noticed that I began with a model where you fetch some data, change it locally, and write it back. This is a fairly obvious thing to do given that you are going to want to do the write-back in probably a single transaction but it’s important to do this even if you aren’t working in a transacted system. Consider an alternative: if you were to provide some kind of proxy to the data to each client and then RPC each property change back to the server you are in a world of hurt. Now the number of round trips is very high and furthermore it’s impossible to write correct code because two people could be changing the very same object at the same time in partial/uncoordinated ways.

This may seem like a silly thing to do but if the authoritative store isn’t a database it’s all too common for people to forget that the database rules exist for a reason and they probably apply to any kind of store at all. Even if you’re using (e.g.) the registry or some other repository you still want to think about unit-of-work and make it so the each normal kind of update is a single operation.

Whatever you do don’t create an API where each field read/write is remoted to get the value. Besides the performance disaster this creates it’s impossible to understand what will happen if you several people are doing something like Provider.SomeValue += 1;

2. Consider your locking strategy

Implicit in the discussion above is some notion of accepting or rejecting writes because the data has changed out from under you. This is a normal situation and making it clear that it can and does happen and should be handled makes everyone’s life simpler. This is another reason why an API like Provider.SomeValue = 1 to do the writes is a disaster. How does it report failure? And if it failed, how much failed?

You can choose an optimistic locking strategy or something else but you’ll need one. A sure sign that you have it right is that the failure mode is obvious, and the recovery is equally obvious.

I once had a conversation with Jim Gray where I told him how ironic it was to me that the only reason transactions could ever succeed at all in a hot system was that they had the option of failing. Delicious irony that.

Remember, even data from a proxy isn’t really live. It’s an illusion. The moment you say “var x = provider.X;” your ‘x’ is already potentially stale by the time it’s assigned. Potentially stale data is the norm, it’s just a question of how stale and how do you recover. That means some kind of isolation and locking choice is mandatory.

3. Don’t forget the queries

Even if you did everything else completely correctly you’ve still only built half the system if all you can do is read and modify entities. The OLAP part of your system will want to do bulk reads like “find me all the photos for this week”. When doing these types of accesses it is vital to take advantage of their read aspect. Do not create transactable objects just bring back the necessary data in a raw form. Simple arrays of primitives are the best choice; they have the smallest overheads. Do not require multiple round-trips to get commonly required data or the fixed cost of the round trip will end up dwarfing the actual work you're trying to do.

These queries are supposed to represent a snapshot in time according to whatever isolation model your data has (which comes back to the requirements created by your use cases and your unit of work). If you force people to use your object interface to read raw data you will suffer horrible performance and you will likely have logically inconsistent views of your data. Don’t do that.

One of the reasons that systems like Linq to SQL were as successful as they were (from various perspectives I think) is that they obeyed these general guidelines:

you can get a small amount of data or a large amount of data
you can get objects or just data
you can write back data chunks in units of your choice
the failure mode for read/writes is clear, easy to deal with, and in-your-face (yes, reads can fail, too)

Other data layers, while less general no doubt, would do well to follow the same set of rules.

Performance Guidelines for Properties

2011-12-19T20:27:00Z

I can’t say I’ve asked the framework guidelines folks about this but I’m fairly sure there would be a lot of agreement from the guidelines gurus; so in the spirit of approximately correct advice I give you Rico’s Guidelines for Performant Properties.

I should start by saying I wish more people just used fields instead of committing these terrible sins with their properties. If we had the notion of a type-safe, read-only field the world would be a better place. Alas…

So, you’re using a property, the most important thing to remember is that it will seem very much like a field in all ways. It looks like a field and feels like a field, people will expect it to perform like a field. So with that in mind:

Accessing the property should not allocate any memory; people are going to use this thing in loops walking over whatever data structure you are presenting, they expect that this data is sitting around and being accessed.
Accessing the property should not “synchronize” – if there is any locking to be done it should be done at a higher level than a single field, by the time you’ve acquired whatever object has the property on it, it should already be safe to read it.
Accessing the property should not do any I/O, especially not any network I/O, again see above, by the time you’re reading the property the object offering it should have done whatever I/O was needed.
Accessing the property should not be an operation with complexity greater than O(1) – that means no loops. At all. You could haggle me as high as O(lg(N)) if we’re talking about an property that is an “indexer” but no higher.
Accessing the property should not have side-effects (i.e. it's strictly a read operation, it changes nothing)

What you’re left with is you can use your object state and the argument (if an indexer) to do a constant-time lookup, or log-time at worst, in an already existing data structure and immediately return the result. That’s it.

Why?

Pit of Success is the only reason you need. In my opinion, patterns more complicated than the above are doomed to fail. If you allow say network access, an RPC or the like, on each property fetch, then you promote things like field-at-a-time access to remote data. Not only is this astonishing to users of the API it is incredibly inefficient. If you follow the rules above you soon realize that you must allow some kind of query to get the data you need and then you can use properties all you like to access that data. That is a much better pattern, it leads to success.

Remember that if you allow things like I/O or synchronization you can easily get property access times that are measured in milliseconds, this simply will not do. A typical interactive scenario might have a budget of say 100ms for prompt response and it might require access to several dozen properties just to paint the screen properly. This has several issues:

If property access is say 1 millisecond then accessing 10 properties (not at all crazy) would be 10% of your overall budget – that’s nuts!
The interactive code must be able to use your properties with confidence, that means they cannot block and should not be subject to large variability, else you get UI that is non-responsive for extended periods of time

Frequent access to properties is very common and therefore must be astonishingly cheap if only to keep a handle on the Joules your program consumes – an increasingly important metric for modern platforms.

From a time perspective, I like to see properties that have access time measured in nanoseconds. Let’s say 10^-8s. I can reasonably see systems with tricky indexing that might go has high as 10^-7 or even 10^-6. When your time is getting to be 10^-5 or larger, you really need to start thinking about your design. That’s too much work to put inside a thing that looks like a field access. Give your users a fighting chance to understand where their costs are and let them query for a batch of results, probably asynchronously and then access them quickly when they arrive. That’s going to be a successful pattern.

It goes without saying that under these conditions, the flexibility to version/replace/subtype virtual-call properties is actually illusory. If you were to try to do something new and costly, or even just very different in a subtype you would likely cause all manner of problems for callers. So while you do get to change things a somewhat, you cannot and must not, use that flexibility to access new and different subsystems or the like inside of a method that is supposed to be as small as a getter. You are necessarily very constrained.

When I consider all of this I arrive squarely at the conclusion that more fields and fewer properties would be better. Don’t hide real work inside a property and don’t use them as a synchronization tool. Once you decide that you need a fetch/read pattern you soon find that that reading thing can be very simple/fast indeed and that perhaps fields were just fine after all.

Something to think about anyway.

Going to Internet Explorer

2011-11-12T17:21:00Z

Many of you have commented that I've been kind of quiet lately. This was largely a reflection of my job, I was working on an internal team that I could not write much about for the last couple of years. I had a great experience there but that trek is now coming to an end. In a few days I'll be working on Internet Explorer and back in the thick of the public facing products. IE is a very exciting place to be and I'm sure it will be loaded with issues I can write about that will be of general interest without giving away product plans and so forth.

I hope to be writing a lot more in the coming months.

Thank you for your support over the years :)

Measure!

2011-09-20T19:41:12Z

I know I haven't posted for a while; I'm hoping that will change soon. However in the mean time my daughter forwarded me this link because she thought it was cute.

The thing is, she didn't realize it should be my theme song.

http://www.youtube.com/watch?v=MMsQYjYlBEo

It's rule #1

http://blogs.msdn.com/b/ricom/archive/2003/12/02/40779.aspx

Now even Sesame Street agrees! It must be right!

Less Loosely Coupled Than Meets The Eye

2010-09-28T04:35:00Z

I don’t know that it is possible to write anything like a unitary software system in a way that is truly loosely coupled. It’s not that you can’t make boxes and lines that are cleanly separated in all sorts of pretty ways, though that’s hard enough. The problem is that even if you manage to do that, what you end up with is actually still pretty tightly coupled.

What do I mean?

Well, let me use Visual Studio as an example. It’s made up of all kinds of extensions which communicate through formal interfaces. Putting aside the warts and just looking at how it was intended to work it seems pretty good. You can make and replace pieces independently, there is a story for how now features light up; it’s all pretty good. Yes, it could be better but let’s for the moment idealize what is there. But is it loosely coupled, really?

Well the answer is heck no.

These systems all share resources in fundamental ways that actually tie them together pretty tightly. It starts with the address space, each extension can be victimized by the others, and it goes on. In fact every important resource on the machine is subject to direct interference between nominally isolated extensions. Is this avoidable?

I don’t think it is actually. I think unitary software systems are fundamentally tightly coupled when you look at them through a performance lens. And that is why I’m so often telling people that they need to look at their overall system and see if, when it’s all put together, it has any hope of working the way you want. Two subsystems that both (loosely) use 2/3 of the L2 cache are going to use 4/3 of a cache… that’s not good. There may be no lines between them in the architecture diagram but they are going to destroy each others ability to work.

So, does that about wrap it up for agile then? Should we just waterfall our way to victory?

Hardly.

Everything you’ve ever read about requirements changing and not over-designing systems that are loosely specified (much less loosely coupled) still applies, but just because you’re going agile and you want to keep your flexibility is no reason to not think about what it’s all going to look like when you put it all together. Some kind of blend is needed.

If you do run into problems that loose architecture is going to help you. But waiting until you’re done and counting on your agility to save you at the finish line isn’t so smart either. Agile development doesn’t prohibit you from planning out the parts that need planning and understanding your key constraints. In fact I think it encourages this – knowing your constraints keeps you honest and lets you make adjustments smartly as you go.

Performance is never loosely coupled. Don’t be fooled by your diagram, the profiler doesn’t know where your little boxes and lines are. Trust me :)

Debugging Multi-threaded Applications: Some Tidbits

2010-06-04T20:06:00Z

I was lamenting that we haven't really done terribly much to make multi-threaded debugging easier in say the last decade and I was fortunate enough to be able to have a conversation with Brain Crawford about it. Brian is a long-time friend an colleague, we worked on the VC++ debugger years ago, he's the lead architect for all things debugger these days. Anyway, he gave me this succinct information about VS2010 and was kind enough to allow me to share it with you all.

In his own words Brian writes:

While I agree that there is more we could do to make multi-threaded debugging even better, VS 2010 has a number of features that help with multi-threaded debugging for both native and managed code:

Stepping has affinity with the thread being stepped. Whatever thread is active (has the yellow IP marker next to it in the Threads window) will be stepped and no other threads will complete the step started on that thread. However, by default, breakpoints will fire on any thread, so you can step one thread and hit a breakpoint on another.
Any time you started execution (step or go) on a thread and execution stops on another thread (e.g. you hit a breakpoint or exception), there will be a small blue marker overlaid on the yellow IP arrow shown in source code to indicate a thread switch occurred.
Per thread breakpoints can be created by setting a breakpoint and then right clicking on the breakpoint glyph and choosing Filter. In the Filter dialog, you can enter an expression like “ThreadId=5” to restrict the breakpoint to a particular thread.
The threads window in VS 2010 has a number of features to help manage multiple threads:
- Threads can be flagged manually so it’s easy to find the subset of threads you’re currently working with.
- Threads can be flagged according to just my code or by particular modules.
- Call stacks for each thread can be viewed by hovering over the Location cell or the call stack can viewed inline by expanding the Location cell (so multiple call stacks can be viewed simultaneously).
- Threads can be grouped according to different criteria, such as process, name, category, flag, etc.
- Threads, including the call stacks for each thread, can be searched using the search box in the window. For example, if you’re looking for all threads with Foo on the stack, enable Search Call Stack, and type Foo in the Search box.

For multiple threads running the same/similar workload, the new Parallel Stacks view is very useful. While primarily designed for use with the parallel libraries and runtime, it can also be useful for traditional multi-threaded debugging. It consolidates threads with common stacks into a “cactus stack” view. I don’t know the kinds of scenarios you’re debugging, but it might be worth checking out. Here’s some additional info on this view: http://msdn.microsoft.com/en-us/library/dd998398.aspx

Thank you Brian, and please get a blog soon :)

A few words about Micro-Benchmarks

2010-04-27T00:57:00Z

It’s been a long time since I included my “this discussion is only approximately correct” disclaimer so I’ll just preface it here. In the interest of space and clarity, this discussion is only approximately correct. OK, now we can move on…

I love micro-benchmarks.

Really.

I rely on micro-benchmarks to help me understand what is going on in my system on a day to day or even build to build basis. Understanding which micro-benchmarks are really vital is essential to keeping your performance under control. But you have to be careful not to do more than this.

One thing you should never do is use your micro-benchmarks to drive prioritization of your performance problems. Remember, a micro-benchmark’s mission in life is to magnify some aspect of your programs performance. Because of this anyone can readily generate a micro-benchmark that makes any given issue seem like a major problem. Do not be fooled! Micro-benchmarks in-and-of-themselves tell you almost nothing about what is important. What they help you do is manage and track the important things once you have discovered them with representative workloads.

Once you have established which micro-benchmarks are key performance drivers in your system, you can use this to help you make progress on the large problems more quickly, but you must be careful to re-validate. Seemingly positive-looking changes in the micro-benchmark can easily be negative under your real workload because the micro-benchmark is often much less resource constrained. For instance, you could make a change that doubles your metric at the expense of addition L2 cache usage. That might be a good thing but it could also be a disaster, the collateral damage you would do to the rest of the program could easily erase all those gains.

So, should you use these quick running scenarios? Yes, but certainly not exclusively. And not as the primary planning tool, but as a planning aid.

My History of Visual Studio (Epilog)

2010-04-14T19:19:00Z

Visual Studio 2010 Launched on Monday. Wow! It’s HUGE. A major round of congratulations are in order for everyone involved, not just on the Visual Studio team but also on the Frameworks team and the supporting teams and of course the customers whose feedback was so vital to the success of the product.

I’ve already written a lot about VS2010 previously in the series and I don’t want to go over all that stuff again. In my last history posting, back when Beta 2 came out, I covered all the major things that we wanted to get into this version of Visual Studio from an architectural perspective. You could look down that list and get a feel for whether or not the release was a success from that perspective. I think key architectural features like a full WPF surface, MEF extensibility, extension management generally, and of course multi-targeting were delivered. General purpose multi-targeting is a huge thing because it means we don’t have to do a VS SKU release for every new target platform that comes out – that was a critical thing for our own sanity. And WPF has certainly (finally?) proved itself as a viable platform for a flagship application. We set out on a multi-release remodeling plan and I think we delivered an excellent first-installment.

You might be wondering why I was so quiet during the home stretch. I’m afraid my perspective on the VS RC and RTM milestones was an outsider's. The Beta 2 push was my last major hands-on involvement with the product; probably the simplest diagnosis is that I burned myself out at that point. In any case I took a vacation after that Beta with the goal of returning to find a new challenge. I’d been in the Developer Division about 7 years, for the second time, and I like to move around about that often, so it was time. I’d love to take some credit for fixing the daunting memory leaks that were left in Beta 2 but I’m afraid all credit for that goes to my capable colleagues to whom I am forever grateful for taking this particular release across the finish line.

Some things were lost on the way in the name of getting the necessary stability and basically locking things down. It’s kind of a sad thing when that happens but it’s a necessary thing – they’ll just have to wait for the service release. Nearly ready to go things that just needed a little more bake time are a great thing to have in your pocket at that point, but there are a few especially delightful things that I wanted to mention, sort of unexpectedly delightful, for me anyway.

We started with an ambitious goal to make a major dent in Visual Studio’s setup experience. A lot of what we wanted to do ended up having to be pared back and I feared we would end up making little progress but in the final analysis I think the setup experience is substantially better. Seeing tweets on how people had successfully installed in less than 30 minutes was very nice. More than I hoped for in the darkest hours. My own experiences during the beta were the best of any VS release I’ve ever installed – and that’s a lot of releases.

I feared the historical debugging feature, IntelliTrace, was going to be too bulky to use all the time but that’s not proving to be the case at all. I’m just getting used to having it available all the time and it’s proving to be much more of a game-changer than I ever imagined. I wrote more about the history of that feature previously so I won’t go on other than to give some kudos to those that worked hard to make that feature reality.

I didn’t think TFS was going to be so universally important but the ease with which TFS Basic can be brought online was just staggering. In just a few minutes I configured my own home TFS installation – it’s great to have that kind of quality ALM system so easily available to everyone.

The rate at which new extensions are appearing in the gallery is incredible. I’m so glad to see this hub creating energy for all our customers, but there was one extension in particular that made my day. Dmitry Kolomiets produced, I think, the first Solution Load Manager extension. Why do I care so much about that particular extension? Well there’s a story there: from very early on in the history of VS2010 I was espousing a philosophy that we could not universally load every project in a solution and achieve our memory goals – many solutions are just too large and it’s entirely normal to not need every project. But there is no reasonable way to manage solutions. Unfortunately there was just not enough time to make this feature a reality – but there was enough time to add the necessary APIs so that a 3rd party could make us load less and manage what was loaded. With that support they could build their own UI to drive the experience. It was hoped that the performance benefits from such a system would motivate the creation of a variety of managers with different policies – potentially tied in to the great refactoring engines engines already out there (e.g. CodeRush and Resharper). But it was just a hope. Now there is one!

I could go on for ages. The diagramming support and of course our uses of it. Integrated Silverlight support, ASP.NET 4 with MVC2 (read about it on Scott’s Blog). Language improvements in C#, VB, and C++. Did I mention Windows Azure support as well? Sharepoint support? F#? It’s huge.

This was only “My” History of Visual of Visual Studio. For me this is the "epilog" but for everyone else, VS2010 out the door, it’s time to begin your own History.

Congratulations and Enjoy!

Variability in Benchmarks

2010-04-07T22:43:00Z

I’m not especially a great fan of micro-benchmarks, they’re handy as a compliment to the larger tests but I often find that they lead to confusion because they are interpreted as reflecting reality when nothing could be farther from the truth. The fact is that micro-benchmarks are intended to magnify some particular phenomenon to allow it to be studied. You can expect that normal things like cache pressure, even memory consumption generally, will be hidden – often on purpose for a particular test or tests. Nonetheless many people, including me, feel more comfortable when there is a battery of micro-benchmarks to back up the “real” tests because micro-benchmarks are much more specific and therefore more actionable.

However, benchmarks in general have serious stability issues. Sometimes even run-to-run stability over the same test bits is hard to achieve. I have some words of caution regarding the variability of results in micro-benchmarks, and benchmarks generally.

Even if you do your best to address the largest sources of internal variability in a benchmark by techniques like controlling GCs, adequate warm-up, synchronizing participating processes, making sensible affinity choices, and whatnot, you are still left with significant sources of what I will call external variability.

The symptom of external variability is that you’ll get a nice tight results from your benchmark (a good benchmark might have a Coefficient of Variation of well below 1%) but the observed result is much slower (or, rarely, faster) than a typical run. The underlying cause varies, some typical ones are:

Scheduler variability – the scheduler makes an unfortunate choice and sticks with it

The entire run is much slower than normal

Uncontrolled background activity external to the test

Part of the run is slower, maybe several consecutive test cases in a suite

Interference from attached devices

Incoming network packets or other sources of interrupts

These problems are often not the dominant sources of variability initially but as the benchmark is improved, controlling for other factors, they become dominant.

Statistically the underlying issue is this: repeated benchmark measurements in a run are not independent: a phenomenon affecting a particular iteration is highly likely to also affect the next iteration. A closer statistical model might be independent measurements punctuated by a Poisson Process that produces long lasting disruptions. "Waiter, there is a fish in my benchmark."

My experience with this sort of thing, and I expect the battle-scarred will agree, is that reducing external variability is an ongoing war. There isn’t really a cure. But it does pay to not have hair-trigger reflex on regression warnings, and to screen for false positives in your results.

My Annual Personal Posting

2010-04-06T06:26:00Z

I try to keep the content on this blog strictly professional (although sometimes it's like an editorial but at least it's topical) About once a year I break down and write something that's basically just some personal thing I felt like sharing. This is my link for this year.

A couple of years ago my family became involved in animal rescue (mainly cats and dogs). It all started when my wife asked me if we could get a third dog and I told her that was a bad idea -- "too many" I said. But I thought fostering would be a good thing. Ha! I had no idea I was saying "no" to three and now sometimes we have nine in the house!

Still it was the best stupid thing I ever said.

One of the organizations our family is associated with is Furbaby Rescue here in Washington. If you are a pet person, or think you might like to be one some day, I invite you to watch this.

http://www.youtube.com/watch?v=YExxNEpEdJY

Warning: it's quite sad as are many of rescue stories. But I hope this one has a happy ending when all is said and done.

Why you really want to avoid catching and rethrowing exceptions

2010-04-05T19:18:00Z

I like processes that simply drop-dead fail when they have an unrecoverable fault. Trying to continue is often dangerous and unlikely to actually help anyone. This policy is all fine and well but in all cases it is vital that the “death” stack of a process be the true problem, not a generic error handler. Those kinds of handlers ruin the lives of developers who are trying to analyze error logs, and effectively make services like Watson useless. I can’t stress enough how critical it is to NOT mess with the failure stack.

I’ve seen many of these in the wild, including of course Microsoft code, and it means that instead of getting a nice actionable stack in the log (which would have been a Watson report in the field) you have to go back to the code and try to repro. In the case below I stared at the source for a while trying to figure out what was likely wrong and it turned out the true problem was quite distant from the likely candidates.

2010/04/05 11:02:14 AM BEGIN Peek_Range TEST
2010/04/05 11:02:14 AM ---- Exception in [StackTest] thrown type=[TestCodeException], eip=[00000004`4716C93E], message=[Index (1) is out-of-range [0..1]!]

That was the failure right there, and that stack was the one I needed! Nothing after that should have happened... But... execution continues until finally....

2010/04/05 11:02:15 AM Test completed exceptionally [Peek_Range]: Index (1) is out-of-range [0..1]!

The stack reported at this point is useless… the real problem was millions of cycles ago. I'm not even including the stack from the log because it's just stupid.

The real stack can be extracted with a repro (at great pain)

And it ends up looking something like this (cleaned up)

StackTest!TestUtilities::ParamCheckIndex<int32>
StackTest!CollectionTestUtils::TestUtilities_SimpleList<int>::RangeCheckIndex
StackTest!CollectionTestUtils::TestUtilities_SimpleList<int>::get_Item

The actual problem is in the test code! The only useful information from the original log was basically the test that I should run. Thank goodness it’s a 100% repro.

VSX Keynote posted

2009-12-09T01:55:00Z

Just a quick plug, if you missed the VSX conference you can see it all online on Channel 9. Some great talks including my keynote speech on VS Futures.

VS2010 Beta2 performance and other issues

2009-10-28T20:08:00Z

Just a few words of encouragement today: I can't emphasize enough how valueable your feedback is to us at this point; no matter how hard we try we simply cannot duplicate the diversity that is the real world. We need to know what's really working out there -- what works for us isn't enough. Whether it's performance, stability, or any other issue, your feedback will help us to "get it right" for the release.

I've seen some super-positive comments about product performance and I've seen some negative ones. Your comments will help us to figure out which areas we've missed, what combinations aren't working, what hardware are we using poorly, or any other goofy thing we might have done.

You can use http://connect.microsoft.com/ or you can provide comments here. We're even watching twitter and the blog-o-sphere generally but those are less reliable.

Brian Harry is also reaching out http://blogs.msdn.com/bharry/ and I know all the other executive bloggers have their ears on -- and they'll get help. Connect bugs are the best of all but don't be shy, we'll take the feedback however we can get it.

Thank you!

Inside Visual Studio Beta 2 - Performance and Reliability

2009-10-20T19:13:00Z

After my keynote speech yesterday at the Development Tools Ecosystem Summit Charles Torre caught up with me and we made this video. Hot off the presses: Rico Mariani: Inside Visual Studio Beta 2 - Performance and Reliability

It's super-exciting :)

If you missed the keynote you can get virtually all the content by watching this video and reading the last entry in my History of Visual Studio series My History of Visual Studio (Part 10, final).

My History of Visual Studio (Part 10, final)

2009-10-19T21:00:00Z

[All the other Parts: History of Visual Studio]

[Visit the Microsoft Visual Studio 2010 and .NET Framework 4 Beta web site for the latest info]

Visual Studio 2008 Winds Down, Visual Studio 2010 Begins

Just as things were starting to wind down on VS2008, I got a new assignment. I was going to be Chief Architect of Visual Studio. Wow. So here this series takes a bit of a turn because “my” history is quite a bit different at this point – I actually stopped working on the current product at all, and much as I would have liked to write about what I was doing back then, you can imagine there wasn’t a whole lot of enthusiasm about starting blog discussions on what we should do in “Dev10” (the term “Hawaii” never stuck and was soon abandoned) while we were trying to launch VS2008.

Remodeling

From a technical perspective, the idea of VS2010 was to begin a remodeling process that would span several versions. Making key investments that would help long-standing problems and at the same time offer some immediately compelling features as well. I gave a lot of thought to what enabling steps we would have to take to make our experience be something great in say three versions. It was hard to see exactly what that future might be, so I tried not to think about specific features at all. I had seen all sorts of “flashware” concepts for things we might do and I tried to think about what key problems we would have to solve to implement things of that nature. Using all that, and what I already knew, I came up with a list of fairly low-level considerations for a “VS of the Future” that I thought we should work on, and why.

I think the actual document I created is pretty dry, it’s full of observations about the size of processor caches, the number of cores, the memory wall, the locality of our data structures, blah, blah, blah. It goes on like this for pages, but I think I can summarize it into a couple of major themes.

Concurrency

The first major theme is concurrency, and the need to increase our ability to use it successfully. You might think that with all of those threads Visual Studio spawns that there is a great deal of parallelism but, for the most part, it’s only superficial (there are exceptions), what happens more often than not is that those threads run something like relay race with only one thread having the baton at any given moment. One key reason for this is that due to the single-threaded history of our code-base, and indeed many Windows applications, a lot of the objects are tied to the STA that is the main thread.

To create opportunities for concurrency you have to start to disentangle the important data structures from the STA and give them better access models that are friendly to concurrency. A great example of this is the new editor in VS2010. It allows you to access editor buffers via immutable micro-snapshots. Previously, if you wanted to do a something like say background analysis on a live text buffer, you had to do crazy things like copying the entire editor buffer on every keystroke. Concepts familiar to database programmers, like isolation, need to be present in the key structures of any concurrent sytem.

The STA concerns are a bit like a plague, they can cause a massive “entanglement” of data processing and presentation. Transitions into or between STAs can result in unexpected message pumping, leading to bizarre re-entrancies which in turn result in astonishing bugs that are very hard to find and fix.

Additional entanglements occur between processes trying to create parent/child relationships between their windows where the single pump effectively fuses together certain STAs – a necessary step given typical application assumptions but a total bug nightmare and a parallelism disabler.

Ultimately the most important thing is to make it as easy as possible to move work off of the UI thread. I think it’s vitally important that there is a single thread responsible for the UI – it’s the sanest way to preserve the order of operations the user initiated – the causality from their perspective if you will – but as their requests become non-trivial you need the flexibility to do them asynchronously.

There are two major kinds of background activity that come up in an IDE; it is easiest to illustrate these two needs by example:

Search for “foo”
- This can be done in a totally asynchronous fashion with different “threads” proceeding in parallel against different editor buffers or other searchable contexts. All the foos light up as they are discovered for instance.
- Foos that are not visible need not be processed, so this sort of parallelism has the option to be “lazy”
Rename “foo” to “bar”
- We can still do this asynchronously but now there is an expectation that we will eventually do it all and of course we must do all the foos not just the visible ones
- Generally this requires progress, and a transaction
- It can fail, or be cancelled, these are normal things

Both of these represent two cases where we can use a “coarse” parallelism model (e.g. one file at a time, independently). Of course we also would like to do fine-grained parallelism in places and in fact searching could be amenable to this as well (e.g. search odd lines with one thread even lines with another). Fine-grained parallelism has the opportunity to keep data locality good because you can access the data in one stream rather than access disparate data streams all at once, at the expense of more complex scheduling. Locality is good for performance, more threads are good, sounds good but you need some fundamentals to make any of this happen:

Limited memory usage in the face of many threads running
Ability to safely access the data in slices that are durable and limited in span
Isolation from changes that happen during the operations
No STA requirements in the data access and a asynchronous way to deliver updates to the UI

Putting the text editor on a plan that can satisfy these needs is itself a big step forward but on a long term basis it would be nice to do many things on this plan.

New UI Model

Again, I went on at great length about this in my original memo but I’ll treat this topic even more briefly than concurrency. The long and the short of it is that your average developer’s machine has the equivalent of a few 1990s supercomputers for a GPU (we could haggle over how many) and it’s important to be able to get good mileage out of that unit. It’s hard to imagine achieving those kinds of successes at reasonable cost using the good old WM_PAINT + GDI model – a high quality retained mode graphics system is basically a must.

In addition to a making it easier to use the GPU, there is also a tremendous desire to do good separation between the UI and the data elements of your design. Not just because this is architecturally sensible and clean but, as I discussed in the previous section, because to do otherwise significantly hinders your ability to achieve a good degree of parallelism. The retained mode graphics model actively encourages badly needed separation and you all know I love the Pit of Success.

Ultimately that leads to the choice of WPF for presentation in Visual Studio because the long term choices are to either use that or roll our own. However WPF and Silverlight evolve in future frameworks, and you’d have to be crazy to think they won’t, we’re better off using WPF notions than relying on the past, or worse, on something of our own making.

What came of all this?

I’ve deliberately avoided going into feature discussions so far, just as I did in my original memo, but ultimately we had to choose particular product areas where we would take these notions and invest in moving them into a good direction. That’s the point of the “remodel” after all.

These are some of the major initiatives that got an investment; they had impact throughout the entire product.

Multi-targeting

A new framework, and in particular a new CLR, essentially mandated that we do this work. We had been able to do this more cheaply in previous releases because all the target frameworks shared a CLR (the CLR had been on v2.0 even though the framework moved on to 3.0 and 3.5). This base level of compatibility allowed a simpler “onion-skin” kind of model to deliver an effective solution in Orcas – that wasn’t going to work in Dev10. In Dev10 we had to be able to target CLR 2.0 and CLR 4.0 and we had to do it in such a way that any given solution might use the related frameworks in any combination. We certainly couldn’t assume that the mscorlib that the IDE had loaded was the one that the user wanted to target.

Then there’s the matter of tools. There are different compilers for different frameworks and we can’t assume that, for instance, the C# compiler for version 4.0 is the right tool for the job when building a project designed to run on version 3.5. And even if we were super careful with the compilers, what about all the other tools that create outputs, like say resources? Those assets might be in a format that is serialized with the expectation that a particular framework will load it. So no one tool could do the job.

And let’s not forget the native tools, they have all these problems too, though I’ve been using managed examples, you can search and replace the runtime names in my discussion with native tools and libraries and you get basically all the same phenomena.

So in VS2010, based on the project target, we have to select the right libraries and tools to create all your outputs and we have to be able to vary these on a project by project basis. That sounds good.

That’s just the start.

The build chain is in some ways the least of our problems, what about all those designers? What about Intellisense? As you change focus from from file to file, depending on the framework you’re targeting in that file we need to show you Intellisense for just the methods you can actually use in that context. Likewise designers should include only choices that are reasonable for the current target, so for instance, the items in the Winforms designer toolbox should not include controls that are available in the version of the framework the IDE happens to be using, but rather the ones that are available on the desired target, and of course those are not likely to be the same.

Add to this mix the desire to target platform subsets like the “client profile” of framework 4.0 and you’ve got quite challenge on your hands.

I remember a very compelling breakdown that the teams used to explain the levels of success to management (and me)

Good: Designers and Intellisense prevent you from making mistakes; you see only things that you can really use for your target.

Not Good: Designers didn’t warn you, but at least you got a compile time error that was helpful and can be corrected.

Bad: There was no compile time error but when you ran the code you could readily see that it had some target specific aspect that shouldn’t be there and you could correct the code to avoid the problem.

Ugly: As above, but there is no reasonable diagnostic information and/or no reasonable way to fix it.

Obviously we want all aspects to be “Good” and if that can’t be achieved then as few as possible cases in each successively lower category. I think you’ll find that as of Beta 2 we’ve been very successful.

New Editor

Without going into a full discussion of the problems with the “old” editor it is worth mentioning these basic facts:

The old system was blocking innovation because of limitations in how it does the drawing – adornments are painful to add
It was “tricky” to use in non-trivial cases, often requiring extreme interventions such as wholesale copying of buffers to achieve stable state, the locking mechanism is prone to mistakes
When combined with its undo-manager it sometimes (always?) highly wasteful in terms of memory
It has assorted algorithms that do not scale well, generally those to do with region tracking – increasingly critical to editor consumers

Many customer performance complaints boil down to lack of scalability in the current editor structure. These issues in turn boil down to non-linear memory usage, and non-linear algorithms, where sub-linear choices are possible, and necessary, to create the proper experience. We get real scalability complaints from customers on an ongoing basis and these problems can’t be solved by “tuning.”

To address these problems we had already built a new editor platform, including visualization and text management services. The design had proven scalability and extensibility and had already shipped (in Blend). It was a very real technology. However, there was a great deal of code in Visual Studio that depended on old editor behaviors.

To simplify adoption of the new editor we created “shim” services that supported the most popular of the old interfaces allowing existing clients to continue to use those interfaces. Since most editor consumers actually use only a subset of the whole API we were able to substantially mitigate the cost of conversion across Visual Studio – shims weren’t a universal solution, but they helped.

Now I say that like it was some easy thing but wow it’s a ton of work even at the “reduced” cost. And the number of shims seemed to keep growing, and growing, and… it took a while to get this under control.

There were other problems. The new editor wasn’t identical to the old, despite being more functional overall it was missing some things – like column select – that would need to be added (it’s in Beta 2). Then there are the complicated editor interactions in cases like those presented by the Web Forms designer – the presence of markup intermixed with a language (like VB or C#) makes Intellisense very tricky. The language service needs to see the file without the markup to provide coloring and Intellisense but of course the user edits the composite file. Getting this to work on the new editor was quite a trick.

Having gone through all this I think I can say that the new editor has a very bright future; it has unprecedented extensibility – everything from text replacement, to object insertion, to HUD–like overlays. In this version we’ve barely begun to tap its capabilities.

New Shell

It would be a mistake to think of the new shell as just a redo of the old in WPF, it’s quite a lot more than that. There is a whole new toolbar system, a whole new menu system, both based on WPF. The docking controls, easily taken for granted, those are all new. There are the bits that allow document windows to completely float for better multi-monitor support. There are all the decorative bits, the window tabs, the output window tabs. It’s starting to add up but these things are only one aspect of the Shell, its UI.

Behind the scenes, there to enable pieces like the new editor that I discussed above, and the project system that will be next, are the Shell Services. You might see screenshots of the new Extension Manager and the Visual Studio Gallery, they are in some ways the face of much of the underlying work: a new deployment model for extensions centered around the VSIX packaging format and the MEF extension model.

But those dialogs really are just the face, to make the systems work there are many invisible things, like the MEF catalog and services that manage extension state. In fact, if you were to visit the Shell Services link I provided above, you could go down that entire list and you would find very few of the major systems that were not affected in this rewrite. From the Start Page, to Intellisense, most services have had significant work. MEF itself lays out a roadmap for future levels of extensibility in the product, simultaneously more comprehensive, easier to use, and in more areas.

That last notion is perhaps the most important one, I’ve said this many times in various speeches but it bears repeating: It’s hard to look at Visual Studio, a product that is substantially ALL extensions, and say “it needs more extensibility” – but it does. The original Visual Studio shell provided excellent extensibility of the core services – VS itself is the proof of that – but what is lacking is general extensibility of the extensions themselves. These extra levels are what we sought to add in Editor and the Common Project System, and hopefully they will pervade the product over time.

“Common” Project System

I’ve alluded to this system several times so far in the document; it’s also an important step on a long-term plan.

Anyone who has created a project type for Visual Studio knows that it can be a serious pain to try to keep up with the Joneses. The flagship languages add new features (ClickOnce is the canonical example) and since there is no central place to put project features, they end up having to be re-implemented in the code for many different project types. The ability to create project “flavors” (described a little bit here) helps alleviate this somewhat but certainly does not provide complete relief.

So, at the start of the cycle we find ourselves in a situation that is something like this:

We’d like to create project system that would provided the needed extensibility so that we could unify many project types
We think it would be nuts to create this thing and then try to change all the projects to that plan in one release
We have the C++ project system which needs to move to MSBuild anyway to be more like the others and gain MSBuild's incremental build and scale benefits

So the solution seems obvious – we’ll start building “CPS” and we’ll do C++ first, that way we have a test case, we don’t have to change the world all at once, and the C++ team will no longer have to be in the business of managing all its own project stuff. The reason that I quoted “Common” is that, at least for now, it’s only the C++ project system – though it has ambitions to be a lot more.

How hard can that be? It’s only an all new project system with virtually all of its attributes encoded cascaded XML files, general purpose editing UI, with pervasive MEF extension points, and enough scale to handle the largest C++ solutions we have. Why it’s triviality itself J

Other Highlights

When I wrote about “Dolphin” I gave a litany of changes, it was an impressive list, but I think I can say with candor that I gave a fairly complete list of the major new pieces of work in that release. I was able to do that because the team was a lot smaller and the whole release could fit inside one person’s head. I’m not even going to pretend that I can write about everything that’s in VS2010 – So far I’ve stayed very close to my own team and the things that were most dear to me – this is only “my” history after all, the complete history would require input from thousands. So even though I can’t possibly be complete, there are a few of my favorite new things that I want to talk about.

The New C++ Code Model

The VS2010 C++ Code Model is essentially a complete redo of the old technology. Now I’m a funny guy to be writing about this because probably more than any other one person I’m responsible for creation of the old technology. All those .bsc files, that was my baby, we came up with the current system when we were working on Minimal Rebuild (previously discussed) and several of my friends had the joy of maintaining that code in the years after I left the C++ team (thanks, you know who you are, and sorry!). The .ncb code was the “no compile browser” and the original flavor of that was created by another fellow who I supervised, no name dropping, this was another case of “give the new guy this really hard project, I’m sure he’ll do fine”. Gee I find myself apologizing a lot in this history.

Anyway suffice to say they were fine in say 1993 but I think it’s possible to do a lot better in 2009 and nobody is happier to see my old code leave the building than me. The new system offers improvements almost across the board, especially in reliability and performance.

Historical Debugging

You know I couldn’t do another entry without talking about debugger technology. Historical Debugging is a feature that’s been incubating in the minds of debugger people probably for as long as they’ve been debugging.

I was the debugger lead in the early 90s and I used to explain the utility of debuggers and debugging tools in this way: Imagine a program with a bug, it has been running along, everything is fine, everything is going wonderful, the flow of execution arrives at a point we’ll call Albuquerque, where it turns right. Now as every Bugs Bunny fan knows, the correct thing to do at Albuquerque is to turn left. The program’s decision to go right has led it down an incorrect path and sometime later we will observe a problem.

Now if we’re very lucky “sometime later” will be very soon, like for instance it might be that we just de-referenced a null pointer and we’re going to take an exception about 2 nanoseconds after the mistake. That’s an easy bug to fix. On the other hand it could be that “turning right” was more subtle – maybe we corrupted a data structure in a minor way and it might be days before we can see an observable effect – that kind of bug is a nightmare.

Finding “Albuquerque” is what I call The Fundamental Problem of Debugging. The debugger provides you with tools (e.g. breakpoints) that allow you to stop execution while things were still good and slowly approach the point where things first went wrong. The debugger likewise provides you with tools to examine the state afterwards, hoping to find evidence of what went wrong in the recent past that will help you to see the origin. The callstack window is a great example of looking at the past to try to understand what might have already gone wrong.

To find the problem, you might start after the failure and try to look back, finding a previously unobserved symptom and moving closer to the original problem or you might start before the failure and try to move forward slowly, hopefully not overshooting the problem by too much. Or you might do a combination of these things. You might add assertions or diagnostic output to help you to discover sooner that things went wrong, and give you a view of the past. It’s all about finding Albuquerque.

Historical debugging directly addresses the Fundamental Problem by giving you the ability to look at a lot more of the past. And it’s far superior to the limited ideas we had back in the 90s for accomplishing the result. I know a couple of people who have had this idea in their head for over fifteen years – it’s great to see it out there for programmers to enjoy. I think you’ll love it, it’s like a drug! J

Help

The Help system was another case where we saw very early on that it should get a remodel. I think the energy around that effort began with a very well written memo from a friend of mine. Frankly the sad truth is that it was pretty hard to find people who had anything nice to say about the Help system in late 2008. I think many users had given up and were just using their favorite online search to find what they needed, which, as a doubly tragic statistical truth, was probably not our search.

The Help team didn’t have an architect so I loaned myself out to them and together we made what I think are some pretty cool plans that have borne fruit. The new system is much lighter weight and is standards based. Taking a cue from MS Office the content files are basically XHTML in a .ZIP file with a custom extension MSHC. This is an archivist’s dream, but it also opens up the Help authoring space tremendously – with no documentation at all you can now just look at one of the archives and pretty much figure out how to get your content anywhere in the system.

For offline viewing, the customer help viewer is gone, replaced with a mini localhost web-server that delivers the help to the browser of your choice from the archives. We tried various indexing strategies for full-text and other searches but ultimately settled on a custom indexer that I wrote in a few days (well, to be fair they did a lot of work on it after I finished my parts, but I wrote the core). The indexer is actually very similar in operation to my old bscmake program, the C++ program database generator – those techniques fit Help content well, and it produces MSHI files.

Importantly, you don’t have to create our index format to publish Help for your topics; the MSHC file has all the information in the XHTML. We ship pre-built MSHI files to save installation time because of the size of our corpus, but you don’t have to. Additionally, the MSHI files can actually be merged as much or as little as desired for even more performance. But don’t merge MSHI’s from different locales because there is no usable universal sort order that looks decent to customers.

A 3^rd party help viewer need not use MSHI at all, you could easily build your own index.

The content itself is delivered in a much lighter weight format, the big tree control is gone. In fact most of the styling for the pages is itself stored in the content as a topic and it can all be updated like any other content. You can have a lot of help on your system, or just a minimal set of your favorite books, your choice.

Overall I’m very happy with how this effort went. We replaced aging indexing technology with something new and much more precise. We removed the viewer application entirely. We drastically simplified the interface between the IDE and help (basically it’s a URL at this point). And it’s faster!

The Benefits of Dogfooding

It’s fairly well known that we try to dogfood all our own products. In this particular release I think there were some nice significant benefits from this practice. I’ll highlight just a few of the ones that I think are the most important here.

.NET Framework 4.0

It’s hard to imagine not dogfooding this in the release given its importance but there were several unexpected benefits for customers. First, the general character of the release was profoundly affected by Visual Studio’s early adoption, and our need to create designer assets for many different framework versions. The effects were felt in everything from binding policy to metadata format. As an architect I was very pleased by these results because in many cases we were able to find and prevent problems on the whiteboard – where corrections are a lot a cheaper.

A secondary benefit came about because Visual Studio 2010 had to implement so many existing COM interfaces in managed code – resulting in a large number of COM Callable Wrapper (CCW) objects. Likewise much new managed code needed to call existing interfaces that were not being converted, hence significant use of Runtime Callable Wrapper (RCW) objects. That’s all well and good, these technologies had existed for years, however, fairly innocuous seeming choices, like “when should RCW’s be cleaned up” were having profound consequences. The additional CLR cleanup code present in past releases introduced unexpected reentrancy that was not present in the strictly native implementation. Importantly Visual Studio gave us fairly easy and important test cases to work with and ultimately resulted in significant improvements in COM interoperability – these kinds of experiences make the Framework better for everyone.

Windows Presentation Foundation

The WPF team had many headaches to deal with thanks to having VS for a customer but I’d like to think WPF also got more than a few benefits. Some of the corrections we asked for were fairly minor – like the toolbar panel’s layout code which was fairly weak, and we have a lot of toolbars. But in other cases there were significant weakness.

I’d say that if there was a theme to our WPF problems it was this: that the complexity of Visual Studio’s use of hybrid traditional Win32 and WPF elements was a new challenge. But ultimately this resulted in a lot of goodness. It’s possible to do good focus transitions now between WPF elements and Win32 elements, something that would make a person’s head hurt. And, perhaps even more importantly, different combinations of hosting WPF in Win32 UI and Win32 UI in WPF were resulting in extra drawing delays. These paths were exercised by the many window combinations present in Visual Studio and the abundance of big display transitions (like switching to debug mode and back). Generally speaking the hosting services got a lot of exercise and were significantly improved as a consequence.

I’ll talk a little bit more about WPF below specifically as it relates to Beta 2.

Team Foundation Server

Of all the technologies that we beat on, I think TFS probably got the beating worse than any other system. For sheer volume of users, size and number of shelf-sets, number of branches, merges, automated builds, reports, to say nothing of bugs, issue tracking, and basically every other TFS feature our entire division put a level of load on our TFS systems that makes me think that anyone with any load that is even remotely normal is going to have no trouble at all. Sometimes it was painful for us, like dogfooding is (and is supposed to be) but it’s going to be that much better for you.

From Beta 1 to Beta 2

I’ve saved the best news for the last section of the last installment. Beta 1 of VS2010 represented some really great progress towards our goals but there were a lot of issues, I’m happy to report that we’ve been hard at work on the most serious ones and I think we’ve got great results for you. I’ve listed just a few of the improvements that I think are most important below:

Beta 1 got many complaints due to “fuzzy text”, in response to this WPF adopted Direct Write to give a much better overall experience. Those changes accrue to everyone (and yes you’ll get lovely text in VS!)
Performance in Remote Desktop situations is much improved
Startup time is significantly reduced in virtually all cases
Raw typing speed is vastly improved, I know that for instance, C# raw typing speed has never been faster
The new editor includes column select, as well as dozens of other underlying improvements that help region management, outlining and so forth
Debugger performance, with and without historical debugging, is much improved, stepping is quite snappy
The Visual Basic project system can now emit an executable while it is compiling a different one, those should yield the best build times ever for VB solutions
Assorted shell changes in important panels, in data caching, in MEF catalog construction, and in style application will help virtual all cases but notably transitions between modes
The new offline Help system is ready to you to use
Find in Files uses a parallel search algorithm, and last, but not least…
The hated Add References dialog begins on a sensible tab and populates the full list of references asynchronously should you ask for them

There are so many improvements I can’t possibly list them all. I lost count of the number of performance issues we addressed, there were probably over 100 performance analysis reports counting only my efforts on the most critical scenarios and I was hardly alone in this!

Generally, looking at the full battery of our performance tests we find that VS2010 outperforms VS2008 in the majority of cases, and, if you weigh the scenarios by importance, VS2010 looks even better than that. Our remaining performance battles are around reducing overall memory consumption, and I’m sure that will continue to be a focus area between now and the final release as we fight for every byte.

Internally, folks are very surprised and pleased by how far we have come in this Beta. I hope you are equally pleased.

To everyone that got this far, thanks so much for reading My History of Visual Studio, and I look forward to your feedback on either the Beta or the series.

[See The Documentary on Channel 9!]

[All the other Parts: History of Visual Studio]

[Visit the Microsoft Visual Studio 2010 and .NET Framework 4 Beta web site for the latest info]

My History of Visual Studio (Part 9)

2009-10-17T00:35:00Z

[All the other Parts: History of Visual Studio]

In the last posting I talked about the “Whidbey” release, VS2005, but I feel like I left out two really important aspects so I’d like to start this part by rewinding a bit for those two topics.

I had mentioned in Part 7 that some of the UML support in the “Everett” time-frame was starting to foreshadow the release of VSTS. Well, in Whidbey there is no foreshadowing disclaimer necessary – VSTS arrives and I think at that point it’s clear that Microsoft is going to try to have a real offering in the ALM space. I think it’s fair to say that the venerable VSS was not going to be the product to carry us into the 21^st century.

I don’t want to understate the difficulty of bringing just TFS to market, and that’s only one part of the equation, but even giving those hard-working folks a well-deserved nod, I felt like the first VSTS was just a taste of ALM at this time – whetting the appetite as it were. But even so, with so many shops wanting to formalize their practices in some way – even lightweight practices like Scrum – this kind of support was welcome relief. I think those years included a much greater level of attention to the process of creating software than at any other time in my experience.

Just 4 years after its initial 2005 release, VSTS is a huge part of what we deliver – and in 2002 it didn’t even exist.

I started my discussion about the Whidbey release with an observation about how big of an influence Windows Vista was on our tools. I think I should have ended with my single biggest regret about that release because it’s highly related. Despite its huge influence, our support for Windows Vista was lousy in the initial release, like really lousy. It wasn’t until we delivered SP1 and especially the Vista support for SP1 pack that we got to a reasonable level of support. I could make a bunch of excuses but they all sound pathetic to me so I’ll just stick with “It’s my #1 regret.”

OK so much for the downer part of the article; let’s move on to Visual Studio 2008 “Orcas.”

The thing that I most remember about “Orcas” is “MQ” – the Quality Milestone. This is where we spent the equivalent of an entire coding just taking care of nagging issues that otherwise seem to not get handled. I’ll name-drop again since I just linked her video, I’d have to say it was Carol more than anyone else that was keeping us honest about the “debt” we had accumulated in our product and popularizing that meme in our division. Debt is a great way to think about trade-offs: every time you make a choice that isn’t right in the long term, any short-cut, accumulates some debt. Any bug you choose to defer, that’s debt. Some debt you should write-off, that fix just isn’t happening, some you should address, but always you should be aware that you’ll have to deal with it sooner or later, and it may as well be sooner.

That mind-set, which was the focus of MQ was pervasive for the whole release, and I think it shows. From a stability and performance perspective VS2008 was universally better than 2005 – a very hard thing to achieve in any major release with significant additions.

I think the “star” of VS2008 was Linq – my favorite flavor was “Linq to SQL” which I’ve written about at some length before. Perhaps the most amazing thing about Linq is that its introduction into both C# and VB allowed mainstream programmers to start using functional concepts in a natural way, often without even realizing it! But a close second are two other amazing things:

1) Linq variants (Linq to XXX) started appearing like crazy – the notation was incredibly useful

2) You could actually get great performance out of this kind of layer without inflicting craziness on users (see my article again for the Linq to SQL discussion)

In the same way that the needs of high quality designers drove advances like partial classes (critical for code separation) Linq variants and their needs, drove the type inference, extension methods, and expression tree features, including concise lambda notation, necessary for its success into the mainstream languages. This brings great collateral benefits because those exact same notions are helpful in so many other scenarios – I use them pervasively in my WPF code for event handling for instance.

As usual this all comes at a cost, it’s not just language notation: it’s also the debugging support, profiling, garbage collection, FXCOP, and so on. After all, just because a method is dynamically generated, has no name, and no .pdb file information, doesn’t mean you don’t want to analyze it like the others. And of course the ability to generate expression trees in addition to IL represents another significant undertaking for the compilers. All of this is fundamentally necessary for Linq to succeed but it’s kind of invisible – it’s the stuff you expect to just work.

“Orcas” also included the “Cider” designer for WPF. Remember that during this period there had been several runtime releases including 3.0 and it shipped alongside 3.5 of the framework. Well by the WPF was a first class application target and it needed designer support and the richness of WPF resulted in perhaps one of the most complicated designers ever created. It’s doubtful that a successful designer could have been created if not for the lessons from “Sparkle” aka Expression Blend, and language advances like partial classes to ease things along.

The abundance of framework subsets led to the need for multi-targeting – that is the ability of the tool to target any one of several framework versions or to have solutions with projects targeting arbitrary combinations. This was tricky enough in Orcas but at least the frameworks all shared a common core – simplifying things significantly. That grace would be absent in the follow-on release VS2010.

Other application models were popping up as well – Silverlight had made its debut, taking WPF concepts to the web, and Windows Workflow (WF) also delivered a release and a designer. If it wasn’t already de facto reality, the general theme that “every application model needs a suitable design experience” was certainly cemented during this period.

Debugging technology takes itself to new amazing places like XSLT debugging (XSLT makes my head hurt even more than APL) and gains the ability to debug into the BCL and dynamically download the source as needed to do the job. Very slick, but by now you know I’m partial to debugging technology J

And on the native code front, MFC 9 makes its appearance – adding support for new Vista concepts, and representing the first significant update to MFC in a very long time.

But for me the biggest memories are just working on performance across the board, from MQ to the finish. In the framework, in the tools, we made big dents in many scenarios and overall, even with lots of new features, Orcas felt snappier than Whidbey which was a great accomplishment. I loved using it!

I just hit 1200 words so I think it’s time to stop, having skipped vast regions and barely mentioned VSTS, I’ll pick up again in a bit.

Thank you for all the feedback and comments, they’re much appreciated.

[See The Documentary on Channel 9!]

My History of Visual Studio (Part 8)

2009-10-15T02:32:00Z

[All the other Parts: History of Visual Studio]

I can’t really talk about what was going on in the IDE without covering what was happening in the runtime because their fates are so intertwined, so even though it’s off topic a little bit, allow me to cover some details from Framework 2.0.

“Longhorn”, which became Windows Vista, was probably the single greatest influence on the Developer Division during the years when .NET Framework 2.0 and Visual Studio 2005 (collectively “Whidbey”) were being developed. I think I could write several books on those years as a study in being too successful for your own good.

As I wrote earlier there was a certain mania in managed code adoption during that time and, though ultimately some of those efforts had to be scrapped, many were a positive influence on the tool chain.

Anecdote: It’s not really important to the story but just for fun I can’t resist telling you about my first day on the job when I joined the CLR performance team. I had been working for the last 7 years in the MSN area, with a great emphasis on server workloads, data architectures, and so forth. So I was a bit surprised when on day one I was told I was going to be working on our client performance problems – that’s a Whisky Tango Foxtrot moment. Pays to be flexible I guess J

Client workloads were a lot harder on the .NET Framework at that time. We did have an ngen story but it needed a lot of work. Putting as much code as possible into readily shared DLLs is fundamentally necessary to getting reasonable memory usage on the client – much less of an issue on dedicated servers for instance. Contrariwise, code sharing is even more essential on a Terminal Server with potentially hundreds of users running client applications.

Sharing also vitally important to flagship applications, like Visual Studio, that are trying to get their code loaded as quickly as possible using technologies like Superfetch. It’s a pretty simple chain of events, jitted code can’t be shared, unshared code uses more memory, use too much memory and you’re dead. Sharing is good. We had to do more of it. It was doubly important because major new Framework elements were being developed while this was all going on: things like Windows Presentation Foundation (WPF) and Windows Communication Foundation (WCF) to name a few.

The Base Class Library (BCL) was gaining support for Generics (starring my favorite Nullable<T>) and XML use was exploding in all parts of the stack. I used to joke that if angle brackets <> had never been invented I would have no performance issues to work on. If only it were true.

In the universe of technologies requiring tool support we really should add at least two more. There were yet more improvements to ASP.NET, there were compelling 64 bit architectures (x64 and ia64), and, probably most difficult of all the considerations, there was SQL Server 2005 “Yukon” which introduced SQL CLR allowing users to write stored procedures in managed code.

Again, I could write whole books on what was going on in the runtime components, but I’m trying to stay focused on Visual Studio for this series so I think that’s probably enough landscape setting, as it is with all those changes it’s easy to see why Whidbey took nearly three years.

For starters, there were whole new project and deployment types needed to support the new ASP.NET with its own, new, developer server; and likewise for the SQL scenarios. And while I’m on the topic of project and deployment this is also where “ClickOnce” makes its debut.

ClickOnce had to permeate our project systems but its introduction also highlighted an inherent weakness: despite the extensibility offered in the project space there was (and indeed is) no one central place ClickOnce deployment could be added so that all project types would benefit. This is a classic example of why ongoing refactoring/remodeling of architecture is so important.

Personal note: my first direct interaction with the Visual Studio team during this time period was working on the performance of the Create New Project dialog – something which can never quite go fast enough to make us happy.

Ok so the project system needed work. What else? Well, anytime you add pretty much anything you can expect the debugger to be affected, this was no exception. Generics – check, needs debugging support, SQL CLR – check needs debugging support, it was a long list. But there was even more than just that going on: I have written earlier about the challenges of what we call “interop debugging” (where you debug both managed and unmanaged code at the same time) – in this release both the debugger and runtime teams put considerable effort into making interop debugging more reliable. That meant taking a very hard look at all the communications logic, the locking model, the safe-stopping points vs. “skid” points as I called them. It was a huge endeavor, but as a result interop debugging got noticeably better in VS2005.

All this hard work actually had an unexpected extra bonus. We also introduced a managed code profiler in this timeframe and as it turns out, trying to stop managed code as it runs so that you can walk the stack the way profilers like to do tends to have all the same kinds of problems as trying to stop the code in a debugging context. As a result, much of the hard work that went into getting the debugger working resulted in a more effective profiler solution.

I should come back to SQL 2005 support – it isn’t enough to just add project and debugging features but ADO.NET 2.0 was making its appearance and that requires suitable designers. Visual Studio perhaps earns its name more so than its antecedent Visual Basic because it’s Visual in many different ways – just visual forms no longer suffices. New SQL engine means data designers are a must.

As if all of that wasn’t enough, there were two brand new pieces of technology that appeared during this time as well. These were part of our Office programming story: VSTA and VSTO. In a few words, VSTA is tooling for ISV’s to provide scripting features to their users, allowing their users to write managed language programs that automating their product. The primary user is clearly Office but in principle anyone can participate – you get a complete authoring experience with it. VSTO, in contrast, allows you to write office extensions, intended to be written as native code against a COM API, using managed languages instead. That poor sentence hardly does justice to the difficulty of achieving this result, and I’m glossing over the fact that there were other VSTO solutions as far back as 2003 but this is where I remember the spotlight first shining on that technology with any kind of luminosity.

But, even with all this good work, there was one thing that stands out in my memory as being more important than all the others.

Edit and Continue was back – and there was great rejoicing.

[See The Documentary on Channel 9!]