Van Kichline's WebLog : VWD Performance

Code Complete

vank — Sat, 28 Aug 2004 21:08:00 GMT

After an unexpectedly long process, I'm finally code complete on the Beta 2 VWD performance suites. We identified our performance needs a couple of months ago, set our goals early in August, and now we've finally got the infrastructure in place to monitor these test cases from here to Beta 2 release, and on to RTM.

Monday I'll be busy getting the 486 files involved code reviewed (twice) and checked in. I'm glad to have time to take a break first.

Suite Frameworks

vank — Thu, 26 Aug 2004 05:52:00 GMT

It's been a busy week of writing test cases.

We have so many test suite frameworks I can't believe it. Our checkin suites are written against one framework (but other teams use other frameworks.) Our QA tests use another framework, and the perf test cases I'm writing use yet another framework. None are complete or robust; if you use a new function of feature of an existing method you better check the source code for the infamous "TBD." For example, I needed to open a local host web instead of a file system web. The function existed in the header file, but not the .cpp file. I fleshed it out, but another function which pushed the appropriate button for local host as opposed to file system button was implemented only as TBD.

My last test case involves modifying an aspx file and viewing it in the browser. How do you terminate the test? We usually use CodeMarkers, which are simply window messages with a few tests to see if they're enabled or not, to terminate timings. The alternative was to poll and search, so I decided to write an ActiveX control which fires a CodeMarker when rendered. But it turns out that XPSP2 may disable displaying ActiveX controls! You have to click on an "action bar" to enable the control to be loaded. But I've found a way to save the registry setting, change it, and restore it after the test, as long as the test is running with admin privileges.

I'm hoping to finish up five test cases tomorrow, after six weeks of committee meetings defining the test cases, experimentation to set performance goals, and wresting with the test framework and security precautions to implement the tests. Party time!

Other things that have gotten slower by design

vank — Thu, 19 Aug 2004 16:04:00 GMT

Another feature of Visual Web Developer which is certain to seem slower than VS 2002 or 2003 is the opening of "miscellaneous files." These are files which (should) sport Intellisense but are not part of a web or a project. In earlier versions of VS these simply didn't have Intellisense hooked up. So, for example, if you typed "Response.", nothing would happen.

VWD 2005 creates a hidden, "miscellaneous" project to wrap around the miscellaneous file. It searches upward, looking for an actual web the file may be a part of, looking for a /Code folder, web.config file, or other clues (I can dig these up and enumerate them if anyone's vitally interested) and figures out where to "root" the web. If a web is discovered and it's not from a previous version of VS, the project system does what it would for a normal web; /Code Intellisense is generated, global.asax is honored, everything.

If you open two miscellaneous files, you may get one or two miscellaneous projects. If you opened c:\foo\defalut.aspx and c:\foo\info.aspx they'd both share the same hidden project. If you opened c:\foo\aspx and c:\bar\data.aspx there'd be two different miscellaneous projects. Most of the spin-up is asynchronous but the memory footprint of the Intellisense and type resolution services is noticeable.

Has anyone out there run into problems with this? Is double-clicking on a file on the desktop to make a tweak too slow? Do you get unexpected Intellisense (or none at all) with these types of files sometimes? Does the Intellisense make you happy to leave notepad behind at last for this type of operation, or does the startup time make you right-click and "Open With..."?

What's Different About the VWD Build System?

vank — Wed, 18 Aug 2004 21:30:00 GMT

There have already been several posts about build being slower in VWD than in Visual Studio 2003. This is true.

With Visual Studio 7.0 and 2003 there was a huge gulf between the SDK version of examples, starter kits, etc., and the Visual Studio versions. The organization of the source code differed mostly in single-file SDK versions and code-behind VS versions. But there was an even more substantial difference in deployed webs. SDK versions seemed to simply and magically compile themselves, while the VS versions needed to be built and prop'd to the website. One big assembly was generated in the /bin directory. The SDK versions could be modified right on the web server using only notepad; the VS versions had to be modified in the VS Solution, built, and prop'd. Strangest of all, you could make some changes in the running VS webs using notepad, like modifying some text on an aspx page, and it would show up the next time you loaded the page in your browser, but other changes (like modifying the code) simply wouldn't "take."

This really didn't delight everybody. Modifying a sample meant fixing the C# & VB version for both the SDK & VS versions. It was difficult to exchange code. It was confusing. It looked broken. And having to resurrect a project to make a tweak and replace every executable bit to implement a one-line change was simply not a decent model for many kinds of web development.

Early in 8.0 we decided to rip the extensive build system out of the web project system, and instead rely on the managed Client Build Manager, an ASP.Net class wrapping the Build Manager, the ASP.Net piece that compiled the SDK-style webs. The advantages seemed pretty significant: excellent fidelity between development and deployment code generation, any-language support, no-solution-file web projects (my favorite) and simplicity. What the build system used to do was to spin up its own compilers and keep most code built to the state just before writing the actual assembly, so doing a build actually meant just emitting the assembly file.

Now we rely on the Build Manager itself, which isn't really aware that Visual Studio is even running. We've still got our own compilers building the symbol tables used by Intellisense, but they are not used to emit the executable code. And thus we incur some substantial costs in latency and working set.

Was it a good decision? Only time and your comments will tell. The new web project model is a delight to work with; all you have to do is select a folder and it behaves like a project; no files are added to it at all. Could we have accomplished this without duplicating compilation costs? Will version independence be a substantial win going forward? Will the cumulative compilation hit make old and bitter people out of us all? I'm interested in your feedback.

Here's a hint. For "consistency" we currently do a full build internally when you hit F5 before we request the target page, which will trigger a Build Manager build. If the delay is bothering you, try this:

Right click on your project node in the Solution Explorer.
Select Preferences.
Select the Build tab.
Uncheck "Build web site as part of solution."

Do this per web; there's no global setting.
If this makes a noticeable difference in your day-to-day experience, let me know.
Do you feel that this setting should be off by default?

Visual Web Developer (Whidbey) Performance

vank — Tue, 17 Aug 2004 16:59:00 GMT

Hello! I've been a dev at Microsoft for eight years now, and have been working on Visual Studio for the last five or six of them.
Recently I've changed roles. I've always worked on the web development part of Visual Studio (from Visual InterDev through Visual Web Developer) but I'm moving off the feature area and into performance.

Until recently this responsibility was spread over four people on our team, and it was one of many responsibilities for each of them. Now virtually all the performance testing, monitoring and investigation are my responsibility. I'm new at this; I've had a few narrow performance responsibilities and a little training in the past, but I've got a huge amount to learn and a thousand tasks to complete. It's like heaven. It's hard to beat getting paid to learn new skills, particularly interesting ones. My old friend John Pence and I used to say "Always get paid for learning." It was the company slogan of MacMan, Inc.

Let me clarify my role: my responsibilities are specific to Visual Web Developer, the development environment. If you've downloaded VWD Express then the Project System (represented by the Solution Explorer window and a vast expanse of invisible plumbing behind it) and the ASPX document windows cover a lot of the area I'm primarily concerned with, along with a few additional areas which are harder to visualize; the Cassini-based web server which starts up when you press F5, the Build command, Publishing, Copy Web, the acts of opening and closing web sites, and basically any function which pertains to the website as a whole or the web pages within that site. Intellisense hookup, too, although the Intellisense itself is implemented by language teams. The Shell is the framework of Visual Studio that’s wrapped around our package; it’s the part that’s the same for C# Express and VWD Express.

I'm not involved with performance of the Shell, the ASP.Net runtime, the .Net class library, C#, VB, or SQL, but I am getting to know most of these people, so if you have comments on any of these areas I might be able to pass them on but probably won't be able to follow up on them unless they're major issues which impact the entire product. If you're a Visual Web Developer beta user, I'd love to hear about your performance-related experiences with the beta.

For you precious beta testers (I am one of you) I've got some questions. Please let me know:

Have you used other Visual Studio products for web development before? VID, VS7, VS7.1? How does VWD stack up performance-wise?
How much RAM are you using? Are you memory bound? What do you think the minimum RAM requirement should be?
Have you run into limits where performance breaks down? Say, one or two web forms are OK, but once you hit 317...
Are there spots where you reach for the reset button, thinking the system is hung? Where are those spots?
Most important (if you have one of these please drop what you're doing and post a reply) have you ever hit a painfully long pause right in the middle of a thought, that is, have you lost your train of thought while your hard disk flashes and you CPU moans? Where was that point? What do I need to know to reproduce that experience?

If you've got performance complaints with Visual Web Developer, you've found the right guy to notify. I can't guarantee I can solve every problem of course, but I can get your problem areas on the map. I can make sure test cases cover them. I can assure that fixing them or shipping them becomes a conscious decision. I know it's not always easy to tell, but these decisions aren't made lightly. If I don't hear from our users, my focus will be directed solely by my best guesses and suggestions of our team and division. Your input could make a big difference in the product we ship.