Performance Improvements in Visual C++
Hi, my name is Jim Springfield, and I’m an architect on the Visual C++ team. I recently spent two months working on some core improvements to how VC deals with Intellisense as well as overall UI responsiveness.
We observed a strong correlation between the severity of these performance issues and the size of the projects and solutions exhibiting these problems. As a result, we worked closely with some large ISV customers who were reporting problems with our IDE. These customers typically had solutions with over a hundred projects comprising thousands of files. While I can’t identify them by full name, I want to give a shout out to them and thank them for their time and effort. So, thank you to Bob, Don, Dick, Rainer, Kelly, and Mike, you know who you are.
While I was working on these changes as a Quick Fix Engineering patch (or, “QFE”) for Visual Studio 2005, I was also tracking the changes for Visual Studio 2008 “Orcas,” and I am happy to report that all of these changes will be available in VS2008 as well as available in a publicly available QFE (also called a General Distribution Release, or “GDR”) for VS2005.
The GDR can be downloaded from the link below. You will be prompted to login with a Windows Live ID or Passport first and then taken to the actual download location. http://connect.microsoft.com/VisualStudio/Downloads/DownloadDetails.aspx?DownloadID=9436
I wish I could say everything in these areas is fixed as a part of our efforts. However, the reality is that there were some issues that needed more substantial work and which was impossible to accomplish in this time frame. We are already working on the next release beyond VS2008 and addressing these issues will be one of our top priorities. We are fundamentally changing how we approach Intellisense and we are designing with the largest solutions in mind. We will be blogging about the direction this work is going, so watch this space for additional information on these efforts as the work we are doing here is quite interesting.
To give you a feel for the scope of the changes we made to address these issues in VS2005 and VS2008, this work touched 46 source files across three DLL’s. Overall, 4664 lines of code were changed or added. I would like to take this opportunity to give you a bit of insight into the nature of problems we identified and the solutions we implemented.
Different lock implementation for accessing the NCB (Intellisense)
The .NCB file (an in-memory Intellisense data) was protected with a simple critical section to prevent simultaneous access from multiple threads. However, often there were multiple threads that just wanted to read information the NCB, not modify it, so we replaced the NCB critical section with a multi-reader/single-writer lock. This allows multiple threads to read the NCB at one time which can happen when an Intellisense request is happening on the background thread and the foreground thread needs to access it as well. A classic example of this problem was getting a QuickInfo tooltip. The foreground thread would need to do some basic querying of the NCB to create the QuickInfo request, which would then be processed on the background thread. However, if the background thread already had the lock, then the foreground would block until the background was done. This was really noticeable when scrolling around a large function. Similar problems happened with AutoComplete requests as well.
Reduced the number of “throw-away” requests
When quickly navigating around a file, we would generate QuickInfo and CodeDefintionWindow (if the window was open) requests as the cursor touched different areas of the code. This could result in a bunch of background requests that would run and then be ignored. We made a change to only allow one of these types of items in the queue at a time.
Improved speed when changing active configuration and other options
There were a few things that were incredibly slow to change in a solution. Modifying certain options (such as adding an include directory) or changing the active configuration would take a really long time for large solutions. There was some really inefficient work going on in these cases which has been removed or redone.
Goto Definition improvements
“Goto Definition” in many situations will parse the file to the cursor in order to know the exact type of the identifier under the cursor. This was happening on the foreground thread and if there was a low-priority item on the background thread running, the foreground would have to wait for the background one to finish. This was changed to queue the item to the background queue, which causes any low-priority items to be aborted. As a result of this work, we had one customer reporting a two minute delay drop to 10-20 seconds.
Reduced CPU consumption
There is now throttling of the background thread when doing most of the parsing. Even though the background was running at lower priority, there was concern about it using 100% of the CPU and some reported issues of it interfering with other applications. We now process low-priority background items (such as Intellisense population) using only ~80% of a CPU’s time. Any high-priority items (i.e. Intellisense requests) will still be processed during this time.
File lookup in projects
We changed the project system to make looking up files in projects much more efficient. This improves several scenarios such as adding files to projects, changing configurations, etc.
More information in status bar
There is now more information displayed in the status bar. It looks like this: “Updating Intellisense… (xxx)”. The number in parentheses shows how many background thread work items are in progress. For customers that aren’t seeing Intellisense ever complete, this information may be useful.
Wait cursor
We now display the wait cursor at times when the IDE may not be able to respond immediately. This gives better feedback to people that the IDE isn’t just hung, and may be displayed in the following situations:
· Closing a project
· “Goto Definition” is invoked or when a foreground parse is required
· After loading a solution and doing initialization
· Reparsing a solution after a configuration change
Improved idle-time processing
We made a variety of improvements to how the IDE handles its idle-time processing. This work involved changes to ensure that lower priority idle tasks are treated as such by the IDE, enabling longer-running idle tasks to be spread out over multiple idle cycles, and ensuring our idle logic is smart enough to break early for later completion when a higher priority task is waiting. Overall, these changes make the whole UI/editing experience smoother.
Map of files added to project
Looking up files in a project was doing just a linear search of the files. This caused big slowdowns when doing certain operations such as adding a file to a project that already contained lots of files.
Miscellaneous performance and correctness improvements
· We identified a few internal algorithms that exhibited poor performance (i.e. O(N^2) or O(N^3)) at large scale and replaced these with algorithms that scale more linearly.
· We found instances of function-static data being used in multi-threaded scenarios and made these thread safe by moving this data to Thread Local Storage.
· We improved performance of an internal hash table with a better-performing hashing function that exhibits fewer collisions.
Exposed a mechanism to control some aspects of Intellisense
Some customers have learned to manually disable Intellisense by marking the NCB file as read-only or deleting the Intellisense engine, feacp.dll, from the vcpackages directory, but there has not been a way to control any of this from the IDE. While working on this QFE, I added a few flags that can be set using VS macros that do things such as disabling Intellisense almost completely or disallowing updates to the NCB while allowing queries. The second mode is pretty useful for large projects where Intellisense works and is useful but reparsing is painful. You can now disable updates until you have finished a bunch of edits and then turn it on and get everything parsed up-to-date. The macros that control these settings can be attached to toolbar buttons for convenient access while coding. Given how long this entry already is, I will write a separate blog on this soon.
PS: Jim wrote the subsequent macros blog and here it is: http://blogs.msdn.com/vcblog/archive/2007/11/19/controlling-intellisense-through-macros.aspx
Thanks
Damien
