Hi, my name is Andy Rich and I’m a QA on the C++ front-end compiler. The IntelliSense system in Visual Studio 2010 comes with far greater power, flexibility, and accuracy, but these improvements come at the cost of greater complexity. The goal of this article is to assist you in troubleshooting this complex system, and give you a peek under the hood at how it works (and what to do when it doesn’t).
Having spent a lot of time helping customers with slow IntelliSense, I have found that their performance issues are almost always related to PCH being disabled. For large C++ translation units (and most of the ones that you care about are going to be large), IntelliSense PCH is vital to ensuring fast IntelliSense. Getting your PCH settings right are also vital to having fast builds – so getting this right can potentially be a boon on two fronts. I have previously written a blog post on the PCH model and how to configure it within the IDE: http://blogs.msdn.com/vcblog/archive/2010/01/26/precompiled-header-files-in-visual-studio-2010.aspx. This blog post will be focused on what to do when you’ve followed those steps and things still aren’t working for you.
In VS2010 RTM, errors in your PCH will prevent the IntelliSense compiler from creating a PCH. This is something we have addressed in SP1, but even still, the error window can be a good place to start investigating performance issues.
One of the new features in VS2010 is “red squiggles” for C++ - these diagnostics are provided by the IntelliSense compiler. These same diagnostics are also provided to the error list window. If this window is not visible, you can bring it up using View->Error List or the hotkey chord “Ctrl +\, E.” In this case, you should be looking explicitly for errors in header files, and starting at the top of the error list window. With VS 2010 RTM, any errors (even ones that the compiler can typically recover from) will prevent your PCH from being built, and cause severe IntelliSense slowness. (This is addressed in SP1, which I discuss in a later section.)
It is important, here, to call out that the IntelliSense compiler is different from the build compiler. We have made every effort to give these two compilers parity. (For more information about how this works with C++/CLI please check this blog post.)
However, there are still differences, and occasionally, a file that compiles without error using our build compiler will not compile properly with our IntelliSense compiler. Often, this is because the IntelliSense compiler has a more strict interpretation of the C++ standard than the build compiler. In these cases, you can usually work around this problem by fixing the error reported by the IntelliSense compiler. (In most cases, the build compiler will happily accept the more-conformant code being required by the IntelliSense compiler.)
Additionally, if you are targeting an architecture other than x86, you may notice that the IntelliSense compiler is always operating in x86 mode. This can produce errors that are very difficult to work around, and while these errors will not prevent you from working with most code, they can cause PCH generation to fail as mentioned above.
If you are unable to find a code workaround for your problems, there is one further stopgap measure that can help: the compiler macro __INTELLISENSE__, which is only defined when using the IntelliSense compiler. You can use this macro to guard code the IntelliSense compiler does not understand, or use it to toggle between the build and IntelliSense compiler.
This is a good opportunity to discuss context in our IntelliSense engine. The IntelliSense engine provides accurate results by always having as correct a view as possible of the source file being compiled. This is fairly straightforward in the case of .cpp files: these are natively compiled and understood by the compiler. However, the situation is less clear for .h files, as these files are compiled only in the context of an associated .cpp file.
In previous releases of Visual C++, header files were only parsed by the IntelliSense parser and included in the NCB once, based on the single context they were compiled in. An older post by Jim Springfield discusses this so-called “multi-mod” problem in greater detail. We address this issue in Visual C++ 2010 by having all header files compiled in the context of your current .cpp file, so that this highly contextual information can be more accurate.
However, what is the proper recourse for the IntelliSense engine when an .h file is active in the editor? It cannot compile the .h file by itself – this would not be the correct context. The .h file is almost certainly included by multiple .cpp files – which one should be compiled to get the proper context for the .h file?
In Visual C++ 2010, we introduced a bit of technology called the include graph. This allows us to know, for an .h file, all of the .cpp files that have included that .h file, either directly or indirectly. This gives us all of the possible contexts for the .h file, but we still have very little idea which .cpp file is the one you want.
Ideally, this is something that would be configurable by the user, but this seems heavyweight for IntelliSense. What we settled on was looking through your most recently used .cpp files (controlled by the “TU cache” setting in Tools->Options->Text Editor->C/C++->Advanced) and seeing if any of those were reported by the include graph as being a valid context for your .h file. If so, we use that context. If no such context is available, we must fall back on choosing an arbitrary context for the .h file.
Let’s get back to diagnosing IntelliSense issues. Assuming your header files are free of IntelliSense errors, we should look into verifying that your PCH is being built. The most foolproof way of accomplishing this is to actually look on your hard drive for the iPCH file to ensure it is being built.
Browse to your solution directory, and find the “ipch” directory underneath. In here, you should find one directory per project. And within those directories, the “ipch” files themselves. Looking at the timestamps of the files can be informative, but for me, proof positive is to delete the iPCH files (you’ll probably need to shut down your solution first) and ensure the iPCH files are being recreated when IntelliSense is executed on the .cpp file in question.
If you aren’t seeing the iPCH file being generated, this is a good time to go back and review the PCH options blog post and ensure your settings are really configured correctly.
One huge caveat is in the case of makefile projects. By and large, settings in your makefile project are opaque to the Visual Studio project system, and therefore by extension, the IntelliSense system. In these cases, your include directories may not be correct, macro defines may be not set, and any compiler switches you are using in your makefile (including those that control PCH!) will not be on.
For these cases, we have added an extra configuration section to makefile projects. Right-click your project, choose Properties, and go to Configuration Properties->NMake. The “IntelliSense” subsection in here is for options that are specific to your IntelliSense compiler. These options will be passed ONLY to the IntelliSense compiler, and should be of the same format that you would pass to the build compiler. You should ideally set these according to the same options used in your makefile. In particular, preprocessor definitions, include search path, and forced includes are important to have right. For our purposes, of course, you should also have your PCH options included in the “Additional Options” section.
As a quick and dirty workaround for PCH, you can often just specify “/Yu” with no parameters, and the IntelliSense engine will create a default PCH for you. But in the long run, you will have better overall performance and less issues if you mirror your build system’s PCH settings here.
Goto-Definition (GTD) is one of the most complicated operations performed by our IntelliSense engine, and one of the most common to suffer IntelliSense slowdown. The big issue with Goto-Definition is that, typically, the definition of the function is not contained in the translation unit currently being parsed. The declaration is naturally required by the compiler – the prototype in your .h file – but the .cpp file that provides the implementation of this prototype is often not in your current TU; often, it isn’t even in your current project! (And in some cases, it is buried in a static lib or DLL, and no actual code for the definition is possible.)
At a high level, Goto-Definition is implemented like this:
The operation that tends to take a long time is step 3. In a previous blog, I discussed how our preparse model can negatively impact performance. Step 1 is typically not a problem because we have nearly always already generated a preparse for your current source file. In Step 3, however, we are going to a new, unrelated file; and this file typically does not have a preparse generated. The gating factor on the speed of these operations is the preparse, and the only good way to speed the preparse up is with PCH. So getting your PCH working (as mentioned above) is probably the most important thing you can do for performance.
Sometimes, it can help to pull up Task Manager, as this will provide some insight into which piece of our complicated IntelliSense/browsing system is causing the problem. When you perform a long-running Goto-Definition, you can take a look at which process is consuming CPU cycles. If it is devenv.exe, the problem is more likely in the browsing system (a database query, most likely). This is usually due to some kind of complexity of your solution, and is something we’re interested in finding out more about when you encounter it.
If you find that the process eating up resources is vcpkgsrv.exe, then the problem is in the IntelliSense compiler – and once again, most likely to be a long-running preparse (which is best solved by having PCH turned on and working).
Visual C++ 2010 has some additional logging options which can help to pinpoint problems. To turn this logging on, go to Tools->Options->Text Editor->C/C++->Advanced and change the options under “Diagnostic Logging”. To enable full logging of all events (which can be a LOT of logging data), you set Enable Logging to True, Logging Level to 5, and Logging Filter to 255, as in the screenshot below.
You can view the output of this logging in the Output window (you may need to change the “Show output from:” dropdown to “Visual C++ Log”). Briefly, I will go through what the log looks like for a typical QuickInfo operation.
This is the log for a QuickInfo call:
[WorkItem] >> [eNowQueue] class CQuickInfoWorkItem [WorkItem] [eNowQueue] - Executing class CQuickInfoWorkItem [IntelliSense] translation unit: c:\users\arich\documents\visual studio 2011\projects\example_project\example_project\example_project.cpp [WorkItem] [eNowQueue] - class CQuickInfoWorkItem (1ms) [General] [UI] - class CQuickInfoWorkItem (1ms)
[03/15/2011 @ 18:32:22.704] Quick Info : Success : 3 ms : class MyClass
The first thing to help understand IntelliSense operations is to note the “>>” character, which indicates the IntelliSense engine has placed an item on the worker queue. For the case of QuickInfo, the work is called “CQuickInfoWorkItem”. In this scenario, you can see it took 1ms for the QuickInfo workitem to be created and queued, and an additional 3ms for this item to be pulled off the queue, processed, and the result returned. (This was nearly instantaneous because a preparse for this translation unit had already been built.)
The most helpful part of an IntelliSense log is usually looking at the translation unit the IntelliSense compiler has chosen to satisfy your IntelliSense request. If the IntelliSense compiler is loading this TU for the first time, you will also get output that indicates the command-line options this file is being compiled with, which can sometimes be helpful in diagnosing problems, especially the /Fp parameter (which indicates where the ipch for this file is located).
Note that, in the case of Goto-Definition, because the compiler will probably need to compile multiple translation units in order to provide an answer, you may see multiple “Translation unit:” info statements for a single operation. (Also, if you have red squiggles on, you will see an additional workitem fired off as a result of navigating to a new source file, in order to check for compilation errors.)
We have added three improvements/mitigations in Visual Studio 2010 SP1 that are designed to provide an improved IntelliSense experience. These are:
Of these improvements, the third should give the most immediate benefit to our users. Previously, we would only create an iPCH file if the PCH header compiled without any errors. In some scenarios (especially with non-x86 code) it was impossible to get the PCH header completely error-free, resulting in very poor IntelliSense performance, as the preparse could not take advantage of PCH speedups. (This was most noticeable during Goto-Definition, when it was more likely that the ‘target’ TU did not previously have a preparse created.)
With this feature, we have added a few special diagnostics in cases where we were still unable to generate an iPCH. These are mostly the result of project misconfigurations, missing files, and other such catastrophic errors. The text of the error should say something like “An IntelliSense PCH file was not generated.” If you see these diagnostics in SP1, it is almost certain you will suffer poor IntelliSense performance until the error in the diagnostic is resolved.
I am always interested in hearing specific feedback about poor IntelliSense performance. My hope is that with the additional diagnostic information provided in this blog post, you can help us pinpoint the performance issues you are having, which component the issues are coming from, and get closer to the actual root cause of the issues. Supplying this additional information (as much as is available) in your connect bug will help us to understand and address these problems.
Andy Rich Visual C++ QA
Lars: When I said 'x86,' I meant the native x86 compiler, not the cross (x86_amd64) compiler. As mentioned by petke, you should be able to use the 'Rescan Solution' option in Visual Studio to get the browsing database updated if you make siginificant changes (such as syncing a large directory tree or updating a 3rd-party library).
Andrew: The call graph is not directly exposed in the UI, but you can see the information it is built from. Allow the browsing database to be built in the IDE, shut down, and then open the .sdf file in the server explorer as a SQL Express Database. If you look at the code_item_kinds table, you'll see an entry for #include directives, then select from the code_items database for that kind. With some clever SQL queries, you should be able to get the information you desire. Another way to get this information would be to use the front-end compiler switch /showIncludes, which you may wish to programmatically parse.
Asbjørn: it sounds as though you were able to get PCH turned on, and it has improved your intellisense experience. Are you certain that you did not actually enable PCH somehow? Even setting /Fp (name pch file) is usually enough to get the intellisense compiler to generate a PCH.
PleaseFixYourBugs: Your assertion is that the intellisense is slow because of your project size, but it is rare for this to be the case. The section in my article about using task manager to pinpoint the source of the slowness may help you understand more. If it is indeed the size of your browsing database (devenv.exe taking all the cycles), I'm interested in understanding more about your particular solution. If it is vcpkgsrv.exe, and you are certain PCH is on (you are seeing iPCHs be generated on disk and updated, for example), I recommend collecting some logs and sharing them with me. You can email me directly at arich [at] microsoft [dot] com, and I'd be happy to take a closer look with you.
Karen: some intellisense operations should be available even while the IDE is parsing (QuickInfo, Autocomplete, Memberlist, and Parameter Help). However, Goto-Definition will not be available until the browsing database has been fully updated.
Sergejs: the browsing database will find all source files that are somehow included in your project, either directly or as a result of other #include directives. This is not configurable, and is necessary in order for the IDE to be able to provide accurate answers. As mentioned, some intellisense operations will be available, even during the browsing database update, but Goto-Definition will not.
Cristoph: I'm following up with our team regarding your log, I'll post back here when we've done more investigation.
Cristoph: After discussions, here's what we think is happening:the initial Quick Info for goto-definition is failing and the IDE is falling back on a database-only query to satisfy the goto-definition request. This likely means that the slowness you are experiencing is due to database complexity. While the IDE is hung, does the status bar display "Searching Visual C++ Projects"? This would confirm my team's suspicions.
It may be helpful to examine why the quick-info itself is failing, as getting that to work would probably increase your goto-definition speed, but we are also curious to see more about the shape/complexity of your solution that is causing our database queries to also be slow. If you are interested in helping us understand this issue better, please contact me at arich [at] microsoft [dot] com. Thanks!
Goto definition is dead slow on my computer. I have the latest quad core i7, 4GB RAM and an Intel SSD running 32-bit XP. This slowness is unacceptable.
My solution consists of about 25 projects. I have enabled creation of PCH in my startup project (but not in the other projects). A .pch file and some .ipch files are generated on disk. Goto definition takes about 6-11 seconds to run. Meanwhile the process "vcpkgsrv.exe" takes about 10% CPU time in the task manager (I guess it's only using one CPU core). When running the exact same goto definition command repeatedly without any changes in the code, each one has the same execution time (6-11 secs)! Is there no caching performed by IntelliSense? I mean, the result is not gonna change between goto definition lookups if there are no code changes.
These are the last lines of the log when running goto definition:
[WorkItem] [eNowQueue] - class CWorkerThreadExecWorkItem (8895ms)
[General] DisambigAndNavigate with 2 items.
Any solution to this annoying problem?
For as long as **Very Slow 2010** and next releases will be written in managed code there is absolutely **NO CHANCE** for them to perform as VS2008 performs. No tips, no tricks will be of any real help.
What's happening here is what happend long time ago with java. After every release there were tons of papers _first_: proving that current release is much faster than the previous one, _second_: that this version is as fast as C++ and sometimes even faster (whole sheets were provided with very nice graphics etc, etc), _third_: promising that slow performance of current version is only temporary and next release will definitely fix it. And what happend? We all know how java performs and we all know how apps written in MJ (Microsoft's Java aka C#) performs.
The simple true is this:
If you want your program to perform as well as a program written in native code you **MUST** write it in native code.
Jesper: It sounds like it is the intellisense compiler (vcpkgsrv.exe) is the one causing your gotodef problems. For a single translation unit, we build a 'preparse' and keep the compiler instance (vcpkgsrv.exe) persistent in memory to continually serve intellisense requests for that translation unit. The instances of vcpkgsrv.exe are cleaned up when we've reached our limit of cached TU's. It's possible that somehow your gotodef operation is exceeding this number, causing the compiler to need to preparse. You can try increasing the cached TU count (Tools->Options->Text Editor->C/C++->Advanced, increase the "max cached translation units" count). Note that doing so will increase total memory usage, as more instances of vcpkgsrv.exe will be kept persistent in memory. This is why this limit is set low (2) by default. The only other caching we have is in the form of precompiled headers.
As explained in my article, gotodef by default requires at least two preparses (one for the source - hopefully already built, and one for the destiation - almost certainly not built the first time). You might want to verify that both the source and destination files for your gotodef have PCHs successfully built.
"Knowing me": while it is true that the IDE itself is built in .NET, the intellisense compiler itself is 100% native.
I agree with both posts by @PleaseFixYourBugs. We also have a large 98% C++ project that has many issues with intellisense in VS2010, most notably performance. I realize it is a hard problem but I can open up the same solution(s) in SlickEdit and instantly do a go-to-definition or find-all-references operation (I think they have a demo version you can download if you want to see how much faster it really is). Find-all-references in VS seems to behave the same as a basic search (and is about as slow). It doesn't work right at all. And I am always scared to hit F12 (goto-def) in VS because there is a good chance that it will completely hang VS 2010SP1.
Just tried hitting F1 in VS2010 and @PleaseFixYourBugs was right. Tooks about 15 seconds. Google takes 0.15 seconds (100x faster).
The daily crashes I was seeing pre-SP1 have gone away though which is good.
The compilers are amazing but the IDE needs a lot of work.
We have a production environment large solution that has about 70 projects written mostly in C++/MFC/ATL and used to work decently fast with Intellisense in VS.NET 2005. This is without even having PCH enabled at all. Now GoToDefinition is horribly shot in the head and it takes forever to complete, even when the symbol in question is in the very same source file and header! This key operation used to take a fraction of the second in 2005, now it takes 2 minutes to complete. Most developers use GoToDefinition as their bread and butter feature all the time - it is absolutely essential that feature be lightening fast!!! I really don't care about rest of Intellisense - just make the GoToDef work!!! Performance, as it stands right now, for this feature is absolutely horrible! Intellisense really sucks in 2010 to the point of making the tool unusable. It often hangs the product or it indictes that internal operation is being done - how does that help? I'd live with NCB file and updating tread any time over this. Please have MSFT studio team put their act together and get this fixed!!!
Have you tried to Enable Browse Information to generate the bsc file?