Testing the IntelliSense engine

vcblog — Thu, 05 Jul 2007 19:01:00 GMT

Hi, I am James Wang, an SDET on the VC++ Compiler Front End team. Currently, I am working on designing the test architecture for the new IntelliSense engine. I am responsible to design tests that make sure the IntelliSense engine gives correct answer for quick info, member list, parameter help, and etc.

Currently we have a test suite that directs testing the IntelliSense feature. But one of the drawbacks is that it only targets a very limited amount of scenarios. Testing converge becomes a major issue. But due to vast varieties of C/C++ constructs, testing every scenario becomes an impossible or at least a very time consuming task. In the meantime, we do have ~100,000 of tests that targeting compiler / libraries that covers a majority of C/C++ constructs. Would it be better to adapt these tests for the purpose of IntelliSense engine testing?

To accomplish this, I wrote a tool that will analysis any arbitrary C/C++ code and generate IntelliSense test site. The tool will also generate the expected results for each test site. So for any arbitrary C/C++ code, we can generate a set of self verifying IntelliSense tests on quick info, member list, parameter help, and etc. As the result, we can reuse the ~100,000 tests for the purpose of IntelliSense testing.

Two things make this possible:

1. Better intermediate language (IL) representation. After parsing the source code, the new IntelliSense engine generate a better IL and expose a wealth of APIs that allows traversing and querying various compiler artifacts. This allows me to figure out where to create IntelliSense test site and what the expected results should be. For example: while traversing through the IL, if I see a field operator (->), I may want to generate a member list tests. For the expected results, I can query the type for the left operand of the field operator and retrieved the expected member list.

2. Componentized the IntelliSense engine. The new IntelliSense engine exposes API that allows engine level testing. So I can do IntelliSense testing without the IDE. Otherwise, running a large amount of IntelliSense tests become impractical due to the time needed to start / shut down the IDE for each tests.

Hopefully, this new approach will improve the test coverage for the new IntelliSense engine and drive up its quality.

Thanks,

James

IDE Feature Work

vcblog — Fri, 16 Jun 2006 23:51:00 GMT

Hi, my name is Ameya Limaye and I am a developer on the VC++ IDE Team, working mainly on the Intellisense, Browsing and Editor feature areas. Prior to this I was on the VC++ Compiler Team. I would like to take this opportunity to discuss several of the improvements we have made in VS 2005 and some of the stuff we are working on for VS 2005 SP1 and Orcas (the next version of VS).

Some Whidbey New Features & Improvements:

Live Browsing (Function Caller-Callee Browsing, Find Symbol & Find All References):
In VS 2005 we introduced a new feature called live browsing which provides a lot of the functionality of BSC based browsing which I am sure a lot of our users are familiar with. The main advantage of Live Browsing over BSC based browsing is that Live Browsing does not require you to build a project successfully to generate the browse information database. In fact it works even on incomplete code or code that is not in a fully compilable state currently and is always up to date with the latest code changes made by the user. You can read more about this cool new technology in my MSDN Pure C++ column article titled “Live Source Browsing in Visual C++” co-authored by Boris Jabes (Program Manager on the IDE Team).
More correct and always up to date Intellisense data for all code including SDK and Libraries included by user code (Macro based dynamic parsing):
Before VS 2005, we would use a pre-built NCB file to produce the information for SDK headers, and libraries like ATL, MFC, CRT and STL. This means that the information was always the same and could not be tailored to the specific include state / context of the user code. To give you a simple example, if you “#define UNICODE” then the windows headers are slightly different than if “UNICODE” is not defined. Previously there would be one state (the state used to create the pre-built NCB) that would be used irrespective of whether you defined the macro “UNICODE” before “#include <windows.h>“. In VS 2005 we got rid of pre-built NCB files and we generate the correct contextual information for the SDK and library headers when included from your project.
Intellisense Support for Makefile Projects:
The Intellisense parsing engine just like the compiler needs to know the command line switches and environment variables to make sense of the code when parsing user sources. When using a makefile project, the command line switches etc. are in a custom makefile and not available to the parser. Hence Intellisense may not always be able to make sense of the code. In VS 2005 we have added a configuration properties pane that contains a section for Intellisense where you can set these command line options and include search paths etc. in order to get Intellisense to work correctly on your makefile project.
#ifdef Code Graying:
Since we moved to macro based parsing with full information for VS 2005, this feature was an obvious add on as a way to visually show the user in the editor, which parts of the source code are being parsed and which ones are “excluded code” or #if 0 code according to the preprocessor.
Code Definition Window:
This is a nice value add that I find really useful to know exactly what I am looking at in code when browsing through code that I am not intimately familiar with.
Several small features and improvements (in some cases little known productivity enhancements):
Better support for several language constructs and features like namespaces, namespace using directives, templates and explicit and partial specializations, Unicode, etc.

New XML Doc Comment support for Visual C++. This is available for C++/CLI code when doing Quick Info and Parameter Help on symbols from a #using or /FU dll that has been built with XML Doc Comments and has the XML file beside the dll on disk.

Edit.ListMembers (CTRL-J) when invoked at a location in code that is not in a direct member list context (on an identifier following a ./->/:: in code), now shows only the members of the enclosing class or namespace scope. This can be used to effectively get the member list that you would get if you typed “this->” inside a member function, without having to actually type it. If you are in a namespace scope function, then it will display the members from the enclosing namespace scope.

Being able to open a file in your solution very easily. This is a productivity enhancement that I use a lot when in large solutions with thousands of files. Rather than browsing to the file using the File -> Open dialog, you can simply type “>open FilenameInSolution.Extension” in the find combo. Basically you can simply type CTRL-/ to get to the combo with “>” (this is a quick place to type commands) and type “open FilenameInSolution.Extension”. You even get completion for the commands usable there as well as the filename.

For more information on most of these as well as other improvements like Unicode Support, etc., you can look at “What’s New in Visual C++ 2005 Intellisense”. For a high level view of the key internal workings of the Intellisense sub-system in VS 2005 and some of the limitations of Intellisense, you can read “Intellisense Support in Visual C++”.

Over the last few months we have been working on designing Orcas features and on VS 2005 Service Pack 1.

In the IDE space we are going to address several bugs and some key issues for the Service Pack. One of the key ones is, Intellisense uses 100% CPU with a thread running at normal priority causing sluggishness on single processor machines for IDE and other processes (which are also running at normal priority). This fix should make the IDE and other applications much more responsive when intellisense uses the CPU because it will do all background parsing on a lower priority thread.

Right now we are in the design and early implementation phase for several Orcas features for the IDE. We can discuss these in the coming months as they become more concrete. You can send me E-Mail (scrambled E-Mail is at the bottom of this post) with feedback about this post as well as what other things you want to discuss with the VC++ IDE team, on this blog in the future. I really enjoyed writing up this post and am looking forward to your comments and feedback as a VC++ user.

Thanks,

- Ameya Limaye

Software Development Engineer, VC++ IDE

AmeyaL(thisIsTheAtSign)(MSFTFullForm)(dot)com

Visual C++ Team Blog : IDE

Testing the IntelliSense engine

IDE Feature Work