Testing the IntelliSense engine

Testing the IntelliSense engine

  • Comments 20

Hi, I am James Wang, an SDET on the VC++ Compiler Front End team. Currently, I am working on designing the test architecture for the new IntelliSense engine. I am responsible to design tests that make sure the IntelliSense engine gives correct answer for quick info, member list, parameter help, and etc.

Currently we have a test suite that directs testing the IntelliSense feature. But one of the drawbacks is that it only targets a very limited amount of scenarios. Testing converge becomes a major issue. But due to vast varieties of C/C++ constructs, testing every scenario becomes an impossible or at least a very time consuming task. In the meantime, we do have ~100,000 of tests that targeting compiler / libraries that covers a majority of C/C++ constructs. Would it be better to adapt these tests for the purpose of IntelliSense engine testing?

To accomplish this, I wrote a tool that will analysis any arbitrary C/C++ code and generate IntelliSense test site. The tool will also generate the expected results for each test site. So for any arbitrary C/C++ code, we can generate a set of self verifying IntelliSense tests on quick info, member list, parameter help, and etc. As the result, we can reuse the ~100,000 tests for the purpose of IntelliSense testing.

Two things make this possible:

1. Better intermediate language (IL) representation. After parsing the source code, the new IntelliSense engine generate a better IL and expose a wealth of APIs that allows traversing and querying various compiler artifacts. This allows me to figure out where to create IntelliSense test site and what the expected results should be. For example: while traversing through the IL, if I see a field operator (->), I may want to generate a member list tests. For the expected results, I can query the type for the left operand of the field operator and retrieved the expected member list.

2. Componentized the IntelliSense engine. The new IntelliSense engine exposes API that allows engine level testing. So I can do IntelliSense testing without the IDE. Otherwise, running a large amount of IntelliSense tests become impractical due to the time needed to start / shut down the IDE for each tests.

Hopefully, this new approach will improve the test coverage for the new IntelliSense engine and drive up its quality.



Leave a Comment
  • Please add 4 and 5 and type the answer here:
  • Post
  • On a good recent machine, working on huge code size, intellisense can still hangs VC8 for 20 secs if you do a right click and "jump to definition".

    That's simply not acceptable.

  • The biggest problem with VC intellisense is that it doesn't merge the results of multiple projects into one big hash table that can do a quick lookup. When people use large projects with lots of files, the performance is terrible.

    Another problem is that the background parsing that IntelliSense does sometimes causes the IDE to hang for a while. You should ensure that a change, especially to a shared header file, does not cause the IDE to hang.

    The third thing is that Intellisense doesn't resolve identifiers through templates very well. A canonical example is with CComPtr<T>. This works with small projects but rarely is able to process large projects. This may be because of forward declarations that cannot be followed, or the actually definition of T is in another project. So when you type the arrow as part of:

    CComPtr<T> pT; pT->

    you don't get a completion list.

  • Boris,

    We are a C++ house and are on a mix of VS 6.0, 2003 and 2005. Until now, I have been holding off on upgrading everyone, and every project, until VS 2008 comes out.

    However, this was predicated on Intellisense not sucking.

    On July 10, you stated that intellisense will not be fixed until "Orcas+1". I am seriously pissed off about this news. I have suffered with the shitty C++ support in VS2005 since it was in beta, and a big reason is the extremely crappy intellisense support. To push this off is to stab your loyal customers in the back while spitting in their faces. It is simply unnacceptable.

    Since upgrade pain is costly and serious, with this news I am on the verge of cancelling the planned purchase of VS 2008 entirely.

    (I don't blame you, I blame the idiot product managers and executive staff. I used to be a major Microsoft booster and apologist. In the last two years, the assholes in charge of Microsoft have almost completely alienated me and other loyal Microsoft customers. Too bad Borland/CodeGear is so messed up that they are unable to make a competitive product.

    Please forward this to the boneheads in charge and tell them to get their collective heads out of their asses.)

  • When we are working with Visual Studio IDE, sometimes we encounter issues related to intellisense. The

  • When we are working with Visual Studio IDE, sometimes we encounter issues related to intellisense. The

Page 2 of 2 (20 items) 12