Visual C++ Team Blog

Visual C++ Code Generation in Visual Studio 2010

vcblog — Mon, 02 Nov 2009 18:56:00 GMT

Hello, I’m Ten Tzen, a Compiler Architect on the Visual C++ Compiler Code Generation team. Today, I’m going to introduce some noteworthy improvements in Visual Studio 2010.

Faster LTCG Compilation: LTCG (Link Time Code Generation) allows the compiler to perform better optimizations with information on all modules in the program (for more details see here). To merge information from all modules, LTCG compilation generally takes longer than non-LTCG compilation, particularly for large applications. In VS2010, we improved the information merging process and sped up LTCG compilation significantly. An LTCG build of Microsoft SQL Server (an application with .text size greater than 50MB) is sped up by ~30%.

Faster Pogo Instrumentation run: Profile Guided Optimization (PGO) is an approach to optimization where the compiler uses profile information to make better optimization decisions for the program. See here or here for an introduction of PGO. One major drawback of PGO is that the instrumented run is usually several times slower than a regular optimized run. In VS2010, we support a no-lock version of the instrumented binaries. With that the scenario (PGI) runs are about 1.7X faster.

Code size reduction for X64 target: Code size is a crucial factor to performance especially for applications that are performance-sensitive to the behavior of instruction cache or working set. In VS2010, several effective optimizations are introduced or improved for X64 architecture. Some of the improvements are listed below:

· More aggressively use RBP as the frame pointer to access local variables. RBP-relative address mode is one byte shorter than RSP-relative.

· Enable tail merge optimizations with the presence of C++ EH or Windows SEH (see here and here for EH or SEH).

· Combine successive constant stores to one store.

· Recognize more cases where we can emit 32-bit instruction for 64-bit immediate constants.

· Recognize more cases where we can use a 32-bit move instead of a 64-bit move.

· Optimize the code sequence of C++ EH destructor funclets.

Altogether, we have observed code size reduction in the range of 3% to 10% with various Microsoft products such as the Windows kernel components, SQL, Excel, etc.

Improvements for “Speed”: As usual, there are also many code quality tuning and improvements done across different code generation areas for “speed’. In this release, we have focused more on the X64 target. The following are some of the important changes that have contributed to these improvements:

· Identify and use CMOV instruction when beneficial in more situations

· More effectively combine induction variable to reduce register pressure

· Improve detection of region constants for strength reduction in a loop

· Improve scalar replacement optimization in a loop

· Improvement of avoiding store forwarding stall

· Use XMM registers for memcpy intrinsic

· Improve Inliner heuristics to identify and make more beneficial inlining decisions

Overall, we see an 8% improvement as measured by integer benchmarks and a few % points on the floating point suites for X64.

Better SIMD code generation for X86 and X64 targets: The quality of SSE/SSE2 SIMD code is crucial to game, audio, video and graphic developers. Unlike inline asm which inhibits compiler optimization of surrounding code, intrinsics were designed to allow more effective optimization and still give developers access to low-level control of the machine. In VS2010, we have added several simple but effective optimizations that focus on SIMD intrinsic quality and performance. Some of the improvements are listed below:

· Break false dependency: The scalar convert instructions (CVTSI2SD, CVTSI2SS, CVTSS2SD, or CVTSD2SS) do not modify the upper bits of the destination register. This causes a false dependency which could significantly affect performance. To break the false dependence of memory to register conversions, VS2010 compiler inserts MOVD/MOVSS/MOVSD to zero-out the upper bits and use the corresponding packed conversion. For instance,

cvtsi2ss xmm0, mem-operand   à           movd xmm0, mem-operand
                                                                         cvtdq2ps xmm0, xmm0

For register to register conversions, XORPS is inserted to break the false dependency.

cvtsd2ss xmm1, xmm0                  à           xorps xmm1, xmm1
                                                                        cvtsd2ss xmm1, xmm0

Even though this optimization may increase code size we have observed a significant positive performance improvement on several real world code and benchmark programs.

· Perform vectorization for constant vector initializations: In VS2008, a simple initialization statement, such as __m128 x = { 1, 2, 3, 4 }, would require ~10 instructions. With VS2010, it’s optimized down to a couple of instructions. This can apply to dimensional initialization as well. The instructions generated for initialization statements like __m128 x[] = {{1,2,3,4}, {5,6}} or __m128 t2[][2]= {{{1,2},{3,4,5}}, {{6},{7,8,9}}}; are greatly reduced with VS2010.

· Optimize __mm_set_**(), __mm_setr_**() and __mm_set1_**() intrinsic family. In VS2008, a series of unpack instructions are used to do the combining of scalar values. When all arguments are constants, this can be achieved with a single vector instruction. For example, the single statement, return _mm_set_epi16(0, 1, 2, 3, -4, -5, 6, 7), would require ~20 instructions to implement in previous releases while it’s only one instruction is required in VS2010.

Better register allocation for XMM registers thus removing many redundant loads, stores and moves.

· Enable Compare & JCC CSE (Common Sub-expression Elimination) for SSE compares. For example, the code sequence below at left will be optimized to the code sequence at right:

ECX, CC1 = PCMPISTRI                                   ECX, CC1 = PCMPISTRI
JCC(EQ) CC1                                                       JCC(EQ) CC1
ECX, CC2 = PCMPISTRI                   à           JCC(ULT) CC2
JCC(ULT) CC2                                                     JCC(P) CC3
ECX, CC3 = PCMPISTRI
JCC(P) CC3

Support for AVX in Intel and AMD processors: Intel AVX (Intel Advanced Vector Extensions) is a 256 bit instruction set extension to SSE and is designed for applications that are floating point intensive (See here and here for detailed information from Intel and AMD respectively). In VS2010 release, all AVX features and instructions are fully supported via intrinsic and /arch:AVX. Many optimizations have been added to improve the code quality of AVX code generation which will be described with more details in an upcoming blog post. In addition to AVX support in the compiler, the Microsoft Macro Assembler (MASM) in VS2010 also supports the Intel AVX instruction set for x86 and x64.

More precise Floating Point computation with /fp:fast: To achieve maximum speed, the compiler is allowed to optimize floating point computation aggressively under /fp:fast option. The consequence is that the floating point computation errors can accumulate and a result could be so inaccurate that it could severely affect the outcome of programs. For example, we observed that more than half of the programs in the floating points benchmark suite fail with /fp:fast in VS2008 on the X64 targets. In order to make /fp:fast more useful, we “down-tuned” a couple of optimizations in VS2010. This change could slightly affect the performance of some programs that were previously built with /fp:fast but will improve their accuracy. And if your programs were failing with /fp:fast in earlier releases, you may see better results with VS2010.

Conclusion: The Visual C++ team cares about the performance of applications built with our compiler and we continue to work with customers and CPU vendors to improve code generation. If you see issues or opportunities for improvements, please let us know though Connect or through our blog.

Channel 9 Video: Stephan T. Lavavej - Everything you ever wanted to know about nullptr

vcblog — Tue, 27 Oct 2009 16:11:00 GMT

Stephan T. Lavavej is back in front of the Channel 9 cameras once again for a discussion on our recently implemented C++0x feature “nullptr”. In a previous channel 9 appearance, Stephan spoke about the C++0x language and library features we were implementing for VS2010 and the various interdependencies between various features (for example, how rvalue references – a language feature – enable move semantics and perfect forwarding in our Standard Template Library implementation – a library feature.) In this video Stephan describes how rvalue references exposed a few loopholes in the C++ type system around the NULL macro (or more specifically around the value of the NULL macro, the integer constant 0, and how this value is treated “differently” by the compiler to other integer constants.) The issue had already been identified by the C++ Language Committee and a solution had been added to the C++0x language specification (the aforementioned “nullptr”). And to add even more good news, customers can see and use this feature (and all our other C++0x Features) in the Visual Studio 2010 Beta 2 which was released this week. We hope you enjoy Stephan’s latest theatrical release.

Thanks

Damien

Visual Studio 2010 Beta 2 Is Now Available For Download

vcblog — Thu, 22 Oct 2009 16:32:00 GMT

We are very pleased to announce we have released Visual Studio 2010 Beta 2. You can read the official Beta 2 announcement on Soma’s blog. The Visual C++ team has added a few new features and, of course, many bug fixes. The additional features include some substantial new functionality in the MFC library and the return of the MFC Class Wizard. We are currently filming some Channel 9 videos and writing a few VC Blog posts. Our first two Channel 9 videos on Beta 2 are already online. The video on the C++0x Language feature “nullptr” is here and the video on the MFC features is here. You can download Beta2 from this location. Please be sure to continue to watch this blog for updates. As always we welcome your comments/suggestions/criticisms on our blog.

Thank you,

Damien Watkins and Kelly Evans

Visual C++ Team

Visual C++ Code Model in Visual Studio 2010

vcblog — Wed, 14 Oct 2009 21:03:00 GMT

Hello, I’m Vytautas Leonavičius, a developer on the Visual C++ IDE team. Today, I’d like to discuss how the new principles we’re applying to code browsing in Visual Studio 2010 (see here and here) will affect VCCodeModel. General information about VCCodeModel can be found here.

Improvements

Because of our new incremental update architecture, VCCodeModel implicitly gets certain benefits:

• Availability: Once initial population is done, VCCodeModel is pretty much always available. That is a positive result of design decisions to parse files independently. No change of a single file (even the omnipresent windows.h, if you like) will cause “rebuild of the world”. That means, if issued, calls to VCCodeModel.Synchronize will return almost instantly.

• Reliability: We no longer use the old NCB database. Instead, we now use the same redistributable SQL CE components that we provide to our end-users. Suggestions to “delete your .ncb file” (due to DB corruption) in order to restore functionality of Browsing/Intellisense are a thing of the past.

• Independence from a macro state: Prior to Visual Studio 2010, a file snapshot was taken in a certain undefined macro state. Depending on macro state active in a translation unit at the moment file contents were analyzed, certain constructs would or would not make it into the NCB database. That caused a lot of confusion. In Visual Studio 2010 macro state is ignored and all source file declarations and definitions make it into database, and therefore into VCCodeModel.

// Both A and B will make it into database and VCCodeModel, no matter if X is
// defined or not.

#ifdef X

class A {};

#else

class B {};

#endif

See Dealing with preprocessor conditional directives and Hint files sections of Thierry’s post.

• Improved eventing model: VCCodeModel edit Events are more precise and correct, thanks to improvements to our code difference algorithms. The new difference algorithms have better understanding of user edits. This is especially true in the presence of templates. This improves the reliability of the Visual C++ Wizards and clients who listen to VCCodeModel Events.

• Performance: VCCodeModel is faster in scenarios where an object reference is taken and passed through external third party code (your code!) which may perform looped inquiries. This is thanks to some targeted caching we added to VCCodeModel. All clients should benefit from this since many of VCCodeModel services are implemented using its externally exposed APIs for correctness.

Limitations

The aforementioned benefits are primarily brought about by the design decision to treat every single file independently and through some targeted performance work. However, a downside that this design decision exposes is a lack of absolute precision in full symbol resolution in Visual Studio 2010 Browsing. It applies the following limitations on VCCodeModel:

• No symbol resolution: Type Strings for functions (return type), typedefs, base classes, macro definitions (except those specified in hint files) and parameters are coming directly from source code. They are no longer resolved by compiler as it used to be. This problem may surface if a caller wants to figure out precisely what type a function is returning:

Example 1:

using namespace X;

using namespace Z;

Type foo() { return Type(); }

The Type String for VCCodeFunction object referring to function foo is Type. We don’t really know if it is X::Type, global Type or Z::Type. Note that using namespace in this sample, should not necessarily be in same file, it can be in a header (far away). Without building the full translation unit, there’s no efficient way to resolve the symbol Type deterministically and to maintain the performance improvements that our customers desire and that we have achieved with this new design.

Example 2:

File.h

class Base {};

File2.h

namespace A {
class Base {};
}

namespace B {

class Base {};

}

File.cpp

#include “a_header_including_many_headers_and_introducing_using_directives.h”

class Derived : Base // Which Base is that? Base, A::Base or B::Base?

{};

Lack of precise fully-qualified symbol resolution affects TypeString properties of VCCodeParameter, VCCodeTypedef and FullName property of VCCodeBase.It also indirectly affects several APIs of the CodeTypeRef object that is used as a link between object in source code and type.

VCCodeModel provides heuristic APIs that attempt to resolve a name to a type, even if name is not fully qualified. For incompletely qualified names (like ”Type”, if Type belongs to namespace X) VCCodeModel.CodeTypeFromFullName and VCCodeModel.CodeElementFromFullName prefer names defined at global scope and only resolve to a name defined in a namespace if a global one is not found.

We’ve added a helper API called VCCodeModel.CodeElementFromFullName2 in Visual Studio 2010 to help clients detect this ambiguity. See below.

• No code elements from imported assemblies in C++/CLI: Code elements from imported assemblies are not added to the Browsing database. As a consequence, the types coming from these assemblies would not be resolved. For example, if you have a function with a parameter type specified as Exception in source code, VCCodeModel won’t be able to resolve it to System::Exception, since currently such class is not present in database.

• We’ve removed Managed C++ support from VCCodeModel in Dev10: Support for C++/CLI is still there.

New APIs in Visual Studio 2010

We have introduced the following APIs to VCCodeModel in Visual Studio 2010. All APIs are additions to the VCCodeModel interface:

• CodeElementFromFullName2: Is identical to CodeElementFromFullName, except that it will disregard namespaces during lookup. Because there is no symbol resolution in Visual Studio 2010 version of VCCodeModel it is sometimes beneficial to know whether a particular symbol is ambiguous. The primary source of ambiguity in VCCodeModel is using namespace directives. CodeElementFromFullName2 API looks up the name disregarding namespace. For the following source code:

class X {};

namespace NS1 {

class X {};

namespace NS2 {

class X {};

}

Calls to VCCodeModel.CodeElementFromFullName2(“X”) will yield {X; NS1::X; NS1::NS2::X}.

• CodeTypeFromFullName2: Is identical to CodeTypeFromFullName except that it will attempt typedef resolution. That is, for the following code:

class X {};

typedef X TD;

call to VCCodeModel.CodeTypeFromFullName2(“TD”) will yield class X.

• IsSynchronized: returns true or false depending on whether VCCodeModel is in sync with solution’s source code. Useful for avoiding blocking the calling thread for an undefined period of time while Browsing database is being populated.

• SynchronizeFiles: If VCCodeModel is not in sync with source code, it is not safe to query for a VCFileCodeModel for a project file (caller will get null reference if project files are not yet registered in Browsing database). Call to SynchronizeFiles makes sure that FileCodeModel property on a project file is guaranteed to be not null.

• SynchronizeCancellable: if caller invokes this API while Browsing database is being populated, and there’s significant delay till population completes (for example: initial population of solution Browsing database), the user will see a dialog box with a progress bar. The dialog box also allows users to cancel wait and API will exit unblocking thread.

We’re looking forward to hearing your feedback. Any suggestions, comments and feedback about what can we do to make VCCodeModel better are welcome.

Vytas/Visual C++

The ATL/MFC Trace Tool and the Tracing Mechanism

vcblog — Thu, 08 Oct 2009 21:20:00 GMT

Hi, I am Pat Brenner, a Software Design Engineer in the Visual C++ Libraries group. Some time back I wrote about Spy++. Today, I am going to write about another Visual Studio debugging tool, the ATL/MFC Trace Tool, and the tracing mechanism that it interacts with in ATL and MFC.

The tracing mechanism

The tracing mechanism is used to control the type of information, and the amount of that information, that is dumped to the output window during execution of a program. There are a number of categories of information, and different levels of that information, that can be

displayed.

The tracing macros

An application can output tracing messages to the output window by:

· using the ATLTRACE macros (for ATL), defined in atltrace.h.

· using the TRACE macros (for MFC) defined in afx.h.

There are uses of the ATLTRACE and TRACE macros sprinkled throughout the ATL and MFC source code. For example, in CStringT.h in the atlmfc\include folder, in the CStringT::CheckImplicitLoad method, you can find this line of code:

ATLTRACE( atlTraceString, 2, _T( "Warning: implicit LoadString(%u) failed\n" ), nID );

This will dump the message to the output window if level-2 messages in the string category are turned on.

The ATL/MFC Trace Tool

Below is a screen shot of the ATL/MFC Trace Tool. An MFC application named Editor.exe is running. The “atlTraceString” category is selected for the MFC100UD.DLL module in the Editor.exe process. Since a category is selected in the tree, all three of the groups (Process, Module and Category) are enabled. If the Editor.exe process was selected in the tree, only the Process group would be enabled, and if the MFC100UD.DLL module was selected in the tree, only the Process and Module groups would be enabled. With this tool, you can configure exactly what categories you would like to see trace messages for, and what amount of messages in those categories. Here I have indicated that I would like to see a fairly minimal number of trace messages for the entire process, and that the module should inherit the settings from the process, but I have overridden those values and indicated that I want to see a moderate number of trace messages in the string category.

ATL tracing categories

The categories of trace information that can be dumped by ATL:

· atlTraceGeneral: general and miscellaneous trace messages

· atlTraceCOM: COM object and method trace messages

· atlTraceQI: QueryInterface trace messages (category not used in ATL or MFC)

· atlTraceRegistrar: registration trace messages

· atlTraceRefcount: reference count trace messages (category not used in ATL or MFC)

· atlTraceWindowing: Windows message trace messages

· atlTraceControls: ActiveX control related trace messages

· atlTraceHosting: in-place client/site related trace messages

· atlTraceDBClient: database client related trace messages

· atlTraceDBProvider: database provider related trace messages

· atlTraceSnapin: snap-in related trace messages

· atlTraceNotImpl: “interface not implemented” trace messages

· atlTraceAllocation: memory allocation trace messages

· atlTraceException: “exception thrown” trace messages

· atlTraceTime: COleDateTime related trace messages

· atlTraceCache: caching related trace messages (category not used in ATL or MFC)

· atlTraceStencil: stencil related trace messages (category not used in ATL or MFC)

· atlTraceString: CStringT related trace messages

· atlTraceMap: CAtlMap related trace messages

· atlTraceUtil: thread and thread-pool related trace messages

· atlTraceSecurity: CSecurityDesc/CAccessToken related trace messages

· atlTraceSync: synchronization object related trace messages

· atlTraceISAPI: ISAPI related trace messages (category not used in ATL or MFC)

· atlTraceUser: user-defined trace messages (obsolete category not used in ATL or MFC)

· atlTraceUser2: user-defined trace messages (obsolete category not used in ATL or MFC)

· atlTraceUser3: user-defined trace messages (obsolete category not used in ATL or MFC)

· atlTraceUser4: user-defined trace messages (obsolete category not used in ATL or MFC)

Note: the categories that are not used internally by ATL or MFC will probably be removed in a future version of ATL, in order to clean up the interface in the ATL/MFC Trace Tool.

MFC tracing categories

The categories of trace information that can be dumped by MFC:

· traceAppMsg: main message pump trace messages, including DDE

· traceWinMsg: Windows message trace messages

· traceCmdRouting: Windows command routing trace messages

· traceOle: special OLE callback trace messages

· traceDatabase: special database trace messages

· traceInternet: special internet client trace messages

· traceDumpContext: trace messages from CDumpContext

· traceMemory: generic non-kernel memory trace messages

· traceHtml: HTML trace messages

· traceSocket: socket trace messages

Tracing levels

The information dumped by the trace mechanism is assigned a level from 0 (zero) to 4, where 0 is the most important level and 4 the least important. These levels correspond to the five ticks on the sliders in the ATL/MFC Trace Tool.

Tying it together

So, based on the settings I set in the ATL/MFC Trace Tool above, the source line above in CStringT.h will dump out the message to the output window, because although I have indicated that I want only level-0 and level-1 messages from the process, I want level-2 messages in the string category.

How it works

When a module is loaded that is using debug ATL (ATLSD.LIB), part of the initialization process is the initialization of the global CAtlAllocator object g_Allocator (see externs.cpp in the atlmfc\src\ATL\ATLS folder). This method creates a named shared memory area, and part of the name is the process ID (e.g., for process EB0A, the shared memory area is named “AtlDebugAllocator_FileMappingNameStatic_100_EB0A”). This shared memory area is used to contain all the settings for the process, modules and categories that can be modified by the ATL/MFC Trace Tool.

When the ATL/MFC Trace Tool is started up, it first enumerates all the process in the system, and for each, checks to see if a named shared memory exists for that process (using the naming scheme mentioned above. If so, then the tool loads up all the settings for that process and from then on is able to modify the settings for the process in the shared memory area, thus affecting the runtime trace behavior of that process.

An interesting recent discovery

We have implemented support in ATL and MFC for preview, thumbnail and filter (search) handlers. These are loaded by the Windows Explorer and other Windows components (including the Windows Search service). Recently we had an issue where the Windows Search component could not load our debug DLL, so we never got search filter results in indexed locations. As it turns out, this was because the search filter host (which loaded the ATL filter handler DLL in order to do the indexing) was running without any file system permissions. The ATL tracing mechanism, however, tries to set up the shared memory area (for communication with the ATL/MFC Trace Tool) using the CreateFileMapping API. The lack of file system permissions caused this to fail, and the DLL initialization was aborted, and thus our filter handler was not called. Apparently this is an issue that has lurked in ATL since the tracing mechanism was invented. So, in order to fix this issue, I had to allow DLL initialization to continue if the tracing initialization failed, and then simply bail out of any further calls into the tracing mechanism if the initialization had failed. This then allowed the DLL to load and the filter handler was called correctly, and the bug was fixed.

I hope this has been interesting. Let me know if you have any questions.

Pat Brenner

Visual C++ Libraries Development

Ribbon Designer

vcblog — Fri, 02 Oct 2009 03:23:00 GMT

Hello, my name is Samatha Mannem and I am a QA with the IDE team.

The world has become sophisticated and the time has come to make every application geeky as well as fancy. That is where ‘Ribbon’ has evolved. The recent UI designs that people are attracted to are Microsoft Office and Windows 7 ribbons.

While Visual Studio 2008 SP1 included the ability to create an application that has a ribbon UI, it was difficult for you to configure it as desired. Detailed Information on Ribbon Designer VS2008 is available at http://msdn.microsoft.com/en-us/library/bb386089.aspx. The Visual Studio product team received a lot of feedback on this issue. With Visual Studio 2010, designing a ribbon-based UI is made much easier with the “Ribbon Designer”.

During project creation, the Application Wizard allows you to select the ribbon style for your application. In addition to Office, Visual Studio and Windows Native which were available for Visual Studio 2008, Windows 7 ribbon style is also available in Visual Studio 2010.

Changing the style of the application can easily be done on the fly. At any time during application development, the style of the ribbon’s UI can be changed easily via the ‘style’ dropdown shown below. Changing the style of the ribbon only affects the ribbon’s appearance – it does not in any way disturb the functionality of your application.

With Dev10 creating a ribbon of your style or adding/deleting few tools from the existing Office/Windows ribbon is just a drag and drop action. Writing and debugging complicated UI code is now a thing of the past. Adding behavior to the ribbon’s tools is easily done by adding an event handler to each (explained later).

The following images show the variety of controls that can be used on the Ribbon.

Each control shown below can be designed using Ribbon Designer’s tool box shown here. This tool box can be viewed either by hovering the mouse over the ‘Toolbox’ in the Designer window or by using the menu View->Toolbox. You can add the ribbon like any other resource (dialog, icon) with the Add Resource->Ribbon menu in the Resource view.

A ribbon resource created can be added to existing MFC application. To do so, modify the application to load the ribbon resource.

CMFCRibbonBar m_wndRibbonBar; //declare it

if (!m_wndRibbonBar.Create (this)) //create and initialize the ribbon control

{

return -1;

}

if (!m_wndRibbonBar.LoadFromResource(IDR_RIBBON))

{

return -1;

}

Adding various properties to the control can make it function the way user wants it to serve the purpose.

The image and Menu Items of the Button can be set using the properties window and can be viewed by Right clicking on Ribbon ->properties. Setting the properties in this window is same as writing the following code in CMainFrame.cpp.

You can double-click any control on the designer to open an Items Editor and add more items in its sub menu. You can create event handlers for all other control events by using either the Properties window or right click on a control and choose ‘Add Event Handler’.

A Resource file in the solution, ribbonname.mfcribbon-ms, contains the property values of each control on the Ribbon.

For example following properties are equivalent to the property window shown. The values modified in the properties window reflect the values in this resource file.

With this Ribbon designer, our goal is to make your UI creation easy and flexible to change. Overall we believe that with this designer you will have your Application Ribbon easy to play with. We are excited about this feature and would like to hear back from you.

User Feedback

vcblog — Wed, 23 Sep 2009 22:10:00 GMT

Hello! My name is Joshua Baxter, and I am a programming writer on the Microsoft team that produces Help content for Visual C++. I am writing this article to explain how Microsoft collects and handles user feedback.

User feedback is an important part of our documentation improvement program. We maintain over 33,000 topics about C++, and we receive an average of 350 comments from Help users every month. Not only does your feedback help us to improve the quality of existing documentation, it also helps us improve the quality of future documentation.

Most of the feedback we receive comes from ratings and comments that users submit through the MSDN Web site. At the top right corner of every topic is a link that you can use to send us feedback.

We encourage you to leave feedback that describes specific sections in the topic that are wrong, misleading, or confusing so that we can better understand how to fix the topic. Perhaps it goes without saying, but we cannot address generic comments (“bad topic”, “needs work”, “unclear”).

Occasionally, we receive user feedback from other sources. For example, Microsoft MVP Joseph Newcomer maintains MSDN Documentation Errors and Omissions, which at present contains more than 400 issues. Although we also address this kind of feedback regularly, we encourage you to submit your feedback on the MSDN Web site so that it gets to us faster.

When we receive feedback that a topic is inaccurate, we verify whether the concern is valid for both Visual Studio 2008 and Visual Studio 2010. Verification might involve testing the reported inaccuracy by using a code sample or by contacting a member of the development team. If a concern is verified, we revise the documentation as appropriate. Sometimes the topic is actually technically accurate, but it requires clarification or additional information.

Note: We do not maintain versions of the documentation that are earlier than Visual Studio 2008. If we receive feedback about earlier versions, we determine whether it also applies to Visual Studio 2008 or Visual Studio 2010, and then revise the documentation for those versions as required.

Although we address feedback and revise topics regularly, changes do not necessarily appear immediately on the MSDN Web site. All revised topics are reviewed to ensure technical accuracy, and this may take awhile. Also, topics may have to wait in the MSDN publishing queue until the next scheduled update, which occurs every few weeks.

Again, we appreciate the feedback that we receive from our users. If you have feedback about a topic on the MSDN Web site, please click the feedback link at the top right corner of the topic and send us your comments.

Linker throughput

vcblog — Thu, 10 Sep 2009 19:12:00 GMT

Hello, my name is Chandler Shen, a developer from the Visual C++ Shanghai team.

We have made some changes in the upcoming Visual C++ 2010 release to improve the performance of linker. I would like to first give a brief overview of the linker and how we analyze the bottlenecks of current implementation. Later, I will describe the changes we made and the impact on linker performance.

Our Focus

We were targeting the linker throughput of large scale projects full build scenario because this scenario matters most in linker throughput scalability. Incremental linking and smaller projects will not benefit from the work I describe in this blog.

Brief Overview of Linker

Traditionally, what’s done by linker can be split into two phases:

1. Pass1: collecting definitions of symbols (from both object files and libraries)

2. Pass2: fixing up references to symbols with final address (actually Relative Virtual Address) and writing out the final image.

Link Time Code Generation (LTCG)

If /GL (Whole Program Optimization) is specified when compiling, the compiler will generate a special format of object file containing intermediate language. When linker encounters such object files, Pass1 becomes a 2-phase procedure. From these object files, the linker first calls into compiler to collect definitions of all public symbols to build a complete public symbol table. Then the linker supplies this symbol table to the compiler which generates the final machine instructions (or code generation).

Debug Information

During Pass2, in addition to writing the final image, linker will also write debug information into a PDB (Program Database) file if user specifies /DEBUG (Generate Debug Info). Some of this debug information, such as address of symbols, is not decided until linking.

Bottlenecks

In this section, I will show how we analyze some test cases to figure out bottlenecks of performance.

Test Cases

To get an objective conclusion, four real world projects (whose names are omitted) differ in scale, including proj1, proj2, proj3 and proj4, were chosen as test cases.

Table 1 Measurements of test cases

		Proj1	Proj2	Proj3	Proj4
Files	Total	55	27	168	1066
	.obj	4	6	7	882
	.lib	51	21	161	184
Symbols		6026	22436	69570	110262

In Table 1, the number of “symbols” is the number of entries of the symbol table which is internally used by linker to store the information of all external symbols. It is noticeable that “proj4” is much bigger than others.

Test Environment

Following is the configuration of the test machine

· Hardware

o CPU Intel Xeon CPU 3.20GHz, 4 cores

o RAM 2G

· Software Windows Vista 32-bit

Results

To minimize the effect of environment, all cases were run for five times. And the unit of time is in seconds.

In Table 2 and Table 3, it showed that for each test case, there is always one (usually the first, marked in red) run which takes much longer than others. While one run (marked in Green) may take a much shorter run. This is because following two reasons

l OS will cache a file’s content in memory for next read (called prefetch on Windows XP, and SuperFetch on Windows Vista)

l Most of modern hard disks will cache a file’s content for next read

Comparing Table 2 with Table 3, we can notice that if /debug is off, the time of Pass2 is much shorter. So it indicates that the majority of Pass2 is writing PDB files

Table 2 Test result of Non-LTCG with /Debug On

		Pass1	Pass2	Total
Proj1	1	4.437	2.328	6.765
	2	0.266	1.218	1.484
	3	0.265	1.188	1.453
	4	0.265	1.219	1.484
	5	0.235	1.375	1.610
Proj2	1	9.484	15.766	25.250
	2	1.531	8.188	9.719
	3	1.579	8.078	9.657
	4	1.625	7.890	9.515
	5	1.610	8.297	9.907
Proj3	1	27.266	43.687	70.953
	2	4.250	17.672	21.922
	3	4.141	17.265	21.406
	4	4.203	18.500	22.703
	5	4.688	19.078	23.766
Proj4	1	47.453	70.172	117.625
	2	17.250	59.813	77.063
	3	17.547	55.672	73.219
	4	16.516	47.172	63.688
	5	14.937	44.079	59.016

Table 3 Test result of Non-LTCG with /Debug Off

		Pass1	Pass2	Total
Proj1	1	0.187	0.078	0.265
	2	0.218	0.031	0.249
	3	0.187	0.047	0.234
	4	0.203	0.031	0.234
	5	0.187	0.031	0.218
Proj2	1	6.209	0.297	6.506
	2	1.310	0.187	1.497
	3	1.295	0.187	1.482
	4	1.342	0.203	1.545
	5	1.310	0.203	1.513
Proj3	1	15.382	0.764	16.146
	2	3.541	0.546	4.087
	3	3.650	0.562	4.212
	4	3.557	0.546	4.150
	5	3.588	0.562	4.150
Proj4	1	12.059	1.856	13.915
	2	10.811	1.778	12.589
	3	10.874	1.809	12.683
	4	12.855	1.794	14.649
	5	10.796	1.778	12.574

It is highly recommended that users use/LTCG (Link-time Code Generation) to optimize applications. The test results with /LTCG are shown in Table 4.

Table 4 Test Result of LTCG with /Debug On

		Pass1	Pass2	Total
Proj1	1	178.797	1.734	180.531
	2	155.593	0.954	156.547
	3	153.750	1.031	154.781
	4	152.562	0.891	153.453
	5	153.156	0.797	153.953
Proj2	1	120.375	5.546	125.921
	2	102.343	5.172	107.515
	3	102.203	5.235	107.438
	4	102.016	5.343	107.359
	5	102.250	5.078	107.328
Proj3	1	222.859	20.719	243.578
	2	185.281	22.437	207.718
	3	184.984	21.422	206.406
	4	185.203	22.656	207.859
	5	186.078	22.844	208.922
Proj4	1	522.329	122.984	645.313
	2	490.188	54.406	544.594
	3	441.125	51.860	492.985
	4	430.609	51.813	482.422
	5	437.344	49.750	487.094

Observations

Based on above results and other investigation, we have the following observations

1. If /LTCG is used, most of linking time will spend on code-generation (a compiler task) in Pass1.

2. OS caching of input files will decrease the time spent in both passes quick a lot

3. The majority of time spent in Pass2 is writing the PDB file

Linker changes and impact in VS2010

Multi-threading during Pass2

After some investigations, we decided to introduce a dedicated thread to writing PDB files because

1. Most users normally specify /debug when linking, irrespective of whether the application is built under “debug” or “release” configuration.

2. The data written into final binary does not depend on the result of writing PDB file, and vice versa: i.e., the binary writing task is independent of the PDB writing task

3. When the project is big, linker has much other work to do during Pass2 in additional to writing PDB file, such as reading data from object files and libraries.

Results

Following is the table that compares the linker performance results between VS2010 and VS2008 SP1. To remove the effect of cache, we rebooted our test machine (with SuperFetch disabled) before each run. For ease comparison, the time cost by old linker (from Table 2 and Table 4) are also listed (no caching).

Table 5 Test Result of new linker, Non-LTCG with /Debug On

	New linker (VS2010)			Old linker (VS2008 SP1)			Pass2 Improved	Total Improved
	Pass1	Pass2	Total	Pass1	Pass2	Total	Pass2 Improved	Total Improved
Proj1	3.547	1.859	5.406	4.437	2.328	6.765	20.15%	20.08%
Proj2	9.797	10.266	20.063	9.484	15.766	25.250	34.89%	20.54%
Proj3	17.078	22.609	39.687	27.266	43.687	70.953	48.25%	44.07%
Proj4	47.500	54.281	101.781	47.453	70.172	117.625	22.65%	13.47%

Table 6 Test Result of new linker, LTCG with /Debug On

	New linker(VS2010)			Old linker(VS2008 SP1)			Pass2 Improved	Total Improved
	Pass1	Pass2	Total	Pass1	Pass2	Total	Pass2 Improved	Total Improved
Proj1	153.516	0.953	154.469	178.797	1.734	180.531	45.04%	14.44%
Proj2	119.703	5.391	125.094	120.375	5.546	125.921	2.79%	0.66%
Proj3	225.688	16.594	242.282	222.859	20.719	243.578	19.91%	0.53%
Proj4	525.375	80.375	605.750	522.329	122.984	645.313	34.65%	6.13%

From Table 5 and Table 6, it can be seen that multi-threading the linker has improved the performance of Pass2, and it is especially effective for bigger projects.

Future

We will continue to look into linker throughput even after 2010 release to find areas to improve. If you have any suggestions and feedbacks, feel free to let us know.

Compiler Warning C4789

vcblog — Tue, 01 Sep 2009 19:29:00 GMT

When Visual Studio 2010 ships, it will have improvements to warning C4789; allowing it to catch more cases of buffer overrun. This blog post will cover what C4789 warns about, and how to resolve the warning.

What does C4789 mean?

When compiling your source file, you may receive the warning: “warning C4789: destination of memory copy is too small.”

This message means that the compiler has detected a possible buffer overrun in your code.

Example 1

Let’s say we have the source file a.cpp that contains the following:

1: #include <memory.h>

3: int p[1];

5: void bar() {

6: memset(&p[1], 1, sizeof(int));

7: }

From the ”Visual Studio 2008 Command Prompt”, if you compile this with the command:

cl /c /O2 a.cpp

You will receive the warning:

a.cpp(6) : warning C4789: destination of memory copy is too small

For the example above, the compiler has detected a buffer overrun for the variable ‘p’. 'p' has been allocated as an array with one element. Arrays are zero-indexed, so the memset on line 6 is taking the address of the second element of an array; this means that we are actually writing to memory outside the array, corrupting memory!

In this case, the user most likely meant to memset the first element, and thus to fix this issue, the memset would be changed to

memset(&p[0], 1, sizeof(int));

Typical User Scenarios

In practice, a lot of buffer overruns will not be as obvious as Example 1, so I’ll provide some more examples to help you in your investigations.

Example 2

Let’s say we have the source file a.cpp that contains the following:

1: short G1;

3: void foo(int * x)

4: {

5: *x = 5;

6: }

8: void bar() {

9: foo((int *)&G1);

10: }

From the ”Visual Studio 2010 Command Prompt”, if you compile this with the command:

cl /c /O2 a.cpp

You will receive the warning:

a.cpp(9) : warning C4789: destination of memory copy is too small

In this example, we’ve created a variable 'G1' of size short (which is only two bytes), but we’ve taken the address of it and casted it to 'int *' to pass to 'foo’. 'foo' then writes 4 bytes to the memory location pointed at by 'x’. As 'G1' is only 2 bytes in length, the store “*x = 5” will write past 'G1', resulting in a buffer overrun.

There are a couple of important things to note about this example. This buffer overrun will only be caught with the improvements made in Visual Studio 2010. Also, this warning is caught by inlining 'foo' into 'bar'. This means that this buffer overrun is only caught when optimizations are enabled.

To fix the buffer overrun in Example 2, we declare 'G1' as int. If that isn’t an option, we can create a new variable to pass to ‘foo,’ and assign that variable to ‘G1’ (which truncates the int to a short):

int y;

foo(&y);

G1 = y;

Example 3

Let’s say we have the source file a.cpp that contains the following:

1: int G1;

2: int G2;

4: void foo(int ** x)

5: {

6: *x = &G2;

7: }

9: void bar() {

10: foo((int *)&G1);

11: }

From the ”Visual Studio 2010 X64 Cross Tools Command Prompt”, if you compile this with the command:

cl /c /O2 a.cpp

You will receive the warning:

a.cpp(10) : warning C4789: destination of memory copy is too small

This example is exactly like Example 2 with a key difference. We’ve casted “int” to “int *”. On x86, this is a harmless cast (int and int * are the same size, 4 bytes). However, on x64, “int” is 4 bytes, and “int *” is 8 bytes, so this code is no longer correct when this code is run on x64.

C4789 False Positives

You may hit cases of C4789 where the warning is incorrect. This can happen because the compiler detects a buffer overrun along a code path that will never fire.

Example 4

1: __int64 G1;

2: int lengthOfG1 = 8;

4: void foo(char * x, int len) {

5: if (len > 8) {

6: x[8] = 1;

7: }

8: }

10: void bar() {

11: foo((char *)&G1, lengthOfG1);

12: }

From the ”Visual Studio 2010 Command Prompt”, if you compile this with the command:

cl /c /O2 a.cpp

You will receive the warning:

a.cpp(11) : warning C4789: destination of memory copy is too small

In this example, the compiler thinks that ‘G1’ can be buffer overrun because of “x[8] = 1” would assign outside of the size of ‘G1.’ However, as long as ‘lengthOfG1’ is the correct length of ‘G1,’ “x[8] = 1” will never fire for ‘G1,’ and thus a buffer overrun will never occur.

For some of these false positives, the only option will be to disable the warning. In this particular example, however, changing "int lengthOfG1 = 8” to

const int lengthOfG1 = 8;

would solve the problem.

Workarounds

If you have proven that the warning is a false positive, there are a couple of different ways to disable the warning.

1. Disable the warning for one function (recommended)

2. Disable the warning for all functions

Disable the warning for one function

The compiler allows you to disable a warning for a particular function. This is done by putting

#pragma warning ( disable : 4789 )

before the function, and putting

#pragma warning ( default : 4789 )

after the function. This will disable the warning (in this case warning 4789) for that function (and any functions which inline it).

Example 5

1: __int64 G1;

2: int lengthOfG1 = 8;

4: #pragma warning ( disable : 4789 )

5: void foo(char * x, int len) {

6: if (len > 8) {

7: x[8] = 1;

8: }

9: }

10: #pragma warning ( default : 4789 )

11:

12: void bar() {

13: foo((char *)&G1, lengthOfG1);

14: }

15:

16: void bar1() {

17: foo((char *)&G1, 9);

18: }

With the #pragma around ‘foo,’ you will receive no warnings; while without it you will receive the warnings:

a.cpp(13) : warning C4789: destination of memory copy is too small

a.cpp(17) : warning C4789: destination of memory copy is too small

You can also choose to disable the warning for one of the functions where the warning occurs. In the example above, we could put the #pragma around ‘bar’ instead of ‘foo’, and then we’d eliminate the warning for line 13, but still receive the warning on line 17.

Disable the warning for all functions

If you need to ignore warning 4789 completely, you can specify /wd4789 on the command line.

cl /c /O2 /wd4789 a.cpp

This option isn’t recommended as it will hide potentional buffer overruns in your code.

Windows SDK V7.0/V7.0A Incompatibility Workaround

vcblog — Fri, 28 Aug 2009 00:15:00 GMT

Hi,

My name is Nada AboElseoud and I am a QA in VC++ Libraries team. I joined MS in February 2009. I would like to talk here about an incompatibility issue with WinSDK v7.0*.

If you are a developer who has recently migrated to WinSDK v7.0 (standalone SDK) or v7.0A (inbox with VS 2010), you may encounter these kinds of errors “The procedure entry point K32*** could not be located in the dynamic link library KERNEL32.dll” while running your application. This implies that you are running your application on an OS other than Windows7 or Windows Server 2008 R2. This blog will explain this blocking issue and provide the workaround.

Let me explain first why this issue happens. For performance reasons, some APIs have been moved from Psapi.dll to Kernel32.dll in Windows7 and Windows Server 2008 R2. WinSDK v7.0* is reflecting these modifications to be compatible with the new system dlls. This is by design, but wait! If you are trying to link your application to Psapi.lib and then targeting any pre Windows7 or pre Windows Server 2008 R2, you will get this runtime error. Breaking this down, all APIs from Psapi.dll are copied to Kernel32.dll in Windows7 and Windows Server 2008 R2 (Psapi.dll remain unchanged though). Linking to Psapi.lib marks these APIs as Kernel32 APIs to load them from Kernel32.dll instead. Following is the list of these APIs.

//Snapshot from Psapi.lib – WinSDK V7.0*

#if (PSAPI_VERSION > 1)

#define EnumProcesses K32EnumProcesses

#define EnumProcessModules K32EnumProcessModules

#define EnumProcessModulesEx K32EnumProcessModulesEx

#define GetModuleBaseNameA K32GetModuleBaseNameA

#define GetModuleBaseNameW K32GetModuleBaseNameW

#define GetModuleFileNameExA K32GetModuleFileNameExA

#define GetModuleFileNameExW K32GetModuleFileNameExW

#define GetModuleInformation K32GetModuleInformation

#define EmptyWorkingSet K32EmptyWorkingSet

#define QueryWorkingSet K32QueryWorkingSet

#define QueryWorkingSetEx K32QueryWorkingSetEx

#define InitializeProcessForWsWatch K32InitializeProcessForWsWatch

#define GetWsChanges K32GetWsChanges

#define GetWsChangesEx K32GetWsChangesEx

#define GetMappedFileNameW K32GetMappedFileNameW

#define GetMappedFileNameA K32GetMappedFileNameA

#define EnumDeviceDrivers K32EnumDeviceDrivers

#define GetDeviceDriverBaseNameA K32GetDeviceDriverBaseNameA

#define GetDeviceDriverBaseNameW K32GetDeviceDriverBaseNameW

#define GetDeviceDriverFileNameA K32GetDeviceDriverFileNameA

#define GetDeviceDriverFileNameW K32GetDeviceDriverFileNameW

#define GetProcessMemoryInfo K32GetProcessMemoryInfo

#define GetPerformanceInfo K32GetPerformanceInfo

#define EnumPageFilesW K32EnumPageFilesW

#define EnumPageFilesA K32EnumPageFilesA

#define GetProcessImageFileNameA K32GetProcessImageFileNameA

#define GetProcessImageFileNameW K32GetProcessImageFileNameW

#endif

Now, it should be obvious why by calling some API (say EnumProcessModules) you get this runtime error “The procedure entry point K32EnumProcessModules could not be located in the dynamic link library KERNEL32.dll” pointing to a different API name.

However, did you notice the IF condition involved?

#if (PSAPI_VERSION > 1)

This means that these APIs are defined/tagged only if the PSAPI_VERSION > 1. By default this value is set to 2 and _WIN32_WINNT is set to _WIN32_WINNT_MAXVER (which is 0x601 for Win7).

Workaround

After reading about this issue, you may be able now to figure out the solution. Simply,

if you target any OS prior to Windows7 and Windows 2008 R2, what you need to do is to define _WIN32_WINNT to a previous version (before 0x601) or to define Psapi_version to 1.

For example:

cl /MD /EHsc /D _WIN32_WINNT=0x501 mytest.cpp /link Psapi.lib

cl /MD /EHsc /D PSAPI_VERSION=1 mytest.cpp /link Psapi.lib

Does this mean that this generated exe will work fine on Windows7 and Windows Server 2008 R2 as well? Definitely. As stated above, Psapi.dll is not modified and the workaround is just loading the APIs from Psapi.dll.

Otherwise, if you are just targeting Windows7 (or Windows 2008 R2), to take advantage of this performance boost, you can simply use the default predefined macros or explicitly define them as below.

cl /MD /EHsc /D _WIN32_WINNT=0x601 mytest.cpp /link Psapi.lib

cl /MD /EHsc /D PSAPI_VERSION=2 mytest.cpp /link Psapi.lib

Hope this is helpful J

Nada AboElseoud

Tag Parsing C++

vcblog — Wed, 12 Aug 2009 20:59:00 GMT

Hello, my name is Thierry Miceli and I am a developer on the Visual C++ Compiler Front End team. Although our team is mostly known for writing and maintaining the part of the C++ compiler that analyzes your source code and builds an internal representation from it, a great deal of our effort in the last few years has been directed into servicing the IDE and improving the intellisense experience (refreshers here, here, and here).

Today, I am going to write about a new parser that has been specifically created to provide a fast and scalable way to extract information from C++ source code. This parser is one of our new additions to Visual Studio 2010 and we call it the “tag parser”.

The tag parser is used in Visual Studio 2010 to populate the SQL database that supersedes the NCB file. All of the browsing features of VC++ rely in some way on results provided by the tag parser. These include Class View, Call Hierarchy, Go To Definition/Declaration, Get All References, Quick Search, the Navigation Bar and VCCodeModel.

A Fuzzy Parser

It is a fuzzy parser, which means that instead of trying to strictly recognize and validate the full C++ syntax (we have an excellent compiler front-end to do that) it lazily matches an input stream of tokens with some patterns. This parser doesn’t populate a symbol table during parsing, it has no notion of types apart from built-in ones, it doesn’t build a full macro context and its unit of translation is a single file (i.e. it doesn’t follow through #include directives). But nevertheless, the parser is able to deal with all of C++, C++/CLI and IDL.

High level of tolerance to incomplete code and errors.

The tag parser doesn’t try to make sense of every symbol or identifier in the source code. It will be satisfied with being able to recognize the different parts of a declaration and their positions in the source file. If a name in the type specification of a declaration couldn’t get resolved by our C++ compiler this would not prevent the tag parser from recognizing the declaration and it will show up in Class View as in the example below.

The tag parser is somewhat analogous to a human reader of the source code that would just be looking at one unique declaration without knowing much about the rest of the project. He may not know what most identifiers actually represent but he can tell with a high level of confidence what the declaration is and locate its subparts.

In addition to the tolerance to ‘semantically’ incorrect code which is a property of fuzzy parsers, the tag parser has heuristic based error recovery for the most common causes of erroneous code during editing. For example, it will try to detect incomplete declarations or unclosed body of functions definitions as shown in the snapshot below.

Dealing with preprocessor conditional directives.

The tag parser’s main role is to extract information from the source code that is then consumed by the IDE browsing features. Because browsing features closely relate to the editing experience it is more useful that the tag parser generates a structured representation of the full source code as it appears in the editor rather than a representation of the code that would get compiled under a specific project configuration.

The tag parser deals with preprocessor conditional directives (#if, #ifdef, #ifndef, #else, #elif, #endif) in a special way. It incorporates the full code in each of the branches of preprocessor conditional directives but still only parses complete declarations. For example in the image below, both the inactive and active branches are parsed and Class View shows both function declarations.

The tag parser is also able to deal with more complex cases where a declaration is interrupted by one or more preprocessor conditional directives. For example in the image below both of the declarations that can be induced by the 2 branches are parsed and reported.

Faster and scalable

Tag parsing scales because it is incremental – it doesn’t need to re-parse hundreds (or thousands) of compilation units after a header file is changed, as is often the case in an actual build. It is also faster than a full compiler (despite its heuristics) because it is not burdened by macro expansion and full semantic resolution. Thus it is well suited to capture real-time information for even the largest projects.

No built-in semantic resolution

Since the tag parser operates strictly on a per-file basis, certain semantic resolutions are left to its clients. For example, since function declarations and definitions typically appear in separate files, the tag parser reports a function declaration and its definition separately without any binding information. Therefore Class View has to match a function declaration and its definition so that they appear as a single entry in the Class View tree.

The tag parser is light-weight and this comes with some responsibilities on the side of the consumers of the parser results. The good thing here is that clients only have to incur the cost of building the semantic knowledge that they need and they can dig into the data with SQL now.

Hint Files

We tried to make the tag parser as standalone as possible. It doesn’t need to know about any kind of project configuration (include paths, compiler switches, etc…). In many cases the tag parser could be invoked with a source file name as its only argument and it would do an excellent job at extracting detailed information about the code in this file. The only caveat is preprocessor macros that interfere with the C++ syntax so badly that fuzzy parsing and error recovery heuristics cannot make sense out of the code. One example of such macro is STDMETHOD, when expanded it will generate a member function signature from something like:

STDMETHOD(OnDocWindowActivate)(BOOL fActivate)

You’ll have a hard time guessing what the above line means if you don’t know what STDMETHOD is. Since the tag parser doesn’t follow through #include directives and doesn’t perform SQL lookups into the symbol database*, it cannot discover by itself macro definitions. Nevertheless, its macro state can be preconfigured with what we call a ‘hint file’. A hint file simply contains the definitions of macros that are needed for the tag parser to correctly recognize your source code in the presence of macros that fundamentally interfere with the C++ syntax.

If you have Beta1 installed, you will find a “cpp.hint” file in your Visual Studio 2010 install directory under vc\vcpackages, this is the hint file for the VC and SDK library headers. Very often the tag parser will do just fine with only this preset hint file. Nevertheless, if your code or some third party library code you are using contains macros that tamper with the C++ syntax, you may need to setup your own hint file. The IDE will look for files named “cpp.hint” in the directory where your source files are located and in all the parent directories up to the root directory or until a file named “cpp.stop” is found. All the hint files that are found will be preprocessed to build the macro context before your files actually get to be parsed. I won’t go into more details about hint files for now but feel free to ask questions and, by the way, they will be thoroughly documented on MSDN.

Don’t worry too much if this machinery seems complex, most of the time you won’t have to define your own hint files or you’ll just need to drop a “cpp.hint” file with a few macro definitions in your project or solution directory.

In the future we are planning to work on tools that will help you decide where hint files are needed and possibly generate them for you. And we will also work on making the tag parser act smarter in the presence of macros so that fewer hints need to be added to a hint file.

*In theory the tag parser could query the database for macros definitions when additional information is needed to recognize or disambiguate a declaration, but a reliable implementation of symbol lookups (even if it was only for macros) would push the tag parser in the opposite direction of being light-weight, standalone, incremental and independent from project configurations.

Active Template Library (ATL) Security Updates

vcblog — Thu, 06 Aug 2009 00:51:00 GMT

Hello,

On Tuesday 28 July we released guidance and updates to assist developers using our Active Template Library (ATL) to prevent creating controls or components with potential security vulnerabilities. Vulnerabilities in libraries are a rare, but industry wide issue, that requires broad collaboration and action by the community at large to effectively resolve.

Developers who have built controls using the ATL should take immediate action to review their control to identify if it is vulnerable and take appropriate action to rebuild their control using the updated ATL library and distribute a non-vulnerable version of their control to their customers.

If you develop any controls with ATL then please take a look at the guidance on MSDN detailing the steps required to identify and address affected controls. Also, we cover the issue in a Channel9 video.

Thanks

Damien

Visual C++

Linker warning LNK4221, and some tips to avoid it

vcblog — Tue, 21 Jul 2009 19:57:00 GMT

Hello, my name is Chandler Shen, a developer from Visual C++ Shanghai team.

There is some confusion about warning LNK4221 and whether it is safe to ignore it. I would like to give an explanation on when you may encounter this warning and provide tips for some typical scenarios and workarounds.

What does LNK4221 mean?

When building static/import libraries, linker might emit the following message:

“warning LNK4221: no public symbols found; archive member will be inaccessible” (The message will be improved in VS2010 to ”warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library”).

Just as the message says, this warning occurs when an object file was added to an archive library without any previously undefined public symbols, so it will not be accessible in subsequent linker commands.

Though this warning can be safely ignored sometimes, user should be aware of what’s happening beneath the surface.

Example

Create two source files, a.cpp and b.cpp, with following contents

// a.cpp

#include <atlbase.h>

// b.cpp

#include <atlbase.h>

int func1()

{

return 0;
}

In ” Visual Studio 2008 Command Prompt”, enter the following commands (Note: We use command line here to specify the order of .obj files; Visual Studio 2008 will supply linker with .obj files in alphabetical order)

1. cl /c a.cpp b.cpp

2. link /lib /out:test.lib a.obj b.obj

And LNK4221 will be thrown for a.obj as the following.

a.obj : warning LNK4221: no public symbols found; archive member will be inaccessible

For the above case, atlbase.h (shipped with Visual Studio) contains some definitions of symbols, which will be included in both a.obj and b.obj. Additionally, there is a function, func1, defined in b.obj. Linker will process OBJ files in Last In First Out manner, so when it is processing a.obj, it cannot find any new public symbols in it because b.obj provide all the public symbols that a.obj has, LNK4221 will be thrown. If command line 2 is replaced with following

link /lib /out:test.lib b.obj a.obj

Now the warning is gone!

Typical user scenarios

In practice, most LNK4221 warnings occur on stdafx.obj, which is the name of the precompiled header. In this section, we will focus on two typical “real-world” scenarios and provide corresponding workarounds.

Build a static library with object files

Generally, LNK4221 should not be thrown on stdafx.obj. But in some cases, such as migrating from other IDEs or project systems, there are more than one source files which are specified with /Yc, hence the .obj files of them will all contain the content from stdafx.obj, if any of them is processed prior to stdafx.obj, a LNK4221 will be thrown. If this occurs, try following steps to check your configuration of Precompiled Headers.

1. In the Property Dialog of the project, select “Configuration Properties”->”C/C++”->”Precompiled Headers”. Select “Use Precompiled Header (/Yu)” for “Create/Use Precompiled Header”, and accept the default values for the other two options.

2. Right click “stdafx.cpp” in solution explorer. Select “Properties”, then select “Configuration Properties” -> ”C/C++” -> “Precompiled Headers”. Select “Create Precompiled Header (/Yc)” for “Create/Use Precompiled Header”, and accept the default values for other two options.

3. Save the changes and rebuild the project. If the problem remains, check the build log to confirm that only one source file (“stdafx.cpp” here) is compiled with /Yc and the others are compiled with /Yu.

Build a static library with other static libraries

Sometimes, the user would like to build a single static library from existing libraries for convenience. This approach might “lead to” multiple LNK4221 warnings, especially when the user changed the settings of output path of these libraries. To begin with, create a simple test case as follows

1. In Visual Studio 2008, create a “Blank Solution”, and name it “solution1”.

2. Add three new “Win32 Projects”, lib1, lib2 and wrapperlib, into solution1, and in the “Application Settings” page of the wizard, select “Static library” for “Application type”, check “Precompiled header” for “Additional options”, and check “MFC” for “Add common header files for”.

3. Add a new source file, lib1.cpp, into project lib1 with following code

#include <stdafx.h>

int func1()

{

return 1;

}

4. Add a new source file, lib2.cpp, into project lib2 with following code

#include <stdafx.h>

int func2()

{

return 1;

}

5. In the Property Dialog of wrapperlib, select “Configuration Properties” -> “Librarian” -> “General”, enter “lib1.lib lib2.lib” in “Additional Dependencies”, and enter “$(OutDir)” in “Additional Library Directories”.

After all projects are created, build all libraries in the order of lib1, lib2 and wrapperlib.lib, everything should go well except an informative message as following will be shown:

“Replacing .\Debug\stdafx.obj”

The reason why you see the message is because the linker builds a library from some object files and static libraries. It will attempt to replace archive members of input libraries with those in the command line with the same name. In this case, because “.\Debug\stdafx.obj” was used as the name of the corresponding archive member of stdafx.obj for all three libraries. When linker is processing the “.\Debug\stdafx.obj” in command line (the command line used to build wrapperlib.lib would look like “link /lib /out:wrapperlib.lib .\Debug\stdafx.obj lib1.lib lib2.lib”), it finds that both lib1 and lib2 contain a member with same name, so linker will exclude these two from linking. Therefore, no other files will contain symbols defined in this “stdafx.obj”.

To verify the above explanation, perform the following steps:

1. In the Property Dialog of lib1, select “Configuration Properties” -> “General”, set “Intermediate Directory” with “$(SolutionDir)/intermediate/$(ConfigurationName)/$(ProjectName)”, which is a common style of project hierarchy for some users.

2. Rebuild lib1 and wrapperlib, and following warning will be shown.

stdafx.obj : warning LNK4221: no public symbols found; archive member will be inaccessible.

Is it (both the replacement mechanism and the LNK4221) a bug of linker? The answer is no, because:

1. Linker should be very cautious about not dropping any useful object in a library just because its name conflicts with that of another object

2. If something unusual happens (an object contributes no new symbols to archive), user should be informed.

Workarounds

Since this warning is widely reported , the following workaround is recommended

Perform following steps to replace “stdafx.obj” in input libraries manually.

1. In “Visual Studio 2008 Command Prompt”, enter the following command for each input libraries.

lib /remove:.\Debug\stdafx.obj lib1.lib

The name after “/remove” depends on your actual settings, you can use the following commad to confirm

dumpbin /archivemembers lib1.lib

2. If some of “stdafx.obj” are not identical in content, you should merge all “stdafx.obj” into the one in wrapperlib.lib beforehand.

a. Delete all contents in the stdafx.h from wrapperlib.lib

b. Inspect all stdafx.h from input libraries (lib1.lib, lib2.lib, etc.), if there is any content (such as an “#include” clause) that is not included in the one from wrapperlib.lib, add it. If there are conflicting definitions, it’s up to you which one is to be used.

c. Repeat step b until all stdafx.h from input libraries are inspected, so the one from wrapperlib.lib will contain all contents.

3. Build wrapperlib.lib

If you don’t want to work around this linker warning, just ignore LNK4221. However, you should be sure that all “stdafx.obj” are identical in content.

Intellisense and Browsing with C++0x

vcblog — Mon, 13 Jul 2009 21:09:00 GMT

Hello Everyone,

My name is Sunny Gupta and I am a SDET on VC++ IDE & Compiler team. In this blog I will be talking about the IDE support for Cpp0x.

What is C++0x??

C++0x is the name for the next version of the C++ Programming Language Standard. For VC 10, the C++ 0x features that have been implemented in the compiler so far are “auto”, “lambdas”, “rvalue references”, “decltype”, “nullptr” and “static_assert”.

Details about these can be read from:

· Lambda-auto-static_assert

· Rvalue-reference

· Decltype

It will be super cool if the IDE could provide Intellisense and Browsing for all these new features, is VS team going to do that?

- Yes, Beta2 is going to have all the intellisense and browsing functionality for these c++ language changes.

Brief Overview of what to expect.

Before I explain the details of feature I would like to list the cool things.

· A static_assert getting squiggled (red wavy underline under the word indicating some error) with the assert message as the error message.

· Getting parameter help(tool tip about the parameter of the lambda) on lambdas

· Quick info(tool tip that pops up on mouse hove) showing the actual type of an object that is declared using auto

· Complete intellisense for rvalue references as lvalue reference.

· Quick info, Auto Complete etc ., working inside the decltype statement.

Outline of the Feature

The feature can be better explained with the help of some screenshots: I will be demonstrating just few of the many intellisense/browsing options available.

Design and Development of the feature

We had one main goal while designing this feature. That goal was intellinsense parity for cpp0x with all the other cpp constructs. We had the following things in mind during designing the feature:

·         Writing and using lambda should be as easy as writing and using a regular C++ function.

·         Intellisense on an object declared using ‘auto’ or ‘decltype’ should be the same if the object was declared using the actual type name

·         Writing a static assert should be as easy as making a function call to some inbuilt function.

·         Intellisense for rvalue reference should behave the same was as it used to behave for an lvalue reference

Development involved integration of the new cpp0x features with the current intellisense engine. The challenging part was ensuring the IDE shows the expected intellisense and browses to the correct location. Example: Ensuring that on the IDE a lambda is represented as a function with return type and arguments. Lambdas are actually classes with overloaded operator (). So a lambda statement internally creates a class with overloaded operator (). During development just integrating the cpp0x with the intellisense engine would results into lambda being represented as ‘operator ()’ in the tooltips. Instead of lambda name. We had to makes sure where ever the lambda will be shows, class view, Quick info, Call Hierarchy etc., it should not be displayed as ‘operator()’ and a proper readable signature should be shown.

Testing

I was the QA for the feature. Since this was not an isolated feature, the challenging part was to ensure complete intellinsense and browsing functionality inside the IDE for these new c++ language features. Dedicated testing was needed for browsing (Quick Symbol Search, Class View, Navbar, Gotodef etc.) and intellisense (Quick Info, Auto Complete, parameter help and squiggles). This feature is an IDE feature so equal importance was given to IDE testing along with the engine level testing.

Example of an IDE tests is: Ensuring that on mouse hover over a static assert squiggle shows the correct error message tool tip.

Example of an Engine level test: ensuring that the Intellisense compiler API returns correct intellisense results. These API are called from the editor to display the results.

More information on testing can be gained from the blog : Testing VC++ Compiler and Intellisense

Customer Impact & Summary

Having this will enable customer to easily and accurately use cpp0x. Since cpp0x inevitable creates a thin level of wrapping on the underlying Cpp implementation, having intellisense and browsing will ensure better readability of the code. It would be a pity if such great C++ features are not used because the IDE is not intelligent enough to help the programmer navigate their complexity. With the changes we have made to the IDE and IntelliSense engine, we certainly hope we have taken care of that problem. We are excited about these new features and we hope you will like them too.

VC++ Directories in Visual Studio 2010

vcblog — Wed, 08 Jul 2009 18:51:00 GMT

If you recently started using VS2010 and were wondering what happened to Tools > Options > Projects and Solutions > VC++ Directories UI, Brian Tyler has all the details in his new post on the VSProject blog. Check it out here: http://blogs.msdn.com/vsproject/archive/2009/07/07/vc-directories.aspx.

We hope you’ll enjoy the new approach and adopt its new capabilities. Don’t hesitate to send us your feedback.

Thank you,

Visual C++ Team