Visual C++ Team Blog

Nice to see more exciting features moving into C++. Thanks for all the efforts

Kudos to MS for not waiting until the next edition of VS to add these features.

Auf der TechEd wurde die Katze aus dem Sack gelassen. Ale Contenti stellte die neuen Features der MFC

I would like to see improvements to the linker option Eliminate Unreferenced Data, so that any unused functions and/or data is not included in the final output. I did mention this while VS2005 was in beta and was told that this feature already worked correctly, but unfotunately I didn't have time to reply then. This is simply not true though, as is nicely demonstrated y the following article on CP:

http://www.codeproject.com/library/tlibc.asp

"Compiler fixes are more expensive than you might expect"

In that sentence the word "you" means ...... OK, let's see, what does the "K" stand for?

"Intellisense has to be taught to understand whatever the actual compiler understands"

It does?  When did that happen?  (And if Intellisense really has that improvement then, at the risk of going off-topic, may I ask if the debugger does too?)

"any program using export can be trivially rearranged into a program not using export"

Any program using "for" can be trivially rearranged into a program not using "for".  Athough this discussion has involved a mix of standardization and other issues, I think this question concerned standardization, so rearrangement isn't an answer.  Even if it's like trigraphs, i.e. even if no one in the world likes it, I think this question concerned standardization.

By the way, at risk of going off topic again, I'll ask a question about standardization too.  Will "void main" finally get a warning?

(The reason I don't ask if the concatenation

L"wide string" "narrow string"

will finally get a warning is that I've already been told that's a "won't fix".)

[Craig]

> The very first thing that I do with a new compiler to gauge standards compliance is to try export.

I can think of many better things to try.

> However, like it or not, the undeniable fact is that these are all part of the standard.

To be clear: I personally want two-stage name lookup and exception specification support.  (I also want export to be erased from human memory - I don't always get what I want.)

> And there is still "no movements towards supporting [it]".

Riddle me this: why doesn't GCC support export?

> Rather, you are beginning to implement TR1 from C++0x, which isn't even finalized yet.

Are you seriously arguing that export is more useful than TR1?  (Disregarding the fact that compiler devs aren't library devs and vice versa.)

The most important missing thing in C++98-minus-export isn't export, it's shared_ptr.

> Non sequitur. Intellisense is an IDE feature. VC++ is a compiler separate from any IDE.

> While Intellisense is useful, it is not necessary to compile code.

It is our policy to have Intellisense and the actual compiler support the same code.  As it stands, Intellisense is actually powered by a variant of the actual compiler. (Parsing C++ is hard!)

To fend off responses of "but they don't actually support the same code" - yes, we know that Intellisense has bugs. Many bugs. We're working on that.

(I myself don't use Intellisense because I don't use the IDE to edit or compile code, but everyone else does.)

> I had assumed that Microsoft would be licensing the

> Dinkumware implementation, which is reportedly complete.

We are, and it is.

> Or is there further work required to work with VC++?

As ever, development is not as simple as it might seem.  :-)

For example, we had to add TR1 to our build system (getting the source files picked up into the DLL/LIB and getting the headers/sources copied to the right directories in setup). We had to make TR1 work with /clr. We've had to make the TR1 headers (and their regression tests) /analyze-clean. We're fixing correctness and performance bugs. And we have to add IDE visualizers for TR1 types. I've even found defects in TR1 along the way (Library Issues 726 and 727).

Dinkumware's implementation is great, and they're awesome to work with. But there's lots of VC-specific work to do, and everything benefits from having extra eyes look at it.

> However, it is the big issues mentioned above and their lack of priority that I find most troubling.

In the end, priorities are determined by seeing what the VC team thinks is most valuable in addition to customer feedback. So if you think something is important, keep mentioning it to us!  :-)

(Honestly, though, you'll have much better luck with Anything But Export.)

[Norman Diamond]

> Any program using "for" can be trivially rearranged into a program not using "for".

That's a logic change.

> Will "void main" finally get a warning?

I wish!  (I complain bitterly every time I see that particular heresy.)

Stephan T. Lavavej

Visual C++ Libraries Developer

[Laszlo Szoke]

> Why do you hate export so mutch?

It doesn't buy me anything, and it costs a lot.

Doesn't buy me anything: Export doesn't unlock magical powers of templates. It "maybe" speeds up builds (some say yes, some say no, I say PCHs), and it enables "a different style of code organization" which I find utterly worthless.

Costs a lot: It's difficult for compiler writers to implement. It's also difficult to understand all of its consequences. C++ is already difficult enough to understand, being jam-packed with useful features.

> Why don't you love standard so mutch?

Oh, I love the Standard (I also love capitalizing it). That doesn't mean that I love everything in the Standard unconditionally.

There are things that I adore (everything that makes the STL possible). Things that are complicated, but useful (overload resolution, template argument deduction). Things that I haven't found uses for in my own code, but that are definitely useful (virtual inheritance). Things that I'm not too fond of, but understand the reasons for their existence (things for C compatibility, like the conversion rules). Things that I think should always be avoided (C casts, shadowing). Things that might be useful, but are often abused (the ability to replace the global allocation and deallocation functions). Things that seemed like a good idea at the time, but we know better know (auto_ptr, mem_fun, bind1st). There are plain ugly things that will hopefully disappear (stuff lurking in Annex D). And then there is export.

"Why export? Well, the Standard Library had vector<bool>, so the Core Language felt lonely..."

Stephan T. Lavavej

Visual C++ Libraries Developer

[Nemanja Trifunovic]

> Any chance of removing checked iterators from the

> Release mode

You can disable "iterator checking" in Release mode by defining _SECURE_SCL to 0 . However, you must be VERY CAREFUL. There are rules that you must follow in order to avoid violating the One Definition Rule. If you aren't familiar with ODR violations, the most important thing about them is that they are extremely, horribly bad.

I described the rules here: http://blogs.msdn.com/vcblog/archive/2007/08/10/the-future-of-the-c-language.aspx#4617984

I must stress that rule #2 applies to all of your separately compiled libraries, such as Boost.

Note that "iterator checking" (_SECURE_SCL) is usually not that expensive, especially when you avoid handwritten loops in favor of STL algorithms, which you should be doing anyways.  It's "iterator debugging" (_HAS_ITERATOR_DEBUGGING), obtainable in Debug mode only, that is expensive.

> and the "security warnings" when calling some

> standard C and C++ functions?

Disable warning 4996.

> After all, if we cared for security more than

> performance we wouldn't use C++ ;)

It is possible to be both secure and performant.

Stephan T. Lavavej

Visual C++ Libraries Developer

[Craig]

> it really needs to be implemented in due time.

Does it?

I don't see any particular difficulty in continuing to deny the existence of export forever and ever.

> I do not know. Maybe it will someday.

> Would MS jump on the bandwagon then?

It'd certainly make a stronger case.

> Is the fact that it is in the standard not enough justification and reason to implement it?

Unfortunately, no. The people who ultimately make resource allocation decisions care much more about "return on investment" and "customer demand" (as they should) than paragraph-by-paragraph conformance to the Standard. export seems very costly for very little benefit, and there isn't massive customer demand for it (yes, you and several other people want export, but we don't have legions of customers beating down our door about it).

I care about pure conformance much more, but in the end I completely agree with the non-implementation of export.  Only if all compiler bugs were fixed and all yummy C++0x features were implemented would I want to see export (and maybe not even then; implementing stuff breaks other stuff).

> Now riddle me this: why do Comeau C++ and Borland Builder X support export?

One of Comeau's major selling points is pure paragraph-by-paragraph conformance.  I'm not sure (I don't keep track of anything but VC, GCC, and Comeau), but I think Borland now uses the same EDG frontend that Comeau does, so they would get export mostly for free.

> Surely GCC is not the baseline by which MS designs VC++ against.

No, but it is obviously easier to argue that VC should do something when the clear weight of the C++ community is behind it.

> My main argument is this: the 1998 standard should have been fully implemented before beginning on the 2003 update. And the

> 2003 standard should be fully implemented before beginning on the future C++0x.

First, 2003 completely supersedes 1998.  It is 1998 as it was meant to be (C++03 is shorthand for C++98+TC1, integrated into one document for your reading convenience).

Second, imagine that there was one person at Microsoft responsible for deciding on what the frontend devs are going to do in the next version. (There's not actually just one person.) Now put yourself in this person's shoes, deciding what to do in VC9. You don't have an army of ninja frontend devs; instead, you have one (http://blogs.msdn.com/vcblog/archive/2006/06/30/visual-c-compiler-plans.aspx). The rest are working on unspecified post-VC9 awesomeness. JonCaves is our voting committee member (as you can see from http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2453.html), and knows the language and compiler inside and out - but there is only one of him, and all attempts to clone him have been unsuccessful. Now, where should dev time be spent - on a feature that is "perhaps not so useful", or on making two-step PCHs as documented at http://msdn2.microsoft.com/en-us/library/2yzw0wyd(VS.80).aspx actually work? (They apparently worked at one time, probably in VC7.1, but are profoundly broken in VC8 RTM and VC8 SP1).

As it happens, that particular bug (one of my "favorites") was punted; it'll be broken in VC9 RTM too (although I will agitate for it to be fixed in VC10). There were even more pressing matters to attend to. For one great example, Jonathan fixed a lurking bug, noticed by an internal team, that sped up the compilation of large projects using lots of templates with PCHs by 10-20%.

You can bet that "make the compiler faster" is one of the top customer demands for every version.

Stephan T. Lavavej

Visual C++ Libraries Developer

> No matter how fast you make the compiler, if it can not even compile legal C++ code slowly (such as export), then its value is not so great.

YES!

> It is possible to be both secure and performant.

I was really joking. You are absolutely right - not only it is possible, but in many cases required. But only with the checking iterators off! The difference in speed was just amazing (around 3x) for the processing we were doing.

So my point is (and I saw a similar argument in another thread): with the default settings set to "secure", people will simply avoid using the Standard Library and fall back to much less secure C-style data structures and functions. That way not only the security, but also the productivity goes down.

Not really a part of my discussion, but still:

> If i develop on VC++ first, most likely i will get errors on GCC later due to 2-phase name lookup.

> (Or worse, different behavior)

> As I understand it, you'd get compilation

> failure. If you have an example that compiles

> and produces different behavior, I'd like to > see it.

I wrote an article on the two-phase name lookup, and one of the readers left this sample code:

http://www.codeproject.com/cpp/TwoPhaseLookup.asp?msg=949325#xx949325xx

Back to our discussion:

> The difference in speed was just amazing (around 3x) for the processing we were doing.

>Were you using Standard algorithms whenever >possible (yes, even the underappreciated >for_each())?

No, I use them only when it makes my life easier. As Scott Meyers wrote here: http://www.ddj.com/cpp/184401446 "In the ongoing tussle between algorithm calls and hand-written loops, the bottom line on code clarity is that it all depends on what you need to do inside the loop. If you need to do something an algorithm already does, or if you need to do something very similar to what an algorithm does, the algorithm call is clearer. If you need a loop that does something fairly simple, but would require a confusing tangle of binders and adapters or would require a separate functor class if you were to use an algorithm, you’re probably better off just writing the loop."

Mind you, I am the one who uses the algorithms the most - other developers mostly avoid them.

> So the choice of default is a difficult one, > and I think that "on by default" is  >reasonable, if not perfect for everyone.

I dissagree here :) C++ is meant to be used for high speed and easy access to system resources - putting "security" (and all these security settings make C++ only slightly "less insecure", and still far from "secure") first at the expense of speed violates one of the basic design priniples for this lanuage.

[Nemanja Trifunovic]

> one of the readers left this sample code

Thanks for the example.

> As Scott Meyers wrote here

That's good advice (like everything that he writes).

> Mind you, I am the one who uses the algorithms the

> most - other developers mostly avoid them.

Note that there's an additional reason beyond clarity to use Standard algorithms whenever possible: they may be faster.  In the case of VC8+, they actually are faster, since we can lift out _SECURE_SCL/_HAS_ITERATOR_DEBUGGING checks and proceed with raw pointers (in the case of vector iterators, etc.).

> I dissagree here :)

Fair enough; my point is that VC's default is not gratuitously trying to make your life harder.

[Norman Diamond]

> I agree with the opinion that C++ is meant to be

> used for high speed, so options to assist security

> at the expense of speed should default to being

> off.

Things like /GS (Buffer Security Check) are on by default, and this is absolutely the correct decision - /GS is extremely cheap and this check is definitely worthwhile.

_SECURE_SCL followed this example.  It seems that a lot of customers complain about its performance cost, and some of these concerns are legitimate (remember, there are also people who say "argh, don't slow me down" and disable /GS out of fear/superstition rather than for any good reason - separating fear from real concerns is not always easy).  We're looking at how to make _SECURE_SCL faster in future versions of VC.  I don't think you should expect us to make it off by default (security is a big concern around here), but we'll definitely be retaining the option to disable it.

Good defaults are important, but a default that you don't like isn't the end of the world.

> Object-oriented assembly language

C++ is so much more than that.

> Options to maximize security should be on by

> default in C# and its predecessors.  These are

> higher level languages for a reason.

C# is not "higher level" than C++.  C++ is capable both of "down to the metal" coding as well as highly abstracted coding.

Stephan T. Lavavej

Visual C++ Libraries Developer

"Were you using Standard algorithms whenever possible (yes, even the underappreciated for_each())? Handwritten iterator loops incur a greater _SECURE_SCL penalty. We are *very* interested in cases where you are using STL algorithms and still incur significant penalties."

Simple. The case that the most popular STL container was made for.

Random access in a std::vector.

True, if we're iterating through a vector, you can and should usually use the standard algorithms, but consider for a moment what the vector is. A resizable array. If we can't use it as an array (perform random access), it's not very useful at its primary role.

In a perfect world, I'd agree with your decision. Sure, it makes sense to make code more secure by default, but it assumes that all C++ programmers are perfect and know both the language and the IDE.

Neither of which is the case.

Most C++ programmers suck. A huge proportion don't use, or aren't aware of, the standard algorithms. The containers are a bit more accepted, but there are still many who don't use them either, for fears that they're too slow, because they seem too complex, or because of the "Not Invented Here" syndrome.

Most people are very skeptical of the STL to begin with.

If their test code then shows std::vector to be 4 times slower than a plain array, what conclusion do you think they'll draw from that?

Even if they only see it being 50% slower, or 20%, that's enough to impress beginners, and quite a few intermediate programmers (and aren't those the ones we really want to reach with any security improvement?). C++ developers often obsess over performance, whether or not it's justified in that particular case. And the last thing we need is another generation of programmers deciding that "the STL is too slow for real-world code".

That's one half of the argument. C++ is a complex language, and far from every C++ developer are aware of half the features it offers. If they see their for loops on vectors being slow, they're going to ditch the vector rather than switching to for_each or a std algorithm, because they don't know the latter exists.

The other half is that far from everyone know their IDE. Not everyone know that these macros exist in the first place. (You could at least make them configurable in project settings)

Sure, you or I can just toggle the macros off (usually, anyway. It becomes a bit more awkward to do if I want to download and compile a third-party library such as Boost. I trust Boost to be reasonably robust, and some areas of it may be quite performance-critical in my code. So I have to figure out how to make their build tool insert this macro in every compilation unit, and hope that it doesn't break anything.)

But you are seriously underestimating how many people don't do this, because they simply see "Hmm, the STL is ridiculously slow. All those "experts" who keep telling me to use STL must be stupid. I'll make my own resizeable array instead, how had can it be? And I'll use char* instead of std::string, thank you very much"

I see posts by people online every day, who say "I know I've been told to use the STL, but it's just too slow. I made my own array class, and everything runs smoothly now". Some find out afterwards that they can disable SECURE_SCL, some don't. Of those who do find out, some are by then too skeptical of the STL to give it a second chance. And some have simply invested too much time into their C-style code by now, that they don't really want to switch back to STL, with or without SECURE_SCL.

People like you or I, who are aware of the SECURE_SCL option, and realize that STL code is preferable, make up maybe a few percent of the C++ developers out there. (And we're probably not the ones most liable to write security vulnerabilities in the first place)

Most people have a hard time swallowing STL because it looks so complex that "it must be slow". The last thing you want to do is confirm that fear.

"Good defaults are important, but a default that you don't like isn't the end of the world."

And this is coming from a "security is important" person? Making people do the right thing by default isn't a big deal?

Don't you think it is a serious problem if developers shy away from the STL completely? If anything *does* cause the end of the world, I'd say the odds are good that it'll be because of people hand-rolling their own std::vectors, with all the security issues it entails.

No, I think good defaults are extremely important, for security.

[quote]However, with the default settings set to "super speed", no one would ever use the "secure" setting[/quote]

Wouldn't they? Perhaps that's a sign that you're heading the wrong way with SECURE_SCL?

Remember that you've just spent an impressive number of years focusing on managed C++ which no one really cared about. Does that demonstrate the deep understanding of your userbase that is required to proclaim that SECURE_SCL on by default provides a net increase in security? ;)

When your actual users are standing on the other side, saying that from all they've seen, it doesn't?

Anyway, I'm not asking for "super speed", I'm just asking for "good enough for people to consider it comparable to raw C code". A vector shouldn't be noticeably slower than a raw array, because then people won't use it.

I agree with your /GS example, but that's different. If I'm irrationally scared of /GS, I just disable it. I don't start using char*'s. So enabling /GS by default can never have the same *negative* effect on security that _SECURE_SCL does.

"I think _SECURE_SCL's performance effects are overly feared."

Yes, precisely my point. Those of us who don't "overly fear it" can just disable it when we want to. No problem there. (Even if it gets a bit hairy with 3rd party libs, as I said)

It's everyone who overly fear it (or rather, fear the performance characteristics they observe in STL code because of it), who are the issue.

To all the people who say VC must be 100% conformant before doing anything else:

what exactly do you plan to use 'export' for that you cant do now??  Not all language elements are created equal, in fact no compiler had implemented 'export' at the time it was "standardized".  I say continue to let the VC team prioritize things in a sane manner dictated by customer needs as opposed to slavish devotion to adding flawed or minorly useful corners of the 'standard'.

[Nemanja Trifunovic]

> Thanks! That's great information.

> But why did I find about it by accidentally checking out the VC++ dev blog?

Because our _SECURE_SCL/_HAS_ITERATOR_DEBUGGING messaging hasn't been detailed enough.

[jalf]

In my experience, STL-heavy code has indistinguishable performance in VC8 SP1 with _SECURE_SCL enabled and GCC 4.2.1. Certain constructs are significantly impacted by _SECURE_SCL (my worst-case example from an arithmetic encoder slows down by 10x-22x), but these aren't common.

Making people fear the STL is bad. However, people feared the STL long before VC8, and they fear it with other compilers too. _SECURE_SCL is not the root of all evil.

Had I worked at Microsoft when _SECURE_SCL was implemented (I didn't), and had I been responsible for deciding whether to enable or disable it by default (as a dev, I don't get to decide such things), I might have decided to disable it by default. Personally, I think that it is hard to commit heap overruns and other problems with the STL if you know what you're doing. (In contrast, knowing what you're doing generally won't stop you from committing buffer overruns and other badness with C-style code.) I prefer to assume that programmers will be knowledgeable but human (there is really no way to stop bad programmers from writing bad code, and C++ doesn't go down that road). In practice, not all C++ programmers know how to use the language and library as well as they should, so committing security errors with the STL is probably more common than I'd like to expect. Hence, I can see the rationale for the on-by-default decision.

That's my only point: that the enabled-by-default setting is not *unreasonable*. You are free to disagree with it (and like I said, I lean in that direction too), but the purpose of the default isn't to scare people away from using the STL.

> I see posts by people online every day, who say "I know I've

> been told to use the STL, but it's just too slow.

I doubt that this is because of _SECURE_SCL.  I'd tend to expect that those people are doing something egregiously wrong (my favorite is compiling in debug mode - argh - or maybe copying vectors when they should be passing by const reference).

> Don't you think it is a serious problem if developers shy away from the STL completely?

Yes, that would be bad.

> It's everyone who overly fear it (or rather, fear the performance characteristics they observe in STL code because of it), who are the issue.

In my experience, people who avoid the STL do so because of exception handling.  (Exception denial frustrates me greatly.)

> Why is windows.h so absolutely horrible?

> [...] the whole #define hell you have going now?

Here, "you" = "not VC". windows.h is part of the Platform SDK; while it ships with VC, it's not maintained by VC.

> Or just a WindowsEx.h which does the most obvious fixes, such as not defining the min/max macros

Define NOMINMAX before including windows.h, and then it won't stomp all over <algorithm> and friends.

[Derrick]

> "what exactly do you plan to use 'export' for that you cant do now??"

> Better code organization.

There's nothing stopping you from doing that right now.  Remember, you can declare templates before defining them, and you can even stick the definitions in sub-header files.

Templates *inherently* cannot be separately compiled.  What export does isn't separate *compilation* in the usual sense.

> Rather than even giving a time frame, apparently the standard is so unimportant

> that it is not even on the radar as one developer said he wants "export to be erased from human memory".

I'd like to point out a few things:

1. I'm not a compiler dev.

2. I'm not a compiler PM, or anyone else who is involved with deciding what to implement.

3. I consider the Standard extremely important...

4. ... except for export.

5. VC has *massively* increased its conformance from VC6, through VC7.1, to VC8 and now VC9.

Maybe Microsoft should print T-shirts - one saying "IMPLEMENT EXPORT NOW" and another saying "DOWN WITH EXPORT". I sense a billion-dollar business on the horizon...

Stephan T. Lavavej

Visual C++ Libraries Developer

"""An implementation is either conformant or non-conformant; there is no middle-ground."""

IMHumbleO, wrong.

There's a lot of middle ground. A non-conformant compiler must be fixed, but with reasonable priorities.

"""Not everyone know that these macros exist in the first place. (You could at least make them configurable in project settings)""" [2]

For some programming languages, every compiler is non-conforming.

For some languages including C, every program on the face of the earth is a conforming compiler, whether or not it was intended to be a compiler.  Also every program that has been exported from earth is a conforming C compiler.  Some of us wish that the limits section would be debugged to change this, but meanwhile it's a fact.

Nonetheless, when we know what the standard is trying to say that we should do, and if other parts of the standard don't prohibit it, then we ought to try.

For thos who still insist on an implementation of export "for completeness's sake", you should give the "Export Restrictions" articles by Herb Sutter a serious read.

Part 1: http://www.ddj.com/cpp/184401563

Part 2: http://www.ddj.com/cpp/184401584

(Apologies for resurrecting a weeks-old thread, but I just discovered this while arguing with a co-worker about the same subject.)

Thank you very much for the link to this superb article. I was unaware of all these issues, and now I understand the "... and there is export" comment of the blog author in a post above, which without the proper explanation of Herb Sutter's article, sounded pretty much like a smartxxx remark at best, I'm sorry to say.

I realize now that my link problems are more or less poor behavior of linkers over large programs, not solvable by the use of export. So please, disregard my earlier post of Nov 21.

Still, can you (and CodeGear's) guys stop and work this out once for good? I shouldn't have to solve linker problems in 2007 by shuffling compilation order, dup(multip)licating explicit instantiations, regroup .obj files into smaller libraries and more generally rain dance around my computer. Thank you.

Stephan,

> I should have simply linked those articles, or Sutter's "Why We Can't Afford Export" paper, in the first place.

>Shame on me for trying to argue from first principles instead of citing authorities.

What authority? Like it or not, the simple undeniable fact is that export *is* part of the C++ standard. No matter what argument you make against it, a compiler that does not implement it is non-compliant.

Sutter and others tried to get export removed from the standard. That was rejected by the standard committee.

The only way out of implementing export is for Microsoft to declare that VC++ is not a C++ compiler and thus exempt from implementing the C++ standard.

You can claim to care about standards if you only selectively implement the parts that you like and omit the others.