Q&A on our TR1 implementation

Q&A on our TR1 implementation

Rate This
  • Comments 82

Hello.  My name is Stephan and I’m a developer on the Visual C++ libraries team.  As the Visual Studio 2008 Feature Pack Beta (available for download here with documentation available here) contains an implementation of TR1, I thought I’d answer some common questions about this technology.

 

Q. What version of Visual C++ does the Feature Pack work against?

 

A. The Feature Pack is a patch for the RTM version Visual C++ 2008 (also known as VC9)..  The patch can't be applied to:

 

    * VC9 Express.

    * Pre-RTM versions of VC9 (e.g. VC9 Beta 2).

    * Older versions of VC (e.g. VC8).

 

Q: Can I just drop new headers into VC\include instead of applying the patch?

 

A: No. VC9 TR1 consists of new headers (e.g. <regex>), modifications to existing headers (e.g. <memory>), and separately compiled components added to the CRT (e.g. msvcp90.dll). Because it is mostly but not completely header-only, you must apply the patch and distribute an updated CRT with your application. You can think of VC9 TR1 as "Service Pack 0".

 

Q: If I use TR1, will I gain a dependency on MFC? Or, if I use the MFC updates, will I gain a dependency on TR1?

 

A: No. The TR1 and MFC updates don't interact. They're just being distributed together for simplicity.

 

Q: How complete is your TR1 implementation?

 

A: Our implementation contains everything in TR1 except sections 5.2 and 8 (http://open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1836.pdf ). That is, we are shipping the Boost-derived components and the unordered containers.

 

Q: Because TR1 modifies existing headers, can it negatively affect me when I'm not using it?

 

A: It shouldn't (if it does, that's a bug, and we really want to hear about it). You shouldn't see any broken behavior at runtime, nor any new compiler warnings or errors (at least under /W4, the highest level that we aim to keep the VC Libraries clean under). Of course, TR1 can slow down your build slightly (more code means more compilation time), and if you are very close to the /Zm limit for PCHs, the additional code can put you over the limit. You can define _HAS_TR1 to 0 project-wide in order to get rid of the new code.

 

Q: Does this patch affect the compiler or IDE?

 

A: No.

 

Q: Did you license Dinkumware's TR1 implementation?

 

A: Yes (see http://dinkumware.com/tr1.aspx ), just as we licensed their Standard Library implementation. So, you can expect the usual high level of quality.

 

Q: So, what has MS been doing?

 

A: Roughly speaking, additional testing and "MS-specific" stuff.

 

1. We've integrated TR1 into VC9, so TR1 lives right next to the STL. (This involved unglamorous work with our build system, to get the TR1 separately compiled components picked up into msvcp90.dll and friends, and with our setup technologies, to get the new headers and sources picked up into the Visual Studio installer.) As a result, users don't have to modify their include paths, or distribute new DLLs with their applications - they just need to apply the patch and update their CRT.

 

2. We've made TR1 play nice with /clr and /clr:pure (which are outside the domain of Standard C++, but which we must support, of course). At first, these switches caused all sorts of compiler errors and warnings. For example, even something as simple as calling a varargs function internally triggered a "native code generation" warning. These errors and warnings took a long time to iron out.

 

3. We're ensuring that TR1 compiles warning-free at /W4, in all supported scenarios. This includes switches like /Za, /Gz, and the like.

 

4. We're ensuring that TR1 is /analyze-clean.

 

As usual, we've preferred real fixes to workarounds to disabling warnings (and when we disable warnings, we do so only in the headers, not affecting user code).

 

5. We're identifying bugs in TR1 and working with Dinkumware to fix them. Dinkumware's code was very solid to begin with - but as ever, more eyes find more bugs. I've even found a couple of bugs in the TR1 spec itself (see http://open-std.org/JTC1/sc22/WG21/docs/lwg-active.html#726 and Issue 727 below).

 

6. We're striving for performance parity with Boost (which serves as a convenient reference; we could compare against GCC's TR1 implementation, but then we'd have to deal with the difference in compilers). In some areas, we won't get there for VC9 TR1 (hopefully, we should for VC10), but we've already made good progress. Thanks to MS's performance testing (which Rob Huyett has been in charge of), we identified a performance problem in regex matching, which Dinkumware has sped up by 4-5x. (Note that this fix didn't make it into the beta, which is roughly 18x slower at matching; current builds are roughly 3.8x slower.) And we've achieved performance parity for function (again, this fix didn't make it into the beta).

 

("Okay," you say, "but will regex::optimize make it faster?" Unfortunately, no. The NFA => DFA transformation suggested by regex::optimize will not be implemented in VC9 TR1, but we will consider it for VC10. In my one cursory test, regex::optimize did nothing with Boost 1.34.1.)

 

7. We're identifying select C++0x features to backport into TR1 - for example, allocator support for shared_ptr and function. While not in TR1, this is important to many customers (including our own compiler). This just got checked in, and isn't in the beta.

 

8. We're implementing IDE debugger visualizers for TR1 types. Like the STL (more so, in some cases), the representations of TR1 types are complicated, so visualizers really help with debugging. I've written visualizers for almost every TR1 type (I am secretly proud of how shared_ptr's visualizer switches between "1 strong ref" and "2 strong refs"). Note that the beta doesn't include any TR1 visualizers.

 

9. We've worked with Dinkumware to fix a small number of bugs present in VC8 SP1 and VC9 RTM ("because we were in the neighborhood"). One which was actually related to TR1 was that stdext::hash_set/etc. had an O(N), throwing, iterator-invalidating swap() (discovered because unordered_set/etc. shares much of its implementation). This has been fixed to be O(1), nofail, non-iterator-invalidating.

 

10. Because TR1 lives alongside the STL, we've made them talk to each other in order to improve performance. For example, STL containers of TR1 types (e.g. vector<shared_ptr<T> >, vector<unordered_set<T> >) will avoid copying their elements, just as STL containers of STL containers in VC8 and VC9 avoid copying their elements.

 

This is a little-known feature of the VC8 STL; it's there in the source for everyone to see, except that almost no one reads the Standard Library implementation (nor should they have a reason to).  Basically, this is a library implementation of C++0x "move semantics", although it's naturally much more limited than language support will be.  In VC8, template magic is used to annotate containers (vector, deque, list, etc.) as having O(1) swaps, so containers-of-containers will swap them instead of making new copies and destroying the originals.  (For builtin types, swapping would be less efficient.)

 

We've simply extended this machinery to the new types in TR1. Everything in TR1 with custom swap() implementations will benefit: shared_ptr/weak_ptr (avoiding reference count twiddling), function (avoiding dynamic memory allocation/deallocation), regex (avoiding copying entire finite state machines), match_results (avoiding copying vectors), unordered_set/etc. (avoiding copying their guts), and array (which doesn't have an O(1) swap(), but does call swap_ranges() - so arrays of things with O(1) swaps will benefit).

 

That is to say: if a vector<shared_ptr<T> > undergoes reallocation, the reference counts won't be incremented and decremented. That's really neat, if you ask me.

 

If you have any questions about or issues with the Feature Pack Beta, let us know!

 

Thanks,

 

Stephan T. Lavavej, Visual C++ Libraries Developer

  • Soma annonce sur son blog le support du TR1 dans Visual Studio 2008 . Vous pouvez télécharger la béta

  • [Richard Webb]

    > I've opened a number of bugs @ Connect.

    Thanks!

    > Can you confirm whether these issues are considered bugs in the feature pack?

    I haven't yet looked at all of the bugs you opened. In general - where N2157 clarifies or corrects TR1, we'll want to follow N2157. Obviously, we'll ignore C++0x-specific changes in N2157 (e.g. is_rvalue_reference).

    Stephan T. Lavavej, Visual C++ Libraries Developer

  • @Jared, heh. Might be because the service packs take forever to install nowadays, MS doesn't want to bother with them ;)

    VC 6 ones were a lot faster. But of course, as time moves on, things get bigger and [maybe?] better.

    I've submitted some bugs myself, and hope the fix gets back ported to VC 2005.

  • the regex doesn't work with subsystem:window application,when i did 2 undef about the max() and min(),it then worked.but still it was givin me memory leaks

  • > the regex doesn't work with subsystem:window

    > application,when i did 2 undef about the max() and

    > min(),it then worked.

    We have an open bug about this, thanks. In the meantime, you can define NOMINMAX before including <windows.h>, which will prevent it from defining the min and max macros that stomp over <regex>.

    (In VC7.1, I think, <algorithm> was affected, but it has since been made immune. We'll simply extend this to TR1.)

    > but still it was givin me memory leaks

    Can you provide a self-contained test case that demonstrates this?

    Thanks,

    Stephan T. Lavavej, Visual C++ Libraries Developer

  • this a mfc dialog application

    i put:

    "

    #include "stdafx.h"

    #include "regex_test_dlg.h"

    #include "regex_test_dlgDlg.h"

    #undef max

    #undef min

    #include <regex>

    using namespace std::tr1;

    "

    at the beginning of the regex_test_dlgDlg.cpp file.

    then i put this "regex r("aa");" into OnInitDialog(),after that,i start the debug version program.when it's done,i get this

    "

    Detected memory leaks!

    Dumping objects ->

    {486} normal block at 0x00ED1B20, 20 bytes long.

    Data: <@4C             > 40 34 43 00 15 00 00 00 00 00 00 00 00 00 00 00

    {485} normal block at 0x00ED1AC8, 24 bytes long.

    .................

    .................

    "

    do you get that?

  • 好男人 has touched on something that I've wondered about for years.

    Why can't the VC++ memory leak detection code be extended, so that the Origin (Filename + source code line and/or class name) is captured?  You could then echo the Origin when you report each item in: "Dumping object ->".  

    I appreciate VC++ detecting memory leaks, but notification of the leak isn't worth nearly as much as also knowing what actually leaked (which enables devs to more quickly fix leaks).  In the above post (by 好男人) it may be obvious what caused the memory leak. But, even in that case we are making an assumption and as a CS professor wisely told me: DON'T ASSUME! In large systems, it’s usually anything but obvious what leaked.  

    Am I missing something?  Is there an easy way to reconcile the detected memory leak to the actual source code or class?  (I'm sometimes  kind of dense).

    Thanks much.

  • Confirmed! regex is leaking memory. I've identified the problem and I'll file a bug about it immediately.

    Stephan T. Lavavej

    Visual C++ Libraries Developer

  • @Jared:

    Usually the debug CRT does in fact tell you what leaked, and where. You can even double-click in the IDE and it will take you right to the cause, provided it was identified correctly.

    I don't remember the specifics, but occasionally it doesn't show you the offending line.

    Anyway, here's an example which *does* show you the offending code.

    // compile with cl /D_DEBUG /MDd leak.cpp

    #define _CRTDBG_MAP_ALLOC

    #include <stdio.h>

    #include <crtdbg.h>

    int main()

    {

    malloc(40); // leak 1

    malloc(10); // leak 2

    _CrtDumpMemoryLeaks();

    return 0;

    }

    0:000> g

    Detected memory leaks!

    Dumping objects ->

    leak.cpp(11) : {62} normal block at 0x001A3FB8, 10 bytes long.

    Data: <          > CD CD CD CD CD CD CD CD CD CD

    leak.cpp(10) : {61} normal block at 0x001A1188, 40 bytes long.

    Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD

    Object dump complete.

    ---------------------------------------------

    And another with C++ code, requires some adjustment:

    // compile with cl /D_DEBUG /MDd /EHsc leak2.cpp

    #define _CRTDBG_MAP_ALLOC

    #include <iostream>

    #include <crtdbg.h>

    #ifdef _DEBUG

    #define DEBUG_NEW new(_NORMAL_BLOCK, __FILE__, __LINE__)

    #define new DEBUG_NEW

    #endif

    int main()

    {

      char *a = new char[10];

      char *b = (char*)malloc(20);

      _CrtDumpMemoryLeaks();

      return 0;

    }

    0:000> g

    Detected memory leaks!

    Dumping objects ->

    leak2.cpp(17) : {62} normal block at 0x002011D0, 20 bytes long.

    Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD

    leak2.cpp(16) : {61} normal block at 0x00201188, 10 bytes long.

    Data: <          > CD CD CD CD CD CD CD CD CD CD

    Object dump complete.

  • Nice tricks with _CrtDumpMemoryLeaks(), i didn't know much about it.

    It seems that the documentation is not that good, because i couldn't find _CRTDBG_MAP_ALLOC and DEBUG_NEW new(_NORMAL_BLOCK, __FILE__, __LINE__). But i admit that my search was not extensive.

    If you use _CrtDumpMemoryLeaks() without those macros, would it work? It's a little sad that such features are not easily available from the IDE. Or are they?

  • Sorry about the slightly off-topic posts guys.

    @ikk, these links should be helpful:

    http://msdn2.microsoft.com/en-us/library/k70yt3e2%28VS.80%29.aspx

    http://msdn2.microsoft.com/en-us/library/zh712wwf(VS.80).aspx

    http://msdn2.microsoft.com/en-us/library/974tc9t1%28VS.80%29.aspx

    And all the related sub-topics, etc.

    Also, I recommend John Robbins' excellent book, which also covers the Debug CRT very nicely:

    http://www.amazon.com/Debugging-Applications-Microsoft-Windows-Pro-Developer/dp/0735615365

    (Don't let the .NET in the title fool you, the majority of it is native)

  • @ikk,

    The spam catcher stops me from posting anything containing multiple URLs, so I'll just tell you that you can find more info on MSDN -- just look for the Debug CRT, etc.

    Also, get John Robbin's book "Debugging Applications" which covers the Debug CRT and a lot more.

  • "A: Our implementation contains everything in TR1 except sections 5.2 and 8"

    But the beta reference_wrapper doesn't seem to be assignable. Are we going to have reference_wrapper& operator=(const reference_wrapper<T>& x)?

    Thanks.

  • Here is a piece of my test code:

    struct A

     {

     int f() {std::cout<<" f called\n";return 8; }

     };

    A a;

     int (A::*pAf)()  = &A::f;

     std::tr1::function< int (A)> mf1(pAf);

     //std::tr1::function< int (A*)> mf2(pAf);

     //mf2(&a);

     mf1(a);

     boost::function<int (A)> bf1(&A::f);

     boost::function<int (A*)> bf2(&A::f);

     bf1(a);

     bf2(&a);

     std::tr1::mem_fn(pAf)(a);

     std::tr1::mem_fn(pAf)(&a);

     std::tr1::bind(pAf, a)();

     std::tr1::bind(pAf, &a)();

    My question:

    Why the line std::tr1::function< int (A*)> mf2(pAf);

    won't work?

    Note that boost::function<int (A*)> bf2(&A::f); as well as all other lines, works fine.

    Is there a reason why it shouldn't be allowed?

    I couldn't figure it out from n1836. And I didn't have time to dig through the source code yet.

    Am I missing something?

    Thanks.

  • [H Zhang]

    > But the beta reference_wrapper doesn't seem to be

    > assignable.

    It is:

    C:\Temp>type ref.cpp

    #include <functional>

    #include <iostream>

    #include <ostream>

    using namespace std;

    using namespace std::tr1;

    int main() {

       int x = 100, y = 200;

       reference_wrapper<int> a(x), b(y);

       a += 10;

       b += 20;

       cout << x << ", " << y << endl;

       b = a;

       cout << x << ", " << y << endl;

       a += 1;

       b += 2;

       cout << x << ", " << y << endl;

    }

    C:\Temp>cl /EHsc /nologo /W4 ref.cpp

    ref.cpp

    C:\Temp>ref

    110, 220

    110, 220

    113, 220

    > Why the line std::tr1::function< int (A*)> mf2(pAf);

    > won't work?

    That's a bug. I'll file a bug in our internal database.

    Thanks,

    Stephan T. Lavavej, Visual C++ Libraries Developer

Page 4 of 6 (82 items) «23456