Q&A on our TR1 implementation

Q&A on our TR1 implementation

  • Comments 82

Hello.  My name is Stephan and I’m a developer on the Visual C++ libraries team.  As the Visual Studio 2008 Feature Pack Beta (available for download here with documentation available here) contains an implementation of TR1, I thought I’d answer some common questions about this technology.

 

Q. What version of Visual C++ does the Feature Pack work against?

 

A. The Feature Pack is a patch for the RTM version Visual C++ 2008 (also known as VC9)..  The patch can't be applied to:

 

    * VC9 Express.

    * Pre-RTM versions of VC9 (e.g. VC9 Beta 2).

    * Older versions of VC (e.g. VC8).

 

Q: Can I just drop new headers into VC\include instead of applying the patch?

 

A: No. VC9 TR1 consists of new headers (e.g. <regex>), modifications to existing headers (e.g. <memory>), and separately compiled components added to the CRT (e.g. msvcp90.dll). Because it is mostly but not completely header-only, you must apply the patch and distribute an updated CRT with your application. You can think of VC9 TR1 as "Service Pack 0".

 

Q: If I use TR1, will I gain a dependency on MFC? Or, if I use the MFC updates, will I gain a dependency on TR1?

 

A: No. The TR1 and MFC updates don't interact. They're just being distributed together for simplicity.

 

Q: How complete is your TR1 implementation?

 

A: Our implementation contains everything in TR1 except sections 5.2 and 8 (http://open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1836.pdf ). That is, we are shipping the Boost-derived components and the unordered containers.

 

Q: Because TR1 modifies existing headers, can it negatively affect me when I'm not using it?

 

A: It shouldn't (if it does, that's a bug, and we really want to hear about it). You shouldn't see any broken behavior at runtime, nor any new compiler warnings or errors (at least under /W4, the highest level that we aim to keep the VC Libraries clean under). Of course, TR1 can slow down your build slightly (more code means more compilation time), and if you are very close to the /Zm limit for PCHs, the additional code can put you over the limit. You can define _HAS_TR1 to 0 project-wide in order to get rid of the new code.

 

Q: Does this patch affect the compiler or IDE?

 

A: No.

 

Q: Did you license Dinkumware's TR1 implementation?

 

A: Yes (see http://dinkumware.com/tr1.aspx ), just as we licensed their Standard Library implementation. So, you can expect the usual high level of quality.

 

Q: So, what has MS been doing?

 

A: Roughly speaking, additional testing and "MS-specific" stuff.

 

1. We've integrated TR1 into VC9, so TR1 lives right next to the STL. (This involved unglamorous work with our build system, to get the TR1 separately compiled components picked up into msvcp90.dll and friends, and with our setup technologies, to get the new headers and sources picked up into the Visual Studio installer.) As a result, users don't have to modify their include paths, or distribute new DLLs with their applications - they just need to apply the patch and update their CRT.

 

2. We've made TR1 play nice with /clr and /clr:pure (which are outside the domain of Standard C++, but which we must support, of course). At first, these switches caused all sorts of compiler errors and warnings. For example, even something as simple as calling a varargs function internally triggered a "native code generation" warning. These errors and warnings took a long time to iron out.

 

3. We're ensuring that TR1 compiles warning-free at /W4, in all supported scenarios. This includes switches like /Za, /Gz, and the like.

 

4. We're ensuring that TR1 is /analyze-clean.

 

As usual, we've preferred real fixes to workarounds to disabling warnings (and when we disable warnings, we do so only in the headers, not affecting user code).

 

5. We're identifying bugs in TR1 and working with Dinkumware to fix them. Dinkumware's code was very solid to begin with - but as ever, more eyes find more bugs. I've even found a couple of bugs in the TR1 spec itself (see http://open-std.org/JTC1/sc22/WG21/docs/lwg-active.html#726 and Issue 727 below).

 

6. We're striving for performance parity with Boost (which serves as a convenient reference; we could compare against GCC's TR1 implementation, but then we'd have to deal with the difference in compilers). In some areas, we won't get there for VC9 TR1 (hopefully, we should for VC10), but we've already made good progress. Thanks to MS's performance testing (which Rob Huyett has been in charge of), we identified a performance problem in regex matching, which Dinkumware has sped up by 4-5x. (Note that this fix didn't make it into the beta, which is roughly 18x slower at matching; current builds are roughly 3.8x slower.) And we've achieved performance parity for function (again, this fix didn't make it into the beta).

 

("Okay," you say, "but will regex::optimize make it faster?" Unfortunately, no. The NFA => DFA transformation suggested by regex::optimize will not be implemented in VC9 TR1, but we will consider it for VC10. In my one cursory test, regex::optimize did nothing with Boost 1.34.1.)

 

7. We're identifying select C++0x features to backport into TR1 - for example, allocator support for shared_ptr and function. While not in TR1, this is important to many customers (including our own compiler). This just got checked in, and isn't in the beta.

 

8. We're implementing IDE debugger visualizers for TR1 types. Like the STL (more so, in some cases), the representations of TR1 types are complicated, so visualizers really help with debugging. I've written visualizers for almost every TR1 type (I am secretly proud of how shared_ptr's visualizer switches between "1 strong ref" and "2 strong refs"). Note that the beta doesn't include any TR1 visualizers.

 

9. We've worked with Dinkumware to fix a small number of bugs present in VC8 SP1 and VC9 RTM ("because we were in the neighborhood"). One which was actually related to TR1 was that stdext::hash_set/etc. had an O(N), throwing, iterator-invalidating swap() (discovered because unordered_set/etc. shares much of its implementation). This has been fixed to be O(1), nofail, non-iterator-invalidating.

 

10. Because TR1 lives alongside the STL, we've made them talk to each other in order to improve performance. For example, STL containers of TR1 types (e.g. vector<shared_ptr<T> >, vector<unordered_set<T> >) will avoid copying their elements, just as STL containers of STL containers in VC8 and VC9 avoid copying their elements.

 

This is a little-known feature of the VC8 STL; it's there in the source for everyone to see, except that almost no one reads the Standard Library implementation (nor should they have a reason to).  Basically, this is a library implementation of C++0x "move semantics", although it's naturally much more limited than language support will be.  In VC8, template magic is used to annotate containers (vector, deque, list, etc.) as having O(1) swaps, so containers-of-containers will swap them instead of making new copies and destroying the originals.  (For builtin types, swapping would be less efficient.)

 

We've simply extended this machinery to the new types in TR1. Everything in TR1 with custom swap() implementations will benefit: shared_ptr/weak_ptr (avoiding reference count twiddling), function (avoiding dynamic memory allocation/deallocation), regex (avoiding copying entire finite state machines), match_results (avoiding copying vectors), unordered_set/etc. (avoiding copying their guts), and array (which doesn't have an O(1) swap(), but does call swap_ranges() - so arrays of things with O(1) swaps will benefit).

 

That is to say: if a vector<shared_ptr<T> > undergoes reallocation, the reference counts won't be incremented and decremented. That's really neat, if you ask me.

 

If you have any questions about or issues with the Feature Pack Beta, let us know!

 

Thanks,

 

Stephan T. Lavavej, Visual C++ Libraries Developer

  • - I mean, Is TR1 an specific feature for MFC users? if not , why did you distribute it along with MFC update?

    - after compiling with this new patch ,is distribution of updated CRT enough ?

    No new libraries needed to distribute?  

    Thanks in advance

  • [Dave]

    > I mean, Is TR1 an specific feature for MFC users?

    No. It has nothing whatsoever to do with MFC.

    > if not , why did you distribute it along with MFC update?

    Read the third Q&A above:

    "Q: If I use TR1, will I gain a dependency on MFC? Or, if I use the MFC updates, will I gain a dependency on TR1?

    A: No. The TR1 and MFC updates don't interact. They're just being distributed together for simplicity."

    > after compiling with this new patch ,is distribution of updated CRT enough ?

    Yes.

    > No new libraries needed to distribute?

    Correct. TR1 involves no additional DLLs.

    Stephan T. Lavavej, Visual C++ Libraries Developer

  • Is VC 2005 still going to get service packs and updates? The product lifecycle website says mainstream support ends 2011 and extended support ends 2016. I'm hoping you don't just forget about it.

    After all, VC6 had 6 service packs.

  • Good question scorpion007!!

    I'm also curious if VC 2005 will get more SPs.  I too remember the relatively frequent SPs for VC6 and I miss getting semi-regular service packs.  IMHO, VS has turned into a money-machine where developers need to buy new versions to get fixes that used to be in Service Packs.

    Looking at it from a different angle.  VS6 SPs were sort of a conversation (feedback loop) between VS producers and consumers: Each SP addressed either recent or longstanding significant issues. Continued support was something like iterative.   Then (with VS 2002, 2003, and 2005) continued support appears to be more Waterfall: A customer gets one SP and that's it, so you'd better like it.

    I prefer Iterative development and support.

  • A developer could implement stdint.h/cstdint and test cases for it in an afternoon or less, especially if you have the licensed Dinkumware implementation to work from.

    It shouldn't interact with any of the other STL or TR1 code, there's no reason to refactor any of it to use cstdint. It's isolated from everything else. This is probably one of the easiest TR1 features to get right.

    You would seriously be doing everyone a service by shipping it. Here's hoping you will reconsider. ^_^

  • 缘起: 自VisualC 5.06.0以来一直遭人诟病的是什么?过于简单的界面控件!

    作为一个以VisualC 作为开发工具的程序员,遇到最郁闷的事情是什么?开发一个具有漂...

  • 缘起: 自VisualC 5.06.0以来一直遭人诟病的是什么?过于简单的界面控件!

    作为一个以VisualC 作为开发工具的程序员,遇到最郁闷的事情是什么?开发一个具有漂...

  • <quote>

    Hey - everyone who says Microsoft needs to provide stdint.h. There's a public domain version available (from the MinGW toolset):

    http://www.cs.colorado.edu/~main/cs1300/include/stdint.h

    It required some tweaks for my use

    </quote>

    It is exactly those tweaks that developers should not need to do. That's the point of stdint/inttypes: it should be provided by the implementation so that we can be sure that changes in the implementation are reflected in stdint.

    BTW, boost also has stdint.

  • Pavel Minaev - I am happy to report that the swap-optimization regression you found has just been fixed in mainline (and the comments have been updated to match the code). Thanks again for being so observant.

    Also, a bug has been fixed where replacing "A" with "x" in "AAAAAA" was producing "xAxAxA" instead of the correct "xxxxxx". (I found this one myself, but someone also reported it through vcblog a day later!)

    Keep those bug reports coming!

    Stephan T. Lavavej, Visual C++ Libraries Developer

  • <quote>

    It is exactly those tweaks that developers should not need to do. That's the point of stdint/inttypes: it should be provided by the implementation so that we can be sure that changes in the implementation are reflected in stdint.

    </quote>

    True.  but if you aren't going to get it from Microsoft (and that looks to be the case in the near future) and you want or need it, then it's an option.  And, since it public domain there are absolutely no licensing issues (though Boost's license is pretty dang close to PD as well).

    The boost version is C++ only (which may be fine for some people).  Then again, the version I pointed to doe snot have a cstdint variant for C++, and it has some deficiencies properly defining pointer-sized values on 64-bit builds.  Anyone who wants a correct version for Win64 should maybe get the latest from MinGW at:

    http://www.mingw.org/download.shtml

    As for changes I had to make in my case, I'm pretty sure I'll have no good luck getting a version of stdint.h from MS for VC6, even if they ever deliver one for VC 2008+.

  • VC6 is no longer supported by Microsoft.

    ("Good riddance," I said, referring only to the compiler and libraries, which were hideously nonconformant.)

    Stephan T. Lavavej, Visual C++ Libraries Developer

  • Hi,

    I've tried running the Boost regression tests on VC9 with the feature pack, and they are showing up a number of apparent issues with the beta TR1.

    For example, from type_traits:

    -std::tr1::is_const<type[N]>::value always seems to be true

    -std::tr1::remove_const<const type[N]::type always seems to be 'type const[N]'.

    Using the Boost versions of type_traits, the results are false and 'type[N]'.

    The Boost/VC9/TR1 test results are viewable at <http://beta.boost.org/development/tests/trunk/developer/tr1.html>, in the column "RW_WinXP_VC".

    Does anyone have any comments on these in general, or should i just raise the possible issues on the connect site?

  • [Richard Webb]

    > I've tried running the Boost regression tests on

    > VC9 with the feature pack, and they are showing up

    > a number of apparent issues with the beta TR1.

    Excellent; the more bugs that we can hammer out during the beta, the better.

    > -std::tr1::is_const<type[N]>::value always seems to be true

    > -std::tr1::remove_const<const type[N]::type always seems to be 'type const[N]'.

    N2157 ( http://tinyurl.com/2r54lo ), voted into the WP, clarifies that is_const<int[3]>::value is false, and is_const<const int[3]>::value is true, so VC's wrong there.

    It also clarifies that remove_const<const int[3]>::type is int[3], so VC's also wrong there.

    I have filed a bug about this (internal number 171837).

    > Does anyone have any comments on these in general,

    > or should i just raise the possible issues on the

    > connect site?

    If you could investigate the other issues and file Connect bugs about them, that'd be great. Please provide self-contained repros.

    Thanks,

    Stephan T. Lavavej, Visual C++ Libraries Developer

  • [Pavel Minaev]

    Hi,

    I have had the same problems with the performance of std::vector. Maybe worse because my vectors had an average element count of 5. The result was my own vector implementation. Since I have analized the problem in detail, here are some explanations.

    Look at the folowing code which is called from the consructor:

    bool _Buy(size_type _Capacity)

    {

    // allocate array with _Capacity elements

    _Myfirst = 0, _Mylast = 0, _Myend = 0;

    if (_Capacity == 0)

      return (false);

    else if (max_size() < _Capacity)

     _Xlen(); // result too long

    else

    { // nonempty array, allocate storage

     _Myfirst = this->_Alval.allocate(_Capacity);

     _Mylast = _Myfirst;

     _Myend = _Myfirst + _Capacity;

    }

    return (true);

    }

    First look at

    _Myfirst = 0, _Mylast = 0, _Myend = 0;

    Could be transormed to (on wintel platforms)

    _Myfirst =_Mylast =_Myend;

    Lets have a look at

    if (max_size() < _Capacity)

    Does anybody need that check? Me not, but even if, the result of max_size is typicaly known at compiletime but the compiler leaves a function call and some computations.

    The really funny thing is, that allocate

    makes the same checks again, before calling new. The rest of the vector implementation has similar "features".

    The implementation of vector cannot be called

    efficient for small element counts.

    If you can do without generalized allocators, have a look at yasli::vector, an implementation of Alexandrescu.

  • [Stephan]

    I've opened a number of bugs @ Connect.

    However:

    There are a number of test failures for things where the WP (N2157+) is different from the TR1 draft (N1836). For example:

    -the result of is_base_of<int, int>

    -the behaviour of is_convertible<> with void types

    So the expected results depend on which spec you're following. (see http://tinyurl.com/yw9zcb for a bit more info on the Boost tests in question).

    Can you confirm whether these issues are considered bugs in the feature pack?

    Thanks,

    Richard Webb

Page 3 of 6 (82 items) 12345»