Just What Is This TR1 Thing?

Just What Is This TR1 Thing?

Rate This
  • Comments 54

Hi there. Rob Huyett here again. I’m an SDET on the VC Libraries team. One of the things I’ve been working on lately is the new TR1 add-on for Visual Studio 2008. When VS 2008 TR1 support was announced about a month ago, I’m sure that the reaction varied from “Woo-hoo! I can’t wait!” to “Huh? What’s TR1? Do I care about this?” and all sorts of things in between.

This post is mostly aimed at that second category of folks – the ones who hadn’t heard about TR1, and who maybe haven’t found the time or inspiration to seek out what it’s all about. I’ll talk briefly here about just what TR1 is, and I’ll describe just a few of the TR1 features that I think are particularly nifty. This isn’t going to be an in-depth discussion or how-to manual, but it will hopefully inspire someone out there to take a closer look at what’s in TR1 and see if it can help you in your projects.

So what is TR1? Well, TR1 (“Technical Report 1”) is a set of proposed additions to the C++0x standard. Most of it will likely find its way into the new standard, but in the meantime it provides a useful stepping stone toward C++0x. TR1 is full of very useful new utilities such as new types of smart pointers (called “shared_ptr” and “weak_ptr”), new containers (tuples, unordered maps, unordered sets, and a neat STL-like array), reference wrappers, regular expression support, and function wrappers. You might look at that list and think that much of it sounds familiar. Smart pointers and function wrappers, for instance, already exist. This is true, but TR1’s versions try to be easier to use and more useful than the existing stuff… sort of like the next iteration based on a few years of experience finding out what works and what doesn’t work.

Alright, on to some specifics! First, I’d like to mention the new TR1 tuple and array classes. Then I’ll talk just a bit about shared_ptr. I’ll finish up with some information about regex, TR1’s regular expression utility. Again, people who are familiar with TR1 probably won’t get much out of this, but it will hopefully whet the appetite of those who are new to TR1.

Tuple is very much like the existing pair class, except that it can hold up to ten items instead of just two. Just like you can have pair<INT, char> p;, you can have something like tuple<INT, char, int, double, char*> t;. Handy, no?

TR1’s array class is very much like a fixed-length STL vector. Vector is a very useful class, and it is probably sufficient (even preferable) for most array-type needs. However, there are some situations where the developer is absolutely positive that the array needed will always be a particular size… no more, no less. In these cases, the variable-size feature of vector is not needed and just adds extra overhead. While you could just use a regular old C-style “square-bracket” array, the TR1 array class lets you use all of the STL-type iterators and algorithms. While it lacks some of the flexibility of a vector, it opens up more options than are available with a C-style array.

Shared_ptr is a very easy-to-use tool that greatly simplifies memory management. It all but makes new/delete combinations obsolete. Shared_ptr is a smart pointer class that is pretty easy to use. The syntax is fairly simple… shared_ptr<STRING> sp(new string(“foo”)); creates a shared_ptr called sp to a string containing “foo”. This shared_ptr will act almost just like any “normal” pointer, except that you don’t need to remember to delete it when you’re done. And unlike some older smart pointers, you don’t need to modify the target class (in this case, string) to include reference counting or anything like that… nearly any class you want will work with shared_ptr as-is. Speaking of reference counting, shared_ptr takes care of that (hence the “shared” in the name). If I were to make another shared_ptr that points to the same thing (like shared_ptr<STRING> sp2 = sp;), then shared_ptr’s reference counting is smart enough to only free up the memory when BOTH sp and sp2 are gone. Of course, this barely scratches the surface of what shared_ptr is all about, but it’s a start.

Regex is a class that lets you write complex regular expressions like those commonly used in Perl. While C++ has always had some amount of support for regular expressions, TR1’s regex utilities simplify things by building in the mechanics for parsing, matching, and capture groups. The regex class holds the actual regular expression, and algorithms such as regex_search(), regex_match(), and regex_replace() make it easy to apply that expression to a string. As you can probably deduce from the algorithm names, regex_search() tells the developer if the string contains any substrings that conform to the expression, regex_match() tells if the entire string conforms to the expression, and regex_replace() provides an easy way to change the string to fit a particular format. Regex can do quite a bit more than I’ve outlined here, but this should give you an idea of what regex is all about.

Well, that’s about all I wanted to say here. If any of this kindles some new interest in TR1 in any of the vcblog readers, great! Of course, any comments or questions that you might have will be appreciated.

Rob Huyett, VC Libraries Team

  • [peter]

    > I wish compiler diagnostic and debugger support for tr1 will be also on high level.

    Diagnostics: The compiler errors you'll get from misusing TR1 will be more or less equivalent to those you'll get from misusing Boost. (In the cases where compiler errors are triggered at the TR1 interface, e.g. trying to construct a shared_ptr implicitly from a raw pointer, you'll get identical diagnostics modulo namespaces. Compiler errors that are triggered within the implementation may vary. Without concepts, there is little that can be done at the library level to make template error messages less hideous-looking.)

    Debugger: VC9 TR1 will come with extensive IDE debugger visualizers for almost all TR1 types, including some new visualizers for STL types that should be helpful for TR1 users (e.g. plus<T>() will be visualized as "plus", to make bound functors easier to look at).

    > I don't want to land in the middle of some macro expansion during debugging some tr1:function/bind code.

    > Does the tr1 in vc9 use the same preprocessor tricks as boost does?

    Without looking at the Boost implementation, I can't perform a direct comparison (you can, when the VC9 TR1 beta is released very soon). I would characterize VC9 TR1 as using "some" but not "a whole lot" of macros. Most of TR1's implementation is similar to the STL macro-wise. The exception to this is those parts of TR1 that have to simulate variadic templates; this requires lots of preprocessor machinery (the only thing worse than lots of macros here would be no macros). In particular, tuple, bind, and the other functional stuff are powered by non-idempotent headers and the like.

    [Andre]

    > Coupling factor - meaning if I want to store a templated type,

    > which has a template parameter, in a class as member, I have

    > to make the class containing this type templated too.

    Only if you want to propagate the generality to the containing class. stack<T> contains a deque<T> (by default), but you can also have an Image containing a vector<unsigned char>.

    This particular case is special because TR1 leaves some types unspecified. Usually, you can look at a function (even a function template) and easily figure out what it's going to return.

    > Don't think so. For Intellisense to function constraints are needed.

    Okay - let's call it a consequence of how Intellisense and templates interact (in the absence of concepts).

    > Or can you give me an illustrative example - for the code below.

    decltype allows you to say "give me the type of this expression", where the expression can be something like a + b, or a function call. This is something that C++03 can't do, and it's exactly what you're wanting to do here.

    I don't have a decltype-supporting compiler, but you can read the decltype papers for more details.

    auto allows you to say something similar, "give this thing the same type as its initializer".

    > By replacing boost::function with auto or decltype and without converting the class CallbackHolder to a templated one.

    That's asking for type forgetting without any overhead, which can't be done.  Functors in C++ can contain arbitrary state. Even pointers to member functions have to be invoked differently depending on whether the inheritance is virtual or not, etc.

    According to my extremely limited understanding of managed delegates, they work by restricting themselves to binding only "pointers to member functions" (even if that's not what they call them) to "pointers to objects" (also even if that's not what they call them), and they've restricted inheritance so that you can call everything in a uniform way. That way, branches can be avoided. I might be completely wrong about that.

    C++ works in a different way - it allows great diversity in functors (pointers to member functions with no inheritance, single inheritance, virtual inheritance, or pointers to data members, or pointers to functions, or full-fledged function objects, stateless or stateful), and relys on inlining being done through templates to produce efficiency equivalent to handwritten C. If you demand that something be a non-template, yet cope with so many different functors, you're going to introduce overheads.

    > What would be a proper comparison ? I want to use the style boost::function allows me to use.

    boost::function is extremely simple to use, and permits separate compilation in cases where it was previously difficult or impossible. It does, however, involve small overheads.  Here, you are looking at the overheads to the exclusion of everything else, which is why it appears to be a big deal.

    By "handwritten binder", I mean a functor that is templated on class type, parameter types, and return type, which stores a pointer to member function, an object, and an argument (you can make the object free, or the argument free, or neither for full binding). You can't use this for type forgetting - the bound functor type depends on the class type - but it also doesn't propagate templateness (something storing a bound functor need not be a template).

    Stephan T. Lavavej

    Visual C++ Libraries Developer

  • [(in the absence of concepts)]

    O.k. C++ names them concepts. Sorry get always confused with C++/CLI, C# and C++.

    [decltype allows you to say ...]

    Well I want to express, store a member function pointer at this location to a function with the signature X.

    [binding only "pointers to member functions"]

    I've not discovered any restrictions of delegates yet. But anyways you are correct, managed code doesn't support multiple inheritance of classes. Only of abstract interfaces.

    So they don't have the problems, introduced by C++ supporting multiple inheritance.

    There are other proprietary delegate extensions of other compiler vendors, which also restrict multiple inheritance, to support delegates. VC also has extensions, though IMHO not directly comparable.

    [...you're going to introduce overheads.]

    Yes, at least the delegate would have to be large enough to store the largest member function pointer, which is larger as the most simple member function pointer: this + codeptr.

    [of everything else, which is why it appears to be a big deal]

    Yes, agreed, it's not that simple for a (C++) library - it would be rather hard to implement this for all possible cases.

    And yet there are some implementations, which come quite near to an ideal C++ delegate implementation, which supports the syntax of boost::function, avoiding it's runtime overhead.

    But they have to use dirty hacks, where a little compiler (C++ core) support could help. E.g. Don C. has written such an implementation and article on the codeproject web site - fast delegate.

    Generally std::function will be sufficient for me most times. But, this is where the discussion started, I think with a little standard/compiler help it could be implemented more efficient.

    Andre

  • Stephan T. Lavavej [MSFT],

    Sorry for the late response.

    I was on vacation for new years.

    >> So, what exactly is the Microsoft VC

    >> Libraries team doing?

    > I'll expand on this with a VCBlog post soon,

    > but here's a summary:

    > 1. We're integrating TR1 into VC9, so that TR1's headers live next to the STL's headers, and TR1's separately compiled components go into msvcp90[d].dll next to the STL's separately compiled components. You won't need to do anything special to access TR1 features, and you won't need to ship a whole new DLL with your programs. You'll simply need to update the version of the CRT that's distributed with your program, just like for a VC service pack (you can think of VC9 TR1 as SP0).

    As a developer, managing headers and libraries is one of the basic skills needed in our profession. It is not a difficult task. Hopefully you can accept that I can just as easily do that myself. Boost is just one major example.

    > 2. We're making TR1 play nice with /clr and /clr:pure.

    /clr is not C++ (ISO/IEC 14882:2003). Nor is it part of the forthcoming C++0x, which TR1 is being targeted for.

    While I have given try implementations of C++/CLR a chance, customers continually ask me to do managed development in C#, and mixed interop is essentially non-existent (usually for reasons of portability).

    > 3. We're ensuring that TR1 compiles warning-free at /W4, in all supported scenarios. This includes switches like /clr, /clr:pure, /Za, /Gz, and the like.

    At least in regard to standard code (see above), that sounds like Dinkumware's job. I seriously hope that warnings are not merely being pragma-ed off.

    4. We're ensuring that TR1 is /analyze-clean.

    Sounds like Dinkumware's job.

    > 5. We're identifying bugs in TR1 and working with Dinkumware to fix them. Dinkumware's code was very solid to begin with - but as ever, more eyes find more bugs. I've even found a couple of bugs in the TR1 spec itself.

    Good, but again that sounds like Dinkumware's job.

    > 7. We're identifying select C++0x features to backport into TR1 - for example, allocator support for shared_ptr and function. While not in TR1, this is important to many customers (including our own compiler).

    As C++0x does not exist yet, I hope that you are limiting yourself to the currently accepted subset. However, key features such as export have existed in the standard for almost a decade now. Please implement the existing standards first.

    >> If I really needed TR1, I could always

    >> license it from Dinkumware myself.

    > That would cost you money and time.

    Yes, that is how business works.

    However, while I *can* license Dinkumware's implementation as needed, no matter how many years I plead and beg for standard compliance, I can *not* get Microsoft to implement the existing C++ standard. The biggest issues for me are 1) export, 2) two-phase name lookup, and 3) exception specifications.

    >> Or I even code it myself.

    > Are you a world expert at library

    > programming?

    You make it sound so difficult to build C++ libraries... I am sufficiently confident in my skills to implement required types as necessary for jobs.

    I have used VC++ for many, many years now. However, as more and more customers are beginning to use standard C++ features such as export (etc), increasingly I can no longer compiler existing code-bases with non-conformant Microsoft compilers. Perhaps we do not need some of these features, but people *are* using them (for whatever reasons), and they are part of the existing standard. Often when I try to get around this with #defines I am opposed by senior developers who point me to the ISO/IEC 14882:2003 standard and tell me to replace the compiler if it is incompatible with the standard.

    I really, really want to continue using VC++. I beg and plead with you for years, but while you do not deny that it is part of the existing standard, you ignore it and implement future standards. You make it so incredibly difficult to support your products.

  • """The biggest issues for me are 1) export, 2) two-phase name lookup, and 3) exception specifications."""

    This is starting to sound like kid's quarrel[1], but i would change the order to: 1) two-phase name lookup, 2) exception specifications and 3) export. With export, one can always say: "so don't use it". Not so with two-phase name lookup (unless you avoid templates completely, it's hard to guess when it will happen, or when you forgot the keyword "typname", specially if you never use another compiler).

    """

    >> If I really needed TR1, I could always

    >> license it from Dinkumware myself.

    > That would cost you money and time.

    Yes, that is how business works.

    """

    I appreciate the work that's being done for TR1, because it's the kind of library that should be included with compilers. If i were going to buy a library separately, i would forget TR1 and go directly to Qt, for example.

    [1] Sorry, English is not my native language, but i think you'll get the point

  • [Stephen]

    >> 1. We're integrating TR1 into VC9

    > Hopefully you can accept that I can just as easily do that myself.

    You misunderstand - adding TR1 to the Visual Studio product, right next to the Standard Library, involves more work than a standalone library (like Boost). There's the Visual Studio build system to contend with (it took us a little while to get the TR1 separately compiled components exported from msvcp90[d].dll), and then the setup system (getting the new headers and sources picked up by the installer), etc. Unglamorous work, but work nevertheless.

    >> 2. We're making TR1 play nice with /clr and /clr:pure.

    > /clr is not C++ (ISO/IEC 14882:2003). Nor is it part of the forthcoming C++0x, which TR1 is being targeted for.

    Certainly. I am personally uninterested in /clr[:pure]; however, it is a VC feature, so TR1 must support it. And this took an unbelievable amount of work.

    >> 3. We're ensuring that TR1 compiles warning-free at /W4, in all supported scenarios. This includes switches like /clr, /clr:pure, /Za, /Gz, and the like.

    >> 4. We're ensuring that TR1 is /analyze-clean.

    > At least in regard to standard code (see above), that sounds like Dinkumware's job.

    TR1 arrived mostly /W4-clean. However, Dinkumware hadn't yet thrown (to my knowledge) exotic options like /Za and especially /clr[:pure] at TR1. Our test matrix identified these warnings so Dinkumware could fix them. Similarly, /analyze exposed several warnings.

    > I seriously hope that warnings are not merely being pragma-ed off.

    Generally, no - we preferred true fixes to workarounds to pragmas, in that order. And as usual, we disable warnings only in the TR1 headers, not in user code.

    >> 5. We're identifying bugs in TR1 and working with Dinkumware to fix them.

    > Good, but again that sounds like Dinkumware's job.

    As I said, more eyes find more bugs. It's our job to find bugs, and Dinkumware's job to fix them. (Independently, they also find and fix bugs.)

    >> 7. We're identifying select C++0x features to backport into TR1

    > As C++0x does not exist yet, I hope that you are limiting yourself to the currently accepted subset.

    Anything that has been voted into the Working Paper is almost certainly going to make it into the C++0x standard.

    > You make it sound so difficult to build C++ libraries... I am sufficiently

    > confident in my skills to implement required types as necessary for jobs.

    The more generic the library, the more skill it takes to implement. Application and OS developers aren't library developers, and they don't have the same skills. TR1 is an extremely generic library, and extremely difficult to implement.

    (Within MS, I've seen a half-dozen smart pointer implementations, all deficient in one way or another. My hope is that shared_ptr will sweep them all away.)

    [ikk]

    > Not so with two-phase name lookup

    It's actually "pretty hard" to trigger two-phase name lookup (such that it'll make a difference), if you follow a certain style of code organization. Yes, that's rather vague.

    > or when you forgot the keyword "typname"

    Requiring "typename" is completely unrelated to two-phase name lookup, and VC implements the "typename" rules pretty well. I'm sure that there are bugs that aren't coming to my mind right now, but VC definitely enforces this rule in most situations where it should be enforced.

    Perhaps you're thinking of unqualified name lookup reaching into dependent base classes (it shouldn't, says the Standard) - this rule is related to two-phase name lookup, although not actually part of it. VC actually enforces this rule under /Za.

    Stephan T. Lavavej

    Visual C++ Libraries Developer

  • ikk,

    > With export, one can always say: "so don't use it".

    Only if one owns all of the code themselves. However, when I work with various customer codebases, increasingly they are already--for whatever reasons--using export. Needless to say, this will not compile with any existing VC++ compiler. When I try to either 1) rewrite it without using export or 2) make a special VC++ version with #ifdef, I am often opposed by other developers who point me to the ISO/IEC 14882:2003 standard and tell me to replace the compiler if it is incompatible with the standard rather than replace good code. And that is precisely what I have had to do. More and more I am being forced to use Comeau. Comeau is a great compiler, but I would like to continue using VC++. However, it is less and less possible.

    "so don't use it" is often not an option.

  • '''Requiring "typename" is completely unrelated to two-phase name lookup'''

    '''Perhaps you're thinking of unqualified name lookup reaching into dependent base classes'''

    (snip)

    '''VC actually enforces this rule under /Za'''

    Sorry, i wasn't sure and mixed a few concepts (i wasn't very clear either, mainly because i didn't feel the need to be very specific).

    But i did think about "unqualified name lookup reaching into dependent base classes" too.

    Last time i tried /Za (a long time ago, under VC++.NET 2002), i triggered a documented bug and never tried /Za again.

    If /Za is working OK (and if standard headers compile cleanly under /Za), i will give it a try again. Thanks for the info. :-)

  • [ikk]

    > If /Za is working OK (and if standard headers compile

    > cleanly under /Za), i will give it a try again.

    The Standard and TR1 headers should compile cleanly under /Za. If they don't, that's a bug. (Warnings and compiler errors occasionally creep in - the unqualified-name-lookup-reaching-into-dependent-base-classes thing is really easy to forget - but we now have pretty good test coverage.)

    However, /Za is an obscure option. It doesn't get a whole lot of testing (on both the compiler and library sides), doesn't do a whole lot of stuff, and isn't really being actively developed. The front-end devs I know recommend against it, as it could do more harm than good.

    For portable code, I simply suggest using multiple compilers regularly. Then you'll get the union of their conformance checks (of course, you'll also have to deal with the union of their bugs).

    Stephan T. Lavavej

    Visual C++ Libraries Developer

  • TR1 is a set of additions to the standard library of C++9x (not 0x as the article states) and should become part of C++0x (as the article states correctly).

    TR1 is important as it brings new library features to C++ that don't require a language (and therefor a compiler) change. Thanks and kudos to Microsoft for being quick to bring TR1 to Visual C++.

    However, I should like to add my voice to those clamouring for support for export. The fact that export offers an opportunity to increase separation between the definition and the declaration of code templates means that it is a valuable tool in enabling separation between compilation units and so in managing build times. Even if all the other benefits offered by export turn out to be illusory this one is worth every bit of the development effort that its implementation might require (a couple of lousy man-years is nothing to argue about, really). Yes, Daveed Vandevoorde's proposal for modules offers more, but we will have some years to wait before that is standardized, let alone available in the compilers on our desktop.

    Exception specifications are another matter. The C++ syntax is such that checking of exception specifications at compile time is not possible (a pointer-to-function doesn't have an exception specification, for example, so a call through a pointer cannot be checked), and automatically checking at runtime violates the C++ convention that you should not pay for what you do not use. Exception specifications are essentially useless in C++ and should be removed from the standard.

  • > I'm still not convinced that a core implementation of a delegate type *and* the library extension in combination couldn't be more efficient.

    "Delegates" (lambdas) will likely be in C++09, and from the looks of them they are going to be just as efficient as .NET delegates.

  • Stephen and others - we really do not need "export". Please reade complete N1426 to understand why. If I had a choice, I would vote to remove it entirely from the standard. Not only does not buy us anything, but also comes with difficult to understand problems related to names visibility and makes implementation of other C++ features (present or future) needlessly expensive. Yes, it's good to have another tool to improve code separation/decoupling, IF that tool works.

  • If I install this beta, will I be able to uninstall it / update it to final without problem??

    PS: I agree with those that say that improving standards conformance -- particularly for straight C -- is more important to me than other new features.  The workarounds in boost give a good indication of how far VC++ is away from standard C++.  I am not particularly interested in .NET features.

  • [jrp]

    > If I install this beta, will I be able to uninstall

    > it / update it to final without problem??

    I am told that uninstallation works, which will get you back to VC9 RTM so that you can apply the final patch. I would suggest NOT trying to apply the final patch over the beta patch.

    Stephan T. Lavavej

    Visual C++ Libraries Developer

  • -----------------------

    [Stephan]

    > Are you a world expert at library programming?  (If so, we're hiring!)  We've *got* some of the world experts at library programming, namely Dinkumware, working on TR1 - and it's not easy.  Take a look at TR1 mem_fn()/bind()/function's implementation when we release the TR1 beta, and then tell me that you could write that yourself.  :-)

    >

    > I consider myself lucky to be able to *understand* what most of the TR1 implementation does -...

    -----------------------

    What an arrogance !!

    A good indication why VC is not progressing further and letting the C# and all its peers take precedence.

    Well, what if we ask the same thing? "if the world experts, namely the so called Dinkumware, are doing better things that you cannot barely even "understand" - what is Microsoft Doing? Why are you with Microsoft? Why do not you join the "World experts"??

    Don't try to underestimate your readers, and in turn pull-down your own company. Remember that your company, Micrsoft, has its foundations built on top of those very people you are criticizing with questions such as "Are you a world expert at library programming"....

    If we are not world experts at library programming, you are not either (because this is not "your" library) !! Let us all praise the Dinkumware !!

     So much for an arrogant. Yuck.

    Hello managers - what are you doing when one of your reports is criticizing your customers and bringing down your company values in public?? May be its a time you either give him a break or train him to be respectful. Seems he has forgotten the very core values of being open and respectful.

  • Reading this list, I do wonder what MS' priorities are.

    While it is nice to have a performant IDE, it would be good to deepen standards conformance (for C as well as C++). For example, libsndfile does not compile with VC++.

    It would also be good to update OMP support and improve vectorisation (with the advent of multi-core cpus).  We're now on OMP 2.5 and 3.0, whereas VC++ is only 2.0.

    More pertinently to this thread, the use of parallelism in the standard library would be most welcome.  See what the gcc guys are up to.

    Unless the strategy is to leave the parallel field to Intel?

Page 3 of 4 (54 items) 1234