Hi, I'm Stephan, the Visual C++ Libraries Developer working on TR1.
Recently, I gave 3 presentations within Microsoft about the most novel components of TR1: shared_ptr, regex, and the additions to <functional> (including mem_fn(), bind(), and tr1::function). These presentations explained where to use, how to use, and how to avoid misusing these components, with plenty of examples.
Attached at the end of this post is a ZIP of the slide decks for each of the presentations, in PowerPoint 2007 format.
Each slide deck mostly stands alone, although the "shared_ptr To Noncopyable" and "shared_ptr To Polymorphic" examples require some context and additional explanation:
The "shared_ptr To Noncopyable" example demonstrates how to create a container of noncopyable resource managers. A resource is something that must be acquired from the system and then released afterwards. The most common resource is memory (acquired in C++ with new, and released with delete), but there are many non-memory resources, such as files, sockets, textures, fonts, and so forth. A resource manager is an object that encapsulates a resource, controlling its lifetime and permitting access to it. Resource managers acquire resources in their constructors, and release resources in their destructors (this is the somewhat-mysteriously-named RAII principle: Resource Acquisition Is Initialization, implying RRID: Resource Release Is Destruction). std::vector is an example of a resource manager for memory, while std::ifstream is an example of a resource manager for files. Memory resource managers are usually copyable (and assignable); chunks of memory can easily be copied and assigned. But non-memory resource managers are usually noncopyable (and nonassignable); either their underlying resources can't be copied without additional information (e.g. copying a file requires a new name), or copying them is a relatively expensive operation (e.g. textures consume precious video card memory, etc.). So, it's best to prohibit copy construction and copy assignment of non-memory resource managers. This requires programmers to explicitly copy the underlying resources when that's desired.
The problem here is that STL (and TR1) containers require ordinary value types. An ordinary value type is "something that behaves like an int"; ints are copyable and assignable. vectors copy their contained elements when undergoing reallocation, and so forth. You can't have a vector<ifstream> because ifstream is noncopyable. But sometimes, that's exactly what you want.
In this example (admittedly somewhat contrived), I'm writing a program to interleave text files. The user will have an arbitrary number of text files, and will provide the names of the files to the program. Then, the program will have to loop through the files in order, printing out the first line of each file, then the second line, and so forth. When a text file runs out of lines (they can have different numbers of lines), the program has to start skipping that file. When all the lines from all the files have been printed, the program has to stop.
Now, I could write this by reading each file into a vector<string> (storing each line), and building up a vector<vector<string> > (storing each file), and then looping through that. But that would require reading everything into memory. Suppose I don't want to do that.
Well, if I could use a vector<ifstream>, I could repeatedly loop through the vector, reading one line from each file in turn. When a file ran out of lines, I could remove it from the vector. But ifstream is noncopyable.
To solve this, I can use shared_ptr. shared_ptr can be used as a handle to a noncopyable object; the shared_ptr itself is copyable and assignable.
Slide 1/3 of the example shows the contents of 3 text files.
Slide 2/3 shows the code. To fit the code onto a single slide, I use a queue<shared_ptr<ifstream> >. I begin by reading filenames from cin (with the for-loop). From each filename, I new up an ifstream, held by shared_ptr, and push it into the queue. At the end of this for-loop, the queue contains shared_ptrs to the files mentioned by the user in order.
Then, I loop through the queue (with the while-loop) as long as it contains files to process. I attempt to read a line from the file at the front of the queue (this is the next file to process). If that's successful (the file still contains lines), I print it out. I want to keep working on that file, so I push() a copy of the shared_ptr to this file (q.front()), to the back of the queue. Then I pop() the front of the queue. (At this point, particularly attentive readers should ask whether q.push(q.front()) raises the specter of iterator invalidation; the answer is that it does not. I would have avoided this in the interests of clarity, but it would have taken more lines.) If I wasn't able to read a line from the file, then it's empty, so I simply want to pop it from the front of the queue. Then I continue with the loop.
Slide 3/3 shows the filenames being entered, and the program's output.
The "shared_ptr To Polymorphic" example demonstrates how to create a container of polymorphic objects (i.e. classes with virtual functions). Polymorphic objects should always be noncopyable, because of the danger of slicing. Inheritance means substitutability; a Derived object has to be usable wherever a Base object would be usable. However, copying presents a problem. If Base is copyable, then you can copy from a Base to a Base. But substitutability then implies that you can copy from a Derived (which is a Base) to a Base. This "slices" off any Derived behavior and state, which is almost always undesirable. Therefore, copy construction and copy assignment of polymorphic base classes should be prohibited. (You should provide a virtual clone() function if you need to copy polymorphic objects safely.)
If Base is noncopyable, you can't have a vector<Base>. You could try to use a vector<Base *>, newing up Derived objects and storing pointers to them, but that would be highly likely to leak, especially in the presence of exceptions. As with resource managers, shared_ptr allows us to manipulate noncopyable objects with copyable handles. vector<shared_ptr<Base> > is a powerful construct that can be used in many situations.
In this example (even more contrived; you can mentally substitute your favorite Base and Derived classes), I have a polymorphic base class Animal. Therefore, it has a virtual destructor, no copy constructor, and no copy assignment operator. It stores a string m_name (the name of the Animal), and has an explicit constructor taking that name. Animal uses the Non-Virtual Inheritance idiom, so it has a public, non-virtual member function named noise(), which returns "<name> says <something>". The "something" is customizable by derived classes, through the private pure virtual member function noise_impl(). (Yes, private virtual member functions can be overridden; the difference is that derived classes cannot call the base implementation, whereas they could if the base implementation were protected. As noise_impl() is pure, there is no base implementation, so the difference is theoretical - but I prefer to lock down access control unless I need looser control.)
Slide 2/4 shows three derived classes, Cat, Dog, and Pig, identical in structure except for what their noise_impl()s return.
Slide 3/4 constructs a vector<shared_ptr<Animal> >, which is then filled with a Cat, Dog, and Pig. Then I print out the noises that each Animal makes. I use the STL algorithm transform() on the vector, printing strings to cout separated by newlines. mem_fn(&Animal::noise) uses tr1::mem_fn() to adapt a pointer-to-member-function (&Animal::noise) to a function-object (what transform() wants).
Slide 4/4 shows the output, demonstrating that the virtual function calls worked as usual.
The point of these two examples is that shared_ptr is useful for much more than sharing and exception safety. shared_ptr allows you to use the STL in more situations, by wrapping noncopyable objects in copyable clothing.
If you have any questions, I'll be happy to answer them in the Comments!
Stephan T. Lavavej
> You're *the* only developer working on TR1?
Within Microsoft, yes.
* We licensed Dinkumware's TR1 implementation, and (as I explained in a previous VCBlog post) they're responsible for implementing the majority of fixes to it. I'm acting as a hybrid developer/reviewer/whitebox tester.
* My manager, Ale Contenti (VC Libraries and IDE dev lead) also reviews Dinkumware's and my checkins, but I'm the "boots on the ground", so to speak.
* The TR1 "feature crew" includes non-developers: tester Rob Huyett, program manager Ayman Shoukry, documentation writer Jim Vance, etc.
So, I'm not locked in a closet somewhere, slaving away over TR1 by myself.
> By the way, I know this is Microsoft, but making
> the attachments available as PDF would be nice; I
> don't have Office 2007 or the Office viewers
The PowerPoint Viewer 2007 is freely available (http://www.microsoft.com/downloads/details.aspx?familyid=048dc840-14e1-467d-8dca-19d2a8fd7485&displaylang=en ). (I haven't used it, as I have Office 2007 at work and home, but apparently it works quite well).
As for the lack of PDF - I simply hadn't installed the Save As PDF add-in. If you really want PDF versions, E-mail me (email@example.com) and I'll get the add-in and create PDFs for you. (I don't know if VCBlog attachments can be updated - there's definitely a one-attachment-per-post limitation, so I'd rather not struggle with that.)
Stephan T. Lavavej, Visual C++ Libraries Developer
Oh, and I almost forgot - other VC Libraries devs are working on the Feature Pack packaging, making sure that it installs on all SKUs, uninstalls cleanly, etc.
> Implementing export is very difficult and a waste of time (as the only team that has ever done it admits).
Difficult: apparently. However, it should not take a decade or more to implement. If Microsoft developers are not up to the task, then perhaps software development is not the right industry for them.
Waste: perhaps; nevertheless, people are using it. I often think that about roughly 10-15% of the features in the C++ standard are a waste. But I do not argue for them to be removed after the fact.
>Considering that only EDG-based compilers support it, it is naive to try and use export and then expect to compile on any other compiler.
Your logic and argument is naive. The standard exists solely to ensure interoperability between various compilers. Export IS part of the C++ standard and has been for a DECADE now. There were attempts to remove it and the standards committee rejected them.
I am in a similar situation as the earlier poster: a number of my clients are using export which forces me to drop VC++ even though it is my favorite compiler. Thus, with the exception of my own personal projects, I doubt that I will be able to take advantage of the work being done on TR1, at least Microsoft's implementation of it.
Stephan> Interesting presentations; I was able to install the viewer.
Mainer> Implementing export took 1.5 years of design + 2 man years of coding (according to EDG). Here's the advice EDG gives about implementing export: "Don't.".
The main reason it was kept in the standard is the considerable amount of time EDG spent implementing it; it would have been unfair to remove it.
> 1.5 years of design
A lot of that time was spent seeking clarification on initially unspecified issues with interactions with subsystems. Those issues have since been resolved and need not be repeated.
> 2 man years of coding
That sounds reasonable. There is a new release of VC++ roughly every 2-4 years (1998, 2002, 2003, 2005, 2007).
Even assuming the worst four years and two people, MS should have been able to implement export at least twice in the last decade. Also note that Microsoft has far more resources available to their disposal than EDG, and the design time should not be as long since the above issues have been clarified.
> The main reason it was kept in the standard is the considerable amount of time EDG spent implementing it; it would have been unfair to remove it.
Do you have a resource for that statement? If EDG opposed ("Don't") export, then fairness is hardly very important.
Here are the actual minutes:
There was more than three times the support for keeping export than removing it.
"Remove export: 8"
"Leave in export: 28"
Personally, I neither like nor dislike export; I'd say that I am neutral. However, I have customer code that uses it and need to compile it as is. That pretty much eliminates VC++ as an option.
Wow, look at what i have found (the link points to a google groups search):
"Can't afford export" was debunked! :-O
Too bad i hadn't found this earlier. I that thread, Daveed Vandevoorde states that the claim that export is underspecified is "a serious exageration."
Until today, I thought that the article "Why we can't afford export" was a good reference about the problems with export. Now i am not sure anymore...
Besides Comeau, is there any other compiler with export? (other EDG-based compilers, i mean... becaus non-EDG compilers certainly don't support export)
> Daveed Vandevoorde states that the claim that
> export is underspecified is "a serious exageration."
Vandevoorde is an expert - but the experts disagree. I don't consider Sutter's paper to have been "refuted".
Most important are two words: opportunity cost.
This 'export' freak really needs to go and get some life out there, or buy EDG and be over it.
Frankly, whoever is your manager, and all those .NET PM managers are to blame as much, they should be taken out and beaten by an oakwood stick.
Putting one man to do TR1 just shows how far behind Linux and Google infrastructure MS will be once it gains adoption one of these years.
Someone over there needs a good GC and LINQ invention slapping and some proper Bjarne lectures in metaprogramming and better than half-decent support for value-type generics.
Oh, I forget, they are all mass market architects.. Delphi delegates bloatware style.
A colleague pointed me to this thread. I was a long-time user of Visual C++. It's a very productive environment. Unfortunately, I am unable to use it anymore. Like some of the people above me, several of my long-term clients use export and I have had to abandon VC++ for more compliant compilers. I respect that Microsoft may have other priorities beside the standard; however, it sure would be nice to do my daily work again someday with VC++.
"""I have had to abandon VC++ for more compliant compilers."""
Which ones? Last time i checked it out, only Comeau supported it. Maybe the Intel compiler also supports export since it is EDG-based. Is there anything i am missing?
[Cannot believe it..]
> Putting one man to do TR1
Does "20% of the VC Libraries developers" sound more impressive?
> just shows how far behind Linux and Google
> infrastructure MS will be once it gains
> adoption one of these years.
Linux is a compiler? Google ships a compiler?
GCC, on the other hand, is a compiler. 4.2 has partial TR1 support, but you need the not-yet-released 4.3 for <regex>.
> Is there anything i am missing?
GCC doesn't support export (and although I don't follow their development closely like when I was in college, they don't appear to be moving towards it at all).
Code using export may be theoretically-portable, but it is not actually-portable to the Big Dog compilers, VC and GCC. (Market share numbers would be fascinating - but my understanding is that VC is the most widely used Windows-targeting compiler, just as GCC is the most widely used Unix-targeting compiler.)
STL, I'm glad to see you've landed on the VC++ team. Keep up the good fight.
On export, I can't imagine they're anything but trolls. Comeau and recent versions of C++ Builder support it, sure, but lets face it -- the interaction of ADL and export presents profound difficulties and (probably) demands that exported templates contain almost as much information as the source.
Just change your name to Eric Xavier Paul Othello Robert Trudeau and ship the new compiler from your new e-mail address.
> the interaction of ADL and export presents profound difficulties and (probably) demands that exported templates contain almost as much information as the source.
Fear mongering aside, all of the above is entirely immaterial. As per the decade old standard, the compiler just needs to compile standard code. VC++ does not do that with respect to export.
Plz someone supply slides in pdf format or .ppt format. I don't have windows sry for bugging you in an msdn blog, but as m$ claims for interoperability I will ask for those pdfs too, they sound interesting :)