Inside the C++/CX Design

Inside the C++/CX Design

Rate This
  • Comments 49

Jim SpringfieldHello. This is Jim Springfield, an architect on the Visual C++ team.

Today, I want to give some insight into the new language extensions, officially called C++/CX, which was designed to support the new API model in Windows 8. If you attended //BUILD/, watched some of the sessions online, or have been playing with the prerelease of Visual Studio, you probably have seen some of the “new” syntax. For anyone who is familiar with C++/CLI (i.e. the language extensions we provide for targeting the CLR), the syntax shouldn’t seem much different.

Please note, however, that while the C++/CX syntax is very similar to C++/CLI, the underlying implementation is very different, it does not use the CLR or a garbage collector, and it generates completely native code (x64, x64, ARM depending on target).

Early on in the design of our support for Windows 8, we looked at many different ideas including a pure library approach as well as various ways to integrate support in the language. We have a long history of supporting COM in the Visual C++ team. From MFC to ATL to #import to attributed ATL. We also have a good bit of experience at targeting the CLR including the original managed extensions, C++/CLI, and the IJW support for compiling native code to MSIL. Our design team consisted of seven people and included people who worked on these and who have lots of experience in libraries, compiler implementation, and language design.

We actually did develop a new C++ template library for Windows 8 called WRL (Windows Runtime Library) that does support targeting Windows 8 without language extensions. WRL is quite good and it can be illuminating to take a look at it and see how all of the low-level details are implemented. It is used internally by many Windows teams, although it does suffer from many of same problems that ATL does in its support of classic COM.

  1. Authoring of components is still very difficult. You have to know a lot of the low-level rules about interfaces.
  2. You need a separate tool (MIDL) to author interfaces/types.
  3. There is no way to automatically map interfaces from low-level to a higher level (modern) form that throws exceptions and has real return values.
  4. There is no unification of authoring and consumption patterns.

With some of the new concepts in the Windows Runtime, these drawbacks become even more difficult than in classic COM/ATL. Interface inheritance isn’t vtable-based like it is in classic COM. Class inheritance is based on a mechanism similar to aggregation but with some differences including support for private and protected interfaces. We quickly realized that although there is a need for a low-level tool like WRL, for the vast majority of uses, it is just too hard to use and we could do a lot better while still preserving performance and providing a lot of control.

The #import feature that was available in VC6 provides a good mechanism for consuming COM objects that have a type library. We thought about providing something similar for the Windows Runtime (which uses a new .winmd file), but while that could provide a good consumption experience, it does nothing for authoring. Given that Windows is moving to a model where many things are asynchronous, authoring of callbacks is very important and there aren’t many consumption scenarios that wouldn’t include at least some authoring. Also, authoring is very important for writing UI applications as each page and user-defined control is a class derived from an existing Runtime class.

The design team spent a lot of time discussing what consumption of Windows Runtime components should look like. We decided early on that we should expose classes and interfaces at a higher level than what the ABI defines. Supporting modern C++ features such as exceptions was deemed to be important as well as mapping the Runtime definition of inheritance (both for interfaces and classes) to C++ in such a way that it was natural. It quickly became clear that we would need some new type categories to represent these as we couldn’t change what the existing C++ ABI meant. We went through a lot of different names and it wasn’t until we decided to use the ^ that we also decided to use ref class to indicate the authoring of a Windows Runtime class.

We also spent a lot of time exploring various approaches to how to hold onto a pointer to a WinRT class or interface. Part of this decision was also how to tell the difference between a low-level version of an interface and the high-level version of the interface. We had a lot of different proposals including just using a *, using * with a modifier, and using various other characters such as the ‘@’ symbol. In the original extensions we did for managed code, we in fact did use a * with a modifier (__gc). We realized we would have many of the same problems if we followed that route. Some of the breakthroughs came when we started thinking about what the type of a pointer dereference would be. This made us realize that what we were doing was similar to what we did when C++/CLI was designed. At one point, someone suggested “Why don’t we just use the ^ symbol?” After the laughter died down, it started making a lot of sense. On design point after design point, we often came to the same design decision we had made for C++/CLI.

Many of the concepts we were trying to express were already present in the C++/CLI syntax. Given that reference counting is a form of garbage collection, using ^ to represent a “refcounted” pointer in ZW fits quite well. Dereferencing of a ^ yields a %, also like C++/CLI. While many concepts are expressed the same way, there are a few areas where we decided to deviate from C++/CLI. For example, in C++/CX, the default interface on a class is specified through an attribute in the interface list while in C++/CLI it is an attribute on the class itself.

In C++/CX we have a much better story than C++/CLI when it comes to interoperating references types with regular types. In C++/CLI, managed objects can move around in memory as the garbage collector runs. This means you can’t get the real address of a member (without pinning) or even embed anything except primitive types (i.e. int) into your class. You also cannot put a ^ into a native class or struct. In C++/CX, objects do not move around in memory and thus all of these restrictions are gone. You can put any type into a ref class and you can put a ^ anywhere. This model is much friendlier to normal C++ types and gives the programmer more flexibility in C++/CX.

We will be providing more insight into our design over the coming months. If there are specific things you would like to know more about, please let us know.

  • @Colcord

    You do have to specify /ZW in order for these extensions to be enabled.

    @PleaseFixYourBugs

    All you did was essentially copy the code I had already shown we emit. You weren't able to properly handle the meaning of the "requires" relationship, you punted on the mapping of exceptions to HRESULTs, and you ignored metadata.  If you don't care about exceptions/HRESULT mapping and you want to write more standard c++ code, then use WRL.  That is what it is for.  If you want to write it in more complex ways, you can.  Don't use any libraries or write your own.  For the vast majority of people that want to write simple code that maps very closely to the C++ object model, C++/Cx should work very well for them.  

    We provide three options here:

    1) Write it in plain C++ using your own mechanisms and use MIDL.

    2) Write it in standard C++ using WRL and MIDL.

    3) Write it in C++/Cx with a pretty simple (but nonstandard) syntax.

    We believe these three options provide a good balance.  Are there other things we could have done?  Yes, some of which we have already tried in the pase with COM/TLB.  We believe that providing integrated support in the language provides a lot of benefit for our customers.

  • @Jim Springfield:  "You do have to specify /ZW in order for these extensions to be enabled."

    Thanks, but that doesn't quite address the question I asked; STL already indicated that /ZW enables the extensions.  What I was asking was whether VS2011 would turn /ZW on by default in the project settings.  If it does, then it's out of compliance with the ISO C++ requirement that STL cited.

  • @Colcord

    Sorry I misunderstood.  The IDE will only turn it on automatically when you create specific project types that need it(i.e. Metro apps).

    BTW - Someone here pointed out I used some incorrect syntax in my example.  It should have been "interface class", not "ref interface".  My apologies.

  • @Jim Springfield:

    "All you did was essentially copy the code I had already shown we emit."

    Not so. Look into the code.

    "You weren't able to properly handle the meaning of the "requires" relationship, ..."

    Not so. Look into the code and stop shooting from the hip.

    "... you punted on the mapping of exceptions to HRESULTs, ..."

    Punted? I said this is a controversial feature. I suggested we talk about why exactly you want to map exceptions to HRESULTs and I said I will amend the design to accommodate that after we talk.

    "... and you ignored metadata."

    What?? I said exactly the reverse, that if we follow Ray, we could drop COM and use metadata for interop. There is nothing stopping you or someone else from writing a tool that would take the above code I wrote and emitting metadata out of it, then using that metadata to talk to the class from another language.

    "For the vast majority of people that want to write simple code that maps very closely to the C++ object model, C++/Cx should work very well for them."

    How exactly is the C++ code in my post any less simple than C++/CX code in your post?

    I get an impression that you didn't actually read what I wrote, and that in general you aren't interested in reading what others have to say at all.

  • > If you don't care about exceptions/HRESULT mapping and you want to write more standard c++ code, then use WRL. That is what it is for. If you want to write it in more complex ways, you can.

    Jim, all due respect, nobody asked what he should do if he wants to write more standard C++ code. Everyone will make their own choices, I am sure. The question was whether there is anything about C++/CX which one couldn't do in plain C++. So far it looks like plain C++ indeed works just as well as C++/CX both on the consumption side and on the authoring side. The C++ code posted by PleaseFixYourBugs is of essentially the same complexity than the C++/CX code posted by yourself. I also agree that converting exceptions to HRESULTs is a controversial feature, this might be a necessary evil in some cases, but not others. Your reactions seem very knee-jerk, this attitude is unhelpful.

  • > We provide three options here:

    > 1) Write it in plain C++ using your own mechanisms and use MIDL.

    > 2) Write it in standard C++ using WRL and MIDL.

    > 3) Write it in C++/Cx with a pretty simple (but nonstandard) syntax.

    Missing option, which would be vastly preferred by C++ developers:

    4) Write it in plain C++ with a pretty simple and standard syntax.

  • PleaseFixYourBugs, it would indeed appear that Jim didn't read the code you posted.

    Jim's quote:

    You weren't able to properly handle the meaning of the "requires" relationship ... [that implementing IFoo requires implementing IBar]

    Your code:

    struct IFoo : public IBar

    QED, didn't read.

    I agree with your general point as well: there is absolutely nothing in C++/CX that Microsoft couldn't do with regular standard-compliant C++, achieving the same ease of use for the developer. There is no reason Microsoft could not have generated metadata out of regular C++ instead of generating it out of C++/CX. Once you have metadata, you have basically solved the main problem: achieving interoperability between different languages. Of course, metadata also allows you to wrap exceptions to HRESULTs, should you want to do this.

    It was a really bad day for C++ when Microsoft decided to go with C++/CX. Not only spending time on C++/CX moved the development focus away from C++11 in favor of a language extension (gasp!), but it also seems that in the end C++/CX does not really offer anything we couldn't already have with regular C++. Double whammy.

  • So, I'm seeing a lot of disagreement about whether the C++/CX extensions were really necessary.  I know that Herb Sutter felt that they were, and he's a bright guy, so I'm not inclined to dismiss the idea.  However, I think the comments here indicate that MS hasn't effectively communicated that rationale.  Might I suggest a white paper or somesuch, clearly spelling out how ISO C++ was determined to be inadequate?

  • @PleaseFixYourBugs

    I did read what you wrote, althogh I did miss your comment about throwing exceptions across module boundaries, which we can talk about.  I also didn't connect your previous post on metadata with your code example.

    Is this a valid summary of what you are saying.  

    1) You want to write in C++ without any langauge extensions

    2) You are willing to write your code somewhat differently than you would if just writing C++

    (i.e. write special code/interfaces for statics, constructors/factories).

    3) You are OK using macros to some degree.

    4) You are OK using special base classes.

    5) You don't want to use HRESULT as the return type on every method.

    6) You want to allow the C++ code to throw exceptions

    7) You support using some form of metadata

       a) it shouldn't be generated by the compiler, but by a separate tool.

       b) it shouldn't be directly consumed by the compiler, but .h should be generated by a tool.

    Questions:

    How should additional (i.e. wrapper) code be generated?

     1) Macros, base classes, templates (i.e. normal library approach)

     2) Compiler can generate special code based on special marker types such as IInspectable or

    WinRTBase.

     3) A separate tool should be run that generates wrapper code that is linked into the binary.

     4) Some combination of all is acceptable.

    You say that you do address the "requires" relationship and you used similar syntax to what we use in C++/Cx which is to say that IFoo inherits from IBar.  However, the underlying code gen and some of the semantics are different for standard C++ and what is required for the Windows Runtime.  From a standard C++ perspective, this is normal inheritance and we would generate code for that the same way it has always been done.  For Windows Runtime, both interfaces actually inherit directly from IIinspectable and there is a requirement that any object that implements IFoo also implements IBar.  Similarly, when MyClass inherits from IFoo, the codegen for that is going to be based on our existing C++ ABI, not what WinRT needs.  In my example, the equivalent code I showed had another automatically generated interface for the class itself called __MyClassInterface.  Any public methods (or properties or events) on a class that aren't implementing an interface method automatically get exposed through an interface.  

    Also, protected methods (and properties and events) get automatically exposed through an interface.  This is done so that inheritance across Windows Runtime components can be achieved.  Inheritance in WinRT is quite different from inheritance in C++ as you don't have a full definition of what exactly a base class is.  You know what the public and protected interface is, but you don't even know what the size of the base class is.  This is an advantage in that you can avoid some aspects of the brittle base class problem.  The allocation of the base class is done as an independent allocation and the derived class wires up to it through something akin to COM aggregation.

    As I've said several times here, you can definitely implement this stuff using standard C++ and you could also write some different tools that make some of this easier.  Our goal was to provide an easy way to target the Windows Runtime in a way that feels familiar to C++ developers and doesn't require them to know many of the details of how the runtime works.  I'm sure there are things we could have done differently, but I believe that what we are delivering will make it much easier for developers to write correct WinRT componens.  I've heard questions for years from old school C developers asking why they needed C++.  Their view is that everything you can write in C++ can also be written in C.  You can't really disagree with that, but the value is in the abstractions and the simplicity of expressing complex concepts.  We are simply trying to bring that to native code developers for the Windows Runtime.  Whether our approach is successful can only be determined in the long run.

  • Please fix the supersize syndrome for projects built with MFC statically linked that don't use the MFC feature pack classes. VS2010 ( and possibly vNext) adds about 1.4 Mb to the size of the executable in comparison to the same project built with VS2008. I don't want to switch to VS2008 toolset from the VS2010 IDE, I want to use the new language extensions/enhancements.

  • "Is this a valid summary of what you are saying."

    More or less. The main point is to use plain ISO C++ without any language extensions. If we absolutely have to expose C++ classes for use in other languages -- it's not clear to me if this is even a requirement, we already have interop via C-style functions, it works great, so do we really have to expand on it at this very moment? I would say this could wait until you are completely done with C++11 -- but, anyway, if we absolutely have to, let's use metadata. It's not important whether metadata is generated by a C++ compiler or a separate tool, what's important is that it is generated out of plain ISO C++.

    "How should additional (i.e. wrapper) code be generated?"

    Either of your options is OK.

    "From a standard C++ perspective, this is normal inheritance and we would generate code for that the same way it has always been done. For Windows Runtime, both interfaces actually inherit directly from IIinspectable and there is a requirement that any object that implements IFoo also implements IBar."

    If WinRT absolutely requires singular inheritance (why?), let's compose the metadata as above, then use that metadata to generate classes that expand each regular ISO C++ class which implements several interfaces into a set of classes with a separate class for each interface. This can generate either the source code you include into the original binary, or the source code that compiles into a separate binary, or simply generate the separate binary. Do what's easiest and what makes the most sense (I'd vote for the second option, but either is fine).

    The metadata should include protected methods, yes.

    "As I've said several times here, you can definitely implement this stuff using standard C++ and you could also write some different tools that make some of this easier. Our goal was to provide an easy way to target the Windows Runtime in a way that feels familiar to C++ developers and doesn't require them to know many of the details of how the runtime works. I'm sure there are things we could have done differently, but I believe that what we are delivering will make it much easier for developers to write correct WinRT componens. I've heard questions for years from old school C developers asking why they needed C++."

    First, please don't compare the benefits of C++ over C with the "benefits" of C++/CX over C++. This is beyond joke. I can elaborate if you insist, but this comparison is completely ridiculous.

    It is nice that you agree that you could definitely implement this stuff using standard C++, without inventing C++/CX. Since the main point of this discussion was that using standard C++ would have been as easy for developers as using C++/CX, and you are no longer contesting this point, I take it you agree here as well.

    What you are delivering with C++/CX will make it easy to write correct WinRT components, but will only do so for developers who don't care about being locked into your proprietary syntax. Using plain ISO C++ would have allowed you to make writing correct WinRT components just as easy, but without stepping outside of the standard syntax. (Ironically, it also seems to me that using plain ISO C++ would have been easier for you from the implementation point of view, since you would only have to do part of the work you did while implementing C++/CX.)

    What's more important is that using plain ISO C++ instead of C++/CX would have avoided the situation where you continue to flood the standard C++ with new keywords and semantics. Everyone understands the concept of sharpening their tools so that they fit the job better, but, seriously, a third language extension in 8 years??? With no tangible benefits over using plain ISO C++ with help from metadata (as we have just determined)? And with the history of Microsoft jumping from one language extension to another, leaving all but the newest language extension in the cold (ME C++ is dead, C++/CLI has been neglected for years)? What's wrong with you?

    You failed big time.

  • Thanks for expressing my thoughts, PleaseFixYourBugs. Like / +1.

  • if you want learn about data structure in c++ you can visit   pakitgroup.blogspot.com

  • @PFYB couldn't agree more +1; And just one more observation:

    "Whether our approach is successful can only be determined in the long run." O my God, another decade lost, another hundreds of thousands of people waisting their lives on learning this ... (.NET' history comes to mind). Pathetic. Pathetic. Pathetic.

  • Trying to understand this.

    So if I write "Type^ something" then I can think of that as a shortcut for writing somethign like std::shared_ptr<Type> something" ?

    (I realise it's not a shared_ptr, it's a com wrapper, but is the basic concept right? That it's largely a neat syntax for defining a reference counted pointer rather than anything very new?)

Page 3 of 4 (49 items) 1234