Inside the C++/CX Design

Inside the C++/CX Design

Rate This
  • Comments 49

Jim SpringfieldHello. This is Jim Springfield, an architect on the Visual C++ team.

Today, I want to give some insight into the new language extensions, officially called C++/CX, which was designed to support the new API model in Windows 8. If you attended //BUILD/, watched some of the sessions online, or have been playing with the prerelease of Visual Studio, you probably have seen some of the “new” syntax. For anyone who is familiar with C++/CLI (i.e. the language extensions we provide for targeting the CLR), the syntax shouldn’t seem much different.

Please note, however, that while the C++/CX syntax is very similar to C++/CLI, the underlying implementation is very different, it does not use the CLR or a garbage collector, and it generates completely native code (x64, x64, ARM depending on target).

Early on in the design of our support for Windows 8, we looked at many different ideas including a pure library approach as well as various ways to integrate support in the language. We have a long history of supporting COM in the Visual C++ team. From MFC to ATL to #import to attributed ATL. We also have a good bit of experience at targeting the CLR including the original managed extensions, C++/CLI, and the IJW support for compiling native code to MSIL. Our design team consisted of seven people and included people who worked on these and who have lots of experience in libraries, compiler implementation, and language design.

We actually did develop a new C++ template library for Windows 8 called WRL (Windows Runtime Library) that does support targeting Windows 8 without language extensions. WRL is quite good and it can be illuminating to take a look at it and see how all of the low-level details are implemented. It is used internally by many Windows teams, although it does suffer from many of same problems that ATL does in its support of classic COM.

  1. Authoring of components is still very difficult. You have to know a lot of the low-level rules about interfaces.
  2. You need a separate tool (MIDL) to author interfaces/types.
  3. There is no way to automatically map interfaces from low-level to a higher level (modern) form that throws exceptions and has real return values.
  4. There is no unification of authoring and consumption patterns.

With some of the new concepts in the Windows Runtime, these drawbacks become even more difficult than in classic COM/ATL. Interface inheritance isn’t vtable-based like it is in classic COM. Class inheritance is based on a mechanism similar to aggregation but with some differences including support for private and protected interfaces. We quickly realized that although there is a need for a low-level tool like WRL, for the vast majority of uses, it is just too hard to use and we could do a lot better while still preserving performance and providing a lot of control.

The #import feature that was available in VC6 provides a good mechanism for consuming COM objects that have a type library. We thought about providing something similar for the Windows Runtime (which uses a new .winmd file), but while that could provide a good consumption experience, it does nothing for authoring. Given that Windows is moving to a model where many things are asynchronous, authoring of callbacks is very important and there aren’t many consumption scenarios that wouldn’t include at least some authoring. Also, authoring is very important for writing UI applications as each page and user-defined control is a class derived from an existing Runtime class.

The design team spent a lot of time discussing what consumption of Windows Runtime components should look like. We decided early on that we should expose classes and interfaces at a higher level than what the ABI defines. Supporting modern C++ features such as exceptions was deemed to be important as well as mapping the Runtime definition of inheritance (both for interfaces and classes) to C++ in such a way that it was natural. It quickly became clear that we would need some new type categories to represent these as we couldn’t change what the existing C++ ABI meant. We went through a lot of different names and it wasn’t until we decided to use the ^ that we also decided to use ref class to indicate the authoring of a Windows Runtime class.

We also spent a lot of time exploring various approaches to how to hold onto a pointer to a WinRT class or interface. Part of this decision was also how to tell the difference between a low-level version of an interface and the high-level version of the interface. We had a lot of different proposals including just using a *, using * with a modifier, and using various other characters such as the ‘@’ symbol. In the original extensions we did for managed code, we in fact did use a * with a modifier (__gc). We realized we would have many of the same problems if we followed that route. Some of the breakthroughs came when we started thinking about what the type of a pointer dereference would be. This made us realize that what we were doing was similar to what we did when C++/CLI was designed. At one point, someone suggested “Why don’t we just use the ^ symbol?” After the laughter died down, it started making a lot of sense. On design point after design point, we often came to the same design decision we had made for C++/CLI.

Many of the concepts we were trying to express were already present in the C++/CLI syntax. Given that reference counting is a form of garbage collection, using ^ to represent a “refcounted” pointer in ZW fits quite well. Dereferencing of a ^ yields a %, also like C++/CLI. While many concepts are expressed the same way, there are a few areas where we decided to deviate from C++/CLI. For example, in C++/CX, the default interface on a class is specified through an attribute in the interface list while in C++/CLI it is an attribute on the class itself.

In C++/CX we have a much better story than C++/CLI when it comes to interoperating references types with regular types. In C++/CLI, managed objects can move around in memory as the garbage collector runs. This means you can’t get the real address of a member (without pinning) or even embed anything except primitive types (i.e. int) into your class. You also cannot put a ^ into a native class or struct. In C++/CX, objects do not move around in memory and thus all of these restrictions are gone. You can put any type into a ref class and you can put a ^ anywhere. This model is much friendlier to normal C++ types and gives the programmer more flexibility in C++/CX.

We will be providing more insight into our design over the coming months. If there are specific things you would like to know more about, please let us know.

  • Nice article. The ^ thingy still burns my eyes, but I guess I'd better get used to it. By the way, is there any documentation yet regarding the WRL? I mean, a readable one, not videos (can't see videos at work). Thanks!

  • There aren't any docs on WRL available yet, but it is being worked on.

  • msdn.microsoft.com/.../hh438466%28v=VS.110%29.aspx

  • Here are some questions:

    In C++/CX, what does CX mean?

    If I use XAML and the C++/CX and WinRT classes, am I going to be able to develop

    in anything other than full screen, suspended, async, metro mode?

    msdn.microsoft.com/.../hh464912(v=VS.85).aspx

    Will C++/CX ever run on Windows 7?

    Will I be able to simply recompile my C++ desktop applications for Arm?

    Will C++/CX be adopted eventually as part of the C++ standard?

    Why is the C++ example in "Getting data into an app" much longer in size than C# and VB ?

    msdn.microsoft.com/.../br211380(v=VS.85).aspx

    A "Hello World" program is not the place to show widely diverging techniques between implementing languages.

    I recommend a simple GUI program in which a click of a button either displays "Hello world" or "Hello" depending on a checkbox.

    Will there be a substantial document that describes how C++/CX works?

    msdn.microsoft.com/.../hh454076(v=VS.85).aspx is a start, but there is nothing on:

    The purpose of Windows Runtime struct

    How to do Async

    Inheritance

    Boxing

    How to write a new WinRT component in C++/CX.

    How to inherit from an existing component and change it slightly.

    How XAML interacts with C++/CX

    etc

    p.s. Congratulations on getting this far.  Widespread adoption depends on effective communication to the developers now.

  • Hi Jim!

    I'd have a few questions -- feel free to respond to the extent that you can / feel comfortable with -- if, for some reason, you cannot provide the details on specific parts yet, just giving reasons for that would be still appreciated :-)

    0. (Standardization). Are there any plans on contributing the extensions from C++/Cx to ISO C++ at some point in the future? If not, why not, and (in any case) are there plans to standardize it in some way -- yes, we have ECMA-372 (December 2005) for a similar, managed language called "C++/CLI" already, but as we all know (and this is important) C++/Cx is a different language (native, not managed) with different semantics (e.g., ref-counting in C++/Cx vs. GC in C++/CLI), so it would be very helpful for the community to get a new, full standard-style specification of the new, native thing :) Alternatively, are there plans to update ECMA-372 from its 2005 version to reflect the developments (C++/Cx) in 2011?

    0'. (Somewhat related, C++11 standard and C++/Cx relationship). It seems that some of the extensions of C++/Cx are possibly overlapping with new features in the ISO C++11 standard -- in particular, I mean the new, ref-counted smart pointers, like shared_ptr. I understand and am familiar with the rationale (as explained by Herb) that it was the fact that some of the features were missing from C++98 that has made C++/Cx necessary. Now, that we have C++11, are there plans for convergence of C++/Cx to a larger reliance on those standard features (like using the standard ISO C++11 ref-counted pointers instead of the hat-handle) -- if yes, in what way? If not, why not (e.g., does the new ref-counted C++/Cx hat-handle brings something to the table that the new ref-counted C++11 shared_ptr doesn't from the design point of view?)?

    1. (Interop w/ other MS tech: Excel, XLL). Do you think you could pass on my question on Excel<->C++ interop using new tech, such as C++/Cx? In particular, perhaps the general outlook on how (if possible) to use C++/Cx as a simplification over current state-of-the-art native XLL (C API, but the fastest), and yet not as slow/bulky as the COM add-ins and not managed like VSTO? (I'm having big hopes here, with C++/Cx being native!)

    1'. (Somewhat related, C++/Cx and COM). A generalization of the Excel interop. question -- using C++/Cx instead of the current applications of COM -- is it a good idea, how far can we go, what are the limitations, when is it a particularly good idea _and_ when is it a particularly bad idea?

    Thanks!

  • Hi Jim, I'd have a few questions -- feel free to respond to the extent that you can -- if, for some reason, you cannot provide the details on specific parts yet, just giving reasons for that would be still appreciated :-)

    0. (Standardization). Are there any plans on contributing the extensions from C++/Cx to ISO C++ at some point in the future? If not, why not, and (in any case) are there plans to standardize it in some way -- yes, we have ECMA-372 (December 2005) for a similar, managed language called "C++/CLI" already, but as we all know (and this is important) C++/Cx is a different language (native, not managed) with different semantics (e.g., ref-counting in C++/Cx vs. GC in C++/CLI), so it would be very helpful for the community to get a new, full standard-style specification of the new, native thing :) Alternatively, are there plans to update ECMA-372 from its 2005 version to reflect the developments (C++/Cx) in 2011?

    0'. (Somewhat related, C++11 standard and C++/Cx relationship). It seems that some of the extensions of C++/Cx are possibly overlapping with new features in the ISO C++11 standard -- in particular, I mean the new, ref-counted smart pointers, like shared_ptr. I understand and am familiar with the rationale (as explained by Herb) that it was the fact that some of the features were missing from C++98 that has made C++/Cx necessary. Now, that we have C++11, are there plans for convergence of C++/Cx to a larger reliance on those standard features (like using the standard ISO C++11 ref-counted pointers instead of the hat-handle) -- if yes, in what way? If not, why not (e.g., does the new ref-counted C++/Cx hat-handle brings something to the table that the new ref-counted C++11 shared_ptr doesn't from the design point of view?)?

    1. (Interop w/ other MS tech: Excel, XLL). Do you think you could pass on my question on Excel<->C++ interoperability using new tech, such as C++/Cx? In particular, perhaps the general outlook on how (if possible) to use C++/Cx as a simplification over current state-of-the-art native XLL (C API, but the fastest), and yet not as slow/bulky as the COM add-ins and not managed like VSTO? (I'm having big hopes here, with C++/Cx being native!)

    1'. (Somewhat related, C++/Cx and COM). A generalization of the Excel interop. question -- using C++/Cx instead of the current applications of COM -- is it a good idea, how far can we go, what are the limitations, when is it a particularly good idea _and_ when is it a particularly bad idea?

    Thanks!

  • MattPD> does the new ref-counted C++/Cx hat-handle brings something to the table that the new ref-counted C++11 shared_ptr doesn't from the design point of view?

    Hats and shared_ptrs are as different as apples and oranges. They're both fruit (i.e. deterministically reference-counted smart pointers), but that's about it.

    Hats do QueryInterface stuff in addition to a bunch of magic that I barely know about.

    shared_ptr works with custom deleters, custom allocators, make_shared, allocate_shared, weak_ptr, enable_shared_from_this, and atomic operations, as required by C++11. It knows absolutely nothing about COM.

    If you look at my collection.h in the VC11 Developer Preview, you'll see hats and shared_ptrs working side by side. For example, Platform::VectorIterator stores WFC::IVector<T>^ (as it wraps the interface in STL iterator semantics), while Platform::Vector implements WFC::IVector<T> (actually WFC::IObservableVector<T>) and stores shared_ptr<UInt32> and shared_ptr<std::vector<T>>. The trick there is that the Platform::Vector itself is refcounted, but users can request a WFC::IVectorView (or a WFC::IIterator) from it, and they need to keep the underlying storage alive, in addition to conspiring for other reasons (view invalidation). This is a perfect setup for shared_ptr.

  • First, I like the new design, well done.

    I have a question though, regarding versioning of the interfaces/ref classes. In 'old' COM one should not modify an existing interface in order not to break the compatibility with existing clients. Is this topic obsolete because the components are not installed globally anymore? Or are there situations where the ^/ref class -magic reaches its limits and one must take a grasp under the hood (wrl)?

  • STL: Great answer, thanks!

  • @WalkingCat

    Thanks for the link.  I’m glad they have some WRL docs up.

    @Andrew7Webb – Here are some answers…

    What does Cx mean?  It means the same thing it did in ActiveX and DirectX.

    Can I use XAML, CX, WinRT in non-metro scenarios?  In Windows8, those technologies are only available under the new Metro UI shell.

    Will C++/CX ever run on Windows 7?  Parts of it could, but without Windows 8 features available down-level, there wouldn’t be much value.

    Will I be able to simply recompile my C++ desktop applications for Arm? Many things just work, but I believe there are some things that won’t.

    Will C++/CX be adopted as part of the C++ standard?  I doubt that it would be adopted in whole. Much of it is targeting Microsoft specific platforms.  Some pieces could make it in the same way that some small parts of C++/CLI were.  The C++ committee prefers to see prior art before adopting something into the standard.

    The purpose of a Windows Runtime struct is that it can be used in an API.  There will be more information coming on all the other stuff.

    @MattPD

    0.    See above…

    0’.   I believe STL answered this

    1.    I will see…

    1’.   Overall, I think that the Windows Runtime and C++/Cx are a better solution than COM.  For the most part, it provides a richer model with inheritance, constructors, and static methods/properties. It is a little more limiting than classic COM in that it doesn’t support arbitrary data types.  With COM/RPC you can describe just about any conceivable type and how to marshal it.

    @Tavi Cacina

    Thank you, I’m glad you like it.

    In Windows Runtime, you should still not modify an existing interface.  Also, interface inheritance is very different.  In COM, interface inheritance meant vtable-based inheritance.  In C++/Cx, interface inheritance is not done that way.  In low-level WinRT, all interfaces inherit directly from IInspectable.  There can be a “requires” relationship between two interface that requires an object that implements one interface to also implement the other interface.  In C++/Cx, inheritance between interfaces maps to this low-level requires relationship.  This is actually very beneficial for versioning as you can create a new version of an interface without putting an undue burden on the implementer. For instance, in classic COM, let’s say you had this.

    //Classic COM - version 1

    interface IFoo;

    // My code

    interface IMyFoo : IFoo;

    // In version 2, there is a new version of IFoo called IFoo2.

    interface IFoo2: IFoo;

    // I can’t create an IMyFoo2 that inherits from IMyFoo and IFoo2.

    interface IMyFoo2 : //???

    In WinRT, this can actually be done.  Here is what it looks like in C++/Cx.

    //WinRT - version 1

    ref interface IFoo;

    // My code

    ref interface IMyFoo : IFoo;

    // In version 2, there is a new version of IFoo called IFoo2.

    ref interface IFoo2: IFoo;

    // I can’t create an IMyFoo2 that inherits from IMyFoo and IFoo2.

    ref interface IMyFoo2 : IMyFoo, IFoo2

  • I must express my disappointment with such custom syntax...

    The semantics of * and & operators can be easily derived from type they are referring to and the type can be declared to belong to some group by having some strongly defined base class. To make things more obvious to programmers, you could have made those types have different text editor colouring and/or naming convention.

    In my opinion, you should have done this much better...

  • @Goran

    I'm not sure exactly what your point is.

    Are you arguing against the use of any language extensions or is there something about C++/Cx you specifically don't like?

    Also, are you saying that you support special semantics (i.e. compiler has special knowledge) on types that inherit from some marker?  Or are you saying that everything can be done directly in standard C++?

    For a standard C++ approach you can use WRL, ATL, or just straight C++, although you lose many benefits as I mentioned.

  • @Jim Springfield:

    I am also disappointed with C++/CX, so:

    "Or are you saying that everything can be done directly in standard C++?"

    This.

    "For a standard C++ approach you can use WRL, ATL, or just straight C++, although you lose many benefits as I mentioned."

    Correct. The thing that is bothering me is that we lose many benefits because you have explicitly chosen NOT to implement these benefits for standard C++ and instead went for yet another language extension. You *could* have done everything in ISO C++. You have *chosen* not to do this. As a result, me and my team have recently decided that we are not going to buy VS2011 and are moving to Intel's compiler. I believe we are not alone.

    Good luck in your endeavors with VS2011. At this point we are parting ways and moving on. Enough is enough.

  • @PleaseFixYourBugs, standard C++ would require users to write much more code to achieve same goals.

    I think creating C++/CX was excellent decision.

  • I think COM sucks compared to Java, .NET, Objective-C.

    Whoever decided to resurrect that fossil is bozo.

    Microsoft sinks.

    In a few years what's left of it will be sold for scrap. Unemployed COM devs will finally die out for good.

Page 1 of 4 (49 items) 1234