A first hand look from the .NET engineering teams
This post was authored by Morgan Brown, a Software Development Engineer on the .NET Native team. It is the first post in a series of five about Runtime Directives.
If you've read our previous blog posts on .NET Native, you might be wondering how all of this works. We've discussed the performance benefits of .NET Native and dug into the tool-chain but we never answered a basic question: how does .NET Native do dynamic things when it's compiling your code statically? A big part of the reason developers love .NET is the fact that you can do things like reflect on your code and call code on the fly. But when you're compiling all your code statically, as .NET Native does today, how does the compiler know what code your app might want to call at runtime? The answer is something called Runtime Directives.
It's actually our goal to make sure you never need to look at Runtime Directives—our compiler tries to predict what your program will need at runtime—but if you're interested in knowing how .NET Native allows dynamic features in static code, you have to understand how it uses Runtime Directives. And if you're using .NET Native while it's still in preview, this information might be useful for diagnosing issues.
This first blog post is an introduction to Runtime Directives. We'll be publishing four additional posts on Runtime Directives:
First, we need to understand the differences between static compilation and dynamic compilation. In a statically compiled app, all code that will ever run gets compiled ahead of time. That's how C++ compilers work. In contrast, dynamic compilation means there's a compiler in the app's runtime. In the .NET runtime, that's the JIT compiler. Having dynamic compilation is extremely powerful since it lets your app call code on the fly using reflection. Many libraries and apps take advantage of that to provide functionality such as serialization and UI frameworks. So why build static compilation into .NET Native? Static compilation lets the compiler spend more time on advanced optimizations and doesn't require it to run on the end user's device.
How does .NET Native let you call any code through reflection and still stick to static compilation? In part, the .NET Native compiler has a number of algorithms to predict what classes and methods your code might dynamically use. To cover the situations that the compiler doesn't pick up on its own, .NET Native Compilation includes a new file format for Runtime Directives you can add to your app or library called rd.xml.
There are lots of things that can go in a Runtime Directives document and we won't go through the whole list here (they're all documented under the Runtime Directives Configuration File Reference topic on MSDN), but let's talk about a couple of terms that will help you write the documents (and that we'll use in other blog entries about Runtime Directives).
When you create a .NET Native project, Visual Studio adds an rd.xml that looks something like this:
<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata"> <Application> <!-- Your directives go here --> </Application> </Directives>
A directive is a line of XML that talks about a type (a class, interface, or struct), a group of types (a namespace or assembly), or a type member (a method, field, property, or event). For example,
<Type Name="MyNamespace.MyClass" />
talks about just one class while
<Namespace Name="MyNamespace" />
talks about all types in MyNamespace.
Just naming a type or member alone isn't enough to have any effect. To get the compiler to add new runtime behaviors for it, you also need to include one or more degrees. A degree is an XML attribute that declares what you're planning on doing with that type or member at runtime. For example, the Dynamic degree means that you're planning to create objects or invoke methods dynamically (like the C# dynamic keyword might – that's one of the cases where you'd use the Dynamic degree). The DataContractSerializer degree means that you'll serialize a class using the DataContractSerializer.
There's also one other part of a degree – a policy, which says how to apply the degree to the type or member. Policies are expressed as the value of the XML attribute. A policy contains two things:
Full documentation on Runtime Directive Policy Settings is available on MSDN. Intellisense also provides tooltips within Visual Studio.
Here are a couple examples:
<Type Name="MyNamespace.MyClass" Dynamic="Public" />
This means that if MyClass gets used by something else in my program, then make it available to dynamic reflection. Also, make its public members (but not its internal or private members) available to reflection if they're used by something else in the app.
<Namespace Name="MyNamespace" Serialize="Required All" />
For all types in MyNamespace, make sure they're all kept and include the reflection data usually used by reflection-based serializers (such as many NuGet serialization libraries).
There's a lot of great information coming up in our series of deep dives into how .NET Native using Runtime Directives. We hope you find these blog posts useful—let us know what you think about Runtime Directives. And we always welcome your feedback on .NET Native. You can leave comments on this post or send mail to firstname.lastname@example.org.
I hope you will also tell us what happens when dynamically generated code is produced, loaded and execute. I guess this is the most tricky part. Its interesting if .Net Native is capable to insert JIT compilation fallback for the cases when its impossible to do static compilation (like in mentioned case).
@sich: In the current preview, .NET Native doesn't have a JIT to fall back on. We're doing all Windows Store apps as fully static. In the future, we can take what we learn from this exercise and improve on our scenarios where we fall back on JIT compilation for more dynamic methods (such as Windows Phone Triton and desktop NGen.) JIT fallback is a great capability and we're preserving our ability to add it into .NET native in the future. But right now we're focusing on doing an excellent job on fully static code.
@Andrew Pardoe, thanks. Do I understand correctly that Store apps just don't support emitting dynamic code (never checked that), and since .Net Native for now is supported for Store apps only, you don't have to care about such scenario at this stage?
@sich That's about right -- Windows Store apps don't have access to System.Reflection.Emit. You can use LINQ expression trees -- the JIT runtime does emit those as dynamic code, but .NET Native can interpret those. Interestingly, it turns out to be often faster to interpret them for a reasonable number of iterations. Overall, we've found that there's a lot of dynamic functionality that we've done using a JIT that we can precompile, such as serialization.