A first hand look from the .NET engineering teams
This post was authored by Morgan Brown, a Software Development Engineer on the .NET Native team. It is the second post in a series of five about Runtime Directives. Please see the first post in this series, Dynamic Features in Static Code, before reading this post.
So you've started out with .NET Native and you're seeing great performance gains, but you've run into a MissingMetadataException. MissingMetadataExceptions happen in certain reflection scenarios because of new optimizations in the .NET Native compiler. We'll talk about why they happen a minute. We are continually making this better – if you hit a MissingMetadataException please let us know!
Let's start by making those pesky exceptions go away. First though, are you working on an app or a library? If you're working on an app, keep reading; if you're working on a library, we'll have another post, Making Your Library Great with .NET Native, available within the week.
In some cases, there's just one type (class, interface, or struct) causing you trouble and you can fix it with a quick change. Try these steps first to see if you can get moving again fast:
Your project should already include a file with a name ending in .rd.xml (usually the file is default.rd.xml) that should look something like this:
<?xml version="1.0" encoding="utf-8"?> <Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata"> <Application> <!-- We’re going to put more stuff here in step 3 --> </Application> </Directives>
Inside your Application element, add a Type tag with the name of the problem type you got from your MissingMetadataException and a degree attribute that tells the .NET Native Compiler what to do with it. There are lots of options for these that we'll cover later, but since we're trying a quick fix let's pick one that's likely to make the particular type get all the metadata you could want for it:
<Type Name="MyNamespace.MyProblemType" Dynamic="Required All" />
That tells the compiler that MyProblemType must be compiled and that it and all of its members must be available for reflection.
Build your app again, run it and see what happens. You won't get a MissingMetadataException for the same type or method again, so your problem might be solved. If you just get a MissingMetadataException for something else, it's time to move on from the quick fix to:
If you haven't worked with runtime directives before, this is going to use some concepts defined in the first post in this series, Dynamic Features in Static Code. Usually, we use reflection for cases where we don't know all the types or methods we might be working with. That often means there are hundreds or sometimes thousands of things that require reflection metadata. The .NET Native Compiler has smarts to try to figure out what most of those will be, but it doesn't necessarily figure everything out (we keep working on making it better and we're very interested in any feedback on cases you run into). Fortunately, as a human, you're much smarter about what your app needs to do, so you can give the compiler hints about what you really want. Before we dive into syntax, let's talk a little background on what's different about .NET Native that would make us go through all this trouble.
One of the reasons .NET Native makes code so much faster is that it has a global view of everything that runs in your program – that includes code you wrote, any NuGet libraries you might be using and the portions of the .NET Framework that you and your libraries use. That lets it do some pretty serious optimizations. If the compiler compiled all of that stuff, it would take many times longer than it does today, even for a very simple app and produce huge binaries. To avoid that we use a technique called "tree shaking" that gets rid of all the code you don't need and only compiles what the compiler thinks you need. .NET Native's static compiler compiles all code ahead of time and doesn't include a JIT, so what you compile is what you get at runtime.
"Wait a second," you say, "reflection lets me use methods and even whole classes that aren't statically mentioned in my code at all!" To ensure .NET Native allows all of the dynamic functionality that makes .NET so great, we added RD.xml (Runtime Directive) files so you can tell the compiler all the stuff you need that it can't figure out on its own. (It turns out there are a few categories of runtime behavior that get informed by this file. Besides reflection, rd.xml can also make the compiler generate support code for the built-in serializers and advanced interop scenarios).
If you're here, you've probably noticed it's not practical to try to specify every single type or method. There are ways to include everything, but this will increase your binary size and build times. A good balance between including too much stuff and wasting development time is to try to think about RD.xml like the code that uses it.
To make your app work, check out the stack trace in your MissingMetadataException. The exception will be thrown from somewhere in the .NET reflection APIs, but take a look at what's calling into reflection. Is it:
Library-like code that you do control (e.g. code that isn't too specific to your app or code you might use in multiple apps)?
Sometimes it's helpful to factor your own code into libraries that use reflection to simplify patterns like MVVM. We'll follow up with another blog post, Making Your Library Great with .NET Native, within the week.
A library you use, but didn't write?
That's a pretty common case since libraries often use reflection to find out about types in your app that they don't know about. If a library is the caller, a good first step is to check for a new version of the library that's been built with .NET Native in mind and might include its own RD.xml file to fix any issues. If there's not one, point the library author here so they can make it better for all of their users. In the meantime, you can still write an RD.xml that makes your app work.
Plain old app code?
You're using reflection in a targeted way on a known set of things. In that case, you might be able to get a good idea of what those things are, which should make it much easier to get your RD.xml right.
So now that you've found what's calling reflection, there are two questions to answer:
What kind of information does that library or app code likely need?
You might not know how it's implemented, but you might be able to make a good guess about how it maps to the directives in an RD.xml file. For example, if it's a serializer, the Serialize degree is probably what you want. If it probably dynamically calls back into your classes, try the Dynamic degree.
What are the inputs to that code (if it's not your code, what parts of your app code does the library interact with)?
For instance, what are the types of objects or System.Types you're passing to that code? These types are going to be the ones you will specify in your RD.xml.
Putting that together, it's time to draw some conclusions and write some directives. Let's say you find that lots of the types in your MyApp.Data namespace are getting serialized by a serialization library that's throwing MissingMetadataExeptions. In that case, rather than listing out all of the types it might find, try the simple directive:
<Namespace Name="MyApp.Data" Serialize="Required All" />
Or maybe many of the methods on types in your GameScriptObjects.dll assembly get dynamically called:
<Assembly Name="GameScriptObjects" Dynamic="Required All" />
As a rule of thumb, we usually find that an RD.xml file that strikes a pretty good balance between performance and hassle is 10-30 lines. If you're still finding problems that might be reflection related, but aren't throwing exceptions, we'll have a post, Help! I Didn't Hit a MissingMetadataException!, available later this week.
We welcome your feedback on the developer experience using .NET Native. Please leave comments on this post or send mail to firstname.lastname@example.org.
Can you build a tool to support that effort, that traces program execution in JIT mode, and writes down all dynamically accessed metadata automatically into a RD.xml file, which can than be reused in native compilation?
If you like the idea, send kudos for vimilova
I'm wondering about how this will fit in with automated testing.
I can imagine that some people might want to run tests against a .net native compiled library to make sure that their reflection or serialization code still works. But I think you'd want it to compile the libraries under test separate from the libraries doing the testing because otherwise the smarts in .net native might change what gets taken out of the libraries when compiled along with the test libraries.
@vimilova - instrumenting the desktop CLR to capture reflection is an idea we looked at in the early days of .NET Native. It turns out that this doesn't work as well as you would like for a variety of reasons. For instance, much of the implementation of the framework is refactored in .NET Native, so many reflection patterns turn out to be different due to different implementation details. We also found that the technique relied too much on having individual developers manually walk through their application in the reflection profiler, which is not a task we would like the correctness of a program to depend upon.
Instead, there was a fundamental shift in how we approached rd.xml, and we moved more toward a technique where an API could describe the reflection requirements it has on its callers. That shift allows us to both build better tooling and better support for statically getting rd.xml right. Morgan's got another post coming next week that gets into more details on this technique.
@Mark - you're correct that unit testing and .NET Native has an interesting intersection. We're still early in the development of .NET Native so we don't have a direct unit testing story for the output of .NET Native code just yet. Some of the tricky problems include the fact that many unit testing frameworks have specific dynamic requirements which would in fact change the analysis that our tools might do on the program.
Will applications using Prism/MEF need to define entries in RD.xml for all the types that are dynamically discovered? Or will compiling the assemblies that contain the dynamically discovered types with .NET Native suffice.
@Ganesh R: The default rd.xml may work for Prism/MEF. However, my expectation is that Microsoft should create a library rd.xml for Prism so that its dynamic behavior can be tracked by the .NET Native toolchain.
Is there a published Store app you can point us to that uses Prism for the Windows Runtime?
@Andrew Pardoe I was speaking in general for Prism as such. I was more interested on the implications of using .NET Native for Desktop applications. Hence the question.
@Ganesh R: As other comments indicate, we're focused on Store apps right now.