Suzanne Cook's .NET CLR Notes : Loader Info

Channel 9 Interview

Suzanne Cook — Sat, 12 Feb 2005 02:01:00 GMT

Charles Torre and Robert Scoble (behind the camera) dropped by my office to chat with me about the loader for MSDN's Channel 9: part I and part II.
Also, check out the PDC video I was in for Channel 9: http://channel9.msdn.com/ShowPost.aspx?PostID=79591

New Assembly, Old .NET (and Vice-Versa)

Suzanne Cook — Thu, 27 Jan 2005 01:40:00 GMT

I typically recommend that you build and test your assemblies against the same version of .NET that you will be running them against. That way, you'll have correct references and avoid surprises from behavior differences between builds.

Older assembly, newer .NET

But, sometimes you don't run against the same version that you built against. For example, the latest available CLR is the default when interop causes it to be loaded. Or, if the version of the CLR the assembly was built against is not available, the latest version can also be used instead. If there is a compatibility problem with that for your application, you can force the use of a preferred CLR version by using an app.config.

Compatibility

Yes, there are going to be behavior differences between versions - no way around it. It is the .NET team's goal to be as backwards compatible as realistically possible between versions. But, some changes are required - new features are added which sometimes require (hopefully minimal) behavior changes in other features. If we were required to never break any app, ever, we wouldn't be able to fix any bugs or add any features for fear that someone, somewhere relied on the broken behavior in some strange way. As Dll Hell has shown in the past, new versions of components aren't really compatible. The only guaranteed 100% compatible version is the same one you tested against.

V2+ assembly, older .NET

Say you see a BadImageFormatException for hresult 0x80131107 (CLDB_E_FILE_OLDVER) with message "Version [...] is not a compatible version." That means that you've tried to load an assembly built against a newer version of the CLR than was running.

This fails because new files can't be opened on an older CLR that does not understand them. You will need to either use the v2 CLR to load the v2/v1 assemblies, or load only v1 assemblies on the v1 CLR. That's just good practice, anyway. If you try to use an older .NET than you built against, a method you built against may not exist, etc.

Versioning/Deploying Unmanaged Files

Suzanne Cook — Fri, 29 Oct 2004 00:37:00 GMT

An unmanaged dll can be wrapped in a managed assembly by adding it as a file of a multi-module assembly. Then, it can be deployed and versioned in the same way as managed assemblies. (So, that assembly could contain nothing but metadata and unmanaged code - no managed code, if you prefer. It can also contain multiple unmanaged files in the same assembly.)

If your compiler does not support this directly, you can get this to work by adding that file as a linked managed resource. For example, see Visual Studio's /linkresource option (if using it for command line compiling).

This is useful in the case where DllImport() is used to access a function in that file. That call should be updated with the new assembly's info. For example, in the place of "unmanagedfile.dll", change it to include the display name like this: "unmanagedfile.dll, managedassembly, Version=1.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089".

If that unmanaged file needs to be loaded by LoadLibrary() outside of DllImport(), however, it will need to follow the rules of LoadLibrary(), like the usual unmanaged file use outside the CLR. (See Junfeng's blog for an extra tip regarding that when using v2.)

Load(AssemblyName)

Suzanne Cook — Wed, 04 Aug 2004 23:20:00 GMT

Calling Load(AssemblyName) is not necessarily the same as calling Load(String). If the AssemblyName.CodeBase is not set, then they do do the same thing. So, if you've set the AssemblyName.Name, CultureInfo, public key token / public key and/or Version properties, it would be the same as if you had specified those properties in a String (as a display name) and passed that to Load(String).

If the CodeBase is set, but the Name is not, however, then it's the same as calling Assembly.LoadFrom() on that CodeBase.

When both the CodeBase and the Name are set, then the bind is tried with all the given binding information except the CodeBase (so, again, just like calling Load(String)). If that succeeds, we're done. But, if that fails, then the bind is tried again with just the CodeBase (just like LoadFrom()). If it fails again, then, of course, the whole bind fails. But, if it succeeds, then we verify that the binding properties in the AssemblyName match the found assembly. If they don't match, a FileLoadException will be thrown for hresult FUSION_E_REF_DEF_MISMATCH.

So, setting both the CodeBase and the Name is useful for when you want to both load an assembly at a given path into the LoadFrom context, and verify that it has the public key token, etc. that you expect. Of course, as described above (and due to binding context rules), keep in mind that just because you call Load(AssemblyName) with a CodeBase, it does not mean that it will be loaded from that path.

App.config Examples

Suzanne Cook — Fri, 14 May 2004 16:25:00 GMT

Below are three examples of useful application configuration files.

Forces the v1.0 CLR to be run. If the v1.0 CLR is not installed, the app will fail to run.

<?xml version ="1.0"?>
<configuration>
    <startup>
         <requiredRuntime version="v1.0.3705"/>
         <supportedRuntime version="v1.0.3705"/>
     </startup>
</configuration>
Redirects “assemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=25283151a234958d“ to version 2.0.0.0 of that assembly. This is only useful for strongly-named assemblies, since versions don't matter for those that are simply-named.

<?xml version ="1.0"?>
<configuration>
<runtime>

        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

                <dependentAssembly>
                        <assemblyIdentity name="assemblyName" culture="" publicKeyToken="25283151a234958d"/>
                        <bindingRedirect oldVersion="1.0.0.0" newVersion="2.0.0.0"/>

                </dependentAssembly>
        </assemblyBinding>

</runtime>
</configuration>
Redirects “assemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=8968ee41e78ce97a“ to codebase “http://www.yourwebsite.com/filename.dll“. 'Href' can also be set to something like “file:///c:/localfile/filename.dll“. Note that redirecting to a codebase causes a System.Net.WebPermission or System.IO.FileIOPermissionAccess.Read + PathDiscovery demand when loading that assembly.

<?xml version ="1.0"?>
<configuration>
<runtime>

        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

                <dependentAssembly>
                        <assemblyIdentity name="assemblyName" culture="" publicKeyToken="8968ee41e78ce97a"/>
                        <codeBase version="1.0.0.0" href="http://www.yourwebsite.com/filename.dll"/>

                </dependentAssembly>
        </assemblyBinding>

</runtime>
</configuration>

Determining Whether a File Is an Assembly

Suzanne Cook — Thu, 18 Mar 2004 08:37:00 GMT

A file is an assembly if and only if it's managed and it contains an Assembly entry in its CLR metadata.

Determining by hand

A fast way to determine whether a file is an assembly is to run ildasm.exe on it. If it immediately gives an error saying that it may not be a PE file, then it's not a managed file. But, if it is an assembly, then ildasm will show an entry for the Assembly definition (“.assembly“ in the MANIFEST window or at the bottom of the original window).

Determining programmatically

From unmanaged code, you can call GetAssemblyFromScope() on the IMetaDataAssemblyImport interface for the file. If it returns S_OK, it's an assembly. If it returns CLDB_E_RECORD_NOTFOUND, it's not an assembly.

From managed code, if AssemblyName.GetAssemblyName(), Assembly.Load*(), etc. succeeds when given that file, then it's an assembly. If the load failed with a BadImageFormatException, then it may not be an assembly. There are other reasons, however, why that exception may have been thrown (maybe it's an assembly but could not be loaded because it has an incorrect format). Coming soon in v2, if the hresult for BadImageFormatException is COR_E_ASSEMBLYEXPECTED, then it's because it's not an assembly. Catch the exception and call System.Runtime.InteropServices.Marshal.GetHRForException() to get its hresult to find out.

Note that this assumes that you are not concerned about performance. If you are concerned about that, then the way to optimize for it is to not try to determine whether files are assemblies at runtime at all. (Some people are concerned about the exception that is thrown if the file is not an assembly. They overlook that in order to determine that it is not one, it has to be loaded and then thrown away, which can be even more expensive than catching an exception!) Instead, require that only assemblies are given to you. Then, if a file is not an assembly, it will be an exceptional situation. That way, you avoid the bad perf of both the exception and the unnecessary file loading.

LoadFile vs. LoadFrom

Suzanne Cook — Sat, 20 Sep 2003 04:34:00 GMT

Be careful - these aren't the same thing.

LoadFrom() goes through Fusion and can be redirected to another assembly at a different path but with that same identity if one is already loaded in the LoadFrom context.
LoadFile() doesn't bind through Fusion at all - the loader just goes ahead and loads exactly* what the caller requested. It doesn't use either the Load or the LoadFrom context.

So, LoadFrom() usually gives you what you asked for, but not necessarily. LoadFile() is for those who really, really want exactly what is requested. (*However, starting in v2, policy will be applied to both LoadFrom() and LoadFile(), so LoadFile() won't necessarily be exactly what was requested. Also, starting in v2, if an assembly with its identity is in the GAC, the GAC copy will be used instead. Use ReflectionOnlyLoadFrom() to load exactly what you want - but, note that assemblies loaded that way can't be executed.)

LoadFile() has a catch. Since it doesn't use a binding context, its dependencies aren't automatically found in its directory. If they aren't available in the Load context, you would have to subscribe to the AssemblyResolve event in order to bind to them.

So, which one should you use? That depends on which binding context is right for you and whether it matters that the LoadFrom context may redirect the bind. Regarding the latter:

If no other code can load assemblies in your AppDomain, your LoadFrom() call can prevent a redirect* by just not loading anything else with a duplicate identity.
If you do need to load a duplicate, you could get away with it by loading it into anything other than the LoadFrom context (or into another AppDomain) instead.
But, maybe the redirect is really okay. Are they identical assemblies, just at different paths? If yes, it may be safe to risk the redirect unless its dependencies aren't available in the dir of the first one loaded.
Or, are they possibly different bits, just with the same assembly identity? If so, it may be unsafe to risk the redirect: maybe you'll get an more-updated/not-as-updated version which you didn't expect. Or, if it's not strongly-named, maybe this is a third-party's assembly, totally unrelated to the one you expect.

LoadFrom's Second Bind

Suzanne Cook — Wed, 17 Sep 2003 03:49:00 GMT

Pre-v2, when you load an assembly by path through Fusion (LoadFrom(), ExecuteAssembly(), etc.), it can actually cause two binds, not just one. The first bind loads the file at the given path. If that is successful, another bind is done with the display name of that assembly to see if it's available in the Load context. If it's not available, the LoadFrom() call is still successful - we throw away the results of the second bind. If it is available, we check the path returned from the first bind against the path from the second bind. If they're identical, we keep the Load context IAssembly and throw away the LoadFrom one. (That means that that assembly will be in the Load context, not the LoadFrom context.) Otherwise, we keep the LoadFrom context IAssembly.

This is to ensure that the Load context remains order-independent - an important part of the design for that context. It's an implementation detail, but I bring it up because it has some side-effects that are surprising to customers:

The second bind shows up in the Fusion log. (It's just another bind to Fusion.) It's unsettling to see a failure in the log (for the second bind) when the LoadFrom() call succeeded.
Doing a second bind by display name causes probing which slows down the performance of LoadFrom(), especially over http, when the file isn't found immediately.

Fusion has stopping doing the second bind in v2. But, when using an older CLR, customers can avoid both side-effects by putting a redirect from the display name to the codebase in a config file. That will skip probing, helping perf, and, assuming it's the correct codebase, avoid a Fusion log noting an ignorable failure.

Avoid DevPath

Suzanne Cook — Sat, 16 Aug 2003 04:22:00 GMT

I hesitate to talk about this because I don't want people who don't know about it to think, "Hey, what's this DevPath thing? I need that." But, maybe if I don't explain how to use it, it will be too much effort for people who don't already know how. :) (And, for those who already know how and are, in fact, using it, hopefully, they'll see this and get off that plan.)

The intent of DevPath was to make the development environment less painful. Assemblies could be put there and bound to at runtime, ignoring the assembly version and overriding the GAC.

It turns out that that's badness for several reasons (below). So, DevPath is soon to be deprecated. Don't use it - not even in the development environment.

Why It Should Never Be Used
Versioning alone is why you should never, ever use it in a production environment. It subscribes your users to dll hell. See Avoid Partial Binds for details (DevPath causes partial binding, since the version is ignored for the bind).

It's not good for the dev. env., either - it makes it unnecessarily different from the shipping env., which may lead to uncaught versioning or deployment bugs in the shipping env.

What to Do Instead
If DevPath was a development environment-only solution for you:
I strongly recommend not changing your assembly versions between non-shipping builds instead of using DevPath.

If you are shipping code relying on DevPath:
One thing you could do is create a new AppDomain with the ApplicationBase set to the path you care about. Once you do that, see Executing Code in Another Appdomain.

If there are multiple paths you want to load assemblies from, use PrivateBinPath or multiple AppDomains. If that's not reasonable for you, consider using the LoadFrom context.

Binding to .NET Frameworks Assemblies

Suzanne Cook — Thu, 31 Jul 2003 04:30:00 GMT

By "Frameworks assemblies," I mean the assemblies that ship with the CLR. But, I'm not counting mscorlib.dll, since it's special in a different way.

With v1.0 SP3 or later, Frameworks assemblies are unified. That means that the version of those assemblies that you request is ignored - you get the version that matches the loaded CLR. This only applies when the assembly is loaded by version - that is, by assembly display name or static reference (AssemblyRef in the CLR metadata).

For example, if the v1.1 CLR is loaded and you request v1.0's system.dll, then you'll get the v1.1 system.dll back. If you load it by path from c:\foo\system.dll, however, then you'll get c:\foo\system.dll back, not the v1.1 system.dll.

Those assemblies are unified because the owners of the code feel that they are so closely tied to the CLR/mscorlib.dll that they should not be loaded with a different version than they were built against. Additionally, there are some cases where they expect that only one copy of the Frameworks assemblies be loaded in a given process.

If the v1.0 (pre-SP3) CLR is loaded, though, you will get the assembly that you ask for. Unification isn't done for Frameworks assemblies in that version.

Do not rely on this behavior. You should still fully-specify assembly references with the correct version. That's just good practice, in general. Besides, if your assembly is loaded in a CLR later than v2.0, those references may not be unified, so you may get unexpected behavior.

Assembly Identity

Suzanne Cook — Tue, 22 Jul 2003 01:12:00 GMT

There are two types of assembly identity that the loader deals with: bind-time and after bind-time. The identity is used to determine whether we will consider a certain assembly to be the same thing as an assembly reference or another assembly.

Assembly Binding
At assembly-bind time, the components of the assembly display name determine identity. This will be used to match an assembly reference (the assembly that the user wants to load) to a manifest file.

Not all of the components matter in all cases. If the assembly is not strongly-named, then the version is ignored for binding. But, if it is strongly-named, the entire version in the assembly reference needs to match the found assembly.

There are other details about what is and isn't necessary to specify for binding, but please always give the entire display name EVERY time, anyway. Otherwise, you are asking for trouble!!

Comparing Already-Loaded Assemblies
Unless this comparison is for binding, the path to the manifest file of this assembly is used for determining whether this is the same assembly as another one.

For example, path matters when determining whether a type is castable to another type. Even if the assemblies containing the types are identical, if they're loaded from different paths, they're considered different assemblies and therefore their types are different. This is one reason why using contexts other than the Load context is risky. You can get into situations where the same assembly is loaded multiple times in the same appdomain (once in the Load context, once in the LoadFrom context, and even several times in neither context), and their corresponding types won't be castable.

Some assemblies don't have paths associated with them, such as those loaded by Load(byte[]) or created by Reflection Emit. Those are always just considered different assemblies, even if all of their bytes are identical.

Unloading an Assembly

Suzanne Cook — Wed, 09 Jul 2003 05:11:00 GMT

There's no way to unload an individual assembly without unloading all of the appdomains containing it. (See here for why not.) This can by done by calling AppDomain.Unload() for each AppDomain that has it loaded. (You could also use UnloadDomain() on the v1 unmanaged hosting API, but in general, I recommend using managed code whenever possible.)

I should point out that this means that even if your managed Assembly/Module/etc. instances go out of scope, GC may clean those instances up. The actual file will remain loaded until you unload all of the appdomain(s) containing it, however. (In general, that's better, anyway. Loading a file, especially by http, is too expensive to be done over and over when it can just be cached.)

If you want to unload some assemblies but not others, consider creating a new AppDomain (requires ControlAppDomain permission), execute the code using the temporary ones from within that AppDomain, and then unload that new AppDomain when you're done. I recommend minimizing the number of AppDomains, though, since there is overhead involved in creating them, transitioning between them, and unloading them. So, it's best if you do all of that work at once, in one AppDomain, minimizing the cross-domain communication, and just unload that when possible (as opposed to creating a new AppDomain for each assembly to unload, etc.).

Gotchas
Watch out for references to that assembly leaking to other appdomains. If that happens, that assembly will remain loaded if you just unload one AppDomain. For example, passing a Type object to another appdomain will cause its assembly to be loaded there.

Calling Unwrap()/AppDomain.CreateInstanceAndUnwrap()/etc. may also cause the runtime to try to load the assembly in the calling appdomain. In that case, call CreateInstance*() on a MarshalByRefObject type from a different assembly. For example, using AppDomain.CreateInstanceFrom() on the calling assembly (the one currently calling Unwrap()) may be the most convenient way. Then, call a method on it which will do all of your assembly loading. That way, those assemblies can be loaded in just the target appdomain.

Also, note that assemblies loaded as domain neutral will not be unloaded until the process exits, since other appdomains are technically still using them.

Alternatives
But, maybe you don't need to load that Assembly, anyway. If you just need info about it, you could call AssemblyName.GetAssemblyName() with the path to the file. That will temporarily load the file to get the info and then immediately unload it. That call will not cause it to show up as an Assembly in the AppDomain.

Or, if you're only worried about keeping the file locked, you could use shadow copying. That will make a copy of the file on disk and load it from the new location. The original file will not be locked by that load. To do that, set AppDomainSetup.ShadowCopyFiles to "true" when creating the AppDomain or set AppDomain.ShadowCopyFiles to true after it's already been created.

Mscorlib.dll

Suzanne Cook — Tue, 01 Jul 2003 02:14:00 GMT

At least for v2 and earlier, mscorlib.dll is a special case. That causes it and its types to be loaded differently from other assemblies.

Loading Mscorlib.dll Without a Path
It and the execution engine are so closely integrated that it's required that they match each other. So, once a version of the CLR is chosen, the version of mscorlib is also chosen automatically. That means that, when late binding (Assembly.Load(), etc.) for it without a path, the 'official' mscorlib Assembly will always be returned - regardless of the version that was requested. The same thing goes for loading by static reference to it.

This doesn't mean that it's okay to give an incomplete or incorrect reference for it! If/when mscorlib.dll is factored away from the execution engine, it won't be such a special case and bad references will start to fail to load.

Reflecting On Another Mscorlib.dll
It's possible to load another mscorlib.dll if it's loaded by path. But, it's not possible to load its types, due to optimizations done by the CLR's execution engine. (Same goes for ReflectionOnlyLoadFrom() on another mscorlib.dll.) This means that even if you rename the mscorlib.dll that you want to reflect over, it may be loaded, but its types will still be unloadable. So, the options for working around this are:

Reflect on the 'official' mscorlib.dll. You could change the version of the CLR that your app loads in order to reflect on the desired mscorlib version.
Use the unmanaged metadata API to get the desired info.
Create an asmmeta file for the desired mscorlib.dll and parse it at runtime.

Assembly.CodeBase vs. Assembly.Location

Suzanne Cook — Fri, 27 Jun 2003 03:27:00 GMT

The CodeBase is a URL to the place where the file was found, while the Location is the path from where it was actually loaded. For example, if the assembly was downloaded from the internet, its CodeBase may start with "http://", but its Location may start with "C:\". If the file was shadow copied, the Location would be the path to the copy of the file in the shadow-copy dir.

It’s also good to know that the CodeBase is not guaranteed to be set for assemblies in the GAC. Location will always be set for assemblies loaded from disk, however.

Determining an Image’s CLR Version

Suzanne Cook — Sat, 21 Jun 2003 04:34:00 GMT

To get it programmatically, from managed code, use Assembly.ImageRuntimeVersion. From unmanaged, use mscoree.dll's GetFileVersion(). (From the command line, starting in v2.0, ildasm.exe will show it if you double-click on "MANIFEST" and look for "Metadata version".) Those will give you the CLR version that the image claims it wants. By default, it's the version that the image was compiled against. That's not necessarily what it will be, however, since compilers can be configured to put any string there.

Also, the CLR version that will be run by an exe is not necessarily the same as what is in its image runtime version. The chosen version can also depend on its hosting application's choice, a config file, environment variables, and registry settings. If you need to override those, you can set the supportedRuntime/requiredRuntime in the app.config for the process exe.