A first hand look from the .NET engineering teams
This is the first in a series of posts where we discuss the reasoning behind “obsoleting” specific APIs.
If you use AppDomain.AppendPrivatePath, or look at MSDN, you’ll notice it’s obsolete. This frustrates people because the alternative suggested (AppDomainSetup.PrivateBinPath) requires you to do something entirely different (spin up a new AppDomain). It also doesn’t shed any light on why the API is obsolete.
You can use this general rule to determine whether you should continue to call an obsolete method:
If a method is obsolete, stop using it!
Generally, the reason we obsolete methods is because we’ve identified something problematic about the method that isn’t fixable. In this case, we leave the method in place for compatibility. But remember, there’s still something wrong with it!
So, what’s so bad about AppendPrivatePath? The short answer is that it lets you get into situations that introduce load order dependencies. In other words, your app can become a time bomb that can crash under seemingly random circumstances. Load order can be affected by anything from local, machine specific factors (CPU speed, # of processor cores), to external factors like transient network latency. So, if you are dependent on load order, you can end up with “straightforward” problems like loading unexpected assemblies, or failing to load an assembly.
Consider the following scenario:
Now, an optimization is made by the JIT team, and bar.dll is now aggressively loaded due to some inlining that occurs in the JIT compiler. Now, the order of the 2 steps is reversed and bar.dll fails to load because c:\foo wasn’t yet added to the probing path.
In addition to the straightforward (easy to explain) problems, there are a number of unexpected (hard to explain) issues that can arise. Consider this scenario (same as above, but with step 3 added):
Now something occurs that disrupts the load order, and Assembly.LoadFrom occurs first (perhaps it’s called by a component that is loaded earlier due to user interaction). Can you guess the result?
The reference to “bar, Version=18.104.22.168, Culture=neutral, PublicKeyToken=asdfghjkl” will fail to load, despite the fact that it is available in the probing paths at the time the reference is resolved. Why does this happen?
When you do Assembly.LoadFrom, we attempt to do something called “context propagation”. Load contexts (covered on Suzanne Cook’s blog) are another mechanism meant to avoid load order dependencies (among other things). For LoadFrom, in order to determine whether an assembly gets propagated to the load context, we actually attempt a load. In this case, that load fails because the assembly is in “c:\foo”, which is not yet in the probing path, so the assembly is not propagated to the load context. By the time the reference is resolved, “c:\foo” is in the probing path, but we fail. Can you guess why?
We already tried the same load (during LoadFrom) and it failed, so it’s using the cache result of the previous bind!
So, hopefully you agree not to call this API anymore. If you’re looking for an alternative, here are the scenarios you might fall into:
Great, just use a config file to set the path or AppDomainSetup.PrivateBinPath as suggested.
You can use the assembly resolve event. Hook up a handler that probes wherever you want via Assembly.LoadFrom. These will be visible to the load context (because they originated from Load) and will keep you from tripping over loader context problems.
I hope this helps people who might otherwise use the obsolete method because they can’t figure out what else to do.
PingBack from http://www.marklio.com/marklio/PermaLink,guid,b00a172c-cf40-41b1-83b7-4a737648c224.aspx
Are there any alternatives for SetShadowCopyPath?
I need to be able to set this after an AppDomain has been created, because the ASP.NET runtime is creating the AppDomain on my behalf (via ApplicationHost.CreateApplicationHost).
I could just construct the ASP.NET AppDomain myself, but I think there's a large chance I'll get that wrong.
So, I decided to call SetShadowCopyPath even though it's obsolete. Reflector shows that ApplicationHost.CreateApplicationHost uses SetShadowCopyPath under the hood anyway!
How is this any different from PrivateBinPath? They would seem to suffer from the exact same problems.
I come to a different conclusion than this analysis. The current implementation still allows load order dependencies, and the removal of this API makes it more difficult to resolve them.
It would be useful for the CLR team to explore/explain the Load/LoadFrom contexts in detail. My analysis and experience with working with multiple AppDomains and dynamic loading is that these have caused more problems than they are worth (many pain points). Many of the load order dependencies discussed here are caused by having two different lists of loaded assemblies that are considered to be different identities, even if they resolve to the same binary.
There are scenarios where using the assembly resolve event does not work well. One problem is that assemblies loaded this way are almost always loaded using LoadFrom, which puts it into a different context than assemblies loaded using Load (or loaded transitively by the runtime). I've run into problems when a loader engine loads up an addin in a location that is not already part of the private bin path. To put it simply, this can result in the same assembly being loaded twice, once in each load context. This resulted in typecast exceptions. It is also difficult for a loader to do this for an addin running in a secondary AppDomain.
It is difficult to use the config file approach because that only works if you can create the config file before you create the AppDomain. This means that for secondary AppDomains you must know in advance all the private paths that must exist, and for the default AppDomain it is not possible modify this from managed code while it is running. I've used this approach for secondary AppDomains but the benefits are limited due to this requirement.
To work around these problems, to ensure consistency, and the remove load order dependencies, I've had to create the equivalent of a local GAC, which is enormously more complex and error prone than using the AppendPrivatePath API.
Consider some alternatives.
1. Eliminate the LoadFrom context. This was a good idea in theory that IMO turned out bad in practice.
2. Provide a mechanism for promoting an assembly from the LoadFrom context to the Load context.
3. Create a local GAC management class. This would allow applications to define their own common assemblies that are shared across multiple addins, with full versioning support, and still be xcopy deployable.
Expect SetShadowCopyPath to be covered in more detail in a future post. That said, what is the scenario that requires setting it in ASP.net?
PrivateBinPath is a member of AppDomainSetup. AppDomainSetup is provided to an AppDomain when it starts up, so its properties take effect early enough to eliminate the load order dependency.
Thanks for the feedback. A collegue of mine is working on a post regarding the loader contexts, so hopefully that will be a helpful post.
Loader contexts actually mitigate load order dependencies, rather than cause them. They do, however, introduce the class of issues you are referring to where types from the same binary are not considered equal because they were loaded differently. This is not the same as load order dependencies though.
I'll try to address the other good points you bring up below:
*One problem is that assemblies loaded this way (the resolve event) are almost always loaded using LoadFrom....
This is true. However, assemblies loaded via the AssemblyResolve event should be visible from the Load context, even if you loaded them via LoadFrom. Can you provide a repro of when this doesn't happen or causes problems?
*It is difficult to use the config file approach because that only works if you can create the config file before you create the AppDomain.
This is also true. You can also use the properties of AppDomainSetup, which makes this slightly easier. However, you have to know these ahead of time if you are to avoid load order problems. In v4, we've added a feature that makes it possible to modify the settings of the default domain via an AppDomainManager, but you still have to do this at AppDomain startup to avoid load order problems.
*To work around these problems... I've had to create the equivalent of a local GAC.
This sounds very interesting. I'd like to hear more about your solution, and the problems it solves for you. Hit me at markmil@(you know where).
As to your alternatives...
1. I agree. Contexts replace one set of problems for another. It will take a while to get there, unfortunately. App compat is a problem when innovating in this space.
2. The resolve event makes assemblies visible to the load context, and assemblies we would have found via Load are promoted to the Load context. A more general mechanism would be interesting and perhaps useful, but would be prone to load order issues (the promotion could race with other operations)
3. We've actually investigated this feature a few times. We do have a native hosting API today that gives you some of this functionality. SQLCLR uses it to provide the assemblies stored in the database. It's too complicated to be practical in most scenarios.
-mark (the above CLRTeam comments are from me as well)
The reason why I think the existence of two (or more) contexts increases load order dependencies is because user-defined code gets involved in the load decisions, and that increases inconsistencies of assembly loading, not decreases them. User code could return a different assembly for each call to the AssemblyResolve event (not likely but possible). As applications become more componentized and constructed from compositions of small building blocks, each unaware of other components, and each of which can implement their own AssemblyResolve handler, the potential for inconsistent loading is increasing.
The more that the runtime makes the load decision to more consistent the loading will be.
When you say that LoadFrom assemblies are visible to the Load context, are you referring to the LoadFrom assembly that is returned from the AssemblyResolve event?
My experience has been that the runtime will repeatedly issue a new AssemblyResolve event for a bind, even though a previous call to the AssemblyResolve event for the same bind had returned an assembly loaded using AssemblyLoad. I have theorized that it does this because:
1) Assemblies loaded into the LoadFrom context are fixed in that context
2) The runtime does not look at this context when attempting to perform an implicit bind, even though a previous bind attempt succeeded using an assembly from the LoadFrom context.
If there were a way to permanently make the association (in essence, promote the LoadFrom assembly to the Load context) than future implicit binds would be satisfied. This would increase the consistency as the same assembly would always get used for a bind. It would also be faster.
This all started for me years ago when we created an app using plugins, and we somehow managed to get the same assembly loaded twice into an AppDomain, resulting in typecast exceptions. It all boiled down to some binds being resolved through the default load context and others being resolved using LoadFrom.
Part of the difficulty of dealing with this is lack of documentation about the behavior of the runtime, and lack of tools and/or APIs to deal with this. Currently there is no way to determine which context an assembly lives in, let alone change it, but the behavior of the runtime is affected by this.
As to the local GAC, because I could not hook into the low-level fusion layer from managed code I was limited as to what I could do (I did not want to do this from native code using the low-level hooks). Fundamentally, the question was about how to isolate AddIns from each other so that private types remained private and shared data types all resolved to the same assembly. There is a mixture of assemblies, private, shared amongst addins, and shared between all addins and the host. Many types were intended to be shared regardless of version,and so were forced to use a single version, regardless of the version the addin was built against. In this case the latest version found was used. Backward compatibility was required.
The ability to dynamically add an assembly to the Load context even if the path was not in the original Bin Path would have been useful. The API AppendPrivateBinPath provided a mechanism to achieve this benefit. Once that was deprecated I tried to work around this.
I used a combination of hardcoded folder locations for shared assemblies, AppDomainSetup settings and AssemblyResolve handling. I created the equivalent of a private AppendPrivateBinPath for the AssemblyResolve handler so that when a new addin was loaded the loader could participate in loading dependencies and know where to look for them. In the end it was way too complicated for the benefits.
A local GAC, implemented by the CLR, would solve many of these problems, without the overhead of using the actual GAC, and would have provided valuable versioning support.
ASP.NET only shadow copies assemblies from BaseDirectory\bin*, but I need them to be copied from another location instead, specifically, the BaseDirectory itself. As far as I know, there isn't a way to tell ASP.NET to set up the AppDomain with alternative shadow copy paths.
If there is a way to do this safely, I'd love to know :)
FYI, this is the first method I call after ASP.NET creates the AppDomain for me. Can you comment on just how dangerous this is?
* Reflector shows HttpRuntime calling SetShadowCopyPath(BaseDirectory + "bin")
We don't call the user-defined resolve event for the same assembly twice in the same AppDomain (we've already found it). There are, of course, problems that people could get into by writing poor resolve events, just as they could write poor code anywhere in their app.
I am indeed referring to LoadFroms from the resolve event when I say they are visible from the Load context.
We should not issue the resolve event for the same assembly twice in the same AppDomain. Do you have a repro showing this happening?
I'm trying to discover why you need to do this at all. What are you trying to acheive? Why is what ASP.net does by default not meet your requirements?
As to the danger, I'd have to ask someone from ASP.net, since the danger would be in conflicting with something the ASP.net host is doing.
I have simliar problem as Jacob (I think). I have a 'plug in' type framework, if you will. So I have hundreds of clients and I don't want all those dll's in the main /bin directory. This is because each client is a seperate Url/IIS Application/AppDomain, however, all of them point at the same base directory and I can't have all client assemblies in /bin b/c if I update one client, it'd restart appdomain of every client.
So simliar to method described here: http://www.hanselman.com/blog/MovingTheCodeBehindAssembliesDLLsToADifferentFolderThanBINWithASPNET11.aspx I've set up probing to look in another directory.
/bin - core site code
/assemblies - client assemblies
/client1 - client files (aspx/ascx/etc)
/client2 - client files (aspx/ascx/etc)
As you might be guessing now, if I don't set new directory up for ShadowCopy, then I can never update the client dlls.
That is the reason I need to use this method still and in searching web, I'm not finding anyone offering advice except to 'keep using obsolete method'.
Any information would be great.
I have a situation where AppendPrivatePath seems to be the best way to solve my problem.
My application [WinForms app] periodically downloads new DLLs. Each "version" of DLLs goes in a new folder. The assembly names of the DLLs include the version number.
AppDirectory contains base level application files.
AppDirectory/v1/ contains downloaded version 1 files [X.1.dll, Y.1.dll, Z.1.dll].
AppDirectory/v2/ contains downloaded version 2 files [X.2.dll, Y.2.dll, Z.2.dll].
The assemblies are loaded dynamically into the application as needed. Most workflows use the most recent version of assemblies, but an older version can be used in some circumstances. Older versions can be downloaded or deleted as needed.
I used to load these assemblies using Assembly.LoadFrom, which worked fine. No need to resort to private bin paths at all.
Now, I want to load these DLLs using Assembly.Load. I don't have any concerns about ambiguous assembly resolution, because given an assembly name, it will always be located in the directory corresponding to its version.
I can't put the directories into the application configuration file, because the version directories aren't fixed. They potentially could change while the app is running.
I don't want to use AppDomainSetup.PrivateBinPath because creating an app domain each time I want to use these assemblies does not result in good performance.
If I use the AssemblyResolve event, I have to use Assembly.LoadFrom, which is what I'm trying to get away from.
AppDomain.AppendPrivatePath gives me exactly what I want. Before I load one of these assemblies, I ensure the directory has been appended to the private path. Then I run Assembly.Load and everything works perfectly.
I prefer the semplicity of using AppendPrivatePath() even if i understand about the risk, rather than get in troubles refactoryng a working application just for accomodate assembly resolution!
I will continue to use AppendPrivatePath(), that seem exactly what i need for my dynamic plugin system.
Semplicity win over complexity, even if the latter offer a greater level of safety in some uncommon scenario.
Just provide BOTH paths and inform correctly about potential issues. My application is running for years relying heavily on AppendPrivatePath() so i can guarantee that race conditions or loading order issues are not worrying me at all.
Please dont fix what ain't broken for me.