Put the Power Inside: Hosting the CLR in Your Application

Track: Panels   Code: PNL07

Room: Room501ABC   Time Slot: Thu, October 30 10:30 AM-12:00 PM

Speakers: Andrew Murchison, Christopher Brown, Christopher Brumme, Dmitry Robsman, Jose Blakeley, Mahesh Prakriya, Mark Alcazar, Sean Trowbridge, Thomas Quinn

Expose the power of managed code to your customers by using the CLR as your host for scripting and automation. Join experts from the CLR, SQL Server and "Longhorn" design teams to hear their experiences hosting the CLR in SQL Server "Yukon" and Windows "Longhorn." Pick their brains on how to enhance your own application's programming model by integrating the CLR languages including C#, Visual Basic .NET, and Managed C++.


This panel was for hosting the CLR in a process.  It includes folks from SQL Server, DB2, Asp.Net, VS.net, the Windows Shell, etc.  A lot of folks seemed simply confused about how to load managed code or how to write pieces of their application using managed (should use COM interop or MC++).  Other folks had some very good questions about how to do something similar to what the advanced hosts were doing:



Q:  Ximian (Miguel):  I work on an "alternative" platform, so for the "alternative" hoster [DB2], are you planning to target the "alternative" runtime? <laughs>

DB2:  Not yet, targeting MSFT for now.


Q:  What is the memory footprint of hosting to get started?

Dmitry:  Our experience [asp.net] is about 10 MB of overhead to start up empty.  Originally per app domain was pretty bad, but has improved significantly.  We can not host 100 applications in an app domain right now.

Sean:  Really depends on the code you are loading.  There is base infrastructure overhead on the first app domain.  Second is much chepar but then faults in remoting code, and then the third and subsequent get to incremental costing.


Q:  Can you give a rule of thumb about when to share assemblies between app domains.

Sean:  Trade offs:  for single domain code, it is faster for static addresses and perf domain init time burned in at JIT for perf improvmenets.  Multi-domain options goes through fatter code paths (eg: using TLS) which means indirection to touch your statics.  Artificats around loader implementation: dll and assembly generated for the code cannot be unloaded from the process, so you get sharing but may suffer working set.  We need to do more bookkeeping to fix this.  Final cost is that sharing tries to be smart so we do some analysis of the assembly to do good sharing and there is some processing cost to that decision.  We're working on all of these problems in Whidbey (it is more systematically used in Longhorn).


Q:  Hosted the 1.0 CLR, when 1.1 shipped we tried to use it but bad things happened.  What are best practices around this.

Thomas:  Always want to run the latest version since it can execute the older versions.

Jose:  One exception is SQLCLR where there are persistence issues where you want to lock down hard.

ChBrown:  We have an API that allows you to lock down specifically.

Cbrumme:  In the general case doing the latest makes sense.  But for highly coupled hosts with application compat issues, then locking down may make more sense.

DB2:  We also lock things down.


Q:  If you have fragments of user code which you want to expose, how can you do that?

Dmitry:  You can expose methods from the the user class and wire it into your host.


Q:  Is there any effort to unify the programming models that people use?

Panel:  In general this is about defining a unique hosting experience.  But libraries are interesting to write in a safe way.  Examples include not allowing calling TerminateProcess from SQL Server.


Q:  Do I need to use hosting in order to control memory usage from my managed host which is hosting unmanaged things.

Cbrumme:  Hosting is probably overkill to accomplish this.  Instead look at some of the memory pressure classes added for Whidbey to solve that classic example of the light weight managed wrapper for an expensive unmanaged object.

Q2:  What about synchronization?  I have situations where I have crit secs and reader/writer type synchronization.  Today I wrap them.

Cbrumme:  That's a reasonable approach but with some downsides.  You are using p/invoke to use the crit sec api's.  While you are blocked in unmanaged code, the runtime cannot take control of your thread.  So if someone was trying to unload the app domain we'd have to wait for unmanaged code to let you go.  That is an example where managed locks are superior.

Jose:  SQL Server is an unmanaged programming environment with very rich resource management.  So in our case, it makes a lot of sense for us to do this more complicated kind of work and we cooperate with the  CLR to do this.  But it is very complicated, so if you are talking about app level then you should keep it simple.


Q:  If I want to generate code and execute it, should I target hosting?

Sean:  Not necessary.  You can use Reflection Emit to generate code dynamically and execute it.  You may have some interesting issues around unloadability of the code. If you have very fine grained code gen, you should bundle more.

Cbrumme:  Alternatively you could take advantage of the new Whidbey lightweight code generation strategy.  You should also make sure to pay attention to the security of that code you are compiling and running.


Q:  We have unmanaged code but would like to add new features in managed.  Do I have to host?

Sean:  No.  You should approach this through the IJW or COM interop features to make this work.

Mark:  A good example is the Shell side bar.  It is unmanaged, but then does a CoCreateInstance (CCI) to instantiate the unmanaged code.

Cbrumme:  MC++ is a great solution.  But be careful if you have multiple app domains.  It is your responsibility to figure out which app domain you are using.  We're trying to make this better in whidbey.


Q:  What are the licensing issues around hosting the CLR?

Mahesh:  there is no license requirements currently.  For example, the DB2 product simply uses the normal 1.1 redist, so that applies.

Q2:  What kind of ISV's have contacted you?

Mahesh:  Rather not speculate, chat offline.


Q:  How can I CCI an unmanaged object and get it to appear in a non-default app domain?

Cbrumme:  All you should have to do is make sure the thread you do the CCI on is not in the default app domain.  A caveat is that COM threading models apply, so if it in turn marshals you to a different domain then you have to watch that.

Dmitry:  If you create your app domain yourself, then you have to track them.  You can expose a managed activation method in the right app domain to let it create the instance.

Cbrumme:  Generally we don't have a great answer here.  With the COM registration features, it is hard. It is something in the Longhorn timeframe we've been talking about ways to fix this.

Thomas:  We do the right thing for Office integration, but unfortunately VS.Net has not gone that far [jlz: this person was writing a managed control plug in to VS.Net].


Q:  Is Rotor hostable?  If not, how much work would it be to do that?

Mahesh:  Not currently.  We'd like your feedback on if that makes sense.


Q:  Do you have any guidelines on when to use MC++ vs COM interop?

Mark:  Shell team did a lot of looking at this, and with the high level of interop we chose MC++.  [jlz:  there is a published set of guidelines on exactly this topic on gotdotnet, looking for it now...].


Q:  Is there something like VBA for CLR that would make this hosting easy?

Mahesh:  The ~150 docs are very useful when you are trying to target a wide range of features.  You can use the subset.

Sean:  There are two levels to the API: unmanaged code does intrinsics like create app domain, then there are interfaces for callbacks that are optional.

ChBrown:  There is only about 1 page of code required to start the runtime <shows on screen>.  The rest of the doc is about very advanced callback customizations.  I'm also hearing a lot of folks asking about how to bridge between managed/unmanaged.  Hosting is overpowered for that.  You should look at COM interop and MC++.

Thomas:  There really isn't a great replacement for VBA in this world, but you could imagine something like out of the Office integration coming up, send me mail if you are interested.  VSA was our first attempt, but has been deprecated.


Q:  Asp.Net hosts and also has a hosting api.  Can you some examples of requirements or issues you ran into doing this?

Dmitry:  The hosting API has more to do with creating app domains for hosting user web applications.  We do things like Fusion shadow copying operations and web config work.  We have some Cassini [jlz:  feature of Whidbey Asp.Net which is a local only web server for testing your web apps without IIS on the machine] examples.

Cbrumme:  there is a new thing coming (not in PDC bits) callled DomainManager which can help with some of this.  There are issues to sort out and it is early, so stay tuned.


Q:  How will you make sure that Asp.Net will not break on new versions like the 1.0 -> 1.1 automatic upgrade you shipped?

Dmitry:  That was probably a mistake we made in v1.1 [jlz: caused compat errors for lot's of customers, this was actually done by my team on Asp.Net].  Our customers will be happier if we don't upgrade by default, so we will try to lock back in the future.


Q:  Can you ngen at the create stage?  Eg: does SQL Server do this?

Jose:  In general we use Fusion loader hook integration to deal with files that are loaded and where they come from.  In our case, we want to keep the user's data and their managed sproc code together so we store a copy in the database itself [jlz: allows you to xcopy a database with code to multiple machines, easier backup, etc].

DB2: Unlike SQL Sever we don't do the external lookup piece but we are looking at this. [jlz: presenter shows example on screen of DB2, which looks more like a marketing talk <g>].

Sean:  There is a performance issue on some of these api's.  The assemblies are loaded via a stream which is not as efficient as paging it off of disk [jlz:  you can get page sharing if the OS loader get's to do it, read my Ngen blog for details].


Q:  Do you see hosting as an interim plan for DB2 & SQL Server?  Or will you eventually move the whole engine to managed code?

Jose:  For SQL Server we are talking about millions of lines of highly tuned code, so I don't forsee us doing a rewriting in short or near term.  The interesting part for us doing managed code is that the database is a platform for writing apps, so we get better extensibility by allowing other people extend the engine.


Q:  How do you do the localization of managed code and resource work?

Thomas:  VS.net is a good example of a mixed application, and there are different formats and tools for doing the work.

Jose: the pool of managed expertise is increasing dramatically internally.


Q:  I have a managed host, should I rewirte to unmanaged hosting to host other components?

Cbrumme:  No, keep managed code and use the Domain Manager and other features to keep buiding [jlz:  this is what Asp.Net does, it is 95% C# with a small amount of isapi filter unmanaged code]. 


Q:  How much of the 150 page of hosting code is exposed in managed?

Sean:  Typically not, because we're trying to resolve the cycle issues that happen [jlz: for example, to run managed you start the engine, how would you decide which one to load with logic in the startup path?].  We do have parity coming on some of the simpler api's that are higher level.


Q:  How is CLR GC configurable?

Mahesh:  We don't offer many knobs today because the GC is highly self-tuning.  However we do offer some api's for system wide memory usage.  SQL Server can call us back based on global memory view (managed and unmanaged) and it can help decide when to collect.