Thursday, October 30, 2003

CLR Panel Discussion

The following is the "transcript" I took of the "Designing the CLR" Panel discussion.  We had probably around 200 people in attendance, with an open mic forum.  The Panel was swamped with standing room only folks at the end (about 40 up around the stage with lot's of questions).  At the panel we had Brad Abrams (CLR BCL PM), Anders Hejlsberg (Distinguished Engineer, C# architect, Chris Brumme (CLR Architect), Patrick Dussud (lead CLR Architect), Jim Miller (CLR Architect), Sean Trowbridge (CLR Architect), George Bosworth (CLR Architect), and Jonathan Hawkins (CLR PM/Moderator).


Q:  How low can you take the CLR, eg: could it go down into the kernel.

Patrick:  Two versions of the CLR could be made, one with high availability which runs in kernel mode.  The other is the normal CLR as you know on top of that.  AppDomains could become the next version of a Process on the machine.


Q:  Is process recycling really required like

Chris:  We've done a ton of work on fixing internal mistakes and fault injection testing, etc, and we expect it to work given the Yukon work that is being done.


Q:  Have you considered how you can get OS VM and GC to communicate?

Patrick:  Lot of OS system services already there (eg: GetMemoryStatus) to auto tune the load on the machine.  If you are in 90% utilization, CLR GC will back off on allocation pressure.  In XP there is a new low memory notification service.  Finalizer thread in 1.1++ CLR uses this to also back down.  Future directions should include the CLR being able to get a "budget" from the OS, and then the OS could guarantee that without paging and CLR could auto tune within that limit.


Q:  Do you have dynamic code gen limitations that would allow things like writing code for an edit box, etc, like Smalltalk has.

George:  We are introducing lightweight codegen; not as rich as what was in Smalltalk, but the tool (aka VS) could help this.  No fundamental restrictions.

Jim:  Less than restrictions, the bigger issue is integrating every feature across the engine in the right way, such as including security,etc.

Anders/et al:  Facilities do exist for compiling up code on the fly, like Reflection Emit, etc.


Q:  Will ability to add fields via Edit & continue be allowed in managed emit code.

Sean:  Currently only allowed through unmanaged api's, where support exists.  We've talked about various options like this, but have not committed to any immediate plan.


Q:  What kind of safety features do you have around hosting multiple pieces of user code.

Jim:  Use app domains to provide CAS security around specific to each domain.

Brad:  Examples are like wsdl parsing and hosting.

Patrick:  Reflection Emit permission is highly privileged.


Q:  SyncBlock pointer is used for edit & continue, what are you doing with it?

Patrick:  SyncBlock is actually the "Kitchen Sink Block" <laughs>.  It's a general purpose field for lot's of things including locking algorithms, COM interface plumbing, etc.  Dynamically allocated as required to get going.  We optimize for lock and hash because they are the most common case.


C:  Thanks to Chris for an excellent blog <applause>

Q:  Transparent proxy work/interception, what about limitations?

Chris:  We intended for more than remoting, but haven't done a ton with it beyond that.  Interception is very interesting; lot's of options.  Could do code rewriting using prolog method hooks, etc.  Appealing because it is more performance, transparent, and less risky wrt being bypassed.  Conceptually sounds very simple, but digging through details is very complicated (eg: integrating with ngen is tough since everyone shares the same copy, and do you want to impact perf for everyone).

George:  Pay as you go is very important. Considering heavy inlining and how you would intercept that without a perf penalty.


Q:  AppDomains:  currently cannot unload assembly without unloading app domain.  We are dynamically generating assemblies, and do not want to unload app domains.  Why can't we unload assemblies from an app domain?

Sean:  Very common request.  We have to hit a higher bar for secure and reliable programming environment.  In managed, you have to make sure no one is still referencing an assembly, where if it were unloaded you could introduce AV's or security.  To solve requires a  ton of bookkeeping, and you wind up wrapping an assembly with an app domain <stack fault> .  Instead, we're looking at making app domains more performant to use.

Patrick:  There are other systems that can unload code.  However unloading is not free of semantic difficulties.  When you unload, you must make sure you have no dangling references.  You have to define a shut down policy, and that can introduce bad cycles on shutdown.  These kinds of bugs are very frequent.  I've implemented this before, and it is very problematic and not without bugs.  We have a good story on app domains.  If we can make it cheaper to get in/out of app domains, then this is the best container for code.

Jim:  I've worked on systems where we used GC to collect code.  You fix perf issues, but then you add more bookkeeping again.

Sean:  You still have the cycle problem.  If one person holds onto an instance of an assembly which holds onto the other (a->b, b->a), you never get to unload reliably.

Jim:  The system had to build stuff into the GC to fix this, "it wasn't a pretty site", not recommended.

George/Jim:  Security, performance, features.  Lot's of trade offs.


Q:  Why do I have to check if a delegate is null before I use it, why isn't there language support for this?

Anders:  Requested often, spent fair amount of time trying to figure out a ok way.  The suggestion doesn't work well for delegates which return values, etc.  Putting this kind thing "ages" the languages when you add kludges to help solve minor confusion.  In the end, decided to leave things be.  If we had a chance to do it over we might have done something differently, but solution suggested is no better than the problem.


Q:  Is there any plans to given an app domain a CPU budget?

Chris:  Common request in & out of company from folks writing server code.  Folks want quotas around CPU or memory consumption, etc.  Would be good to do something here, but instead of quotas, we may track resource usage instead.  We would accumulate your "charge account" of things you use, and then put the burden back on the sophisticated host app to make decisions.  That way they can apply their policy directly.  Another common request is trying to track memory used by a particular request (say web page compile, sproc, etc).  Very difficult thing to track.

Jim:  Also consider how you would interact with the OS scheduler.  The scheduler doesn't give you any hooks to integrate closer.  Would be good in the future to see if we could exploit this.


Q:  How much do you consider other platforms in the CLI design.

Jim:  when I first joined we were actively developing for multiple platforms.  Doing this kept us reasonably honest, and standards process helped a lot more.  We also have the .NET Compact Framework which can also helps.  There are other systems one could "possibly, possibly consider" <chuckles>.  We also released Rotor which runs on several other platforms and chips which also keeps us honest.  The Rotor team is now part of the CLR team, and those builds are part of our build process.

Patrick:  Example of tension:  in ecma we tried to get a very specific memory model, but we realized that nobody would code to it [jlz: advanced chips like ia64 allow for reordering of reads and writes and let you do scheduling as a code generator; x86 is pretty simple and less flexible, but simpler and easier], because it is really complicated and most people target x86 which has much simpler semantics.  It's an example where the issues are so sublte, that even though the architecture could have been more generic, we need it to be stable across x86, ia64, amd64, etc.  Don't test on x86 and deploy on ia64 and have it broken.

Jim:  show of hands: how many of you have volatile in the right place?  This is required for a reordering architecture, and people don't understand it.

Jim/Patrick:  Java has tried to be inclusive here, but found themselves in a quagmire; very hard to get right.  We decided to simplify instead.

Sean/Chris:  We try to keep complications under the hood for this kind of thing.


Q:  Any plans for things like aspect oriented programming and related techniques?

Chris:  We view this as something you could do with interception, if you wanted ot.

Anders:  Programming by contract is something we've looked at a lot.  MSR has done a lot of work here.  In the end we decided it wasn't quite cooked yet, and so we have abstained here.  If we did aything like this, the solution must allow me to remove up front if () checks for entry conditions.  To be truly viable, you need to have it in the CLR/CLS instead of just a language.  Must be through the libraries as well.  We're not there yet.

Anders:   wrt AOP, it seems ok for debugging and monitoring.  But the notion you can inject code anywhere makes it impossible to understand what your code will do.  Unless you have an IDE that can show you the "aspect weaving" you won't know what is happening.  At this point I'm in wait and see mode until someone can show large scale successes.

Jim:  <Plug for Rotor>  design by contract is one of the RFP projects that we funded.  We are looking at the results of that to see if it is ready for prime time.  Gregor is a close friend of mine and I reviewed it 8 years ago: there are two halfs to this problem:  (1) weaving, and (2) projecting.  As an engineer I need to look at all the code I will run.  I have a degree in MIT, and you don't just do engineering by combining a bunch of things and adding them up without understanding how it works.  If you look at the union as the requirement, you could have come up with a better solution. I think the community is veered in the wrong direction; neither I or Gregor have been able to get them off of this.


Q:  Do you provide any utilities around lock ordering and dead lock detections.

Chris:  prior to Whidbey, the locks you take are your responsibility.  There is no leveling notions or anything like that.  You get essentially a critical section and you can hang yourself with it.  In Whidbey, with integration with SQL Server, they wanted to do deadlock detection and understand the lock graphs [jlz: the hierarchy of locks where cycles can come from].  This is still rocket science, for most people you should still take responsibility for lock leveling.

Q2:  I'd like to do this in C# directly.

Anders:  You should take advantage of the using statement.  We may not have put the lock keyword in if using had proceeded it.


Q:  In new C++, determinsitc finalization is there.  What about other languages?

George:  they've provided scope destructors, not really deterministic behavior per se.  If the scope goes away, you'll get the dispose, but you can still make mistakes.  What people usually mean here is they want to solve the whole thing, and this doesn't do that.  It is still useful.  But it isn't the silver bullet.

Anders:  important to understand the syntax is really no more expressive than the using statement in C#.  The new pattern comes with some restrictions on patterns that using doesn't have.  For example, you have to declare vars in a separate scope instead of inline.  Agree with George that this is not deterministic finalization.

Patrick:  Absolute lifetime we looked at, and keep looking at it, but it is really, really expensive.  You have to do micro-management.  You have to project to all processors the state, which doesn't scale well.  There are tricks, but so far cost > benefit.  We don't believe in general people want to do micro-management, instead they want good enough guarantees that you don't leak and you want limit of population of some resources to a finite number [jlz: like a pool with a quota enforced].  HandleCollector is a good new example where we are providing this kind of functionality.  AddMemoryPressure is also another good example that allows you to help tune the GC (good for cases like allocating bitmaps, where managed side holder class is very small, but unmanaged memory hit of the resource could be huge).

Q2:  My problem is less about memory but the resources [jlz: like database handles, expensive files, etc].

George/Patrick:  but that is what Finalization is.

Jim:  the language provides this from using and this is the right way to solve this problem.


Q:  Love integration idea with the OS.  But how are you going to accomplish that in the long term?  Eg: wait for Moore's law to help, video card processing [jlz:  GPU's are incredibly good at bit vectors schemes].  Or even something like an IL machine ala Transmeta.

Patrick:  Our codegen is pretty good, and getting much better (eg: dynamic and static profiling).  The OS can use this, so we are on track and efficiency is not the key issue.  The real problem is right now we don't cover the entire range of services so you have to p/invoke.  Msft needs to provide a complete range of these api's so that don't need this and Win32 can go away.  My dream is after Longhorn you could do this, removing longhorn p/invoke statements.

Patrick:  We actuallly are thinking the opposite of IL machine.  I did chip design at TI for lisp machines.  The competitor was 6x smaller than ours to run the same applications.  You get better if we program to IL, and then let the JIT optimize for the specific hardware where you can take advantage of subtle advantages in the specific chip.  Several manufactures are serious about adding advanced features along these lines.  With our pluggable optimizing codegen, we can do great on the fly generation this way.

Jim:  I was the PhD advisor for the chief designer at Transmeta.  His general comment was using IL as it stands is not efficient.  We should look instead at something like mips where you look for common codegen patterns from compilers and make those go fast.  Example is optimizing interface dispatch code pattern.


Q:  Threading: do you have intent to write a scheduler in the CLR, for example, something that could work with fiber mode.

Chris:  In Whidbey we did a lot of work to integrate the CLR into SQL Server.  Prior to this we also schedule on pre-emptively OS scheduled threads.  But we knew we might change this, so System.Thread isn't semantically locked this way.  In Whidbey, a sophisticated host can integrate and target fiber code.  We support this.  With a small box, normal thread mode is fine.  If you have very large machines (eg: 32-way), then it might give you value.  The hosting api's do allow you to integrate your own scheduler around fibers just like SQL.  Having said all that, I encourage you to stay away from it <laughs>  The benefits are just getting smaller and it is very hard to use and do correctly.

Patrick:  the OS is making lot's of progress closing the gap here, and its value is becoming less with every new release.

Chris:  If we could get to the pure managed world Patrick was dreaming of, where we don't know when/if we could ever get there, that we will be able to smooth out the rough edges of this kind of programming style.  We could smooth async model and marshalling,etc.  It will be a long time before we could do this.


Q:  Extremely difficult to write a generic library that is exception safe.  Wouldn't it be nice if at compile time you verify code was exception safe.

Anders:  Commonly asked for, not specifically related to generics.  I don't know of a solution that doesn't simply exchange one set of problems for another set.  There is the Java solution where you can declare what you've thrown and require caller to handle or push out further.  There are some serious scalability problems with this model; you have to compute the closure which winds up making people go to catch(…) which is no good..  It also has the versioning problem, where you can't add new exceptions in future versions without breaking the already written clients.

Anders:  what would probably be better is an Fxcop style checking, and we could do a better job on documentation.

Q2:  I understand why you didn't do the Java solution; I don't like it either.  But couldn't you do something around generics because it is very difficult to do otherwise.

Anders:  I'm not sure I see the connection between generics and exceptions.


Q:  There are lot's of type systems now, do you see any chance of integration (eg: CLR & XML).

Anders:  There are differences and we keep whittling away the differences.  If you look at XAML you can extend with tags/attributes that map to class and language elements.  It's true that XSD dreams up extra stuff that doesn't map well.  It was design by committee effort so you see things that don't necessarily see use in real life.  We're working towards equilibrium here.


Q:  Any reason the Reflection API's didn't support reflecting on the IL?

Jim:  No reason at all, we ran out of time <laughs>.