Jim Hugunin's Thinking Dynamic : DLR

The One True Object (Part 1)

hugunin — Wed, 02 May 2007 22:20:00 GMT

I'm very excited by the level of interest that I'm seeing from folks who want to better understand what the DLR is all about. I'm also sorry that we don't have a fully documented detailed story for you today. If you want a detailed whitepaper and documented APIs, you're going to have to wait a while. If you're happy with source code and blog entries, then you can start messing about today. And if you just want to play with working code, download the silverlight bits and start exploring what you can do on top of the DLR with Python and JavaScript together today.

I'm starting my design notes blogging today with my first entry on the dynamic type system that is one of the three key components in the DLR - and the one that I think is most important. The corner-stone of the DLR is support for a shared dynamic type system. This lets these dynamic languages easily and naturally talk to each other and share code. Equally important is that we want these dynamic languages to be able to work with the existing powerful static languages on the platform such as VB.NET and C#. We want to ensure that the huge wealth of both existing and to-be-written libraries designed for .NET all just work from dynamic languages. There's a standard pattern for achieving this kind of interoperability through wrappers and marshaling layers. Here's the pattern as I implemented it for Jython - Python running on the JVM.

Notice that with this pattern that the Python types exist in their own little world (they are in orange) and for every underlying type I need to put the objects into a Python-specific wrapper. This standard pattern is okay if you're only interested in supporting a single language. In my example above, so long as I'm only writing Python code then all of my objects will be PyObjects and they'll all work great together with all the Python-specific information on them. Where this pattern really breaks down is when you want to support integration with multiple languages. In this case every time an object moves from one language to another it needs to be unwrapped from the source language and then rewrapped appropriately for the destination. This can obviously have performance issues as these wrapper objects are created and discarded for any cross-language calls.

The wrapper approach can also have deeper problems. One challenge is just to figure out what object to pass. For example, if Python has a PyString and it calls a C# function that expects an Object, should it pass the PyString or should it unwrap it into a String? These kinds of subtle type issues never have a good answer. Even worse are the nasty problems that can be caused by loss of object identity when objects are silently wrapped and unwrapped behind the programmers back.

This wrapper pattern is also the same one used by most of the popular dynamic languages implemented in C as well. When implementing a dynamic language in C, these kinds of wrappers make a lot of sense because a C pointer doesn't have any useful runtime type information so you need to decorate it with a layer that can provide the runtime type information that's needed. However, managed runtimes like the CLR provide rich type information for their standard objects, so it should be possible to use those objects directly without the confusion and cost of wrappers and marshalling. We exploit this in the DLR, and this is the type system that we actually use.

This means that all of the objects in the DLR are exactly the same as the objects in the CLR. We're adding capabilities to better support common dynamic operations, but we're still deeply rooted in the powerful existing libraries and languages on .NET. We need to have a single unified type system without marshaling and wrapper layers between our languages if we're going to have truly seamless integration between them.

Now that I've reached the end of this first entry, I fear that it might be a little unsatisfying. All that I've explained so far about the dynamic type system is that it uses exactly the same objects and metadata that are already used by the statically typed objects already running in the CLR. Well, I'm not going to let that stop me from pushing this to the web today. More details about how we get a dynamic type system into the CLR without wrapper layers will have come next.

First DLR talk video from MIX

hugunin — Wed, 02 May 2007 21:08:00 GMT

First, my apologies for not posting my first entry on the type system yesterday. I was completely wiped out after preparing and delivering this talk and then talking to all the interested people here at MIX. For those of you who aren't at MIX, you can see a video of John's and my talk.

Also, for those of you who want a more frequently updated blog from an expert on the DLR, be sure to read John's blog.

A Dynamic Language Runtime (DLR)

hugunin — Mon, 30 Apr 2007 19:21:00 GMT

Today, at MIX 07, we announced a new level of support for dynamic languages on .NET that we're calling the DLR.

From the beginning, Microsoft's .NET framework was designed to support a broad range of different programming languages on a Common Language Runtime (CLR). The CLR provides shared services to these languages ranging from a world-class GC and JIT to a sandboxed security model to tools integration for debugging and profiling. Sharing these features has two huge benefits for languages on the CLR. First, it's easier to implement a language because lots of difficult engineering work is already done for you. Second, and more importantly, these languages can seamlessly work together and share libraries and frameworks so that each language can build on the work of the others.

The CLR has good support for dynamic languages today. IronPython-1.0 demonstrates this. The new Dynamic Language Runtime (DLR) adds a small set of key features to the CLR to make it dramatically better. It adds to the platform a set of services designed explicitly for the needs of dynamic languages. These include a shared dynamic type system, standard hosting model and support to make it easy to generate fast dynamic code. With these additional features it becomes dramatically easier to build high-quality dynamic language implementations on .NET. More importantly, these features enable all of the dynamic languages which use the DLR to freely share code with other dynamic languages as well as with the existing powerful static languages on the platform such as VB.NET and C#.

The DLR is about giving you the best experience for your language - true to the language, excellent tools, performance and seamless integration with a wealth of libraries and platforms. The essential benefits of the DLR are about sharing. It lets language implementers share standard features rather than rebuilding them from scratch. This lets them focus on the features that make a given language unique rather than on reinventing yet another GC system. It lets developers share code regardless of the language the code is implemented in and to use whatever language they prefer regardless of the language preferred by the environment they want to run in. Coupled with the Silverlight 1.1 platform announced today, it even lets languages share a sandboxed security model and browser integration. This means that developers building browser-based applications can now use their preferred language even for client-side code.

We're initially building four languages on top of the DLR - Python, JavaScript (EcmaScript 3.0), Visual Basic and Ruby. We shipped today both Python and JavaScript as part of the Silverlight 1.1alpha1 release today. John Lam and I will be demoing all four languages, including VB and Ruby, working together during our talk tomorrow at 11:45.

In addition to the Silverlight release, we've also made the full source code for both IronPython and all of the new DLR platform code available on codeplex under the BSD-style Microsoft Permissive License. All of that code can be downloaded today as part of the IronPython project at codeplex.com/ironpython. If you want to know more about the DLR, you should feel free to download the code. However, you should understand that this is a very early release of these bits and we still have significant work left to do including refactoring, design changes, performance tuning - not to mention documentation.

For the short term, our focus is on using a small number of languages to drive the first wave of DLR development where we can work closely and face-to-face with the developers in order to iron out the worst kinks in the DLR design. After this initial phase, we want to reach out to the broader language community. If you're building a language on top of .NET and are interested in supporting dynamic language features then we want your feedback on the DLR. However, I'd discourage you from trying to implement on top of the DLR today. I don't want you to get frustrated trying to work with these really early bits and then not be interested in working with us when we're better prepared to engage with the language community. We plan to kick off a broader engagement with language implementers at the upcoming lang.net conference in three months - at the end of July. This will be the best place to really engage with the DLR and let us know what we got wrong.

In the meantime, I'll be using this blog to post our design notes for the DLR as they're written and any feedback you have on the design is welcomed. Tomorrow I'll talk more about the shared dynamic type system and the "One True Object".