Jim Hugunin's Thinking Dynamic

IronPython in Action

hugunin — Tue, 14 Apr 2009 20:39:30 GMT

I'm quite excited that IronPython in Action is published. I had the chance to write the foreword to this book and wanted to share it here as well.

Foreword to IronPython in Action

IronPython brings together two of my favorite things, the elegant Python programming language and the powerful .NET platform.

I've been a fan of the Python language for almost 15 years, ever since it was recommended to me by a fellow juggler while we passed clubs in a park. From the start I found Python to be a simple and elegant language that made it easy to express my ideas in code. I'm amazed by Python's ability to appeal to a broad range of developers from hard-core hackers to amateur programmers including scientists, doctors, and animators. I've been teaching my ten year old son to program and even he tells me that, "Python is a great language to learn with." Beyond teaching my son, I've tried to contribute back to the Python community that gave me this great language and continues to drive it forward. Prior to IronPython I started both the Numeric Python and Jython open source projects.

It took a quite bit longer for me to become a fan of Microsoft's .NET platform and the Common Language Runtime (CLR) that forms its core execution engine. I first learned about the CLR by reading countless reports on the web that said it was a terrible platform for dynamic languages in general and for Python in particular. IronPython started life as a series of quick prototypes to help me understand how this platform could be so bad. My plan was to prototype for a couple of weeks and then write a pithy paper titled, "Why the CLR is a terrible platform for dynamic languages". This plan was turned upside down when these prototypes turned out to run really great – generally quite a bit faster than the standard C-based Python implementation.

After getting over my initial skepticism, I've grown to love the CLR and .NET as much as Python. While no platform is perfect, this is the closest that we've ever come to a universal runtime that can cleanly support a wide variety of different programming languages. Even more exciting to me is that the team is committed to the multi-language story and we've got great projects like the DLR, IronRuby and F# to keep extending the range of languages that can coexist on this platform. I've even grown to like C# as by far the most enjoyable and versatile statically typed programming language I've used.

As the architect for IronPython, I like to believe that it's such a simple and elegant combination of the Python language and the .NET platform that it needs no documentation. After all, who could possibly not know that they should use clr.Reference to pass an out parameter to a .NET method. Well, I guess that it's assumptions like that one that make me a poor choice for writing a book teaching people about IronPython. The best choice for writing a book like this would be a long-term user who's deeply engaged with the community and who has been trying to understand and explain the system to others for years.

Now, if only we could find such a person…

I first met Michael Foord in July of 2006. I was preparing an IronPython talk for the OSCON conference in Portland, Oregon. This was going to be an exciting talk where I'd announce that the final release of IronPython-1.0 was weeks away. Of course, this was a terrible time to be preparing a talk since my mind and time were occupied with all the details of the actual release. To further complicate things for me, this was the Open Source Convention, and I knew that I needed to show IronPython running on Linux to have true credibility with this audience. Unfortunately, I didn't have the time to setup a Linux box and get some useful demos running. Oddly enough, I also found that my coworkers (at Microsoft) didn't have any spare Linux boxes running in their offices that I could borrow for a few screen shots.

I did a desperate internet search for "IronPython Linux" and one of the places that led me was a blog called voidspace. There I found a tutorial teaching how to use Windows Forms with IronPython. The reason this tutorial showed up was that it included screen caps of the samples running under both Windows and Linux. This was just what I was looking for! By stealing these pictures for my talk I could both show people IronPython running on Linux and also point them to an excellent online tutorial to help them learn far more about using IronPython than I could ever cover in a 45 minute talk.

I had a few hesitations about including this reference in my talk. I didn't know anything about the author except that his screen name was Fuzzyman and he had a personal blog that was subtitled, "the strange and deluded ramblings of a rather odd person." However, I really liked the simple tutorial and I was incredibly happy to have some nice Linux samples to show the OSCON crowd. I was most grateful at the time to this person that I'd never met for helping me out of this jam.

Fuzzyman turned out to be Michael Foord and one of the authors of the book you have in your hands now. Since that first online tutorial, Michael has been helping people learn to use IronPython through more online samples, presentations at conferences and through active contributions to the IronPython users mailing list. I couldn't think of anyone better to show people how to get started and how to get the most out of IronPython.

I've spent my career building programming languages and libraries targeted at other developers. This means that the software that I write is used directly by a small number of people and it's incredibly hard for me to explain to non-developers what I do. The only reason this kind of stuff has value is because of the useful or fun programs that other developers build using it. This book should give you everything you need to get started working with IronPython. I hope it will make your development more fun or at least more productive. Now go out and build something cool.

Dynamic Language Runtime Talk at PDC

hugunin — Wed, 29 Oct 2008 21:47:08 GMT

I’m at the Microsoft Professional Developer conference in sunny LA where I’ve given an overview talk on the DLR. This was the first talk I’ve given on the DLR where I was able to really dig into some of the technical details underneath and paint a hopefully coherent picture of how the key features fit together. Prior to my talk, Anders Hejlsberg delivered an amazing talk on the Future of C# which he shows for the first time how we’ve added dynamic typing to that language with a static type called ‘dynamic’ – yes, you read that right. This work is all built on top of the DLR, and I’d strongly recommend it as great viewing prior to my talk.

DLR Trees (Part 1)

hugunin — Tue, 15 May 2007 19:32:02 GMT

I'd like to take a detour from discussing the DLR's type system to start talking about implementation. I hope that this will make it easier to talk about some difficult questions like what is the right meta-model for the DLR type system. Let's start by compiling the traditional factorial function for a DLR-based language - using JavaScript for today.

function factorial(n) {
    if (n <= 1) {
        return 1;
    } else {
        return n * fact(n - 1);
    }
}

The JavaScript compiler will take this code and compile it into the following tree form and pass the tree to the DLR.

The key implementation trick in the DLR is using these kinds of trees to pass code around as data and to keep code in an easily analyzable and mutable form as long as possible. Anyone who's programmed in Lisp or Scheme knows this trick extremely well. Much more recently, this idea has resurfaced as one of the central features in C# 3 and VB.NET 9 that enable LINQ. Linq uses these "expression trees" to capture the semantics of a given block of code in a sufficiently abstract way that it can be optimized to run in different contexts. For example, the DLINQ SQL interfaces can convert that code into a SQL query that will be optimized by the database engine on the back-end. More radical projects like PLINQ are exploring how to take these same kinds of trees and rearrange the code to execute in parallel and on multiple cores when possible.

When we started building the DLR we knew that we wanted a shared tree form, but we believed that we wanted purely untyped and late-bound nodes in the tree that would always use dynamic actions for their operations. This matched what we had for IronPython where the only typed node in the tree was the ConstantExpression - which had the type of whatever its constant value was. Because our trees were untyped, we didn't believe that they shared much in common with the always strongly typed expression trees in C# 3 and we went about designing these trees as something completely independent.

As we started adding more language implementations, we found that it was difficult to capture different languages idiosyncratic concepts in the DLR tree. Examples of this range from Python's print statement to JavaScript's regular expression literals. We needed to support these language features, but we couldn't add explicit support for them into our trees. Well, clearly we could have added explicit support for them in the tree, but where would that have ended? Ultimately, if the DLR is going to be successful it will have to be general enough that language implementations can represent their own concepts clearly even if there is no direct DLR equivalent.

For Python's print statement, we knew that IronPython already compiled this into a call to a static method that handled all of the implementation details of printing in Python - including the dreaded softspace. We could support this by adding a static method call node to the DLR tree and by converting the Python print statement into a static method call to this helper method. Similarly, when we went to support the special Python constant for an ellipsis, "...", we knew that the existing IronPython implementation compiled this into a static field access of the singleton ellipsis instance. We could support this by adding a static field access node to the DLR tree and converting the Python ellipsis instance into one of these nodes.

Pretty soon we realized that the nodes that we were adding mapped directly onto the statically typed and bound expression tree nodes from C# 3. This was a good thing, since it meant that there was less for us to create from scratch and more opportunity for us to steal from existing ideas. The DLR trees today in theory include all of the expression tree nodes from C# 3 and add to them several additional kinds of nodes to handle dynamic operations, variables and name binding, and control flow. If you look at the code you'll notice that in fact we only support a subset of the C# 3 nodes and that our APIs don't exactly match the existing expression trees. This is simply an artifact of our late realization that these trees were so closely related and we're now in the process of reconciling our trees with the existing expression trees.

The value of having dynamic languages target this tree form is that we can perform lots of optimizations in the DLR layer on behalf of the language implementations. Some of these optimizations include variable storage where we can often optimize local variables as CLR local variables that can be JITed into machine registers - but where we can also detect explicit meta-programming such as Python's locals() function or JavaScript's arguments feature and fall back to slower and more explicit storage mechanisms such as tuples or object fields when needed. Similarly for the dynamic action nodes we can use different implementation techniques to optimize these important but potentially slow operations.

This post is already too long and it's been over a week since my last post. I'm going to finish up for today and save any more details about the optimizations we perform on these trees for the future. For those of you looking at code, the DLR trees can be found in Microsoft.Scripting.Ast.

The One True Object (Part 2)

hugunin — Sat, 05 May 2007 00:25:19 GMT

The core of the DLR's type system is based on passing messages to objects. This isn't exactly a new idea, but focuses on what has always been the intellectual core of object-oriented systems. This simple notion doesn't explicitly talk about types at all, but instead focuses on objects and messages. Every dynamic and static language has its own notion of what a type is - from C#'s static single-inheritance types to Python's dynamic multiple-inheritance types to JavaScript's prototypes. Trying to reconcile all of these different systems at the type level is ridiculously complicated - although maybe something to tackle for DLR v2. Despite their differences at the type level, all of these languages share tremendous commonalities if you view them at the object-based message-passing level.

In order to support a broad range of languages in this kind of a type system, we need to have a standard set of clearly defined messages that all objects can respond to. The set needs to be rich enough to capture all of the unique properties of the different languages that we're working with while at the same time be sufficiently common so that code written in different languages can work together. This is a balance that I'm sure we're going to need to adjust as we bring more languages to the DLR - but I think we have an excellent start and lots of success with our initial four languages. Here is our current set of operations:

[Get|Set|Delete]Member(name, case-sensitivity) - gets, sets or deletes a named member on an object
Call/CreateInstance(argument modifiers) - standard call or 'new' call to an object with arguments. Modifiers include parameter names, support for expanding argument list or keyword dictionaries and a marker for an argument representing an implicit this.
SimpleOperation(OperationKind enumeration) - catch-all for simple common operations like add and subtract as well as indexing, etc.
Convert(Type) - converts an object to a given static type if possible

Given this set of messages, we also need to have some mechanisms in place to allow objects of different types to respond to these messages. While the DLR lets developers treat objects uniformly whether they come from a static or a dynamic language, under the hood we need to use different mechanisms to implement this message passing for the two different cases.

Standard CLR static types

When an object is of a Type as defined by a statically typed language, the object's behavior is completely described by this Type. We use the runtime Type of the object to determine the correct behavior. For example, with the GetMember message, this will mean looking for a field, property or method with the given name on the object's Type and returning the approriate member for the object if found and otherwise throwing an exception.

The DLR will use all of the existing well known attributes specified in the Common Language Subset (CLS) in order to map members of a type onto a DLR operation. For example, we'll recognize add operator methods to use when responding to an add message. In addition, we will make standard objects behave as expected, for example, any delegate object will respond properly to a Call message.

In addition to these existing well-specified operators, the DLR adds some additional custom attributes that effectively extend the existing set of CLS operations to include support for overriding standard DLR messages. A good example of this is the static attribute that will let a type override member lookup to respond to a GetMember message. This lets a static type support name-based lookup when it is being called from a dynamic language. In our current DLR hosting work, we've found several excellent places to use this attribute. One good example is the FindControl method on page objects. With this attibute we can make working with pages feel more natural to a dynamic language:

page.FindControl("text1") --> page.text1

In fact, we use a slightly more complicated method to override the GetMember behavior of the ASP.NET page objects. We can't modify the actual types because we want to run on existing shipping versions of ASP.NET. This means that we need to extend these types externally. To do this we build on the new feature of extension methods that was added to C#3 and VB9 as part of the suite of features that support LINQ. Extension methods let developers define methods to be added to a type outside of the definition of that type itself. These extension methods can be imported like a standard namespace and will then behave as if they were originally defined on the types that they apply to.

The DLR uses a slightly extended version of extension methods. First, we add support for properties and operator methods. Second, we allow extension methods to be provided at other scopes than just the per-file scope used by C# and VB.NET. For example, we allow languages to choose to apply extension methods to a given type for all code written in that language. This is the way that we support the standard Python methods on core CLR types such as string so that Python programmers can call "s.title()" even though System.String has no method with that name. This same mechanism also allows us to support special Python members on objects such as "__class__" or "__getitem__" by modifying the view of the type from Python code - rather than modifying the type itself. This lets all of our languages that want to use an immutable string object share the same actual instances freely while using extension methods to customize the view of that type to be appropriate to each language.

Fully dynamic types

For types that are defined by a dynamic language, we provide a more dynamic way to respond these messages. These dynamic types will implement a special interface - IDynamicObject - that can provide customized handling for all dynamic operations. One of the key steps in implementing a dynamic language on the DLR is to implement this interface for the standard object type used by a given language. This means that the IronPython implementation implements this interface on Python-defined classes to support the correct dynamic behavior following Python's rules for multiple inheritance and method resolution order. Similarly, the DLR-based JavaScript implementation implements this interface on JavaScript functions/objects in order to implement the correct prototype-based inheritance for that language. Each language doesn't need to know anything about the details of the type system in the other languages, but just needs to know what messages to pass to those objects and counts on the other languages correctly implementing their side of the contract.

One very obvious thing missing from the current DLR is a standard default implementation of this interface. This would be useful to people who wanted to implement simple scripting languages on the DLR who weren't trying to precisely match the semantics of an existing language. This would also be useful for libraries that wanted to create a generic expando object to pass to consumers. I expect that we'll provide this in the future, but at the moment our attention remains focused on the cases where we need to capture the exact semantics of existing dynamic languages where the interface approach provides the needed flexibility.

Obvious things that we haven't gotten to yet

A few obvious questions at this point may be how the different languages can both use these standard messages and yet retain all of the details of their own unique semantics. The simple trick here is that while the messages are standard, each language has to decide which messages to pass for a given piece of source code. This gives sufficent flexibility - although I expect you'll want to hear more details about that. The second obvious question may be that the mechanisms as described above sound awfully slow with a lot of introspection and interface calls going on at runtime. This description tries to capture the semantics of message passing in the DLR - not the implementation. It's the DLR's job to make this run fast and we're quite confident that we do this well today and will keep doing better for many tomorrows, but there's a much bigger explanation owed here as well.

Note for those wanting to explore the source code

I'd still caution most people to wait a few months to really dive into these bits as our design is actively being refactored and documentation is still mostly non-existent. However, if you'd like to look at the code you can find the DLR bits in the IronPython 2.0alpha1 release under the Microsoft.Scripting and Microsoft.Scripting.Vestigial projects. In case it's not obvious, the one with "Vestigial" in its name isn't expected to be around for much longer. The set of standard messages described in this post are found in the Microsoft.Scripting.Actions namespace - where these "messages" are called "Actions". The interface IDynamicObject is currently misnamed. The current version of that interface will soon be removed and the oddly named interface IActionable will be renamed to IDynamicObject. For now, you should look at IActionable.

The One True Object (Part 1)

hugunin — Wed, 02 May 2007 22:20:00 GMT

I'm very excited by the level of interest that I'm seeing from folks who want to better understand what the DLR is all about. I'm also sorry that we don't have a fully documented detailed story for you today. If you want a detailed whitepaper and documented APIs, you're going to have to wait a while. If you're happy with source code and blog entries, then you can start messing about today. And if you just want to play with working code, download the silverlight bits and start exploring what you can do on top of the DLR with Python and JavaScript together today.

I'm starting my design notes blogging today with my first entry on the dynamic type system that is one of the three key components in the DLR - and the one that I think is most important. The corner-stone of the DLR is support for a shared dynamic type system. This lets these dynamic languages easily and naturally talk to each other and share code. Equally important is that we want these dynamic languages to be able to work with the existing powerful static languages on the platform such as VB.NET and C#. We want to ensure that the huge wealth of both existing and to-be-written libraries designed for .NET all just work from dynamic languages. There's a standard pattern for achieving this kind of interoperability through wrappers and marshaling layers. Here's the pattern as I implemented it for Jython - Python running on the JVM.

Notice that with this pattern that the Python types exist in their own little world (they are in orange) and for every underlying type I need to put the objects into a Python-specific wrapper. This standard pattern is okay if you're only interested in supporting a single language. In my example above, so long as I'm only writing Python code then all of my objects will be PyObjects and they'll all work great together with all the Python-specific information on them. Where this pattern really breaks down is when you want to support integration with multiple languages. In this case every time an object moves from one language to another it needs to be unwrapped from the source language and then rewrapped appropriately for the destination. This can obviously have performance issues as these wrapper objects are created and discarded for any cross-language calls.

The wrapper approach can also have deeper problems. One challenge is just to figure out what object to pass. For example, if Python has a PyString and it calls a C# function that expects an Object, should it pass the PyString or should it unwrap it into a String? These kinds of subtle type issues never have a good answer. Even worse are the nasty problems that can be caused by loss of object identity when objects are silently wrapped and unwrapped behind the programmers back.

This wrapper pattern is also the same one used by most of the popular dynamic languages implemented in C as well. When implementing a dynamic language in C, these kinds of wrappers make a lot of sense because a C pointer doesn't have any useful runtime type information so you need to decorate it with a layer that can provide the runtime type information that's needed. However, managed runtimes like the CLR provide rich type information for their standard objects, so it should be possible to use those objects directly without the confusion and cost of wrappers and marshalling. We exploit this in the DLR, and this is the type system that we actually use.

This means that all of the objects in the DLR are exactly the same as the objects in the CLR. We're adding capabilities to better support common dynamic operations, but we're still deeply rooted in the powerful existing libraries and languages on .NET. We need to have a single unified type system without marshaling and wrapper layers between our languages if we're going to have truly seamless integration between them.

Now that I've reached the end of this first entry, I fear that it might be a little unsatisfying. All that I've explained so far about the dynamic type system is that it uses exactly the same objects and metadata that are already used by the statically typed objects already running in the CLR. Well, I'm not going to let that stop me from pushing this to the web today. More details about how we get a dynamic type system into the CLR without wrapper layers will have come next.

First DLR talk video from MIX

hugunin — Wed, 02 May 2007 21:08:00 GMT

First, my apologies for not posting my first entry on the type system yesterday. I was completely wiped out after preparing and delivering this talk and then talking to all the interested people here at MIX. For those of you who aren't at MIX, you can see a video of John's and my talk.

Also, for those of you who want a more frequently updated blog from an expert on the DLR, be sure to read John's blog.

A Dynamic Language Runtime (DLR)

hugunin — Mon, 30 Apr 2007 19:21:00 GMT

Today, at MIX 07, we announced a new level of support for dynamic languages on .NET that we're calling the DLR.

From the beginning, Microsoft's .NET framework was designed to support a broad range of different programming languages on a Common Language Runtime (CLR). The CLR provides shared services to these languages ranging from a world-class GC and JIT to a sandboxed security model to tools integration for debugging and profiling. Sharing these features has two huge benefits for languages on the CLR. First, it's easier to implement a language because lots of difficult engineering work is already done for you. Second, and more importantly, these languages can seamlessly work together and share libraries and frameworks so that each language can build on the work of the others.

The CLR has good support for dynamic languages today. IronPython-1.0 demonstrates this. The new Dynamic Language Runtime (DLR) adds a small set of key features to the CLR to make it dramatically better. It adds to the platform a set of services designed explicitly for the needs of dynamic languages. These include a shared dynamic type system, standard hosting model and support to make it easy to generate fast dynamic code. With these additional features it becomes dramatically easier to build high-quality dynamic language implementations on .NET. More importantly, these features enable all of the dynamic languages which use the DLR to freely share code with other dynamic languages as well as with the existing powerful static languages on the platform such as VB.NET and C#.

The DLR is about giving you the best experience for your language - true to the language, excellent tools, performance and seamless integration with a wealth of libraries and platforms. The essential benefits of the DLR are about sharing. It lets language implementers share standard features rather than rebuilding them from scratch. This lets them focus on the features that make a given language unique rather than on reinventing yet another GC system. It lets developers share code regardless of the language the code is implemented in and to use whatever language they prefer regardless of the language preferred by the environment they want to run in. Coupled with the Silverlight 1.1 platform announced today, it even lets languages share a sandboxed security model and browser integration. This means that developers building browser-based applications can now use their preferred language even for client-side code.

We're initially building four languages on top of the DLR - Python, JavaScript (EcmaScript 3.0), Visual Basic and Ruby. We shipped today both Python and JavaScript as part of the Silverlight 1.1alpha1 release today. John Lam and I will be demoing all four languages, including VB and Ruby, working together during our talk tomorrow at 11:45.

In addition to the Silverlight release, we've also made the full source code for both IronPython and all of the new DLR platform code available on codeplex under the BSD-style Microsoft Permissive License. All of that code can be downloaded today as part of the IronPython project at codeplex.com/ironpython. If you want to know more about the DLR, you should feel free to download the code. However, you should understand that this is a very early release of these bits and we still have significant work left to do including refactoring, design changes, performance tuning - not to mention documentation.

For the short term, our focus is on using a small number of languages to drive the first wave of DLR development where we can work closely and face-to-face with the developers in order to iron out the worst kinks in the DLR design. After this initial phase, we want to reach out to the broader language community. If you're building a language on top of .NET and are interested in supporting dynamic language features then we want your feedback on the DLR. However, I'd discourage you from trying to implement on top of the DLR today. I don't want you to get frustrated trying to work with these really early bits and then not be interested in working with us when we're better prepared to engage with the language community. We plan to kick off a broader engagement with language implementers at the upcoming lang.net conference in three months - at the end of July. This will be the best place to really engage with the DLR and let us know what we got wrong.

In the meantime, I'll be using this blog to post our design notes for the DLR as they're written and any feedback you have on the design is welcomed. Tomorrow I'll talk more about the shared dynamic type system and the "One True Object".

IronPython 1.0 released today!

hugunin — Tue, 05 Sep 2006 23:01:00 GMT

I’m extremely happy to announce that we have released IronPython 1.0 today!

I started work on IronPython almost 3 years ago. My initial motivation for the project was to understand all of the reports that I read on the web claiming that the Common Language Runtime (CLR) was a terrible platform for Python and other dynamic languages. I was surprised to read these reports because I knew that the JVM was an acceptable platform for these languages. About 9 years ago I’d built an implementation of Python that ran on the JVM originally called JPython and later shortened to Jython. This implementation ran a little slower than the native C-based implementation of Python (CPython), but it was easily fast enough and stable enough for production use – testified to by the large number of Java projects that incorporate Jython today.

I wanted to understand how Microsoft could have screwed up so badly that the CLR was a worse platform for dynamic languages than the JVM. My plan was to take a couple of weeks to build a prototype implementation of Python on the CLR and then to use that work to write a short pithy article called, "Why the CLR is a terrible platform for dynamic languages". My plans quickly changed as I worked on the prototype, because I found that Python could run extremely well on the CLR – in many cases noticeably faster than the C-based implementation. For the standard pystone benchmark, IronPython on the CLR was about 1.7x faster than the C-based implementation.

The more time I spent working on IronPython and with the CLR, the more excited I became about its potential to finally deliver on the vision of a single common platform for a broad range of languages. At that same time, I was invited to come out to Microsoft to present IronPython and to talk with members of the CLR team about technical issues that I was running into. I had a great time that day working through these issues with a group of really smart people who all had a deep understanding of virtual machines and language implementation. After much reflection, I decided to join the CLR team at Microsoft where I could work with the platform to make it an even better target for dynamic languages and be able to have interesting technical discussions like that every day.

The first few months at Microsoft were a challenge as I learned what was involved in working at a large company. However, once the initial hurdle was over I started experiencing the things that motivated me to come here in the first place. The team working on dynamic languages in general and IronPython in particular began to grow and I got to have those great technical discussions again about both how to make IronPython as good as it could be and how to make the CLR an even better platform. We began to take advantage of the great new features for dynamic languages already shipping in .NET 2.0 such as DynamicMethods, blindingly fast delegates and a new generics system that was seamlessly integrated with the existing reflection infrastructure.

We were also able to release IronPython publicly from Microsoft with a BSD-style license. In the agile spirit of the project, we put out a new release of IronPython once every three weeks (on average) over the course of the project. This helped us connect well with our daring early adopters and receive and incorporate their feedback to make IronPython better. We've had countless excellent discussions on the mailing list on everything from supporting value types to calling overloaded methods. Without the drive and input of our users, IronPython would be a much weaker project.

IronPython is about bringing together two worlds. The key value in IronPython is that it is both a true implementation of Python and is seamlessly integrated with the .NET platform. Most features were easy and natural choices where the language and the platform fit together with almost no work. However, there were challenges from the obvious cases like exception type hierarchies to the somewhat esoteric challenges concerning different methods on strings. We spent long days and sometimes weeks looking for the best answers to these challenging problems and in the end I think that we have stayed true to both Python and .NET.

To drive our Python compatibility, we run a large portion of the standard Python regression test suite in addition to a large custom test suite we added that runs IronPython and CPython side-by-side to test for identical behavior whenever possible. Despite all of this work, there will still be differences between IronPython 1.0 and CPython. The most obvious difference is that IronPython is missing a number of standard C-based extension modules so things like "import bsddb" will fail. We maintain a detailed list of differences between the two implementations and aim to reduce the size of this list in every release.

IronPython has also striven for deep integration with the CLR. For the implementation this is a great thing as it lets us take advantage of highly-tuned components developed for other languages such as the just-in-time compiler, garbage collector, debugging support, reflection, dynamic loading and more. This integration is also valuable to IronPython developers as it lets them easily use any and all libraries built for .NET from their Python code.

This is the 1.0 release of IronPython. It's more complete and well tested than any other 1.0 product I have personally released in my career. However, like any other software product it's not perfect. You can search for known issues and let us know about any new ones that you find in our public bug database. We're continuing to work on IronPython and we want your input on how to make 1.1 and future releases even better.

It's been an exciting journey for me to see IronPython go from a rough prototype playing around with some ideas to a solid 1.0 release. This never could have happened without all the people who've contributed to this project along the way. My thanks go out to all the users who braved our early releases and passed along their problems and suggestions. My thanks also go out to the amazing group of people here at Microsoft who've come to join this project and drive it to this quality 1.0 release.

Shipping IronPython 1.0 isn't the end of the road, but rather the beginning. Not only will we continue to drive IronPython forward but we're also looking at the bigger picture to make all dynamic languages deeply integrated with the .NET platform and with technologies and products built on top of it. I'm excited about how far we've come, but even more excited by what the future holds!

Thanks - Jim Hugunin (for the IronPython Team)

We're hiring full-time and summer interns!

hugunin — Fri, 06 Jan 2006 00:11:00 GMT

We’re looking for a few exceptionally talented individuals with dynamic language experience (Python, Ruby, PHP, JavaScript, etc.) to come join our efforts to make the Common Language Runtime (CLR) the world’s best platform for dynamic languages and dynamic scenarios. The CLR already has a lot of dynamic support with reflection, runtime code generation, and cross-language interaction. IronPython has shown that the CLR can be a great platform for building dynamic languages. We want you to help us take this support to the next level.

We have one developer (not yet posted), one program manager, one tester and at least one summer intern positions available. If you’re interested, please send me email (jimhug@microsoft.com) that clearly explains why you’d be the best choice for one of these jobs and attach a current resume.

IronPython 1.0 beta 1 is out

hugunin — Wed, 04 Jan 2006 02:59:00 GMT

I'm at least the third blogger to announce the release of IronPython 1.0 beta 1 after Martin LaMonica and Jeremy Jones. Still, it's nice to be working on a piece of software that can be covered on onlamp, cnet and msdn. The original announcement was sent just before the end of 2005.

This release marks a major milestone in that we think we have workable answers to all the major design issues for a 1.0 final. Of course, we suspect that you'll have suggestions and issues we didn't anticipate, so we expect to see these "final" decisions change as we get feedback on the beta releases. Since IronPython ships every 2-3 weeks, the differences between beta 1 and the last 0.9.x release aren't huge. However, we did close on some fairly major design issues to finally reach 1.0beta.

We have a good solution for referencing .NET/CLI libraries and no longer modify the standard sys module
There's a new exception system for better Python compatibility and better .NET/CLI interoperability
We now take full advantage of the new DynamicMethods added in .NET 2.0 to dramatically improve exec and eval
We've made major strides in Python compatibility running a huge set of the standard regression test suite and no longer exposing any .NET/CLI methods to "pure Python" code that never references the CLI.