Ryan Dawson's Web Log

Hosting IronPython 2.0 Alpha 6 via the DLR

rdawson — Fri, 30 Nov 2007 00:59:10 GMT

We previously discussed the hosting model of IronPython 1.1. This time around we'll see how that model was updated in the latest Alpha release by reviewing an updated version of the same example.

It should be noted that the model presented here is changing very soon. So why discuss it? Well, two reasons. The first is practical in that this is the model you'll deal with when using the latest binaries. But also, I think some of you might find the insight into our model's evolution interesting.

Here's our updated example:

using System;
using Microsoft.Scripting; //From Microsoft.Scripting.dll
using Microsoft.Scripting.Hosting; //From Microsoft.Scripting.dll
//You'll also need to reference IronPython.dll
//and IronPython.Modules.dll

namespace Hosting2 {
    class Host {
        public static void Main() {
            //Get the IronPython ScriptEngine for the current domain
            //and create a ScriptModule to execute in.
            ScriptDomainManager sdm = ScriptDomainManager.CurrentManager;
            LanguageProvider pyprov = sdm.GetLanguageProvider("python");
            ScriptEngine pe = pyprov.GetEngine();
            ScriptModule mod = sdm.CreateModule("mymod");


            //Expose host data to IronPython and vice versa
            mod.SetVariable("mystring", "This is a C# string");
            pe.Execute("print mystring", mod);

            pe.Execute("mystring = 'This is a Python string'", mod);
            Console.WriteLine(mod.LookupVariable("mystring"));


            //Expose a host method to IronPython and vice versa
            HostAPI myapi = new HostAPI();
            mod.SetVariable("api", myapi);
            pe.Execute("api.Method()", mod);

            pe.Execute(@"def func():
    print 'This is a Python function'
", mod);
            pe.Evaluate("func()", mod);


            //Use delegates over the same two methods
            HostAPI.Del hostdel = new HostAPI.Del(myapi.Method);
            mod.SetVariable("hdel", hostdel);
            pe.Execute("hdel()", mod);

            HostAPI.Del ipydel = pe.EvaluateAs<HostAPI.Del>("func",
                mod);
            ipydel();
        }
    }

    public class HostAPI {
        public delegate void Del();

        public void Method() {
            Console.WriteLine("This is a host method");
        }
    }
}

What's changed and why?

We've switched from using IronPython.Hosting to using Microsoft.Scripting.Hosting. Microsoft.Scripting is the primary namespace of the DLR, which 2.0 builds of IronPython are based on.

The DLR's hosting model is designed to support many different dynamic languages. So instead of simply constructing a PythonEngine as we did in 1.1, we ask the DLR to provide us with the ScriptEngine mapped to the language "python" in this AppDomain.

            ScriptDomainManager sdm = ScriptDomainManager.CurrentManager;
            LanguageProvider pyprov = sdm.GetLanguageProvider("python");
            ScriptEngine pe = pyprov.GetEngine();

In the process we've introduced two entirely new types: ScriptDomainManager and LanguageProvider. Every AppDomain in a DLR host has exactly one ScriptDomainManager. Among other things, ScriptDomainManager tracks which languages are available on the system. In this case we've asked the ScriptDomainManager to give us the LanguageProvider mapping to Python. If we had an appropriate version of IronRuby installed, we could ask for Ruby in exactly the same manner. We use the LanguageProvider to give us language-specific services including the ScriptEngine.

For an execution scope, we're using a ScriptModule instead of the EngineModule type we used in 1.1. Accessing the contents of a ScriptModule is a little different but shouldn't be too jarring.

Lastly, in 1.1 we were able to create a delegate to IronPython code by calling PythonEngine.CreateMethod with a string of code to wrap in a delegate. That method no longer exists in 2.0. But we can still get a delegate to an IronPython function by evaluating a function as our target delegate.

            HostAPI.Del ipydel = pe.EvaluateAs<HostAPI.Del>("func",
                mod);
            ipydel();

And actually, this same code snippet works in 1.1 as well.

But you could have done...

Some of you may note that there are several ways I could've written this example for 2.0 Alpha 6, and many of them are simpler than the one I chose. For instance, this one line of code:

            ScriptEngine pe = IronPython.Hosting.PythonEngine.CurrentEngine;

Is equivalent to these three lines:

            ScriptDomainManager sdm = ScriptDomainManager.CurrentManager;
            LanguageProvider pyprov = sdm.GetLanguageProvider("python");
            ScriptEngine pe = pyprov.GetEngine();

I chose the latter to provide a more language independent view of the API, which is a central driving force behind our redesigns.

Furthermore, for this very simple example we could've used the Microsoft.Scripting.Script class. Consider the following complete program:

using Microsoft.Scripting;

namespace Hosting2 {
    class Host {
        public static void Main() {
            Script.SetVariable("mystring", "This is a C# string");
            Script.Execute("python", "print mystring");
        }
    }
}

The Script class is intended as an entry level hosting introduction and doesn't allow things like executing in a non-default ScriptModule.

Next...

In this post we've looked at the hosting model of 2.0 Alpha 6 and some ways it differs from the model of 1.1. Next time we'll look at the (hopefully) final hosting model for the DLR and talk more about the design evolution.

Hello World

rdawson — Wed, 28 Nov 2007 23:29:15 GMT

Hello everyone. Welcome to my blog. The focus of this blog will be dynamic languages at Microsoft and specifically the Dynamic Language Runtime, IronPython, and IronRuby. I'll be talking about the design of various features as they take shape and how we're testing everything.

First, a little about me. I'm new to blogging (be gentle), but not new to MS where I've been a "Software Design Engineer in Test", SDET, for nearly 9 years. The first 7 of those years were spent on the Common Language Runtime, CLR, team where I worked on a variety of features over time including MS-IL, P/Invoke, and delegates.

Presently I'm a member of the Dynamic Language Runtime, DLR, team working to improve support for dynamic languages in .NET as well as to provide first-class language implementations of Ruby and Python.

Hosting IronPython 1.1

rdawson — Wed, 28 Nov 2007 23:29:00 GMT

A recurring topic on the IronPython mailing list is the subject of hosting IPy in another application. Part of the reason the topic comes up frequently is that we keep changing the design (we're almost done doing that...we hope).

I thought it might be helpful to discuss where our hosting design has been, where it is, and where it's going within in the context of one consistent example. To that end this is the first in a series of three posts that will focus on one host showing the following basic concepts:

Executing multiple statements of IronPython code in a single scope
Exposing host data to IronPython, and vice versa
Calling a host method in IronPython, and vice versa
Using delegates

I'll start by discussing IronPython 1.1 in this entry, current 2.0 Alphas in the next, and finish with the model coming in future 2.0 Alphas (likely >7).

IronPython 1.1 is the latest production version. Released back in April 2007, this is the last version of IronPython not to be based on the Dynamic Language Runtime. Meaning, this hosting model was designed primarily to support the hosting of IronPython only, and there is no Microsoft.Scripting.dll.

Here's our example host in C#:

using System;
using IronPython.Hosting; //From IronPython.dll

namespace Hosting1 {
    class Host {
        public static void Main() {
            //Create a PythonEngine to execute code
            //and an EngineModule to execute in.
            PythonEngine pe = new PythonEngine();
            EngineModule mod = pe.CreateModule("mymod", true);


            //Expose host data to IronPython and vice versa
            mod.Globals["mystring"] = "This is a C# string";
            pe.Execute("print mystring", mod);

            pe.Execute("mystring = 'This is a Python string'", mod);
            Console.WriteLine(mod.Globals["mystring"]);


            //Expose a host method to IronPython and vice versa
            HostAPI myapi = new HostAPI();
            mod.Globals["api"] = myapi;
            pe.Execute("api.Method()", mod);

            pe.Execute(@"def func():
    print 'This is a Python function'
", mod);
            pe.Evaluate("func()", mod);


            //Use delegates over the same two methods
            HostAPI.Del hostdel = new HostAPI.Del(myapi.Method);
            mod.Globals["hdel"] = hostdel;
            pe.Execute("hdel()", mod);

            HostAPI.Del ipydel = pe.CreateMethod<HostAPI.Del>("func()",
                mod);
            ipydel();
        }
    }

    public class HostAPI {
        public delegate void Del();

        public void Method() {
            Console.WriteLine("This is a host method");
        }
    }
}

PythonEngine and EngineModule

Now let's dig a little deeper. We've used only two types from the hosting APIs: PythonEngine, and EngineModule.

            PythonEngine pe = new PythonEngine();
            EngineModule mod = pe.CreateModule("mymod", true);

The PythonEngine is the primary entry point for code execution. The EngineModule is a scope in which code gets executed. PythonEngine exposes a DefaultModule, which you can use instead of creating your own EngineModule. You will notice that PythonEngine.Execute and PythonEngine.Evaluate methods have overloads that do not take an EngineModule. These overloads work against the DefaultModule.

EngineModule exposes a dictionary of defined names via its Globals property, which both host and scripts have read/write access to. The host is free to insert any .NET type into the dictionary, which then exposes that type to IronPython script code.

            mod.Globals["api"] = myapi;

Delegates

The last major concept here is the usage of delegates. .NET delegates are directly callable from IronPython. No special magic is required on either the host or script's part.

            HostAPI.Del hostdel = new HostAPI.Del(myapi.Method);
            mod.Globals["hdel"] = hostdel;
            pe.Execute("hdel()", mod);

In the reverse, you can get a delegate to IronPython code in 1.1 via the PythonEngine.CreateMethod APIs. Sometimes this is more convenient than calling PythonEngine.Evaluate or PythonEngine.Execute.

            HostAPI.Del ipydel = pe.CreateMethod<HostAPI.Del>("func()",
                mod);
            ipydel();

Next...

In my next entry I'll take this same example, re-write it for IronPython 2.0 Alpha 6, and discuss how and why things changed.

In the meantime, if you'd like to read more about hosting IronPython, the Cookbook has some good examples. And if there are any hosting concepts I haven't touched on that you wish I would, please let me know.