DLR Hosting and related stuff... : DLR

Codeplex.com\dlr is now setup to receive regular source updates

seshadripv — Fri, 05 Dec 2008 02:42:00 GMT

In my previous blog post I had mentioned that we were in the process of setting up regular automatic source pushes to the codeplex DLR project. I am happy to announce that work is now complete and our DLR sources are now available here. Going forward, the sources from our internal main source control branch will be available on a regular basis at codeplex.

We haven't yet decided on the update frequency and should finalize this in a day or two. Currently, we are considering either weekly or daily source pushes.

DLR v0.9 beta released in CodePlex

seshadripv — Thu, 27 Nov 2008 05:16:00 GMT

We are happy to announce the release of the DLR Project in Codeplex. This beta of the v0.9 of the DLR consists of a source and binary release. This excellent release note by our PM Bill Chiles gives a nice short overview of what's in this release. We plan to release the RTW version of the 0.9 shortly in sync with IronPython v2.0.

Going forward, we will update this project regularly with binaries, weekly source drops and related documents.

Please try this release and let us know your feedback. You can contact me directly through comments or email or send an email to dlr@microsoft.com with any questions related to the DLR.

P.S : The codeplex.com site also lets you subscribe and follow the discussions on this project.

Using LINQ and the DLR in the same application

seshadripv — Sat, 20 Sep 2008 05:36:00 GMT

The recent beta release of the DLR allows the usage of LINQ and other CSharp 3.0 features in an application that hosts the DLR. Harry, our IronPython PM has a detailed post explaining why this was not possible before and whats changed now.

DLR Hosting Sample : Simple dlr host using the new App.Config based ScriptRuntime creation

seshadripv — Thu, 11 Sep 2008 20:58:00 GMT

Due to the recent changes to the DLR Hosting API, dlr hosts now have to use the App.Config file to configure the languages they intend to use in the application. It’s probably a little bit more complex than before to get started on Hosting the DLR after this change.

The attached self contained sample(for VS 2008) addresses this and tries to make it easier to ‘get started’. The sample is a very simple console application that -

1) Hosts the dlr

2) Uses the ScriptRuntimeSetup.ReadConfiguration() and the ScriptRuntime constructor to create the runtime based on the entries in the App.Config file

3) Creates 2 language engines – one for IronPython and 1 for IronRuby

4) Creates a variable in IronPython and prints the value in C#.

The sample contains a pre configured App.Config file. You should be able to just extract the files locally and build the sample and execute it. The dlr and language binaries are also included (in the ‘lib’ folder). Note that these post IronPython V2 beta4 binaries are for demo purposes only and come with no warranty or support.

4 different ways to execute IronPython code using the DLR Hosting API

seshadripv — Tue, 29 Jul 2008 06:55:00 GMT

The Hosting API offers 4 different sets of methods to execute a script. These methods provide hosts - managed application in C#, VB and other .NET languages - with powerful, convenient and versatile ways to execute scripts from the DLR languages like IronPython, IronRuby etc. ( Though the title mentions just IronPython, this post applies to others like IronRuby, ToyScript etc as well)

I have put together a short summary of these methods and the types that define them. The descriptions found here are shortened versions of the descriptions found in the Hosting API spec. Please refer to the same for more specific details about the type and the method.

The types that define these method families are ScriptRuntime, ScriptScope, ScriptSource and CompiledCode.

ScriptRuntime

This class is the starting point for hosting. ScriptRuntime represents global script state. This includes referenced assemblies, a "global object" (ScriptRuntime.Globals), "published" scopes (scopes bound to a name on Globals), available language engines, etc.

public ScriptScope ExecuteFile(string path);

This method executes the source identified in the path argument and returns the new ScriptScope in which the source executed. ExecuteFile determines the language engine to use from the path's extension and the ScriptRuntime's configuration, comparing extensions case-insensitively.

The path is an absolute pathname or a filename that naturally resolves with standard .NET BCL file open calls.

ScriptScope

This class represents a namespace essentially. Hosts can bind variable names in ScriptScopes, fetch variable values, etc. Hosts can execute code within scopes for distinct name bindings.

public object Execute(string code);

public T Execute<T>(string code);

These methods return objects that result from executing code in the string argument.

ScriptSource

ScriptSource represents source code and offers a variety of ways to execute or compile the source. You can get ScriptSources from factory methods on ScriptEngine, and ScriptSources are tied to the engine that created them.

public object Execute(ScriptScope scope);

public ObjectHandle ExecuteAndWrap(ScriptScope scope);

public T Execute<T>(ScriptScope scope);

public int ExecuteProgram();

These methods execute the source code and return the result. All but one method executes the code in the specified ScriptScope.

CompiledCode

CompiledCode represents code that has been compiled to execute repeatedly without having to compile it each time, and it represents the default ScriptScope the code runs in. You can also execute the code in any ScriptScope if you need it to execute in a clean scope each time, or you want to accumulate side effects from the code in another scope.

public object Execute() { }

public object Execute(ScriptScope scope) { }

public ObjectHandle ExecuteAndWrap() { }

public ObjectHandle ExecuteAndWrap(ScriptScope scope) { }

public T Execute<T>() { }

public T Execute<T>(ScriptScope scope) { }

Unlike the ScriptSource’s execute methods, these do not compile the code again.

A few of these methods can be found in the samples from the earlier blog posts. For more details and documentation about these methods and types please refer to the Hosting API spec.

IronEditor - Editor for IronPython, IronRuby and other DLR languages

seshadripv — Wed, 23 Jul 2008 04:06:00 GMT

Ben Hall has released IronEditor - an IDE for the DLR languages like IronPython, IronRuby etc as a CodePlex project. Here are the links to the mail threads in the lists where he announced the release - IronPython & IronRuby. You will find some interesting followup comments and conversation about the editor in these threads.

Apart from being a useful editor for the DLR languages, this application is also a practical demonstration of using the DLR Hosting API in a managed application.

DLR Hosting Sample : QikScript - Handle Visual Studio 2008 extensibility events in IronPython

seshadripv — Fri, 18 Jul 2008 00:01:00 GMT

The DLR Hosting API offers a well defined interface for managed applications to host and utilize the power of the DLR (Dynamic Language Runtime) to enable end user scripting. The objective of this sample (and the forthcoming series of posts based on this sample) is to demonstrate and explain the usage of the API in doing those things.

QikScript is a Visual Studio addin that hosts the DLR enabling end users to write python scripts to handle events fired by Visual Studio.

As part of its extensibility model, Visual Studio defines events that can be subscribed to through an add-in. Add-ins can also access ‘live’ objects like the project system, editor, menus, output window etc in the hosting instance of visual studio. By Hosting the DLR and routing the events appropriately, QikScript lets IronPython scripts to handle these events and access VS objects at runtime.

How to Install

1) Extract the contents of the attached zip file to a folder on disk( say C:\QikScript)

2) Copy the ‘QikScript.AddIn’ file to the VS 2008 AddIns folder. is the Addin folder is present in C:\Documents and Settings\<user name>\My Documents\Visual Studio 2008\Addins or C:\users\<user name>\My Documents\Visual Studio 2008\Addins

3) Open the AddIn file in a text or XML editor. Edit the value of the <Assembly> tag to indicate the full path to the ‘QikScript.dll’ file you copied in step 1.

4) If you extracted the files to a disk folder other than ‘C:\QikScript’ do the following. Go to the folder where you extracted the binary files. Open up the ‘Settings.XML’ file. Edit the ‘FilePath’ node in each of the ‘handler’ nodes to refer to the full path of the ‘handler.py’ file. (You can do a search and replace over the entire file).

5) The ‘handler.py’ contains the event handlers. You can write your own code in these methods and it will be invoked the next time VS fires that event.

Once you install the add-in, the next instance of VS that you start will invoke functions defined in the handler file. The simplest way to check if the addin is loaded succesfully is to take a look at the 'outpu' window of the VS. The dropdown box would contain an entry with the name 'QikScript' if loading is succesful.

Depending on your settings, it is possible for the addin not to load on startup. In such cases, you can go to the 'Tools->Addin Manager' menu item and manually select the addin, make it autoload and restart VS. Please do not hesitate to let me know via email or comments if you are stuck with the installation steps.

How to disable or remove the addin

The easiest way to disable this addin is to move or delete the QikScript.addin file from the AddIns folder. You could also change the extension of this file if you want to temporarily stop the VS from loading this. If you want to load VS without *any* addin you can use the ‘devenv /safemode’ from the Run prompt to start VS.

Package details

The deployment zip file contains the binaries that implement the functionality, an event handler definition XML file (settings.xml) and a python file (handler.py) that implements the event handlers. The DLR and ironPython binaries are from the IronPython 2.0 Beta 3 release

Settings.xml

· Settings.xml contains the details about the event handlers. You can edit the individual events to run a specific function in any given python file.

· The settings.xml must reside in the same folder as the QikScript.dll file.

Handler.py

· The Handler.py file contains the event handlers.

· Each of the handler defined in this file uses the correct signature needed to work with VS. Some of the arguments are primitive data types – string, bool , int and some are VS specific types.

· Note the commented out line just above the event handler. That line indicates the types of the arguments passed to the event handler.

The event handlers can also access the host VS object through the ‘hostApp’ variable in the script. The type of the object is DTE2. All output (like print calls) from the script is redirected to a new output window (with the name ‘QikScript’) that the add-in creates.

Notes

· This is strictly intended as a demo tool and comes with zero warranty or support.

· This addin doesn’t make VS an IronPython editor. You would use an external editor to edit the IronPython code. See this post if you are interested in enabling VS to edit IronPython sources

· I have used this addin only as part of Visual Studio 2008. It may or may not work in earlier versions.

· Theoretically, all operations supported by the VS through the extensibility model( the DTE2 interface) can be performed from any of the handlers. However, I did not attempt any testing to see which operations work and which don’t. So you are on your own when trying out adventurous thingsJ.

· Currently only ‘IronPython’ is available as the scripting language

This post is very light on implementation details deliberately. I am planning to put up a separate post with more details about the parts of the Hosting API this component uses.

Please let me know of any comments, questions, feedback or suggestions you have regarding this sample, DLR and/or the Hosting API.

How to redirect output from python using the DLR Hosting API

seshadripv — Tue, 08 Jul 2008 22:58:00 GMT

This is the 3rd in the series of 'How To' posts on using the DLR Hosting API to host languages like IronPython, IronRuby, Toyscript etc in a managed application. The first 2 posts are here and here.

The DLR Hosting API provides the ScriptIO class to give you complete control over the input, output and error streams used by the scripting languages. The Hosting API describes ScriptIO like this - “This class let's you control input and output by default for dynamic code running via DLR hosting. You can access the instance of this class from the IO property on ScriptRuntime.”

Here's the step by step guide to redirect the output to a Stream object using the ScriptIO class.

1) Create the runtime, the python engine and a MemoryStream object

ScriptRuntime runTime = ScriptRuntime.Create();

ScriptEngine pyEngine = runTime.GetEngine("py");

MemoryStream ms = new MemoryStream();

2) Set the memory stream created above as the output stream for this script runtime.( The API can redirect output to any Stream object. So you can also use a FileStream object to capture output).

runTime.IO.SetOutput(ms, new StreamWriter(ms));

3) Execute a python code snippet that would output something.

ScriptSource ss = pyEngine.CreateScriptSourceFromString(

"print 'hello world'",

SourceCodeKind.Statements);

ss.Execute();

4) Read from the memory stream object created in step (2).

string str = ReadFromStream(ms);

5) ‘str’ will now contain the output from the script.

The input and error streams can also be set using the above technique and calling the appropriate method in ScriptIO. Refer to the DLR Hosting API spec to learn more about the ScriptIO and other related classes.

The full sample follows. Normally you wouldn’t care to redirect the output, input or error streams in a console application. In spite of this, I am sticking to a console application as it lets me demonstrate the API usage succinctly and provide a full copy & paste enabled sample

using System;

using System.IO;

using System.Scripting;

using System.Text;

using Microsoft.Scripting.Hosting;

namespace ConsoleApplication1 {

class Program {

public static void Main(string[] args) {

ScriptRuntime runTime = ScriptRuntime.Create();

ScriptEngine pyEngine = runTime.GetEngine("py");

MemoryStream ms = new MemoryStream();

//runTime.IO refers to the current runtime's IO property

runTime.IO.SetOutput(ms, new StreamWriter(ms));

ScriptSource ss =

pyEngine.CreateScriptSourceFromString(

"print 'hello world'",

SourceCodeKind.Statements);

ss.Execute();

string str = ReadFromStream(ms);

ms.Close();

Console.WriteLine(str);

}

private static string ReadFromStream(MemoryStream ms) {

int length = (int)ms.Length;

Byte[] bytes = new Byte[length];

ms.Seek(0, SeekOrigin.Begin);

ms.Read(bytes, 0, (int)ms.Length);

return Encoding.GetEncoding("utf-8").GetString(bytes, 0, (int)ms.Length);

}

How to invoke a IronPython function from C# using the DLR Hosting API

seshadripv — Mon, 30 Jun 2008 22:10:00 GMT

Like I wrote in my previous post, the Hosting API lets you do some very interesting and powerful things with the DLR. One of the things I had listed was the ability to invoke a method implemented in python from C#.

This post describes how to set up the VS IDE and the DLR binaries to create an extremely basic DLR host. So if you are new to DLR you may want to quickly go over the step-by-step procedure to get your dev environment set up.

Here's what you would have to do

1. Create the ScriptRuntime object and get the python engine associated with it

ScriptEngine pyEng = ScriptRuntime.Create().GetEngine("python");

2. Implement a function in python and create a ScriptSource object based on the python code

string pyFunc = @"def isodd(n): return 1 == n % 2;";

ScriptSource source = pyEng.CreateScriptSourceFromString(pyFunc,

SourceCodeKind.Statements);

3. Create a ScriptScope object. This object will act as the context for the python code.

ScriptScope scope = pyEng.CreateScope();

4. Execute the ScriptSource object in the context of the ScriptScope object created in step (3).

source.Execute(scope);

5. From C#, Use the 'GetVariable' method in the ScriptScope to get the delegate to the python function implemented in step 2. ( The GetVariable method lets you access any variable defined in python from C# or VB or any other managed language)

Func<int, bool> IsOdd = scope.GetVariable<Func<int, bool>>("isodd");

6. Invoke the delegate.

bool b = IsOdd(1);

The above steps can be used to invoke any non instance method in python from C#. Instance members can be invoked using a similary but slightly different technique. (I will put up a sample that explains this shortly)

You can find more details about the methods and classes used in this sample in the Hosting API Spec.

Here’s the full sample program.

using System;

using System.Scripting;

using Microsoft.Scripting.Hosting;

namespace ConsoleApplication1 {

class Program {

static void Main(string[] args) {

//create a runtime and get the python engine

ScriptEngine pyEng = ScriptRuntime.Create().GetEngine("python");

//implement a python function and create a ScriptSource object out of it

string pyFunc = @"def isodd(n): return 1 == n % 2;";

ScriptSource source = pyEng.CreateScriptSourceFromString(pyFunc,

SourceCodeKind.Statements);

//create a scope to act as the context for the code

ScriptScope scope = pyEng.CreateScope();

//execute the source

source.Execute(scope);

//get a delegate to the python function

Func<int, bool> IsOdd = scope.GetVariable<Func<int, bool>>("isodd");

//invoke the delegate

Console.WriteLine(IsOdd(1));

Console.WriteLine(IsOdd(2));

}

I hope you find this sample useful. I am planning to post a series of articles on DLR Hosting along similar lines( How To's, step-by-step guides etc). It would be very helpful to get your feedback regarding this. So please leave a comment if you have any questions, suggestions or other feedback.

How to write a simple DLR host in C# using Hosting API

seshadripv — Tue, 24 Jun 2008 21:07:00 GMT

Let me start with a (fair) assumption. You reached here by clicking on a link in a search results page. So, you already have a basic idea of what the DLR is and that it can be hosted in managed applications. This post addresses getting you started assuming no prior knowledge of the API that would let you do this.

Here's a step by step guide to write a simple DLR host

Currently, the DLR is available as part of IronPython. So you can download the latest DLR binaries and/or binaries from here. (You can also get it from the IronRuby project)
Extract the contents of the zip file to disk. The default dir will be 'IronPython-2.0B3'
Fire up VS and create a new C# Console application.
In VS, choose Project-><project name> Properties... menu item to bring up project properties.
Use the 'Output path' option to set the the project's output path to the folder where you unzipped the binaries in step (2).
Make sure the folder you set in (5) is the one that has the unzipped binaries like ipy.exe, Microsoft.Scripting.Core.dll etc
Add a reference to the Scripting dlls. Using Project->Add reference menu to bring up the references dialog.Now use the 'Browse' tag to locate 'Microsoft.Scripting.Dll' and 'Microsoft.Scripting.Core.dll' in the folder you created in step (2)
Create a simple python file. My test python file has just the following line - print "hello simple dlr host"
Paste the following code in the Program.cs file created in step (3)

using Microsoft.Scripting.Hosting;

namespace SimpleDLRHost {

class Program {

static void Main(string[] args) {

ScriptRuntime runtime = ScriptRuntime.Create();

runtime.ExecuteFile(@"D:\test.py");

}

Compile and run
You should see the output from the python script in the output console of this project

By now, I am sure you are thrilled about being able to write simple programs that can host a scripting language like IronPython. This sample is the most simplest DLR host you could write and so doesn't do anything meaningful. The focus of this post is on setting up the environment correctly to use the API ( the classes and methods that are part of the Microsoft.Scripting.Hosting namespace)

Using the Hosting API, You do some very powerful things with the DLR and scripting languages. For example, you can do things like

Use other languages like IronRuby, Managed JScript etc...
Define a method in python and invoke from C#
Declare a variable in C# and access it from the script (and vice versa)
Add user scripting support to your applications.

The Hosting API spec has some samples that demonstrate some of the above items. I would also be putting up some more examples along these lines shortly here.

Further Reading

The Hosting API spec is the most comprehensive resource to learn about DLR hosting in detail.
To get a good understanding of the DLR as a platform visit Martin's blog - http://blogs.msdn.com/mmaly/. You may want to start from the first entry in his blog - http://blogs.msdn.com/mmaly/archive/2007/12/19/dynamic-language-runtime.aspx
This page contains a bunch of very useful links to DLR resources