Charlie Calvert's Community Blog : Dynamic

Videos of C# Authors at PDC

Charlie Calvert — Tue, 24 Nov 2009 08:04:16 GMT

Here are links to a few short video interviews by prebynski with C# MVPs and luminaries speaking while they attended PDC 2009 last week in Los Angeles. This is a chance to hear the interests and opinions of folks who are immersed in the culture and technology surrounding C#. All three of these interviewees are with authors who are currently working on new books about C#. Hear their take on new technologies such as Azure and Dynamic in C#, as well as their thoughts on the latest developments in existing technologies such as LINQ, WPF and Silverlight.

New Article by Bill Wagner on Dynamic Method Bags in C# 4.0

Charlie Calvert — Tue, 03 Nov 2009 22:30:32 GMT

C# MVP and wunderkind Bill Wagner has written an article entitled Dynamic Method Bags which is now available on MSDN. Bill explores the new dynamic feature in C# 4.0. Most posts on this subject explain how to use dynamic to call Python, Ruby or Office. In his article, Bill explains how dynamic can be used not to call another language or tool, but to call your own C# objects without directly using the reflection API’s. The solution he provides combines generics, expression tress and dynamic to create a “type that allows developers to add new methods at runtime” and call those methods through dynamic dispatch.

Resources

Bill’s Dynamic Method Bags article
Bills Blog
Download a Doc file providing an Overview of C# 4.0 Language Features
C# Future Samples
Visual C# 2010 Resources
VS 2010 Walkthroughs
Some Dynamic Walkthroughs
IronPython on CodePlex
IronRuby

Running IronPython Scripts from a C# 4.0 Program

Charlie Calvert — Mon, 26 Oct 2009 06:41:52 GMT

IronPython is a scripting language hosted on the .NET platform. This posts shows how you can use the Dynamic Language Runtime (DLR) and the new C# 4.0 dynamic keyword to call an IronPython script from a C# program.

Before going any further, it might be helpful to take a moment to explore the architecture that makes this technology possible. The key building block is the DLR, which sits on top of the .NET 4.0 CLR and provides the tooling for adding dynamic programs such as Python to the family of .NET languages. C# 4.0 adds the dynamic keyword and some related technologies to integrate support for dynamic binding into the language. The end result is that developers can arbitrary Python scripts directly from C# or other .NET languages.

Download Source from the LinqFarm

Setting up a C# Project to Call IronPython

In two previous posts I described how to install and run a simple IronPython program:

If you don’t already have the latest version of IronPython installed or if you need a little refresher course in writing Python scripts, then you might want to glance at these posts. Otherwise, you can just keep reading to learn how to call Python from C#.

After the IronPython installation, its interpreter and its libraries will be installed in your Program Files directory, as shown in Listing 1. The interpreter is installed in ipy.exe, where ipy is pronounced “Eye-Pie.” The interpreter remains dormant when you call Python from C#, so our focus here should be on the assemblies shown in the second half of the listing, with a particular emphasis on IronPython.dll.

Listing 1: At the command prompt you can easily see the core files that form the basis for the IronPython installation.

C:\Program Files (x86)\IronPython 2.6 CTP for .NET 4.0 Beta 2>dir *.exe *.dll
 Volume in drive C has no label.
 Volume Serial Number is 4C6B-EE0E

 Directory of C:\Program Files (x86)\IronPython 2.6 CTP for .NET 4.0 Beta 2

10/18/2009  01:47 PM            12,624 ipy.exe
10/18/2009  01:47 PM            12,624 ipyw.exe

 Directory of C:\Program Files (x86)\IronPython 2.6 CTP for .NET 4.0 Beta 2

10/18/2009  01:47 PM         1,433,424 IronPython.dll
10/18/2009  01:47 PM           435,536 IronPython.Modules.dll
10/18/2009  01:47 PM           875,856 Microsoft.Dynamic.dll
10/18/2009  01:47 PM            58,192 Microsoft.Scripting.Debugging.dll
10/18/2009  01:47 PM           159,056 Microsoft.Scripting.dll
               7 File(s)      2,987,312 bytes
               0 Dir(s)  16,881,614,848 bytes free

To run a Python script inside a C# program, you should include some or all of the libraries shown in Listing 1 in the References section of your project. To get started, choose File | New Project (Ctrl-Shift-N) from the Visual Studio 2010 menu and create a new console application. Then open the Solution Explorer (Ctrl-W, S) and go to the References section for your project. Right click and choose Add Reference. Take a moment to appreciate how quickly the Add References dialog appeared (the team worked hared on this performance improvement), and then select the Browse tab and navigate to the directory where IronPython resides. The path to the directory hosting IronPython will probably be similar to the one shown in Listing 1. For now, the simplest choice is to select all the assemblies found in that directory, as shown in Figure 1. If you wish to keep things simple, then you only need to select IronPython.dll, Microsoft.Dynamic.dll and Microsoft.Scripting.dll to compile and run the program shown in this example.

Figure 1: Using the Add References dialog to select the assemblies found in that directory.

When you are done, the IronPython libraries should appear in the References section of the Solution Explorer, as shown in Figure 2.

Figure 2: The References node of the Solution Explorer for a C# program that calls an IronPython script.

Writing the Code for Calling IronPython

Now that you have your project set up correctly you need to do two things:

Write or find a script containing Python code that you want to run
Write C# code to call the script.

Listing 2 shows a very simple IronPython script that contains a single method called Simple() that writes a few strings to a console window. Save this script to a file called Test.py, and add it to your project, as shown in Figure 2. There are several ways to do this. One is to right click on your project in the Solution Explorer, and choose Add | New Item. Add a text file, and call it Test.py. Click on the new node when it is added to your project, and set its Copy to Output Directory property to Copy Always. If the Properties window is not visible, you can bring it up by selecting Ctrl-W, P.

Listings 3a and 3b show the C# code for calling the script, and Listing 4 shows the output from the C# program that hosts the script.

Listing 2: A Simple Python script stored in a file called Test.py. This code prints out some basic information about the environment in which Python is running.

import sys

def Simple():
	print "Hello from Python"
	print "Call Dir(): "
	print dir()
	print "Print the Path: " 
	print sys.path

Listing 3a: A simple C# program for calling a Python script

Code Snippet

using System;
using IronPython.Hosting;
using Microsoft.Scripting.Hosting;
 
public class dynamic_demo
{
    static void Main()
    {
        var ipy = Python.CreateRuntime();
        dynamic test = ipy.UseFile("Test.py");
        test.Simple();
    }
}

Listing 3b: The same C# program shown in Listing 3a, but this presentation is easier for you to block copy into your own version of the program.

using System;
using IronPython.Hosting;
using Microsoft.Scripting.Hosting;

public class dynamic_demo
{
    static void Main()
    {
        var ipy = Python.CreateRuntime();
        dynamic test = ipy.UseFile("Test.py");
        test.Simple();
    }
}

The code in Listing 2 is a very simple Python script. Before calling it from C#, you might want to see it in action. That way you can be sure the script works and you can verify what the script is supposed to do. To get started, block copy the code in Listing 2. Use the Windows Start Menu to open up the Python Console window and paste in the code. Finally, you should explicitly call the Simple() method, as shown in Figure 3. When looking at this screen shot, you can see the version of the Python Console on the first two lines. The next 8 lines show the code that I pasted into the console. Finally, you see where I typed in the call to Simple(), and after that the output from the script. Once again, this has nothing to do with calling the code from C#, it is just a way of verifying that you have a valid script, and of confirming how it should perform.

Figure 3: A screenshot showing a run of the Python script that we will be calling from our C# program.

Now let’s look at the 3 lines of code used to call this script from a C# program. This code is shown in Listings 3a and 3b. It first creates an instance of the Python runtime, as shown in line 9:

var ipy = Python.CreateRuntime();

The call to CreateRunTime loads or sets up the core Python libraries and the DLR code needed to to execute the Python script. The second line of code, found on line 10, calls UseFile to load our simple Python script into memory. Notice that the result of this call is a dynamic object. The dynamic keyword is a new feature in C# 4.0 and Visual Studio 2010 which tells the compiler to resolve this particular call not at compile time, but at runtime.

It’s very important to understand that the dynamic keyword breaks the strong type checking that has always been a hallmark of C# development. There is no doubt that the ability to call dynamic objects is a useful and important feature. However, it should be used sparingly since it sidesteps the strict type checking that makes C# such a powerful and useful language.

The dynamic keyword is needed here because Python is an interpreted scripting language where calls are bound at run time, not at compile time. There is no simple or practical way to bind calls to Python at compile time because Python is designed to be a dynamic language which is resolved at runtime. Since C# is a strongly typed language it expects to resolve calls at compile time with strict type checking. In short, we are at an impasse. C# binds statically at compile time, Python dynamically at runtime. Something has to give, and the only valid solution was for C# to allow runtime binding for method calls.

It is important to grasp that calls into Python are not strictly type checked at compile time, and are therefore not guaranteed to succeed at runtime. Just because your program compiled without error does not mean that you have properly bound each call in your program. In this case we call the method named Simple(), which does in fact exist. As a result, the call succeeds, as shown by the output from the program found in Listing 4.

Listing 4: The output from the C# program which calls a Python script.

Hello from Python
Call Dir():
[]
Print the Path:
['.', 'C:\\Users\\Charlie\\Documents\\SyncData\\Source\\ProjectsShared\\Python\\CallPython\\CallPython\\bin\\Debug\\Lib', 
'C:\\Users\\Charlie\\Documents\\SyncData\\Source\\ProjectsShared\\Python\\CallPython\\CallPython\\bin\\Debug\\DLLs']
Press any key to continue . . .

Listing 4 shows the good side of working with dynamic code. Now let’s take a moment to consider the dark side. Suppose we called a method on our dynamic test class that does not exist. For the sake of argument, let’s call the method NonexistentMethod(), which is shown in Listing 5.

Listing 5: This code contains a call to a method called NonexistentMethod() that does not exist in our Test.py file. As a result, the call will fail at runtime.

 var ipy = Python.CreateRuntime();
dynamic test = ipy.UseFile("Test.py");
test.Simple();            // Succeeds
test.NonexistentMethod(); // Fails

The code in Listing 5 contains a call to the method Simple() that will succeed, since the Python script shown in Listing 2 does contain a method called Simple(). However, the call to the method NonexistentMethod() will raise an exception at runtime. This happens because the call cannot be resolved at runtime. At compile time, however, the compiler knows nothing about the object called test. It makes no attempt to see whether or not the call will succeed. It simply takes your word for it and allows the compilation to succeed. At runtime, however, the error raises its head, as shown in Figure 4 and Listing 6.

Figure 4: The exception raised inside the IDE at runtime when you attempt to call a method that does not exist.

Listing 6: The details of the exception that you receive at runtime when you attempt to call a method that does not exist.

Microsoft.CSharp.RuntimeBinder.RuntimeBinderException was unhandled
  Message='Microsoft.Scripting.Hosting.ScriptScope' does not contain a definition for 'NonexistentMethod'
  Source=Anonymously Hosted DynamicMethods Assembly
  StackTrace:
       at CallSite.Target(Closure , CallSite , Object )
       at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid1[T0](CallSite site, T0 arg0)
       at dynamic_demo.Main() in C:\Users\Charlie\Documents\SyncData\Source\ProjectsShared\Python\CallPython\CallPython\Program.cs:line 12
       at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException:

The team went to lengths to ensure that the error you get at runtime looks almost exactly like the error you would have received at compile time had you attempted to call a static method that did not exist. Everything possible is done to make this experience easy for your to understand and handle, including giving you an exception that you can catch with your own custom code.

It goes without saying that everyone on the Visual Studio Languages team at Microsoft understands the implications of introducing the dynamic keyword into the C# language. We understand completely the advantages of statically linked code, and believe that strict type checking is a powerful tool that helps developers create robust code. In most cases, you should still use statically linked calls in your program. There are, however, scenarios that cannot be efficiently supported from the C# language without the introduction of dynamic programming. One is calling a dynamic language such as Python, and another is calling into a some COM objects, such as those used in those used by Microsoft Office. In order to make it easy for you to use dynamic languages and to perform COM Interop, the dynamic keyword was introduced into the C# 4.0 language, and the DLR became a built-in extension of .NET 4.0. Use these tools judiciously, and they will be your friend. Abuse them, and you are on your own.

Summary

In this article you learned how to call a simple Python script from a C# application. You saw how to set up the References section for your program so that it contains the assemblies used when calling an IronPython script. You then saw how to write a simple Python script, and how to call it from a C# program.

The dynamic technology on display in this example makes use of a body of code called the DLR, or Dynamic Language Runtime. The dynamic keyword that is part of C# 4.0 provides supports when you call dynamic languages or COM objects, and particularly when you use C# 4.0 to call into Microsoft Office applications. Coverage of calling into Office is separate subject, and will be covered in another post. A third topic deserving of coverage is using the dynamic keyword in scenarios where you previously used reflection.

Download Source from the LinqFarm

Getting Started with IronPython

Charlie Calvert — Mon, 26 Oct 2009 01:03:46 GMT

I recently spent some time getting IronPython up and running on my system; I will review what I learned in this post.

Hosted inside an interpreter, Python belongs to the same family of scripting tools as VBScript, JavaScript, Perl and Ruby. You can fairly compare Python to a general purpose language such as C#. Developers praise this loosely typed, dynamic language for its ease of use and rapid development capabilities. IronPython is Microsoft’s free version of the open source Python language.

IronPython can be hosted inside a C# program as a scripting language. Nonetheless, Python is a powerful standalone language frequently used as the glue in web based applications. Before I came to work for Microsoft I enjoyed using it as part of my daily toolkit.

In this post I explore the simple steps needed to run IronPython as a standalone tool from the command prompt. In other posts I will show how to call Python from inside a C# application.

Installing Python

I started by downloading IronPython from CodePlex and installing it. The whole process from first visit to the IronPython web site until IronPython was installed probably took me little more than a minute. Below are the links to retrieve the code. If you are reading this article in 2009 or early 2010, consider using the first link, if the date is later than that then use the second link and look for a newer release:

The Beta 2 Release of IronPython (2.6 CTP for .NET 4.0 Beta 2)
IronPython Site

Download the IronPython MSI Windows install file, then double click on it to run setup. When you are done, your IronPython installation will probably be located in a directory similar to this one:

C:\Program Files (x86)\IronPython 2.6 CTP for .NET 4.0 Beta 2

Once you have the IronPython interpreter installed, you need to follow these steps to run a sample program:

Put IronPython on your Path
Create an IronPython program
Compile and run the program.

In the next few sections of this post I’ll walk you through these steps. When you are done, you will have IronPython up and running on your system.

Setting Up Your Path

You can run IronPython from inside Visual Studio, but the tools for doing so are still primitive. Most Python developers work inside a lightweight editor and run Python from inside the IronPython interpreter or from the command line. This post focuses on building Python from the command line. To get started with that process, you will probably want to begin by ensuring that the IronPython interpreter is on your System Path.

They say there must be fifty ways to set the system path. In this post I will outline two of them as practiced by developers on Vista or Windows 7. In both cases you can begin with a four step process to bring up the Environment Variables Dialog:

Method One
- Press the Windows Key to bring up the Start Menu
- Right click on Computer and choose Properties
- Click Advanced Computer Settings
- Click Environment
Method Two
- Press the Windows Key to ring up the Start Menu
- Type Control Panel and press enter
- You should be taken automatically to the search box on the top right of the control panel, if not, press Ctrl-E to get there and type Environment. (The first three letters should be all that’s necessary)
- Hit tab twice to select Edit Environment Variables, and then hit enter.

Which ever technique you use, the dialog shown in Figure 1 should appear. You can use this dialog to edit the system path.

Figure 1: In this screenshot, a variable called Path has been added to the list of User Variables. It lists the path to the IronPython Interpreter.

Once the dialog appears, click New (Alt-N) to add a Path variable for your account. A small dialog appears. Paste the path to IronPython into this dialog and click the OK button. The path should be similar to the path shown above which points to the directory where IronPython was installed. Click OK to close the Environment Variables dialog. From now on, and Command prompt window that you open should have an environment automatically set up to point to the IronPython interpreter. To test that everything is working correctly, go to the command prompt and type ipy. If everything has gone well, you should see a result like the following.

C:\Users\Charlie\Documents\Python\PythonPost>ipy
IronPython 2.6 (2.6.10920.0) on .NET 4.0.21006.1
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

As shown above, you can type exit() and then enter to quit the interpreter. You can also execute Python code from inside the interpreter. Working interactively inside the interpreter is a useful and powerful way to write code, but I will not cover it in this post.

Create a Python Program

Now that you have Python on your path, you can easily compile your projects from the command line. Depending on your tastes, you can edit your program in Visual Studio, in NotePad, or in NotePad++. The goal is to create a simple text-based source file with the letters py as an extension. Here, for example, is the complete contents of a simple Python source file:

print "Very Simple Test"

You can save this source file as VerySimpleTest.py, and process and run it by passing its name to the Python interpreter. The interpreter will process and run the program:

C:\Users\Charlie\Documents\Python\PythonPost>ipy VerySmipleTest.py.
Very Simple Test

Now that you understand the basics, the next step is to understand how to create functions, classes and multiple module projects.

Python uses indentation rather than keywords or syntax to define blocks of code. Here, for instance, is how to define a function in Python:

def WriteTest():
	print "This is a test"

The keyword def followed by a name and a colon defines the header for the function. The next line begins with tab character. This defines the beginning of main block for the function, which in this case contains a single line of code. If you had multiple lines in the function, then you would indent each line with a tab character. An empty line defines the end of the function.

To define a class, you repeat the same process, but use the keyword class rather than def:

class SimpleLocalClass:
	def SimpleMethod(self):
		print "Simple Local Class"

Here you see the definition for a class called SimpleLocalClass which has a single method called SimpleMethod. Not everyone likes the Python syntax for defining classes and methods, but I have found intuitive and easy to use.

Listings 1 and 2 contain a project with two files that demonstrates how to use functions and classes in Python:

Listing 1: Here is the main module for the project. It is a simple source file called Project.py containing a function and a simple class. This source file also has a dependency on a class called SimpleClassTest that is shown in Listing 2

from SimpleClass import SimpleClassTest

class SimpleLocalClass:
	def SimpleMethod(self):
		print "Simple Local Class"

def WriteTest():
	print "This is a test"
	
WriteTest()

simpleLocalClass = SimpleLocalClass()
simpleLocalClass.SimpleMethod()

simpleClass = SimpleClassTest()
simpleClass.SimpleMethod()

Listing 2: The second file in this project is called SimpleClass.py. The file shown in Listing 1 depends on this class.

class SimpleClassTest:
	def SimpleMethod(self):
		print "Simple Class"

Notice the from statement at the top of Listing 1. It is used to import the class found in Listing 2. I should also point out that there is no main() method shown in listing 1. The word main does not have the same significance in Python that it does in C#. The entry point for this Python program is the last set of un-indented statements found in Project.py. In this case the entry point begins with the call to WriteTest().

Compiling and Running the Program

Now that you have the two source files for your project, all you need do is compile and run them. The Python interpreter is designed to make this task as simple as possible. You need only reference the primary file for your project, the interpreter is smart enough to to parse the from import statement at the top of Listing 1 and figure out from that how to include SimpleClass.py in the project.

Listing 3: Here is the command to process and run the files shown in Listings 1 and 2. The output from the command is also shown.

C:\Users\Charlie\Python\PythonPost>ipy Project.py
This is a test
Simple Local Class
Simple Class

Summary

In this post I have reviewed how to download and install IronPython, and how to get some simple Python scripts up and running on your system. You saw how to run a simple program from the command line, how to create:

A simple function
A simple class
And a mutli-module project.

In future posts I will focus on calling IronPython from a C# program.

IronRuby and IronPython for .NET 4.0 Beta 2 Available

Charlie Calvert — Thu, 22 Oct 2009 19:45:17 GMT

The third preview of IronRuby and IronPython running under the Dynamic Language Runtime and .NET 4.0 is now available. You can easily use IronPython and IronRuby objects and types as .NET 4.0 dynamic objects from within the pre-release versions of C# 4.0 and Visual Basic 10.0 found in Visual Studio 2010. The IronRuby and IronPython binaries are roughly equivalent to the functionality found in IronRuby 0.9 and IronPython 2.6.

New Features in C# 4.0 Paper, plus Feedback and Samples

Charlie Calvert — Wed, 05 Nov 2008 02:24:33 GMT

Mads Torgersen, a C# PM and member of the C# design team, has worked with other members of the C# team to create a document that outlines the new features in C# 4.0. Along with Anders’ PDC talk, this is probably the best way to get an overview of what the C# team has planned for the next release of the language. I also want to make sure you know about the C# 4.0 samples. These short projects are designed to demonstrate how to use the new language features in C# 4.0. Finally, I provide a link to the Visual Studio 2010 CTP forums based feedback area, where you can comment on the new features Microsoft has planned for the C# language and the VS IDE. These forums provide a place where you can provide the PM’s who are driving the new features in VS 2010 with your valued feedback.

Here are the links to the:

Here are direct links to

Except for the CTP Feedback Forums, all the items listed above are found on the Code Gallery based C# Future site. That site provides a single nexus of information where you can learn about our plans for the next version of the C# language.

Charlie Calvert's Community Blog : Dynamic

Videos of C# Authors at PDC

New Article by Bill Wagner on Dynamic Method Bags in C# 4.0

Resources

Running IronPython Scripts from a C# 4.0 Program

Setting up a C# Project to Call IronPython

Writing the Code for Calling IronPython

Summary

Getting Started with IronPython

Installing Python

Setting Up Your Path

Create a Python Program

Compiling and Running the Program

Summary

IronRuby and IronPython for .NET 4.0 Beta 2 Available

Links

Recent Videos from the C# Team (2008-2009)

New Features in C# 4.0 Paper, plus Feedback and Samples