Haibo Luo's weblog

IronPython: System.Reflection.Assembly object

2008-03-13T06:12:00Z

IronPython offers a little bit more love to the Assembly object instance: we can directly access the assembly's top-level members (namespace, public type) via the dot operation. System.Reflection provides many ways to let you hold the assembly object, such as Assembly.Load method, Type.Assembly property, ...

For example, given assembly1.dll compiled from the C# file below, dir(a1) in the python below shows the class name "C2" and the top level namespace "NS1" in its' dictionary.

// assembly1.cs
namespace NS1 {
    namespace NS2 {
        public class C1 { }
    }
}
public class C2 {
    public class C3 { }
}
class C4 { }

# part 1 from file/test1.py
import System
a1 = System.Reflection.Assembly.Load("assembly1")

print dir(a1)  # ['C2', ..., 'NS1', ...]
print a1.NS1   # <module 'NS1' (CLS module from assembly1, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)>
print a1.C2    # <type 'C2'>

# access non-public type
print a1.C4    # AttributeError: 'Assembly' object has no attribute 'C4'

# access the down-level namespace, type with the dot operations
print a1.NS1.NS2.C1         # <type 'C1'>
# access the nested type
print a1.C2.C3              # <type 'C3'>

Normally I would use clr.AddReference to bring in the CLR assembly. The python code below gives the similar output.

# part 1 of file/test2.py
import clr
clr.AddReference("assembly1")
import NS1 
import C2

print NS1        # <module 'NS1' (CLS module from assembly1, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)>
print C2         # <type 'C2'>
print NS1.NS2    # <module 'NS2' (CLS module from assembly1, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)>
print NS1.NS2.C1 # <type 'C1'>
print C2.C3      # <type 'C3'>

However the first approach may help keep the global (and local) dictionary less "polluted" and sometimes can be used to avoid the namespace/type name collision. For example, suppose we need to load in another assembly built from the following C# code, and we use the clr.AddReference approach.

// assembly2.cs
namespace NS1 { 
    public class NS2 { } 
}

# part 2 of file/test2.py
clr.AddReference("assembly2")
print NS1.NS2    # <type 'NS2'>
print NS1.NS2.C1 # AttributeError: 'type' object has no attribute 'C1'

NS2 is a type now (no longer the namespace (or python module) NS1.NS2 from "assembly1"), and we lost the access to the type NS1.NS2.C1. If we use the assembly object approach, we can hold both the namespace NS1.NS2 from assembly1 and the type NS1.NS2 from assembly2; basically we can think the assembly variable name adds one more layer to prevent the name collision. Or in the other direction of thinking, what clr.AddReference does for you (related to implicit name merge/override) may not be what exactly you want.

# part 2 from file/test1.py
a2 = System.Reflection.Assembly.Load("assembly2")
print a2.NS1     # <module 'NS1' (CLS module from assembly2, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)>
print a2.NS1.NS2 # <type 'NS2'>
print a1.NS1     # <module 'NS1' (CLS module from assembly1, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)>
print a1.NS1.NS2 # <module 'NS2' (CLS module from assembly1, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)>

Note that such name collision should rarely happen for well-designed framework assemblies. Also IronPython merge types/down-level namespaces under the same namespaces, even if they are from different assemblies. One simple example is System.Int32 from mscorlib.dll and System.Uri from System.dll, both peacefully under the "System" module.

import System
print System            
# <module 'System' (CLS module, 2 assemblies loaded)>
print System.__file__   
# ['mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089', 
#  'System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089']
import clr
print clr.GetClrType(System.Int32).Assembly  
# <Assembly mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089>
print clr.GetClrType(System.Uri).Assembly    
# <Assembly System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089>

IronPython: System.Array

2008-03-13T06:04:13Z

When you start interop'ing with .NET in IronPython, sooner or later, you will find that you are in need of creating an array as argument. There are mainly 2 ways to create array objects:

Array with type indexing (for one-dimensional array object only), and
classic reflection API: Array.CreateInstance

System.Array indexing with a type creates a concrete array type, which can then take a collection of the element objects (or an enumerable) and form the array object. There are half dozen Array.CreateInstance overloads, which allow us to create one-dimensional empty array, also multi-dimensional/non-zero lower-bound arrays.

import System
array_int = System.Array[int]
print array_int               # <type 'Array[int]'>

# list
print array_int([1, 2])       # System.Int32[](1, 2)
# tuple
print array_int((3, 4, 5))    # System.Int32[](3, 4, 5)
# xrange
print array_int(xrange(6,10)) # System.Int32[](6, 7, 8, 9)
# CLR List
print array_int(System.Collections.Generic.List[int]()) # System.Int32[]()

# one-dimensional array 
a1 = System.Array.CreateInstance(int, 5)
for i in range(5): a1[i] = i * 10
print a1        # System.Int32[](0, 10, 20, 30, 40)

# two-dimensional array 
a2 = System.Array.CreateInstance(float, 2, 2)
a2[1, 1] = 3.14
print a2        # System.Double[,](
                # 0.0, 0.0
                # 0.0, 3.14)

IronPython also supports some python list-like operation to the CLR array objects, such as indexing with slice, +, *, in...

a1 = array_int(range(5))
a2 = array_int([100])

# slice
print a1[1:-1]      # System.Int32[](1, 2, 3)
# +
print a1 + a2       # System.Int32[](0, 1, 2, 3, 4, 100)
# *
print a1 * 2        # System.Int32[](0, 1, 2, 3, 4, 0, 1, 2, 3, 4)
# +, upcast the result to object[]
print a2 + System.Array[str]('py')  # System.Object[](100, 'p', 'y')

# slice with step
print a1[1::2]      # System.Int32[](1, 3)
# assignment 
a1[1::2] = [11, 13]
print a1            # System.Int32[](0, 11, 2, 13, 4)

# in/__contains__
print 11 in a1      # True

IronPython: ipyw.exe

2008-03-13T06:03:48Z

Each IronPython binary release ships two executable files: ipy.exe and ipyw.exe. Their (only) difference is, ipy.exe is a console application and ipyw.exe is a windows application. So given the following winform.py,

## winform.py
import clr
clr.AddReference("System.Windows.Forms")
from System.Windows.Forms import *

class SimpleForm(Form):
    def __init__(self):
        self.Text = 'Simple Winform'

Application.Run(SimpleForm())

Running it with ipy.exe will keep the associated console; however, if I launch it with ipyw.exe in a console window, I can continue working on other stuffs within that console.

I found that Shawn wrote some background about WINDOW_CUI/WINDOW_GUI subsystem here.

SignatureResolver (unfinished)

2008-03-09T01:44:00Z

While writing the ILVisualizer for dynamic method late 2005, I'd like to show the local variable information as well; so I started working on the managed signature parser, at least to parse LocalVarSig (Ecma-335 23.2.6). It was 2+ years ago, and never got to a finished state. Now perhaps I will never spend more time on it, so I refreshed the code a bit and you may find it useful for some scenarios.

Here is an example of how to use SignatureResolver to get LocalVarSig in IronPython. The static method SignatureResolver.GetLocalVarSig takes the byte array (the signature blob) and the enclosing method, and returns an array of LocalVar (which includes mainly the runtime type) inside that method.

# show local variables for every Array.CreateInstance overload methods
import clr
import System
clr.AddReference("ClrTest.Reflection.SignatureResolver")
from ClrTest.Reflection import *

for method in clr.GetClrType(System.Array).GetMember("CreateInstance"):
    body = method.GetMethodBody()   # C# won't allow this
    tok = body.LocalSignatureMetadataToken
    sig = method.DeclaringType.Module.ResolveSignature(tok)
    print "> %s, 0x%x" % (method, tok)
    for lv in SignatureResolver.GetLocalVarSig(sig, method):
        print lv

Next is part of the output showing one Array.CreateInstance method's local variable types. The result is, of course, same as what Reflector tells us in the below picture. "(p)" means the local variable is pinned.

> System.Array CreateInstance(System.Type, Int32[], Int32[]), 0x11000009
System.RuntimeType 
System.Int32 
System.Int32& (p)
System.Int32& (p)
System.Array 
System.Int32[] 
System.Int32[] 
System.RuntimeTypeHandle

Yes, I understand the MethodBody.LocalVariables property gives us the similar information; but the dynamic method can not. That was why I wanted to decode the bytes, and to use DynamicScopeTokenResolver to get a meaningful type string. I admit this version of code diverts from its original goal: it only works for RuntimeMethodInfo/RuntimeConstructorInfo, and won't work for dynamic method; but if one day, you are in need of parsing blob, the functions like GetToken/GetInt32Uncompressed could be handy. GetToken basically is the reverse of TypeDefOrRefEncoded (ecma-335: 23.2.8)

int GetToken() {
    int token = GetInt32Uncompressed();
    return s_mapForTokenType[token & 0x03] | (token >> 2);
}

int GetInt32Uncompressed() {
    byte first = _sig[_pos];

    if (first <= 0x7F) {
        return _sig[_pos++];
    } else if (first <= 0xBF) {
        byte[] bytes = new byte[2];
        bytes[1] = (byte)(_sig[_pos++] - 0x80);
        bytes[0] = _sig[_pos++];
        return BitConverter.ToInt16(bytes, 0);
    } else {
        byte[] bytes = new byte[4];
        bytes[3] = (byte)(_sig[_pos++] - 0xC0);
        bytes[2] = _sig[_pos++];
        bytes[1] = _sig[_pos++]; ;
        bytes[0] = _sig[_pos++];
        return BitConverter.ToInt32(bytes, 0);
    }
}

My SignatureResolver also provide another static method GetMethodRefSig to get MethodRefSig (Ecma-335 22.25 and 23.2.2).

If you are interested in this topic, you may download the attached file, unzip it, and take a look at the source. Also try to use VS2008 to build it.

If you are an IronPython user, you may try the accompanying tests (in IronPython). I compare the SignatureResolver.GetLocalVarSig result with the official reflection API MethodBody.LocalVariables: they are same for all method/constructors from all simple-LoadFrom-successful framework assemblies. If you have ClrTest.Reflection.ILReader.dll (or know how to get one), you can try test_methodrefsig.py, which tests GetMethodRefSig.

As stated in the title, this is *unfinished* and could be buggy.

Disclaimer: THE CODE IS PROVIDED "AS IS", WITH NO WARRANTIES INTENDED OR IMPLIED. USE AT YOUR OWN RISK.

ILVisualizer VS2008 solution

2008-03-08T03:11:00Z

I attached the ILVisualizer VS2008 solution in this post. There is no source code update; the only change is to upgrade the Microsoft.VisualStudio.DebuggerVisualizers.dll reference from version 8.0.0.0 to version 9.0.0.0.

Just for your convenience, so you can see a successful build after downloading in VS2008.

Thanks Julien for leaving the instruction here.

IronPython: Accessing the .NET Field

2007-10-29T06:10:00Z

The .NET libraries normally do not expose the public field, but we can still find its' presence for certain scenarios. IronPython treats the .NET fields like python attribute. For python attribute, the typical operations are set, get and delete. In general, IronPython throws TypeError when deleting members of CLR types and their objects (so for fields). To do set and get operation, the IronPython code looks similar to C# code. Updating (setting) values to the read-only or literal fields throws as we expect.

public class C {
public static int StaticField;
public int InstanceField;
}

>>> ...
>>> C.StaticField = 1          # set
>>> C.StaticField              # get
1
>>> c = C()
>>> c.InstanceField = 2        # set
>>> c.InstanceField            # get
2

Guess what you will get for C.InstanceField? It is "field descriptor", same as C.__dict__['InstanceField'], upon which we can call __set__/__get__ or two equivalent helper methods SetValue/GetValue (feel like repeating CLR reflection FieldInfo API here). Although not shown below, we can also set/get the value of the static field too by calling these methods on C.__dict__['StaticField'].

>>> C.InstanceField
<field# InstanceField on C>
>>> C.InstanceField is C.__dict__['InstanceField']
True
>>> C.InstanceField.__set__(c, 3)
>>> C.InstanceField.__get__(c)
3
>>> C.InstanceField.SetValue(c, 4)
>>> C.InstanceField.GetValue(c)
4

IronPython allows accessing static field on instance too. On the other hand, C# compiler requires type name to access static member (otherwise CS0176).

>>> c.StaticField = 5
>>> c.StaticField, C.StaticField
(5, 5)

Updating fields of value types is not allowed. If interested, you may go ahead read the design decision here. Until recently I did not know that as a remedy, the feature of setting fields/properties in the constructor call should be able to set field for value types while still keeping value types effectively immutable after the creation. Unfortunately we had no tests to cover this part (which never worked). This code below illustrates the expected behavior. I expect it to work soon.

public struct MyPoint {
public int X, Y;
public override string ToString() { return string.Format("X: {0}, Y: {1}", X, Y); }
}

>>> p = MyPoint(X=1, Y=2)
>>> p
<MyPoint object at 0x000000000000002C [X: 1, Y: 2]>
>>> p.X = 3
# one kind of exception still throws

VisualStudio as my IronPython editor

2007-10-17T08:00:00Z

The following steps are what I did to get Visual Studio ready as my IronPython (and IronRuby) editor.

Install the latest internal dogfood build of Visual Studio 2008.
- you may use Visual Studio 2005 or download the VS 2008 public beta2;
Download and install ASP.NET futures release (July 2007). This will give me the nice syntax coloring and (well... limited) intellisense for python code.
- you may download VSSDK 2008 CTP, and build the IronPython integration sample, which will give you the similar editing experience;
Download the attached add-in binary zip (poorly named as DlrToolsAddin); for my Vista box, I extract them under "%USERPROFILE%\Documents\Visual Studio 2008\Addins". I wrote this add-in basically to allow me to run the script inside the Visual Studio, and show the output in the output window.
- if you consider using it for VS 2005, you will need put it in "Visual Studio 2005\Addins".

Say, I am trying the example in my previous post, I create a new C# file (sample.cs) without creating solution/project, I can press Ctrl+1 (or the first button in the DlrTools toobar) to get it compiled to sample.dll in the local directory. You see the compile result at the bottom output window. Then I create a python file (method.py) in the same directory, type in some code (shown in color). Again, I can press Ctrl+1, the result are shown in the output window. I do not need leave VS and run these files in the cmd.exe window.

Sometimes I want to run the same code with different tools (for example, to check IronPython compatibility, I often run the same .py file against C-Python 2.5 too). That is the first combo box comes to play.

The second combo box is to set the working directory: "." means the current directory where the active file lives, ".." for the parent directory of the current file, or you may use the absolute path. Such support is needed to run C-Python regression tests. (For the add-in implementation side, I feel what I really want is a combo box of type vsCommandControlTypeDynamicCombo, which seems not available for add-in development).

The delete button is to remove your tool choice for those files you pressed "Run Script". By clicking the last button, an XML file (specifying which tools for which file extension) is opened so you may change it. You must update it with your tool paths, since the default setting is suited for myself.

You may also run part of the file by selecting those lines first (A temporary file is created and will be deleted after VS shutdown).

One bad thing I noticed of this add-in is that sometimes I press the "run script" button to start debugging, both icons have similar shapes.

To uninstall it, delete DlrToolsAddin.* files under "Addins"; and then run once "devenv.exe /resetaddin DlrToolsAddin.Connect".

The source code (VS2008 project) is also attached. Disclaimer: THE CODE IS PROVIDED "AS IS", WITH NO WARRANTIES INTENDED OR IMPLIED. USE AT YOUR OWN RISK. Hope you like Visual Studio as your IronPython editor.

IronPython: explicitly choose one method

2007-10-12T06:32:47Z

C# allows method overloading. Given an argument list, the compiler performs the overload resolution to select the best method to invoke. The chosen method token is baked into the assembly. IronPython does the method resolution at the IronPython run time; for most scenarios, it has no problem in picking up the most reasonable one based on the argument list. The rules are a bit complicated, and I will leave them for a future post. If you believe IronPython chooses the "unexpected" method for your scenario, please do let us know.

IronPython also provides the "Overloads" (python) attribute for the bound method, on which we can do index operation with parameter types to get the exact method we want. In the example below, the .NET reflection API Type.MakeByRefType and MakeArrayType are used in order to get those constructed types. Since these calls are only available for System.Type object, clr.GetClrType is called first on the python type to get the underlying CLR type. However, I am able to use the python type for the first 2 index calls (thanks to the type conversion).

public class ChooseOverload {
   public void M(int arg) { Console.WriteLine(1); }
   public void M(byte arg1, byte arg2) { Console.WriteLine(2); }
   public void M(ref int arg) { Console.WriteLine(3); arg = 10; }
   public void M(int[] arg) { Console.WriteLine(4); }
}

>>> ... # add the assembly
>>> import ChooseOverload
>>> o = ChooseOverload()
>>> o.M
<built-in method M of ChooseOverload object at 0x000000000000002B>
>>> o.M.Overloads
{'M(self, int arg)': <built-in function M>, 'M(self, Byte arg1, Byte arg2)': <built-in function M>, 'int M(self, int arg)': <built-in function M>, 'M(self, Array[int] arg)': <built-in function M>}
>>> o.M.Overloads[int](1)
1
>>> o.M.Overloads[System.Byte, System.Byte](1, 2)
2
>>> o.M.Overloads[clr.GetClrType(int).MakeByRefType()](1)
3
>>> o.M.Overloads[clr.GetClrType(int).MakeArrayType()](None)
4

When generics come to the picture, o.M could mean the generic method. Similar to the generic type, IronPython chooses the index operation to hold the instantiated generic method. Currently IronPython has no support to specify parameter types which are (or made from) generic type parameters (of either the generic method or type). Expect this will be supported one day.

public void M2<T>(T arg) { Console.WriteLine(5); }
public void M2<T>(byte arg) { Console.WriteLine(6); }

>>> o.M2[int](1) # bind to the first M2 given the argument "1"
5
>>> o.M2[int].Overloads[System.Byte](1) # in order to get the second M2
6
>>> o.M2[int].Overloads[???](1) # unable to pick the first M2 currently

IronPython: Provide (or not) Argument for by-ref Parameter

2007-10-05T08:21:00Z

Python function may return multiple objects as a tuple. The .NET method can only return one object as the result of a call; in order to return more than one objects, by-ref parameters are needed. They are decorated with the parameter modifier (ref, out) in C#.

Dictionary.TryGetValue(TKey key, out TValue value) has an output parameter "value". The API returns true if the dictionary contains an element with the specified key (the element value itself is returned to the parameter "value"), otherwise it returns false. When making calls to such methods in IronPython, we may not pass in argument for the output parameter. Instead, the result of such .NET method call in IronPython will likely be a tuple (unless the .NET method's return type is void and the method has only one by-ref parameter), which contains the value of the output parameter (see the example below).

import System
d = System.Collections.Generic.Dictionary[int, str]()
d[1], d[2] = 'One', 'Two'

print d.TryGetValue(1)         # (True, 'One')
print d.TryGetValue(2)[1]      # Two
print d.TryGetValue(3)         # (False, None)

We may also pass a clr.Reference object for the output parameter. The clr.Reference object has only one member "Value", which will carry the output parameter value after the call. "clr.Reference" object is of the type identical to System.Runtime.CompilerServices.StrongBox<T> in the new .NET Framework 3.5. When encountering this type of argument, IronPython codegen/runtime does something special to update the "Value".

x = clr.Reference[str]()             # like "new StrongBox<string>()" in C#
print d.TryGetValue(1, x), x.Value   # True One
print d.TryGetValue(3, x), x.Value   # False None

For reference parameter, we are required to pass in something. If the argument is not clr.Reference object, such reference argument value will also be part of the returned tuple; otherwise, the by-ref change is tracked inside clr.Reference. The following C# code is my twisted way to find the maximum number; the IronPython snippet shows the behaviors of calling that method not using and using clr.Reference.

public class C {
public bool M(int current, ref int max) {
    if (current > max) {
      max = current; return true;
    } else return false;
}
}

import C
o = C()

print o.M(10, 20)             # (False, 20)
print o.M(30, 20)             # (True, 30)

x = clr.Reference[int](20)
print o.M(10, x), x            # False 20
print o.M(30, x), x            # True 30

o.M(1)                        # TypeError: M() takes exactly 2 arguments (1 given)

Note if the .NET method contains more than one by-ref parameters, IronPython expects the user to provide proper clr.Reference objects for either ALL or NONE of them. A mix of clr.Reference object and normal argument/omission (for out parameter) will cause error (as illustrated below).

public int M2(ref int x, ref int y) {...}

o.M2(clr.Reference[int](1), 2) # TypeError

IronPython: Keyword Argument to Set .NET Property/Field

2007-10-02T22:16:00Z

Use keyword arguments to set properties or fields… you likely ask the question: where can we do this? IronPython allows it in the constructor call, the call against the .NET type. Such keyword should be the name of public field, or property (which is not read-only). After creating the object instance, each keyword argument value is then set to the corresponding field or property. Note keyword arguments can be used for the constructor parameters as usual.

The IronPython code below is to monitor any python file change under the directory "C:\temp". I used this special argument passing in the 2nd line.

import System
fsw = System.IO.FileSystemWatcher(r"C:\temp", Filter="*.py", IncludeSubdirectories = True, EnableRaisingEvents = True)
def on_changed(o,e): print e.ChangeType, e.FullPath
fsw.Changed += on_changed

System.Console.ReadLine()

FileSystemWatcher has three constructors (see below). The code uses the second one (not the third one)."c:\temp" is positional argument for "path"; the next 3 keywords arguments are transformed to the property (Filter/IncludeSubdirectories/…) assignments after the object creation.

public FileSystemWatcher();
public FileSystemWatcher(string path);
public FileSystemWatcher(string path, string filter);

I am not going to argue whether this feature is good or bad. You may find it handy in some places. The user could get confused when the constructor parameter name is same as the field/property name (Although it is unlikely as most libraries are following the .NET framework guideline: property name starts with the capital character, while the parameter name does opposite). One of our summer interns also proposed to extend this feature to event setting as keyword argument. It is currently not supported, not sure whether it will ever come in the future.

Note this code above actually doe not work in IronPython 2.0 alpha 4 (due to bugs). The upcoming 2.0 alpha5 will have the fix.

IronPython: Passing Arguments for a Call

2007-10-02T02:06:41Z

After getting a CLR type, we can play it like a python type. We can get the class/static methods using the dot operator ("attribute reference") on the type, or create an instance of it and then get the instance methods. In Python’s word, these methods are callable objects, and we can make "calls" on them using the call operator ("()"). I will write about the method call in the next couple of posts, this one is about passing arguments for calls in IronPython.

Python function definition allows several different parameter styles:

parameter which accepts excess positional arguments,
parameter with default value,
parameter which accepts excess keyword arguments.

Here are some examples showing the common parameter style, and each of 3 styles listed above.

def f1(x, y): print x, y
def f2(x, *y): print x, len(y)
def f3(x, y=5): print x, y
def f4(x, **y): print x, y

When making calls (on these functions), Python allows many different argument styles too. The following type (in C#) defines some methods, each with interesting parameter list.

// sample.cs
public class ArgumentList
public void M1(int x, int y) { Console.WriteLine("{0} {1}", x, y); }
public void M2(int x, params int[] y) { Console.WriteLine("{0} {1}", x, y.Length); }
public void M3(int x, [DefaultParameterValue(5)] int y) { Console.WriteLine("{0} {1}", x, y); }
}

In terms of the parameter style, we will see that M1 is equivalent as pure python function f1, same for M2/f2, M3/f3 (I was unable to find similar parameter kind in C# for f4). When calling these .NET methods, IronPython supports all argument styles as Python does.

M1 is the simplest in the parameter definition, but we can call it with different argument lists. Each call below prints "1, 2", same as if the python function "f1" is called.

import clr
clr.AddReference("sample")

import ArgumentList
o = ArgumentList()
f1 = o.M1

f1(1, 2)                 # positional arguments
f1(1, y = 2)             # positional & keyword arguments
f1(y = 2, x = 1)         # keyword arguments
f1(*(1, 2))              # unpack the sequence as positional args
f1(1, *(2,))             # positional & unpack
f1(**{'x': 1, 'y': 2})   # unpack the dictionary as keyword args

C# parameter array (parameter with "params", or exactly ParamArrayAttribute) is treated like "*y" in f2. The user are allowed to pass 0 or more arguments to the parameter "y".

f2 = o.M2
f2(1)              # print: 1 0
f2(1, 2)           #        1 1
f2(*(1, 2, 3))     #        1 2
f2(1, 2, 3, 4, 5) #        1 4

The C# language does not have built-in support for the default value argument (IL does, with the .param directive inside the method body); even if the parameter is decorated with System.Runtime.InteropServices.DefaultParameterValueAttribute, C# calls on such methods still require argument passed in for that parameter. However IronPython (like Python) honors such default value parameters.

f3 = o.M3
f2(1)            # print: 1 5
f2(x = 1)        #        1 5
f2(1, 2)         #        1 2

Another parameter attribute (OptionalAttribute) received similar treatment from IronPython: you do not need pass in values either; IronPython assign it a special value (the .NET run-time default value for most of primitive types, System.Type.Missing.Value for "object" type, null otherwise). Such supports are useful when using IronPython to call into the COM libraries (such as Office automation). The user can avoid typing in every arguments.

public void M6([Optional] int x, [Optional] object y) {
Console.WriteLine("{0} {1}", x, y);
}
public void M7([Optional] string x, [Optional] Type y) {
Console.WriteLine("{0} {1}", x == null, y == null);
}

o.M6()           # print: 0 System.Type.Missing
o.M6(10)         #        10 System.Type.Missing
o.M6(20, "hi")   #        20 hi
o.M7()           #        True True

IronPython: Grab the .NET Type

2007-09-26T00:58:00Z

After loading the assembly by clr.AddReference and then import namespace, we are ready to take hold of types and down-level namespaces using "attribute references".

>>> import System
>>> System.DateTime                    # type
<type 'DateTime'>
>>> System.Diagnostics                 # still namespace
<module 'Diagnostics' (CLS module, 2 assemblies loaded)>
>>> System.Diagnostics.Debug           # type again
<type 'Debug'>
>>> System.Diagnostics.CodeAnalysis    # namespace again
<module 'CodeAnalysis' (CLS module from mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)>
>>>

We can also access public nested type from the declaring type same way. (I was unable to find such example in mscorlib.dll/system.dll)

// lib.cs
public class C {
public class NestedC {}
}

>>> import clr
>>> clr.AddReference("lib")
>>> import C # C has no namespace here
>>> C.NestedC
<type 'NestedC'>

When the type is generic type, such attribute reference returns the open generic type. We need then call the index operation (subscriptions) to construct the closed generic type, and do something interesting with it. A type tuple is expected as the index, but the parentheses can be omitted most of time.

>>> System.Collections.Generic.Dictionary # open generic type
<type 'Dictionary[TKey, TValue]'>
>>> System.Collections.Generic.Dictionary[int, str] # closed generic type
<type 'Dictionary[int, str]'>
>>> System.Collections.Generic.Dictionary[(int, object)] # closed generic type
<type 'Dictionary[int, object]'>

Also the .NET type names could be "same" or similar; one example is System.Nullable and System.Nullable<T> in mscorlib.dll. Many experimental code of mine has types like C, C<T>, C<T1, T2> . As shown below, "System.Nullable" now returns a type group, including 2 types. In order to distinguish the non-generic type from the generic one, an empty tuple need be passed in as index.

>>> System.Nullable
<types 'Nullable', 'Nullable[T]'>
>>> System.Nullable[()] # get the non-generic type
<type 'Nullable'>
>>> System.Nullable[System.DateTime]
<type 'Nullable[DateTime]'>

IronPython: clr.AddReference

2007-09-26T00:57:00Z

In order to interop with .NET libraries, we need first load in the assemblies we want to play with. The family of AddReference methods in the clr module serves the purpose.

clr.AddReference
clr.AddReferenceByName
clr.AddReferenceByPartialName
clr.AddReferenceToFile
clr.AddReferenceToFileAndPath

clr.AddReference accepts System.Reflection.Assembly objects and/or assembly names. The typical usage looks like:

asm = ... # any approach of getting an Assembly object
clr.AddReference(asm)
clr.AddReference("System.Drawing, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a")
clr.AddReference("System.Drawing")

If a string is passed in, Assembly.Load(string) is used to load the assembly. If failed, it re-tries with Assembly.LoadWithPartialName(string).

clr.AddReferenceByName basically is the wrapper of Assembly.Load, which means the assembly full name is expected. Similarly, clr.AddReferenceByPartialName is the wrapper around the obsolete Assembly.LoadWithPartialName: "System.Drawing" and "System.Drawing, Version=2.0.0.0" are examples of valid arguments.

A typical usage of clr.AddReferenceToFile looks like:

clr.AddReferenceToFile("my1.dll", "my2") # load my1.dll and my2.dll

It expects file name(s) without the directory path as the argument. To locate my1.dll, it searches each directory in sys.path until the specified file is found (it also tries to append ".dll" or ".exe"). Behind the scene, the loading is via Assembly.LoadFile. Normally sys.path includes the current working directory; so if my1.dll and my2.dll exist in the current directory, clr.AddReferenceToFile("my1.dll", "my2") should succeed. If you know the exact full path of a clr assembly, you may use clr.AddReferenceToFileAndPath to load. As a by-product, the path of the assembly file will be appended to sys.path. Note Assembly.LoadFrom is never used underneath by these 5 methods due to the loading context concern.

We can use clr.References to list which assemblies have been loaded. The output shows that mscorlib.dll and System.dll are implicitly loaded/added (without the need of calling clr.AddReference).

>>> import clr
>>> for r in clr.References: print r
...
mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
>>> clr.AddReference("System.Drawing")
>>> for r in clr.References: print r
...
mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
System.Drawing, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

By treating the namespace like the python module, IronPython extends the import statement semantics to bring in the clr namespace. In current implementation the "classical" python module is still preferred by import: given the statement "import MyLib", if the file "MyLib.py" is present under any directory of sys.path, that file will be imported; if not, for each loaded assembly, import tries to find the namespace "MyLib". Although not common in well-designed .NET frameworks, a type could have empty namespace; so "import MyLib" also tries to find type "MyLib" in the loaded assemblies.

The namespace of "MyLib" could exist more than 1 loaded assemblies. The following snippet shows that, "import System" adds a CLS module to sys.modules and this CLS module contains members from 2 assemblies: mscorlib.dll and System.dll (since both assemblies has types and down-level namespaces like System.*).

> ipy.exe
>>> import sys
>>> sys.modules.keys()
['sys', '__builtin__', '__main__', 'site']
>>> import System
>>> sys.modules.keys()
['sys', '__builtin__', '__main__', 'site', 'System']
>>> sys.modules['System']
<module 'System' (CLS module, 2 assemblies loaded)>
>>> System == sys.modules['System'] # better use "is" to check identity as Michael pointed out in the comment
True
>>> System.__file__
['mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089', 'System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089']
>>> System.Console.WriteLine('later')
later

IronPython: import clr

2007-09-26T00:56:00Z

clr is an IronPython built-in module, which provides some functionalities in order to interop with .NET. When writing "import clr" in a python module, it means you intend to leverage the .NET libraries. One classical example is python string's methods. In IronPython, python type str is implemented by the .NET type System.String. We can call System.String's methods on string objects after the line of "import clr" (see the example below); of course those python string methods are still available.

> ipy.exe
IronPython console: IronPython 2.0 (2.0.0.00) on .NET 2.0.50727.1378
Copyright (c) Microsoft Corporation. All rights reserved.
>>> s = "IronPython"
>>> s.upper()
'IRONPYTHON'
>>> s.ToUpper()
Traceback (most recent call last):
File , line 0, in ##14
File , line 0, in _stub_##15
AttributeError: 'str' object has no attribute 'ToUpper'

>>> import clr
>>> s.upper() # python method still works
'IRONPYTHON'
>>> s.ToUpper() # works
'IRONPYTHON'

Such context change is per-python-module. Although lib.py (below) imports the clr module and app.py then imports the lib module, app.py will not see the .NET methods. Running this app.py under C-Python will fail (ImportError) due to the usage of the clr module in lib.py. But if we make lib.py platform-neutral next time, we are ensured that no change in app.py is needed in order to run it under both C-Python and IronPython.

# lib.py
import clr
def my_upper(s): return s.ToUpper()

# app.py
import lib
s = "hello world"
print lib.my_upper(s) # print "HELLO WORLD"
#print s.ToUpper() # AttributeError would throw

DebuggerTypeProxy for IronPython Old-style Class and Instance

2007-08-03T22:37:00Z

First I want to make it clear that this post means nothing related to IronPython's plan about debugging python code in the future; it serves more as an interesting example to demonstrate DebuggerTypeProxyAttribute and how types can be created dynamically and be used in the VS debugger windows to display information the end users are interested in.

The python code below shows one old style class and some operations on the instance. After downloading the binary zip of IronPython 2.0 Alpha 3, we can create a solution, set it to launch ipy.exe for debugging, and step through the code inside Visual Studio; the locals window may show something like this after hitting the breakpoint.

The local variable alpha3's .NET type is IronPython.Runtime.Types.OldInstance, the way this object is displayed in the locals window is based on that type's fields and properties, which exposes many implementation details. While debugging, what the python users really want to know is the object attributes. My goal is to make python object attributes look like C# object's public fields; how can I achieve this?

DebuggerTypeProxyAttribute allows us to specify a proxy type for the target type. VS debugger uses the proxy type to display the target object. We already see such proxy types a lot: when viewing objects of most collection types, we are actually viewing their proxy type layouts.

In current IronPython implementation, every old style instance is of type I.R.T.OldInstance, so my first step is to define a proxy type "OldInstanceProxy" to expose the object attributes only. Clearly old style instances can have different attribute set of different names; but I can only specify one proxy type for the I.R.T.OldInstance type. [DebuggerBrowsable(DebuggerBrowsableState.RootHidden)] is part of the solution to achieve the look-and-feel. The proxy type (OldInstanceProxy) has an object field (m_oldInstance) decorated with that attribute so that m_oldInstance will be hiden in the debugger window, but the fields m_oldInstance has will be shown. What should m_oldInstance look like? A .NET type (let me call it the "real" proxy type) with as many public fields as python object attributes, each having the same name as the attribute name, and we are going to create such types on the fly. Also the "real" proxy type's constructor is emitted with the code to assign these public fields properly. We then instantiate the new type with the original OldInstance object, and assign it to m_oldInstance.

[assembly:DebuggerTypeProxy(
typeof(IronPython.DebuggingSupport.OldInstanceProxy),
  TargetTypeName = "IronPython.Runtime.Types.OldInstance, IronPython, Version=2.0.0.300, ...")
]

public class OldInstanceProxy{
[DebuggerBrowsable(DebuggerBrowsableState.RootHidden)]
public object m_oldInstance;

public OldInstanceProxy(object target) {
try {
   OldInstance obj = target as OldInstance;
TypeBuilder tb = CodeGen.CreateTemporaryType();
ConstructorBuilder cb = tb.DefineConstructor(MethodAttributes.Public, CallingConventions.Standard, new Type[] { typeof(OldInstance) });
ILGenerator il = cb.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Call, typeof(Object).GetConstructor(Type.EmptyTypes));

foreach (KeyValuePair<object,object> pair in obj.Dictionary) {
    string key = pair.Key.ToString();
    Type fieldType = pair.Value == null ? typeof(object) : pair.Value.GetType();
    FieldBuilder fb = tb.DefineField(key, fieldType, FieldAttributes.Pub.Public);

        // emit code for the ctor ...
      }
      il.Emit(OpCodes.Ret);

      m_oldInstance = Activator.CreateInstance(tb.CreateType(), obj);

Few discussions:

The attached solution also includes another proxy type: OldClassProxy for IronPython.Runtime.Types.OldClass. With that, viewing class attributes feels like viewing C# class static fields. Such coding pattern can be applied to other similar scenarios.
It is a bit hard to apply this for IronPython new style instances. For each new style class, IronPython creates a new type on the fly. For example, "class C(object): pass" generates a IronPython.Runtime.NewTypes.Object_1. The proxy type (which will create the "real" proxy type) need be created at the same time.
Types created by Reflection.Emit are not GC-collectible. In my proxy type implementation, a simple caching mechanism is implemented to reuse those "real" proxy types; but considering the python dynamism, object attributes can be added, removed, their values' type can be changed; such "real" proxy type may have to be created frequently, each click to expand in the debugger window could cause one type creation, so use wisely if you use such proxy type in real world scenarios.
Unlike DebuggerVisualizer where VS provides a way to debug it, it is a bit inconvenient to debug DebuggerTypeProxy code. So I found having the constructor code wrapped inside try-catch and saving the exception if thrown is useful. The exception object can be viewed in the debugger window if the type proxy dll is compiled with the "Debug" configuration. The messages from Debug.WriteLine are helpful too.
The proxy type implementation depends on the rapidly changing code of the DLR/IronPython project. It is not surprising that the attached solution could be broken in the future. .

The following pictures show the proxy type in action. Compared to the previous picture, now you can easily check out the values of the attribute "url" and "version", as well as the class attribute "project_name" (shown in the first picture). The new attribute "release_date" appears in the second picture after stepping through the breakpoint line.

To use the attached solution, you need download IronPython 2.0 alpha 3 binary zip, and I assume you unzip it to C:\. Your comments are welcome.