Srivatsn's Blog : IronPython

Hosting IronPython made easier

srivatsn — Wed, 17 Sep 2008 01:30:34 GMT

With 2.0 Beta 5 coming out very soon, there is a new hosting helper class in there called IronPython.Hosting.Python . It has a few helper methods to create a ScriptRuntime or ScriptEngine and adds some Python-specific extension methods to ScriptRuntime and ScriptEngine.

Creating a ScriptRuntime the normal way now requires a LanguageSetup as Sesh explains here. The Python class has helpers called CreateRuntimeSetup and CreateLanguageSetup which return a pythonic ScriptRuntimeSetup and LanguageSetup respectively. But if you are not interested in the configuration options of the DLR then you can simply use Python.CreateEngine/ Python.CreateRuntime.

ScriptRuntime runtime = Python.CreateRuntime();
runtime.ExecuteFile("foo.py");

There are also three extension methods tacked on to ScriptRuntime and ScriptEngine called GetSysModule, GetBuiltinModule and GetClrModule which directly return a ScriptScope for these modules. You can do something like this for example:

using IronPython.Hosting;

ScriptEngine engine = Python.CreateEngine();

ScriptScope sys = engine.GetSysModule();
var platform = sys.GetVariable("platform");
Console.WriteLine(platform);

ScriptScope builtins = engine.GetBuiltinModule();
var pow = builtins.GetVariable<Func<double, double,double>>("pow");
Console.WriteLine(pow(2,3));

ScriptScope clr = engine.GetClrModule();
var getPythonType = clr.GetVariable<Func<Type, PythonType>>("GetPythonType");            
Console.WriteLine(PythonType.Get__name__(getPythonType(typeof(string))));

This will then print out cli, 8 and str respectively.

Passing keyword args to C# methods from IronPython

srivatsn — Wed, 10 Sep 2008 00:13:14 GMT

In Python one can call functions with either the argument itself or a named argument i.e f(3) or f(x=3). Python also provides syntax to absorb excess arguments for either style of argument passing with either * or **. So for example for this function f:

def f(x, *args, **kwargs):
    pass
    
f(3,4,5,y=6,z=7)

The value passed to x is 3 and args then absorbs the extra positional arguments and is passed as a tuple (4,5). kwargs then takes up the keyword arguments and is passed as a dictionary {'y':6, 'z':7}.

Variable number of arguments in the CLS

Now if 'f' were a C# method would this still work? Passing the positional and named parameters from IronPython definitely works. The question is can C# accept variable number of parameters? The CLS clearly states that the only calling convention supported is the standard managed calling convention and variable length argument lists are not allowed. So how does C# support the params keyword then? The C# compiler applies an attribute (System.ParamArray) and passes the extra positional parameters as an array. Great, so can we then do the same thing for dictionary params as well?

ParamDictionaryAttribute

There is a ParamDictionaryAttribute in Microsoft.Scripting.dll that serves the purpose of the System.ParamArray for keyword arguments. But since the C# compiler doesn't know about it there is no syntactic sugar here and it needs to be manually applied. We can, therefore, define a function like this:

using System;
using Microsoft.Scripting;

public class foo {
    public void bar(int x, [ParamDictionary] IAttributesCollection kwargs) {
        foreach (SymbolId key in kwargs.SymbolAttributes.Keys) {
            Console.WriteLine("key: {0}, value: {1} ", key, kwargs[key]);
        }
    }
}

The function bar now accepts a dictionary of excess keyword parameters when called from IronPython. IAttributesCollection is another interface defined in Microsoft.Scripting which represents a dictionary that can be accessed using SymbolIds and also arbitrary objects. The SymbolAttributes property returns just the SymbolIds. This can now be called from IronPython like so:

import clr
clr.AddReference("test.dll")
import foo
foo().bar(2,y=4, z=6)

And you'll see y:4 and z:6 were indeed passed into the method as items in the dictionary.

Static Compilation of IronPython scripts

srivatsn — Wed, 06 Aug 2008 09:28:01 GMT

The ability to compile IronPython scripts into .NET IL and to save them to disk existed in IronPython 1.0 but has been missing in 2.0 so far. With IronPython 2.0 Beta4 this has been added back.

Why would I compile dynamic language scripts?

There are a lot of reasons to compile scripts into a binary form. Shri talks about some of them here. For folks who don't want to distribute source code in plain text this provides one level of obfuscation. To that end, a function called CompileModules has been added to the clr module to compile scripts into executable IL. The signature of the function is:

CompileModules(str assemblyName, dict kwArgs, Array[str] filenames)

So to compile a file foo.py into foo.dll you would do this:

import clr
clr.CompileModules("foo.dll", "foo.py")

This can now be brought in using the regular clr.AddReference. When clr.AddReference sees a compiled assembly, it publishes the module as well. So one can simply import the module into the code.

clr.AddReference("foo.dll")
import foo

Multiple files and main

The function can take multiple python files and compile them into one dll. What if you want it to be a standalone executable? There are two things to be done. First, a stub exe is needed that can load the dll. Second, a way to distinguish the main module is needed. The keyword args that CompileModules can take comes in handy here

import clr
clr.CompileModules("foo.dll", "foo.py", "bar.py", mainModule="main.py")

Now a stub exe can be written that loads up this compiled dll. The IronPython sample pyc.py has code that does shows how to generate a stub exe.

Wait, what is -X:SaveAssemblies mode then?

When IronPython is started with -X:SaveAssemblies, it generates a dll containing IL corresponding to the code it executed. Sounds an awful lot like compilation doesn't it? The difference is one is executable IL and the other isn't.

To understand the difference, one needs to understand that IronPython under normal course of its execution generates IL anyway. Every statement is converted to the DLR AST and IL gets spit out for the ASTs which is then executed. The SaveAssemblies mode simply dumps the generated IL into a dll. It is meant as a debugging device. So what is missing from this IL that prevents it from being re-executable code? The short answer is Dynamic Sites. The sites that are generated during the execution are not persisted. The compilation feature does exactly this - it persists the dynamic sites as well. Lets look at an example here and compare the generated IL in reflector. (Only the relevant code is copied over from reflector). This python code:

print 2 + 5
print 2 * 5
print 3 / 5

when run with -X:SaveAssemblies mode produces this code:

public static CallSite<DynamicSiteTarget<int, int, object>> #Constant207;
public static CallSite<DynamicSiteTarget<int, int, object>> #Constant208;
public static CallSite<DynamicSiteTarget<int, int, object>> #Constant209;

$lineNo = 1;
PythonOps.Print(__global_context, #Constant207.Target(#Constant207, 2, 5));
$lineNo = 2;
PythonOps.Print(__global_context, #Constant208.Target(#Constant208, 2, 5));
$lineNo = 3;
PythonOps.Print(__global_context, #Constant209.Target(#Constant209, 3, 5));

Notice that the Constants defined here are actually defined as fields on the generated type and this type doesn't get instantiated anywhere and therefore the sites don't get assigned anywhere. The same python code when compiled with clr.CompiledModules produces this code:

object[] objArray = new object[] { 
CallSite<DynamicSiteTarget<int, int, object>>.Create(PythonOps.MakeOperationAction(context, "Add")), 
CallSite<DynamicSiteTarget<int, int, object>>.Create(PythonOps.MakeOperationAction(context, "Multiply")), 
CallSite<DynamicSiteTarget<int, int, object>>.Create(PythonOps.MakeOperationAction(context, "Divide")) 
};

line = 1;
PythonOps.Print(context, ((CallSite<DynamicSiteTarget<int, int, object>>)objArray[0]).Target(
    (CallSite<DynamicSiteTarget<int, int, object>>)objArray[0], 2, 5));
line = 2;
PythonOps.Print(context, ((CallSite<DynamicSiteTarget<int, int, object>>)objArray[1]).Target(
    (CallSite<DynamicSiteTarget<int, int, object>>)objArray[1], 2, 5));
line = 3;
PythonOps.Print(context, ((CallSite<DynamicSiteTarget<int, int, object>>)objArray[2]).Target(
    (CallSite<DynamicSiteTarget<int, int, object>>)objArray[2], 3, 5));

You can see that all the dynamic call sites are being created here and their targets are being invoked. This then is perfectly executable code - maybe not as succinct as the python code but it does the same thing :)

IronPython 2.0 Beta 3 is out!

srivatsn — Sat, 14 Jun 2008 01:58:07 GMT

The next stop on the path to IronPython 2.0 is Beta 3 and we pushed it out today. You can get the sources and binaries of IronPython here. As Jimmy mentioned in his release notes for Dynamic Silverlight last week, this release also contains the sources and binaries of IronPython and Dynamic Silverlight needed to build Silverlight applications with IronPython. This release builds against Silverlight 2.0 Beta 2 – so make sure that is installed.

Head over to the release notes to find out what has changed since the last release and the list of bugs that have been fixed. Congratulations to the team and the community on the release.

Accessing IronPython objects from native Javascript

srivatsn — Fri, 09 May 2008 10:16:52 GMT

Today I got a question by e-mail from someone who wanted to implement something like this MSDN article in IronPython. The gist of the article is that there is a property called ObjectForScripting on a WinForms WebBrowser control. If one sets the property to an object, then that object will be available to javascript that executes in the page through window.external. The Form needs to add the ComVisible attribute.

.NET attributes cannot be added to a class from IronPython but fortunately objects created are ComVisible by default. So we can try a direct translation of the code in the article :

import clr
clr.AddReference("System.Windows.Forms")
from System.Windows.Forms import Application, Form, WebBrowser, MessageBox

class MyForm(Form):
    def __init__(self):
        self.wb = WebBrowser()
        self.Controls.Add(self.wb)
        self.wb.ObjectForScripting = self
        self.wb.DocumentText = """
                               <html>
                                <body>
                                 <button onclick="window.external.Test('hello')">
                                   call client code from script code
                                 </button>
                                </body>
                               </html>
                               """
    def Test(self, msg):
        MessageBox.Show(msg)
    
        
Application.Run(MyForm())

Unfortunately this doesn't work because ObjectForScripting is being set to an instance of the python type MyForm. In the CLR-world this type will be generated dynamically and will be something like IronPython.NewTypes.System.Windows.Form_1$0 and will not contain any of the python methods. You can verify this by doing clr.GetClrType(MyForm).GetMembers(). The easiest solution then is to define an interface in C# with the methods that you want on your type and implement that interface on your type. So define a simple interface like this:

public interface IWebBrowserInterop
{
    void Test(string message);
}

and in the python code make your class implement that interface:

from ClassLibrary1 import IWebBrowserInterop

class MyForm(Form, IWebBrowserInterop):
...

Now if you fire up the app and click on the button the MessageBox will popup. You just reached across from unmanaged javascript through COM/.NET to python

Caveat(s): This is not type-safe. The IronPython runtime caches the types that are generated. So if there is another class in the python code that is "similar" (same base classes/interfaces and not different __slots__-wise) then this type can be reused. So if you are not bound by other factors, you should try to use Managed JScript instead and the hosting APIs to interact with the python objects.

Turning your .NET object models dynamic for IronPython

srivatsn — Sat, 12 Apr 2008 05:16:00 GMT

Say you want to interop with a .NET library but you also want it to behave like objects in dynamic languages do. You want to be able to add/delete methods/properties to the object dynamically. In python you can do something like this:

class x(object):
    pass

y = x()
y.z = 42
dir(y)

And the dir(y) will contain z in it. Now if x were a .NET class instead of a python class would you still be able to do the same? Let’s try a simple .NET class

public class TestExt
{
}

From IronPython you would do something like this:

import clr
clr.AddReference("TestExtensions.dll")
from TestExtensions import TestExt
y = TestExt()
y.z = 42

But this code throws a AttributeError saying ‘TestExt’ object has no attribute ‘z’. What now? Enter Stage Right DLR’s extension mechanism. There are five methods that a .NET class can implement that have a special meaning to the binder if you tell it so. The methods are:

GetCustomMember – runs before normal .NET lookups
GetBoundMember – runs after normal .NET lookups
SetMember – runs before normal .NET member assignment
SetMemberAfter – runs after normal .NET member assignment
DeleteMember – runs before normal .NET operator access (there’s no .NET version – so its the only one)

A .NET class can implement these functions and mark them with a SpecialName attribute. The rule generated for binding now makes a call to Getter/Setter before and after the normal .NET binding is done. GetCustomMember/SetMember is called first and if it returns a value that is treated as the result of the member-lookup. This overrides any .NET members that may exist. But if you return OperationFailed.Value from the function, then it will proceed with the normal lookup as well. GetBoundMember/SetMemberAfter is called if the normal binding fails – that is there is no member of the name that it is trying to bind to. So with that in mind let’s modify the .NET class and add this:

Dictionary<string, object> dict = new Dictionary<string, object>();
[SpecialName]
public object GetBoundMember(string name)
{
    if (dict.ContainsKey(name))
        return dict[name];
    else
        return OperationFailed.Value;
}

[SpecialName]
public void SetMemberAfter(string methodName, object o)
{
    dict.Add(methodName, o);
}

Now if I try to do a y.z = 42, it works. I can assign y.z to a function as well and will be able to call y.z(). I could have overridden GetCustomMember and SetMember instead and the behaviour would be exactly the same since OperationFailed.Value is returned for member that are not in the dict but then there is this overhead involved for any .NET member lookup.

The SetMember function can choose to return a bool instead of void and in that case, the returnvalue would control whether further binding lookups happen or not.

So, what exactly is the use of all this, why would I want to do this? Consider writing an object model for something like the following xml file:

<foo>
<bar>baz</bar>
</foo>

Now I want to access this xml as foo.bar and the value of that should be baz. To implement this all that I would need to do is add the GetBoundMember method to the .NET XmlElement class to do a search and return another XmlElement or add a extension method to XmlElement. Now here is the part where things come a little unstuck in IronPython. Extension methods dont show up in reflection, so the IronPython support for that currently isn’t where we want it to be. There is a way around though. You can decorate an assembly with an ExtensionType attribute describing what type you are extending and with what class. After that you would need to register the assembly once so that those methods get injected at the right places. This might change in the future with a better way but this works for now. This is the code you’d need to implement:

[assembly: ExtensionType(typeof(System.Xml.XmlElement), typeof(TestExtensions.ExtClass.XmlElementExtension))]
namespace TestExtensions
{    
    public class ExtClass
    {
        static ExtClass()
        {
            Microsoft.Scripting.Runtime.RuntimeHelpers.RegisterAssembly(typeof(ExtClass).Assembly);
        }

        public static XmlElement Load(string fileName)
        {
            XmlDocument doc = new XmlDocument();
            doc.Load(fileName);
            return doc.DocumentElement;
        }
        public static class XmlElementExtension
        {
            [SpecialName]
            public static object GetCustomMember(object myObj, string name)
            {
                XmlElement xml = myObj as XmlElement;

                if (xml != null)
                {
                    for (XmlNode n = xml.FirstChild; n != null; n = n.NextSibling)
                    {
                        if (n is XmlElement && string.CompareOrdinal(n.Name, name) == 0)
                        {
                            if (n.HasChildNodes && n.FirstChild == n.LastChild && n.FirstChild is XmlText)
                            {
                                return n.InnerText;
                            }
                            else
                            {
                                return n;
                            }

                        }
                    }
                }
                return OperationFailed.Value;
            }
        }
    }
}

Now from IronPython do this:

import clr
clr.AddReference("TestExtensions.dll")
from TestExtensions import ExtClass
foo = ExtClass.Load("test.xml")
print foo.bar

This prints out “baz”. Sweet!

PyCon 2008

srivatsn — Tue, 18 Mar 2008 03:15:00 GMT

PyCon 2008 was a lot of fun. This was my first PyCon and until now my interaction with the python community had been nothing beyond a few mails in the IronPython forum. My experiences of conferences so far has been limited to Microsoft ones like TechEd and TechMela(the equivalent of MIX/TechEd in India). This was so very different from that - the mood was different. PyCon is organized by the community. Everything from stuffing swagbags to debugging WiFi issues was managed by volunteers. There seemed to be more hallway chats between everyone than between just organizers/attendees. The keynotes that interested me were Guido's (I hadn't seen his last year's, so all the Python3k stuff was new for me) and Fitz's keynote on software development in general(it was a very well structured talk). I was pleasantly surprised by the amount of positive buzz about IronPython. Apart from the two regular IronPython talks (Jim Hugunin's talk on IronPython, and Michael Foord's on Silverlight with IronPython) IronPython was mentioned in many other talks as well starting from Guido's keynote to Python.NET to Resolver systems' talks to even Van Lindberg's keynote on IP :). The sessions were at best most illuminating and at worst interesting.

So what did I get out of PyCon (other than four cool t-shirts) in the order of most tangible to least tangible.

Learnt a lot more things about python itself from talks. Some talks that I enjoyed a lot - How Import Does Its Thing, Unicode In Python, Completely Demystified, More Iterators in Action and Core Python Containers -- Under the Hood
Learnt about the different important projects in the community (like Django and its new features, about PyPy, PyGame, SqlAlchemy, zope. It will be a useful exercise to see which parts of IronPython get worked out by these . For example, Django uses metaclasses heavily.
The birds of a feather session was very interesting just in terms of hearing what people wanted in IronPython and what their priorities were. (For example a lot of them said VS integration is important to them, shipping standard libraries would be awesome, better performance with exceptions was discussed at length.)
Got introduced personally to many of the IronPython users and people whose mails I get to see in the mailing list and also met a lot of people in the hallways from various companies using python internally. The variety of companies who had set-up booths varied from YouTube to CCP to LucasArts!
A sense of the kind of stuff being developed in the python community and sense of what is important to the community and what is not. I don't claim to have grokked the community in just 3 days but I'm way better off than I was 3 days back.

I was mostly in listening mode the entire time. In some ways I learned more in the last 3 days than in the last 3 months (I know what the first comment is going to be - So you didn't learn anything in the last three months?)

Decorators to convert return statements to yield statements

srivatsn — Tue, 18 Mar 2008 01:07:00 GMT

We have this really cool tool that parses a python file into a AST and then morphs it according to some transformation and then spits out python code corresponding to the new AST. I've been playing around with it lately to come up with some transformations. One of the things I wanted to do was convert all return statements to yield expressions. That will give a good amount of coverage for yield. So this is the code I want to convert:

def f(a,b):
    return a + b
    
def g(f, *args):
    return f(*args)

f(3,42)
g(f, 3, 42)

Now, its easy enough to change a "return <expr>" to "yield <expr>" but now I cannot just call the function directly as f(). That will return the generator. I need to do f().next(). Now one naive approach would be to go change all the invocations of the function to f().next(). That would work except for the cases where I pass the function as a parameter to another function (like g in the example). Now when g calls calls f, it could blow up because it will get a generator instead of f's returnvalue. I could solve that also by doing replacing every f with lambda : f().next(). So the call to g become this:

g(lambda *args: f(*args).next(), 3, 42)

That almost solves the problem except that even fs which are not references to this function (like the locally scoped f variable inside the g method) will get replaced since in a parsed AST we don't have any idea of scope (that happens during the conversion of the Python AST to the DLR AST). Now come python decorators to the rescue. In python you can wrap a function with a decorator like so:

@dec
def f():
    pass

dec can be defined as a function. Then python will start treating f as dec(f) and any calls to f() will be semantically equivalent to dec(f)(). This is great because now python will take care of that little scoping problem for me. So now the morphed code is simply:

def yieldwrapper(func):
    f = lambda *args, **kwargs: func(*args, **kwargs).next()
    f.__name = func.__name
    return f

@yieldwrapper
def f(a,b):
    yield a + b

@yieldwrappeer
def g(f, *args):
    yield f(*args)

f(3,42)
g(f, 3, 42)

The name of the wrapper is also set to the function's name so that it can masquerade as the real function just in case anyone's looking. Except for a few weird issues, this worked for most of the tests.

DeepZoom in dynamic languages

srivatsn — Tue, 18 Mar 2008 01:03:00 GMT

There's been a lot of talk lately about the DeepZoom technology in Silverlight 2. If you have seen it yet you should definitely goto HardRock's memorabilia site. That is a 2 billion pixels image that you see in that 640X480 segment there. Instead of getting all those pixels down the wire and killing your system, you only get the pixels needed to display the zoom level you are in. To me one of the reasons this is cool is because it introduces a new paradigm of photo sharing. You can have a long shot picture that captures the essence of your album and then have smaller pictures embedded at the right places which can be zoomed into. I wanted to do that for some photos I'd taken when I went rafting with some friends in the Himalayas some time back. Here is the picture of the camp and other pictures embedded in approximate positions of where they were. (Ok I know it sucks, I pretend to have some artistic sense whereas I don't but in my defense I didn't know about deepzoom then or I'd have taken a good long range shot)

So how do I go about doing this? First thing is obviously you need Silverlight. Then if you are going to use a dynamic language (and there is no reason to use anything else) then you need to download dynamic silverlight. John Lam has a series of excellent posts that introduce dynamic silverlight. To create the deepzoom images, you would need DeepZoom composer. The DeepZoom composer is fairly straightforward. Import the pics you want in your project and drop on to the composer. Make sure that you zoom into the main image and drop the embedded images with the right size. Once your images are ready all you need is a simple app.xaml like so:

<Canvas x:Class="System.Windows.Controls.Canvas"
    xmlns="http://schemas.microsoft.com/client/2007"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    x:Name="parentCanvas"
    >
    <StackPanel>
    <MultiScaleImage ViewportWidth="1.0" x:Name="msi" Source="raftingcamp/info.bin"/>
    </StackPanel>
</Canvas>

Now you need some simple python code that hooks the mouse click/wheel event handlers and zooms in and out appropriately. Scott Hanselman has an excellent sample of that in C#. I just wrote python code that does the same thing (ofcourse its python so its simpler and more elegant :)). In the constructor, you need to load the xaml and hook up the events:

class DeepZoomScene(object):
    
    def __init__(self):
        self.scene = Application.Current.LoadRootVisual(Canvas(), "app.xaml")

        self.scene.msi.MouseLeftButtonDown += self.mouseButtonDown
        self.scene.msi.MouseLeftButtonUp += self.mouseButtonUp
        self.scene.msi.MouseMove += self.mouseMove
        self.scene.msi.MouseEnter += self.mouseEnter
        self.scene.msi.MouseLeave += self.mouseLeave

        if HtmlPage.IsEnabled:
            HtmlPage.Window.AttachEvent("DOMMouseScroll", EventHandler[HtmlEventArgs](self.mouseWheel))
            HtmlPage.Window.AttachEvent("onmousewheel", EventHandler[HtmlEventArgs](self.mouseWheel))
            HtmlPage.Document.AttachEvent("onmousewheel", EventHandler[HtmlEventArgs](self.mouseWheel))

Now its just a matter of writing the event handlers. For example, here is the scroll wheel handler (thanks to Pete Blois for his C# sample). Notice how using a dynamic language massively reduces the lines of code needed.

def mouseWheel(self, sender, args):
    if self.IsMouseOver:
        delta = args.EventObject.GetProperty("wheelDelta")
        if HtmlPage.Window.GetProperty("opera") != None:
            delta = -delta
        if delta == None:
            delta = -(args.EventObject.GetProperty("detail"))/3
            if HtmlPage.BrowserInformation.UserAgent.IndexOf("Macintosh") != -1:
                delta = delta*3

        if delta > 0:
            self.zoom(1.1, self.lastMousePos)
        else:
            self.zoom(0.9, self.lastMousePos)

(The entire python file is attached). After this, ideally all I'd have to do should be run chiron and create the xap. But there seems to be a bug in Silverlight beta1 wherein if I have the deepzoom files in both the xap then it refuses to read the read the file (this doesnt happen for normal jpgs - only with MultiScaleImage). So I xap'ed just the app folder by doing chiron /d:app /z:app.xap and then keeping my deepzoom folder along with the xap.

Note: The code is provided "as is" and all that - aka don't blame if it blows up.

What I like about Python

srivatsn — Tue, 22 Jan 2008 06:03:00 GMT

I've been dabbling with IronPython for the past two weeks now. For me it compares quite favourably to the experience I had driving in this country. I used to drive a manual transmission car back in India and I used C# primarily. When I came to the US two weeks back I started driving an automatic transmission vehicle and I started using python. Initially the car felt weird - it was like driving some toy car, every time I'd slow down and my hands used to reach for the gears and my legs kept going towards the non-existent clutch. Similarly, I'd constantly put semicolons and braces everywhere, add type information all over the place and new up everything, indent the code wrongly. Now I'm getting used to the automatic and python and probably somwhere down the line, it might make me lazy enough to start disliking the manuals and static languages. (Although I dont see that happening with C# seeing as how I'm still going to be doing a lot of coding in C#).

Anyway I thought, I'll come up with the list of things I find cool and/or useful (not necessarily the same) in python.

Lists and List Comprehension

Being a fan of LINQ this was one of the first things about the language that interested me. I think the creation and maintenance of lists in python is very light on syntax and though it might not be as powerful as LINQ it does what it does well.

List comprehension is an easy way of constructing new lists from existing ones. To get a list of all IronPython process Ids

procIds = [ x.Id for x in Process.GetProcesses() if x.ProcessName == "ipy"]

Or say you have some functions and parameters and you want to call them in a cross-productish way:

results = [ [y(x) for y in funcs] for x in params ]

It also supports filter, map, reduce (equivalent of where, select and aggregate in C#) and the usual suspects like sorted, reversed etc..

Sequence packing/unpacking:

Sequences (lists, tuples etc) can be constructed from loose elements or can be disintegrated into individual variables. For example this can be done:

t = 3, 30, 300
a,b,c = t

A tuple t is constructed from the loose values and then it is unpacked into three loose variables a, b and c. This enables a bunch of cool scenarios. For examples I can now return multiple values from a function and assign them intuitively:

def f():
    return "a", 3
a, b = f()

The return value of that function was just a tuple packed and unpacked for you. This also lets you do this to swap numbers which I find pretty cool:

a, b = b, a

A related feature is argument unpacking. You can pass a tuple to a function and ask it to unpack the tuple and distribute it to the parameters. All you need to do this put a * in front of the tuple while passing it. For example:

def Multiply(a, b):
    return a*b;
params = [ (4,5), (4, "thisrepeats!") ]
results = [ Multiply(*p) for p in params ]

You can also pass in a dictionary and assign specific values.

Multiply( **{'a':2, 'b':"c"})

Else clause

for loops (or try/catch blocks) can have else clauses: I dont have to have a flag now checking to see if the loop has exited on a break or otherwise:

for item in listToSearch:
    if item == theOneImLookingFor:
        print "Found it"
        break
else:  #loop fell through
    print "Sorry, its not there"

There's more but those probably warrant another post.