I've uploaded the World's Worst Paint Program, as seen in my PDC talk, to IronPython's CodePlex site:

 http://ironpython.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=28125#DownloadId=93331

 This is built using Visual Studio 2010 Beta 2 and the IronPython CTP for that release - so you'll need to download both of those.  I've also included the Python and Ruby code that I used during the demonstration in a file called code.txt.  The paint program it's self is in the final form at the end of the demo.

Ahh, performance…  It’s so much fun!  It’s easy to measure and as you work on it you can see immediate benefits (or not!).  I personally enjoy getting to spend time looking at interesting performance issues and I’ll use this as a time to plug IronPython 2.6 Beta 1 where we’ve continued our focus in this area.  Even more fun yet is when you compare and contrast different implementations - it can be both surprising and amusing.  Recently Robert Smallshire has been doing just that: http://www.smallshire.org.uk/sufficientlysmall/2009/05/22/ironpython-2-0-and-jython-2-5-performance-compared-to-python-2-5/ Obviously Robert has found a case in IronPython where something is performing horribly.  I’d like to take a look at what’s happening here and what we can do to improve this case. 

But before I look at what’s going on here I’d like to briefly say we love to hear feedback from users of IronPython.  We’ve have a mailing list you can join and give us your feedback on and bugs can be reported on our website.  We track both of these very closely and respond as quickly as we can – which is usually pretty fast.  I just mention that because it’s easier for us to improve IronPython when people let us know about a problem vs. us having to scour the web to find issues.  Hopefully this piqued interest will have more people trying out IronPython and giving us feedback.

Ok, but let’s get onto the fun technical details!  It all comes down to one line of code “Node.counter += 1”.  That’s a pretty innocent looking piece of code, how can it cause so many problems?  First you need to understand a little bit about how IronPython works.  IronPython internally keeps versions of type objects and uses this information to generate optimal code paths for accessing those objects.  So the next line of code in this example, “self._children = children” usually turns into something that’s like:

if (self != null && self.GetType() == typeof(PyObject) && self.PythonType.Version == 42) {

     self._dict["_children"] = <new value>;

}

What that one line of code causes then is for the version of the type to change and for this “rule” to be invalidated.  So we’ll figure out what to do, generate a new cached rule, only to have it not work the next time around.  The same thing applies to the calls to Node – we re-generate the “optimal” code again and again only to discover we have to throw it away.

So that sounds pretty bad but if you’re familiar with Python’s member access rules you’ll quickly recognize that this will be much faster than needing to do the resolution again and again.  That involves looking through the class hierarchy to see if we have any data descriptors or classes defining __setattr__ and it's nice to avoid all of that work.   So to get the best performance we require for the world to settle down and not be continuously changing.  We’re well aware of this problem with our implementation and we have continued to focus on improving it.  IronPython 2.0 already includes one mitigation so that we don’t need to continuously generate new code – instead we effectively patch new values into an existing piece of code.  In our IronPython 2.6 branch we’ve continued to improve performance here and significantly reduced the expense when things get mutated.  But it turns out that there’s an easy fix for this particular problem we’ve just never thought it that important of a problem to solve.

One of the amusing things about this is we’ve actually seen it before – Resolver Systems was having some performance issues with Resolver One  which is their advanced spreadsheet which is written in IronPython.  One of those issues turned to be a type which they both used frequently and mutated frequently.  This actually doesn’t seem to be a common idiom in Python code which is the reason we’ve never optimized for it before.  In this case though it turned out that they had found an optimization that improved performance on IronPython 1.0 but was destroying perf on IronPython 2.0!  For Resolver fixing this was a simple change for them to no longer mutate types.  Even if you really want to use a type as a property bag you can always move that to a type which you aren’t creating instances of and executing methods on.

With all that in mind let’s look at some numbers.  Here I’ve compared IronPython 2.01, IronPython 2.6 Beta 1, CPython 2.6 plus IronPython 2.6 with a fix for this issue.

 

Like Robert’s charts I have tree size in number of nodes across the bottom and execution time in seconds across the top and the execution times are logarithmic.  We can see the abysmal performance of 2.0.1, we can see that 2.6B1 has already made significant improvements, and with IronPython 2.6B1 plus a fix for this specific issue we actually beat CPython as the number of iterations scales up.

But there’s another way we can look at this – instead of fixing IronPython let’s look what happens if we fix the benchmark to avoid this performance pitfall.   To do this I’ve replaced the mutation of the class with a global variable.  When running against all of the same implementations again I get these results:

 

Here you can see that all 3 versions of IronPython are performing about the same and are coming in significantly faster than CPython as the number of iterations goes up.  These are now also matching the numbers Robert is getting when he has made the same change.

So you might be curious - what’s the fix?  Well in this case we’re just mutating the type by changing a simple value stored in the type (versus say adding a property or a method to the type).  Today we get a very slight throughput performance benefit because we can burn the value directly into the generated code instead of fetching it from the wrapper the type object keeps it in.  With the fix we’ll burn the wrapper in and fetch the value from the wrapper each time.  Then when we go and modify the type we just modify the wrapper instead of the type and its version is unchanged!  This fix is now checked into the IronPython source tree and you can download the fixed sources right now. 

In conclusion I’d like to wrap up a little bit about our performance philosophy – performance is hard and deciding which numbers matter to you is important.  We have tried to tune IronPython for apps which run for more than a fraction of a second.  We have also tuned IronPython for code which is not constantly mutating types at runtime.  And we’re always open to feedback when users encounter problems with IronPython and we’ll listen to what scenarios are important for them.  But if you look at these graphs I think you can see that we’ve done a good job at making IronPython a fast implementation of Python.

 

Over the past week and a half I've spent a little bit of time getting Django 0.96.1 running on IronPython.  The 1st step in doing this was getting a database provider that would run on .NET that would work with Django.  For DB backends Django basically follows PEP 249 with a few extensions.  Here's the basic DB provider I came up with.  It's not quite complete but it was good enough for me to get Django's tutorial running.  To use this you just need to copy one of the existing DB backends and replace the code in base.py with the code below.  For the rest of the code you can pull it from ado_mssql. 

 Basically this just uses the .NET System.Data namespace to do the communication with the database server.

import clr
clr.AddReference('System.Data')
from System import Data
from System.Data import SqlClient
import System

DatabaseError = Data.DataException

from threading import local

class SqlCursor(object):
    arraysize = 1
   
    def __init__(self, connection):
        self.connection = connection
        self.transaction = None
        self.reader = None
        self.record_enum = None
   
    def execute(self, sql, params=()):       
        parameters = []
       
        # translate to named parameters
        if type(params) in (list, tuple):
            # indexed params, replace any %s w/ @GeneratedName#
            if sql.find('%s') != -1:
                cmd = ''
                sqlSplit = sql.split('%s')
                for text, value, index in zip(sqlSplit, params, range(len(params))):
                    cmd += text + '@GeneratedName' + str(index)
                    parameters.Add(SqlClient.SqlParameter('@GeneratedName' + str(index), str(value)))
                
                sql = cmd + sqlSplit[-1]
        else:
            for name, value in params.iteritems():
                sql = sql.replace('%(' + name +')s', '@' + name)
                parameters.Add(SqlClient.SqlParameter('@' + name, str(value)))
       
        command = SqlClient.SqlCommand(sql, self.connection)
       
        for param in parameters: command.Parameters.Add(param)
       
        self.record_enum = None
        self.reader = command.ExecuteReader()
   
    def close(self):
        self.reader.Close()
   
    def executemany(self, sql, param_list):
        res = []
        for s in sql:
            res.append(execute(s, param_list))
        return res
       
    def fetchall(self):
        return [self._make_record(record) for record in self.reader]           
   
    def fetchone(self):
        if self.record_enum is None:
            self.record_enum = iter(self.reader)
        if self.record_enum.MoveNext():
            return self._make_record(self.record_enum.Current)
        return None
       
    def fetchmany(self, size=None):
        if size is None: size = SqlCursor.arraysize
        res = []
        for i in range(size):
            x = self.fetchone()
           
            if x is None: break
           
            res.append(x)
        return res

    def _make_record(self, record):
        return tuple((self._fix_one_record(record[i]) for i in xrange(record.FieldCount)))

    def _fix_one_record(self, record):
        if type(record) is System.DateTime:
            return datetime.datetime(record)
       
        return record
   
    @property
    def rowcount(self):
        if self.record is not None:
            return self.reader.RecordsAffected
        return -1
   
class DatabaseWrapper(local):
    def __init__(self, **kwargs):
        self.connection = None
        self.queries = []
        self.transaction = None

    def cursor(self):
        from django.conf import settings
        if self.connection is None:
            if not settings.DATABASE_HOST:
                settings.DATABASE_HOST = "(local)"
            if settings.DATABASE_NAME == '' or settings.DATABASE_USER == '':
                conn_string = "Data Source=%s;Initial Catalog=%s;Integrated Security=SSPI;MultipleActiveResultSets=True" % (settings.DATABASE_HOST, settings.DATABASE_NAME)
            else:
                conn_string = "Data Source=%s;Initial Catalog=%s;User ID=%s;Password=%s;Integrated Security=SSPI;MultipleActiveResultSets=True" % (settings.DATABASE_HOST, settings.DATABASE_NAME, settings.DATABASE_USER, settings.DATABASE_PASSWORD)
           
            self.connection = SqlClient.SqlConnection(conn_string)
            self.connection.Open()
           
        cursor = SqlCursor(self.connection)
        if settings.DEBUG:
            return util.CursorDebugWrapper(cursor, self)
        return cursor

    def _commit(self):
        if self.transaction is not None:
            return self.transaction.Commit()

    def _rollback(self):
        if self.transaction is not None:
            return self.transaction.Rollback()

    def close(self):
        if self.connection is not None:
            self.connection = None
            self.transaction = None

Well, now that our RTM and RC-esque builds are starting to make it out into the wild it seems like a good time to discuss a feature we had to make go away...

That feature which most readers of my blog are probably now familiar with is fiber mode.  So why did we cut fiber mode for the released version of the product?  And what changes will you see because of this?

We had gone through great efforts to make sure that all of our core functionality worked great inside of a fiber mode host.  Not only did the CLR team do a lot of work to make sure fiber mode was solid, but so did many other teams who picked up our toolset and verified that worked properly by running their tests inside of fiber mode.  Not to mention the teams that typically run their tests inside of Yukon who also make sure their features work in fiber mode.  We even spent extra time at the end of beta 2 to make sure we got all the last minute fiber bugs fixed from our full fiber mode test pass - but ultimately it wasn't enough.

In the end we had one final exit criteria we'd have to pass in order to make sure fiber mode was solid:  Stress.  Between the choices of spending our time fixing the bugs that we knew ALL of our customers would hit in thread mode, or fixing the bugs that a small percentage of users would hit in fiber mode, we picked fixing thread mode bugs.  In the end I think most customers will be happy as the SQL and CLR teams came together and we've spent a huge amount of time, resources and effort ensuring that SQL/CLR works great under thread mode.

For those of you wanting to develop a fiber mode CLR solution I think you need to first ask yourself why you're doing this.  If you're attempting to conserve stack space then this is not the solution you're looking for.  If you're attempting to reduce the number of context switches experienced then you can still get much the same results using thread mode and blocking "switched out" tasks on an event that gets signaled when it's their turn to run.  And of course there's a 3rd, although usually less desirable, option to redesign the way you're approaching the problem of a large number of work items.

Finally what are the changes in removing fiber mode?  Surprisingly everything will work exactly the same as it did in pre-RC and RTM builds except for one API will now return E_NOTIMPL.  That one API is ICLRTask::SwitchOut.  You still can do all the interesting stuff with integrating the CLR task management with your own environment, and you can still switch a logical task in on a thread that previously had another logical task running on it, but you can't switch out a logical task during the middle of its lifetime. 

I realize removing this feature will cause some of you some pain but I hope you'll see that this was the right call for us to get a super stable Whidbey out the door in a reasonable timeframe.

 

 

In this article we’re going to look at the implementation of Abort and RudeAbort.  In the last article I mentioned that a typical host wouldn’t need to expose these to managed code.  Why is that?  If you have a fiber mode scheduler typically Thread.Abort will be sufficient.  Your scheduler will eventually schedule the task, and the task will then be aborted.

 

But in the CoopFiber sample we don’t have a scheduler.  Therefore a user could call Thread.Abort on a switched out fiber, and it would never get scheduled to be aborted.  What’s worse is that the call into ICLRTask::Abort will block until the thread gets scheduled again.  

 

To handle this peculiarity when an Abort request comes in the Fiber class first determines whether the task is running or not.  If it is already running then we’ll go ahead and abort it right away.  Otherwise we’ll set a bit to notify that on the next switch we should abort the task.  Here’s the Fiber.Abort implementation:

 

bool fAbortNeeded = false;

 

lock (m_syncObj)

{

      m_FiberStates |= FiberStates.AbortRequested;

      if ((m_FiberStates & FiberStates.Running) != 0 && (m_FiberStates & FiberStates.Switching) == 0)

      {

            //the current fiber is aborted, so we can initiate the abort now.

            fAbortNeeded = true;

      }

}

if (fAbortNeeded)

{

      InternalAbort(m_fiberAddr);

}

 

InternalAbort will just turn around and call ICLRTask::Abort on our already switched in task.  In our last article we saw our calls to CheckForAbort during task switch in.  That code simply looks like:

 

// Next see if the user requested an abort on this thread.

if ((m_FiberStates & FiberStates.AbortRequested) != 0)

{

Abort();

}

else if ((m_FiberStates & FiberStates.RudeAbortRequested) != 0)

{

      RudeAbort();

}

           

 

We simply call back to Abort or RudeAbort, which will then call into InternalAbort to finally Abort/RudeAbort the task.  The task is now switched in and running so we’ll call into the host to perform the actual abort (which now won’t block because the task is running).

 

At this point you may be asking yourself “What is this Rude Abort thing?”  Rude aborts are actually a new Whidbey feature, and can be accessed one of several ways.  The only difference between a rude abort and a normal thread abort is that the rude abort will not execute your catch/finally blocks.  In addition to being exposed directly through the hosting APIs rude abort’s can also occur due to escalation policy (also configured via the hosting APIs).  That’ll have to be saved for another blog entry…

 

CoopFiber’s Rude Abort implementation is essentially identical to the Abort implementation.  The only difference is that we’ve replaced the work Abort with RudeAbort everywhere.

 

So that’s it for the Abort/Rude Abort implementation.  We’ve almost reached the end of this serious, but there’s one last API that’s worth covering on the managed size: Fiber.Exit.  I’ll go over that implementation next article.

It seems like a good time to take a breather and look at what we have going on so far. 

 

First, we’ve started the runtime.  We’ve handed our IHostControl off to the runtime.  The runtime has called back to our IHostControl and gotten a couple of managers.  Those are our task manager and our synchronization manager.  Whenever the runtime needs to create a thread it’s called us back, and we’ve created a brand new thread.  Sure, we’ve converted it into fibers, but we’ve done no real fiber switching. 

 

Whenever the runtime needed a manual or auto event we’ve given it to it via the synchronization APIs.  When it needed to block on it, it called us, and we blocked on it via IHostAutoEvent or IHostManualEvent.  If another thread alerted the thread blocked on the event, the runtime called us via the IHostTask APIs, and we queued and APC to it. 

 

If the runtime needed to enter a critical section, it called us, and we blocked or allowed it to enter via the IHostCrst APIs. 

 

We’ve set up everything necessary to replace the CLR’s threading model, but for the most part we’ve merely delegated to the OS APIs.  Sure, there were minor differences here and there…  But for every task the CLR asks for we create 1 physical thread.  And sure every one of those physical threads has already been converted to a fiber, but we never switch those fibers out. 

 

Starting with the next article that’ll start to change.

The synchronization primitives are handled off to the runtime by the IHostSyncManager interface.  We’ve already provided this to the runtime through our GetHostManager callback on IHostControl.  Our IHostSyncManager is implemented on our IHostTaskManager and for the most part we just create objects and hand them off.  For example CreateCrst looks like:

 

      *ppCrst = (IHostCrst *)new CHostCriticalSection(this);

      if(NULL == *ppCrst)

      {

            return(E_OUTOFMEMORY);

      }

      (*ppCrst)->AddRef();

 

And none of them are very different.  Our SetCLRSyncManager does nothing.  A more sophisticated host could use the ICLRSyncManager to help perform deadlock detection.

 

The CoopFiber implements 4 synchronization primitivies: Auto Events, Manual Events, Critical Sections, and a Semaphore.  All of these with the exception of the critical section are just thin wrappers over the equivalent OS API.  For example looking at CHostManualEvent::Wait:

 

      DWORD result;

      if(option & WAIT_ALERTABLE)

      {

            result = WaitForSingleObjectEx(m_hEvent, dwMilliseconds, true);

      }

      else

      {

            result = WaitForSingleObject(m_hEvent, dwMilliseconds);

      }

 

      switch(result)

      {

            case WAIT_OBJECT_0:

                  return(S_OK);

            case WAIT_ABANDONED:

                  return(HOST_E_ABANDONED);                

            case WAIT_IO_COMPLETION:

                  CHostTask::GetCurrentTask()->FlagSet(TASK_FLAG_ALERTED, false);

                  return(HOST_E_INTERRUPTED);

            case WAIT_TIMEOUT:

                  return(HOST_E_TIMEOUT);

            default:

                  _ASSERTE(!"Shouldn't reach here");

                  return(E_FAIL);        

      }

 

We can see an implementation that is nearly identical to Join we saw last time.  Both the auto event and semaphore implementations are nearly identical.  More sophisticated hosts could perform deadlock detection below these events (for reader/writer locks and monitors built on top of the events), or they could choose to schedule other fibers on these threads.  For this simple implementation we simply block the current thread.

 

The critical section is the only synchronization primitive which doesn’t simply wrap the OS API.  This is because the critical section is owned by a specific thread (unlike the events and semaphore).  Because of this the critical section must be aware of the fibers in addition to threads.  This will prevent one fiber from acquiring the critical section, and having another fiber re-acquire on the same thread.

 

The essence of the critical section acquire lives in TryEnter, which Enter builds on:

 

 

      if(m_holderTask == NULL)

      {

            CritSecHolder cs(&m_critSec);

 

            // no one holds the critical section

            if(m_holderTask == NULL && WaitForSingleObject(m_hEvent, 0) == WAIT_OBJECT_0)

            {

                  // we've acquired the crit section

                  m_holderTask = curTask;

                  m_dwEnterCount = 1;

                  *pbSucceeded = TRUE;

                  return(S_OK);

            }

      }

      else if(m_holderTask == curTask)

      {

            CritSecHolder cs(&m_critSec);

            // we already hold the critical section

            if(m_holderTask == curTask)

            {

                  m_dwEnterCount++;

                  *pbSucceeded = TRUE;

                  return(S_OK);

            }                

      }

 

      return(S_OK);

 

There’s a couple of points to note.  First, we use a critical section to protect our state.  We don’t want to worry about other threads entering.  Also we use an event to block the task if the critical section cannot be acquired.  A more sophisticated fiber scheduler would want to re-schedule another fiber rather than block the current thread.

 

Those are the core synchronization primitives that we’re using in CoopFiber.  You can see by limiting the scope of what CoopFiber does we were able to merely rely upon the OS APIs in most circumstances.  Next time I’ll start discussing the managed / unmanaged fiber API interface.

Last time we left off with the CLR calling into the host to create a task.  So far everything’s been very simple.  For each interface we’ve only needed to implement a couple of methods to get the bulk of the work done.  While I’ve certainly left out a couple of lines of code here and there, nearly every other API in the interface merely returns S_OK.

 

Now things start to get really interesting!  The IHostTask interface has just a few functions which are exposed to the runtime.  But there’s a lot of other functionality related to the task embedded in here.

 

We’ll start off at the natural place to start a task: Start.  It’s pretty simple:

 

      FlagSet(TASK_FLAG_NOTSTARTED,false);

 

      _ASSERTE(m_fiberAddr == NULL);

      m_fiberAddr = ::CreateFiber(0,(LPFIBER_START_ROUTINE )CHostFiberProc,(PVOID)this);

 

      // create the new thread

      ::CreateThread(NULL,NULL,reinterpret_cast<LPTHREAD_START_ROUTINE>(::StartNewThread),this,NULL,NULL);

 

    return(S_OK);

 

 

We create a fiber for the new task, and we create a new thread.            This new thread starts in StartNewThread, defined in callbacks.cpp.  It looks something like:

 

      CHostTaskManager *myManager = CHostTask::GetManager();

      CHostTask *curTask = static_cast<CHostTask*>(lpParameter);

 

      LPVOID fiberAddr = ConvertThreadToFiber(NULL);

           

      curTask->SwitchIn();

     

      BOOL fResult = ConvertFiberToThread();

 

Here we simply convert the new thread over to fibers, and then switch in the fiber passed as the argument.  This brings us back to our CHostTask implementation where we need to look at the SwitchIn logic.  This is the most complicated piece of code we’ve encountered yet.

 

       CHostTask *curTask = GetCurrentTask();

       _ASSERTE(curTask != this);

      

       TlsSetValue(CHostTask::CurTaskTlsIndex,this);

       this->AddRef();

 

       // Save our thread handle so we can queue APCs

       if(!DuplicateHandle(GetCurrentProcess(),

GetCurrentThread(),

GetCurrentProcess(),

&m_hCurThread, 0 , FALSE,

DUPLICATE_SAME_ACCESS))

       {

              m_hCurThread = INVALID_HANDLE_VALUE;

              _ASSERTE(!"Duplicate handle failed");

       }

 

       FlagSet(TASK_FLAG_RUNNING, true);

       ::SwitchToFiber(GetFiberAddress());     

 

       // when the fiber switches back we need to switch in our

       // previous task.

       if(curTask->m_pCallback!=NULL)

       {

              HRESULT hr = curTask->m_pCallback->SwitchIn(GetCurrentThread());

              _ASSERTE(SUCCEEDED(hr));

       }

      

       if(curTask->FlagCheck(TASK_FLAG_EXITING))

       {

              _ASSERTE(curTask->GetSwitchingTo());

              curTask->ExitTask();

              curTask->GetSwitchingTo()->SwitchIn();

       }

 

       // release the ref for TLS from the previous thread

       CloseHandle(m_hCurThread);

       m_hCurThread = INVALID_HANDLE_VALUE;

       Release();

             

       SetThreadPriority(GetCurrentThread(), curTask->m_iPriority);

       SetThreadLocale(curTask->m_lcid);

 

 

What’s going on here?  We have a couple of things to worry about when switching tasks.  We have the task that we’re switching to (this) and we have the task we’re switching from (curTask).

 

One of the issues we’re concerned with is the lifetime of the task.  While it’s running we don’t want it cleaned up, so we hold a reference to it (in Thread Local Store).  A more complicated host would probably have a pool of tasks rather than the simple TLS mechanism. 

 

If we need to alert a task we’ll queue an Asynchronous Procedure Call (APC) to it.  Therefore the next thing we do is save the current thread’s handle into this task.  We’ll use this in IHostTask::Alert.  Finally we set the running flag and switch over to the new task.

 

The interesting thing to note about this method is there are 2 halves to it.  After we call SwitchToFiber we are running on a different fiber on a different stack.  We’ll only return to the bottom half after someone has switched back to “curTask”.  This is the task that we switched away from.  When this happens it’s now the bottom half’s job to tell the runtime that “curTask” is once again running.

 

It’s possible when we get to the bottom half “curTask” was only switched in to exit.  If so we’ll notify the runtime and immediately re-run the task we’ve been set to re-run.  This is an implementation detail of CoopFiber to allow calling ExitTask on the managed fiber implementation. 

 

We’re nearly done so we clean up the resources we allocated in the top half for “this”.  It’s no longer running, so we don’t need a reference to it.  Finally we restore the settings for the thread that were stored in the task.  These would have been changed when we did the intital SwitchToFiber which either switched in a task that was at the bottom half of SwitchIn, or the top of CHostFiberProc (in callbacks.cpp).

 

Wow, so that’s how we start a task!  We create a new thread, that thread gets converted over to fibers, and we switch in the newly created task (which has a fiber already associated with it).  That’ll end up in CHostFiber proc which we’ll cover in a future edition. 

 

There’s just one more detail to cap off the life time of a task, and that is our internal SwitchOut API.  All it essentially does is set some state and notify the runtime of the switch out:

 

      FlagSet(TASK_FLAG_RUNNING,false);

 

      if(m_pCallback!=NULL)

      {

            HRESULT hr = m_pCallback->SwitchOut();   

      }

 

 

Next time we’ll go over the remaining APIs on IHostTask that we haven’t covered yet.