Kael Rowan

Code Canvas @ TechFest

In honor of today's historic date of breaking down walls, I'm posting a recording of the live presentation of Code Canvas given at the 2009 Microsoft Research TechFest event on the Microsoft campus in Redmond, WA.

This video demonstrates some of the features of Code Canvas that are applicable when zoomed in close to your code, such as grouping by folders, organization by type hierarchy, references and off-screen previews, screen-constrained search results, code coverage overlays, and code execution traces, including multi-threaded execution traces showing thread synchronization.

Code Canvas @ Lang.NET Symposium

Microsoft recently hosted the 2009 Lang.NET Symposium here in the Microsoft Research building.  A colleague of mine, Herman Venter, asked me to show off a demo of Code Canvas during his Common Compiler Infrastructure talk, and the video is available online:

http://www.langnetsymposium.com/2009/talks/26-HermanVenter-CCI.html (Download)

The segment on Code Canvas starts about 5 minutes into the video and lasts for about 2.5 minutes.   It also shows off some new features not seen on the previous video.  Enjoy!

Softpedia article on Code Canvas

Wow, I’m impressed with how fast the word is getting out.  Marius Oiaga already published an article on Softpedia:

“What if code could leave the boundaries of the traditional editors, and call home an infinite two-dimensional surface as a new spatial development environment? Code Canvas makes this possible. A project put together by Kael Rowan, an RSDE in the Human Interactions in Programming group at Microsoft Research, Code Canvas is designed to offer developers a spatial layout and Deep Semantic Zoom for source code. And an entirely new perspective over building software. Essentially, Code Canvas is meant to catalyze the evolution of the software development experience via spatial memory.”

Read the whole article here: Code Canvas: Spatial Representations of Code

Code Canvas

I’ve been working in Microsoft Research on a project called Code Canvas for a while now, and we finally got through enough of the patent process so that I’m able to release some details publically.

Code Canvas is a research prototype focused on spatial orientation of code as the foundation of an integrated development environment. It is a spatial (2.5D) representation of source code, visual designers, and project-related artifacts that utilizes infinite panning and smooth semantic zoom for navigation. It is also extensible to allow analysis overlays and graph-based relationship visualizations.

Whenever a developer draws their code on a whiteboard, they are applying a sense of space to their software that includes directional relationships and architectural boundaries. Code Canvas lets developers write their code on a two-dimensional infinite canvas instead of in tabbed editors, so all of their source code is arranged in the same way as it would be on the whiteboard.  They can still write code as they do today, in C# or C++ or whatever, but the directional relationships and architectural boundaries are part of the same canvas, and they can easily navigate and zoom smoothly in and out to understand everything at once.

In addition to utilizing spatial memory, the ability to overlay visualizations among multiple files at once is proving to be extremely valuable, especially when the visualizations are drawn directly on top of the same source code that you have already been working with.  This essentially takes the canvas from a simple code editing surface and turns it into a full-blown diagramming surface, but I’ll have to go more into that later…

In the mean time, here is a somewhat old video of Code Canvas that shows some of the first features I implemented when getting started on this project.

Blog Resurrection

Sometimes in the course of software development it becomes best to start over from scratch.  I'm taking this approach with my blog.  I first joined Microsoft in 2000 as part of the Windows XP Embedded group.  In 2004 I joined the Windows Driver Foundation group and created a managed (.NET) interface to the User Mode Driver Framework.  That was also when I attempted my first series of blog posts which were going to describe how to write a driver in C# to control a small USB device.  My management didn't feel comfortable with blogging at the time and 'encouraged' me to discontinue the series.  I left Windows and joined Microsoft Research in 2006 as part of the Foundations of Software Engineering group to work on model-based program analysis and Spec#.  I still wanted to contribute to the blogging community, so I started a second set of blog posts that gave solutions to random problems I had solved that I thought may be helpful to others.  There wasn't any consistency among the topics and eventually the blog became stagnant again.

I'm currently an RSDE in the Human Interactions in Programming group in Microsoft Research.  I'm responsible for the design, architecture, and implementation of applications and tools focused on the next generation of software development.  Over the years I've accumulated a small but useful library which I call "stuff that should have been in System.dll".  For example I've defined simple things like Tuple<T1, T2> through Tuple<T1, ..., T9>, utility things like a generic Cache<...> that uses weak-references, and more complex things like a ThreadQueue which schedules threads on the ThreadPool that can't run in parallel or out of order.  None of these things are rocket science but they prove to be useful time and time again.  I'll try to fill the gap in my blog by posting some of these and pre-dating them to match the date they were created.  Thanks for reading!

Presentation.More

In the same vein as System.More, I've also created a Windows Presentation Foundation counterpart called Presentation.More.  And just like System.More defines classes in the same System namespaces as System.dll and System.Core.dll, Presentation.More also defines classes in the System.Windows namespaces directly.  Not only does this work elegantly for code-based languages like C# and VB, but it also works in XAML by simply adding XmlnsDefinitionAttributes to the Presentation.More assembly.  So far I have classes in six different namespaces under System.Windows, so I have a file called XmlnsDefinitions.cs in my Presentation.More project that looks like this:

using System.Windows.Markup;

[assembly: XmlnsDefinition("http://schemas.microsoft.com/winfx/2006/xaml/presentation", "System.Windows")]
[assembly: XmlnsDefinition("http://schemas.microsoft.com/winfx/2006/xaml/presentation", "System.Windows.Controls")]
[assembly: XmlnsDefinition("http://schemas.microsoft.com/winfx/2006/xaml/presentation", "System.Windows.Data")]
[assembly: XmlnsDefinition("http://schemas.microsoft.com/winfx/2006/xaml/presentation", "System.Windows.Documents")]
[assembly: XmlnsDefinition("http://schemas.microsoft.com/winfx/2006/xaml/presentation", "System.Windows.Input")]
[assembly: XmlnsDefinition("http://schemas.microsoft.com/winfx/2006/xaml/presentation", "System.Windows.Media")]

For example, one of the classes is a NineGridImage which lives in namespace System.Windows.Controls.  Without XmlnsDefinitionAttributes, your application's XAML would have to declare the namespace and use it everywhere you wanted to use the control:

<Window x:Class="WpfApplication1.Window1"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:more="clr-namespace:System.Windows.Controls;assembly=Presentation.More">
    <more:NineGridImage Source="..."/>
</Window>

But by adding the XmlnsDefinition attribute to the Presentation.More assembly for the System.Windows.Controls namespace above you can use a NineGridImage just like you'd use any other built-in control:

<Window x:Class="WpfApplication1.Window1"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <NineGridImage Source="..."/>
</Window>

Some may think this is confusing or even unethical, but I think it makes the resulting application code clearer and more elegant.  I don't suggest overriding the System namespaces for every class, especially when a class is specific to a particular application.  But for classes that have nothing to do with a particular client application and "should have been in System.dll" or "should have been in PresentationFramework.dll", I think it's best to treat them as part of the framework rather than bring attention to yet another namespace.

System.More

I, like most developers out there, have accumulated a set of classes and utility functions that have been used repeatedly in multiple projects throughout my career.  This code has followed me around in various forms; sometimes by copying the files directly to new projects, and other times by creating a shared DLL that's referenced as a dependency instead.  Whether the files were copied in their own folder or compiled into a DLL, the question always remained of "what should these be called"?  They've gone by many names, such as "common", "shared", "utilities" and even just "utils".  I've never been satisfied with any of these, and they don't convey the meaning of what I like to call "stuff that should have been in System.dll".  One day, after looking through the .NET base class library, I noticed that some of the most basic types in the System namespace (like Action and Func) are not in System.dll or mscorlib, but are actually defined in System.Core.dll.  So in the same vein I decided to name my library "System.More" with all of the same expectations thereof.  I stripped out all of the "utility"-like classes and methods, including implementations of algorithms such as traveling salesman and huffman coding, because I couldn't see those belonging in System.dll.  Instead, System.More contains classes like Tuple and PriorityQueue that are defined in namespace System and System.Collections.Generic respectively.  This way, new projects can simply add a reference to System.More.dll (just like adding one to System.Core.dll) to get all of the new classes and extensions easily and elegantly.

Comparator

I recently wrote a generic PriorityQueue class which can treat any type of object as the priority (not just integers).  Specifically, the class looked like this:

public class PriorityQueue<T, TPriority>
{
    public PriorityQueue()
        : this(false)
    {
        ...
    }

    public PriorityQueue(bool invert)
        : this(Comparer<TPriority>.Default)
    {
        ...
    }

    public PriorityQueue(IComparer<TPriority> comparer)
    {
        ...
    }

    public void Enqueue(T item, TPriority priority)
    {
        ...
    }

    public KeyValuePair<T, TPriority> Dequeue()
    {
        ...
    }

    ...
}

Since it allowed any type of object as the priority object, it obviously can't use the standard numerical < and > operators to compare them.  Many classes in the .NET framework (such as List<T>) have this same pattern, so the framework provides an IComparer<T> interface and a Comparer<T> base class to support comparison of arbitrary types.  The interface only has one method you have to implement: int Compare(T x, T y).

Every time I implement IComparer I think it's silly that I have to create a new class (which usually also means a new file) that just defines one method.  In fact, the int Compare method is usually just a one-liner like String.Compare(x.SomeProperty, y.SomeProperty) since all I want to do is sort items based on SomeProperty.  Why do I have to create a whole new class (and whole new file) just to specify String.Compare(x.SomeProperty, y.SomeProperty)?

Obviously someone in the .NET framework team felt the same way, so they introduced a Comparison<T> delegate and added overloads to Array.Sort<T> and List<T>.Sort which accept this delegate instead of requiring an implementation of the IComparer interface.  Unfortunately they only added overloads for those two methods; there are still over twenty other methods in the .NET framework that require the IComparer<T> interface, including LINQ's IEnumerable<T> extension methods.

In order to use the single-delegate idea in all of the places that still require the interface, I created a new class called Comparator<T>:

public class Comparator<T> : Comparer<T>
{
    private readonly Comparison<T> comparison;

    public Comparator(Comparison<T> comparison)
    {
        this.comparison = comparison;
    }

    public override int Compare(T x, T y)
    {
        return this.comparison(x, y);
    }
}

This is probably the simplest class in my library.  It just takes a Comparison<T> delegate in the constructor and implements Comparer<T> (which also implements IComparer and IComparer<T>) by simply calling the comparison function.  This tiny little class allows you to use all of the IComparer-based methods and constructors (such as new SortedList(comparer), new SortedDictionary(comparer), IEnumerable<T>.OrderBy(), etc) without having to create a new class every time.

SynchronizationCallback

It's common for user interface frameworks (such as Windows Forms applications or Visual Studio packages) to restrict API calls to a single thread; specifically the "UI" thread that is usually the original or "main" thread in the process.  This means that if you spawn a background thread then it can't touch the user interface's objects and must use some other synchronization mechanism to touch the user interface on the main thread instead.  For example in the case of WinForms you can call Invoke or BeginInvoke if you have a reference to a Control.  This usually isn't too much of a burden for most application code since it can easily get access to a Control instance, but it makes it harder for reusable library code to offload things to a background thread in a generic way since it may not have a convenient way to get "back" to the main thread.  It might seem like library code wouldn't need to worry about this since it usually doesn't touch the user interface objects, but the problem comes back because of event handlers.  Often the user interface code will attach event handlers to the reusable library's objects, and if those events are raised from a background thread then the event handler will also be executing on the background thread and will be prone to touching the user interface inappropriately.  This can be guarded against by always having every event handler call Invoke, but that's a huge amount of code overhead just to protect against something that the library might do.

Fortunately, there's a generic class called SynchronizationContext with methods Send and Post (which are analogous to Invoke and BeginInvoke) that is always accessible via the static SynchronizationContext.Current property.  User interface frameworks (like WinForms) override the current SynchronizationContext so that the Send and Post simply call the appropriate Invoke and BeginInvoke methods respectively.  This means that if your library code always uses Send when raising events from its background thread(s) then user interface code will be safe by default.  However, there's a catch: SynchronizationContext.Current is only set in the main thread (which is the only thread where you don't actually need it), as shown with the following code:

public class MyLibraryClass
{
    public void CalledInMainThread()
    {
        Debug.WriteLine("Main Thread: SynchronizationContext.Current is " + (SynchronizationContext.Current != null ? SynchronizationContext.Current.ToString() : "null"));

        Thread t = new Thread(new ThreadStart(CalledInBackgroundThread));
        t.Start();
    }

    private void CalledInBackgroundThread()
    {
        Debug.WriteLine("Background Thread: SynchronizationContext.Current is " + (SynchronizationContext.Current != null ? SynchronizationContext.Current.ToString() : "null"));
    }
}

Prints output:

Main Thread: SynchronizationContext.Current is System.Windows.Forms.WindowsFormsSynchronizationContext

Background Thread: SynchronizationContext.Current is null

The obvious way to work around this is to grab the reference to SynchronizationContext.Current in the main thread and then make it available to the background thread by either storing it in a shared field or by passing it as the object state argument to those callbacks that support it.  The drawback with this method is that the code running in the background needs to keep passing the reference around to everyone who needs it, which kind of defeats the purpose of the static global SynchronizationContext.Current property. 

Fortunately there is a SynchronizationContext.SetSynchronizationContext method that sets the value of the SynchronizationContext.Current property, so if the first method in the background thread always calls SetSynchronizationContext with the instance from the main thread then all of the other code in the background thread can use it without having to pass the reference around.  This also makes it so you can write your code without having to care whether it's going to be called on the main thread or on a background thread since you can assume SynchronizationContext.Current will always be set appropriately.

I use this pattern quite often, so I created a new type of delegate that handles marshalling the SynchronizationContext.Current property from the main thread to the background thread automatically.  I call this delegate type SynchronizationCallback, and it's compatible with all of the other delegate types such as WaitCallback, TimerCallback, ThreadStart, etc.  By simply using this delegate type instead of the original type you'll be able to assume SynchronizationContext.Current will always be valid no matter which thread your code is running on.  For example, by changing the above code as follows:

public class MyLibraryClass
{
    public void CalledInMainThread()
    {
        Debug.WriteLine("Main Thread: SynchronizationContext.Current is " + (SynchronizationContext.Current != null ? SynchronizationContext.Current.ToString() : "null"));

        Thread t = new Thread((ThreadStart)new SynchronizationCallback(CalledInBackgroundThread));
        t.Start();
    }

    private void CalledInBackgroundThread(object state)
    {
        Debug.WriteLine("Background Thread: SynchronizationContext.Current is " + (SynchronizationContext.Current != null ? SynchronizationContext.Current.ToString() : "null"));
    }
}

Then you'll see the output as expected:

Main Thread: SynchronizationContext.Current is System.Windows.Forms.WindowsFormsSynchronizationContext

Background Thread: SynchronizationContext.Current is System.Windows.Forms.WindowsFormsSynchronizationContext

IndentedTextWriter

There's a very useful class called IndentedTextWriter in the System.CodeDom.Compiler namespace that automatically adds whitespace before each line depending on its Indent property.  This works great when you have code with several nested (or recursive) subroutines and you want them to print nested output but you don't want to keep track of how "nested" they are by yourself.  Unfortunately, there's a bug in the framework's implementation that prevents the first line from being indented.  To illustrate, take this program:

static void Main(string[] args)
{
    IndentedTextWriter writer = new IndentedTextWriter(Console.Out);
    writer.Indent = 1;
    writer.WriteLine("First line");
    writer.WriteLine("Second line");
    writer.WriteLine("Third line");
}

You'll notice that the first line isn't indented:

The bug is easy to fix with a little reflection.  Before the first Write or WriteLine simply call typeof(System.CodeDom.Compiler.IndentedTextWriter).GetField("tabsPending", BindingFlags.Instance | BindingFlags.NonPublic).SetValue(writer, true);

I've attached a simple wrapper class that applies the fix automatically and moves the IndentedTextWriter class to the more common System.IO namespace at the same time.  Enjoy!

Hello, World!