jaredpar's WebLog : RantPack

BclExtras Library

Jared Parsons — Fri, 23 Jan 2009 16:00:00 GMT

I published a .Net utility library on Code Gallery today called BclExtras. It’s a set of classes meant to be used in addition to the standard .Net base class libraries (BCL). The main focuses of the library are functional programming, multi-threading, LINQ extensions, unit testing and API’s designed to support type inference.

This project evolved from various classes and constructs I was using in personal projects. For the last year or so I’ve kept as a separate tested library. It started out with a lot of multi-threaded code constructs but lately is leaning to a lot of functional style API’s and collections.

The library includes source and binaries for .Net 2.0 and .Net 3.5. The .Net 2.0 version of the library includes many constructs added in 3.5 that don’t rely on any 2.0SP1 CLR features. Examples are Func<T>, Action<T>, the Extension attribute and a subset of the LINQ Enumerable class. It allows for most LINQ expressions in a 2.0 targeted application. These types are removed in the 3.5 version to avoid conflicts with types in System.Core.

I’ve previously released this library under the name RantPack. It originally started out as personal utility library of mine and hence ended up with the somewhat obscure name. But, besides to me, RantPack doesn’t really convey a useful meaning. So I decided to give it a more meaningful name for the general population.

Immutable Collections and Compatibility with Existing Frameworks

Jared Parsons — Wed, 10 Dec 2008 16:00:00 GMT

When developing my immutable collections library, I spent a lot of time on usability. After all, if a library is not useful then what’s the point?

A big portion of usability is being able to work with existing frameworks and technologies. For .Net and collections that means items like Data binding, LINQ etc … Without integrating into popular technologies the usefulness of a particular library is severely impacted and hence usability decreases.

Most of the existing collection based infrastructures use the .Net collection interfaces as their form of abstraction. The most straight forward way to ensure compatibility is to implement these interfaces on the collections in question. In particular ICollection<T>, IList<T> and IEnumerable<T> are the main abstractions. Lets investigate which ones an Immutable collection should be implementing in order to effectively integrate into existing collection based infrastructures.

IEnumerable<T>

This is the easiest decision. IEnumerable<T> represents a read only, one element at time view on a sequence of objects. Immutable collections can easily and reliably implement a IEnumerable<T>. This is a no brainer. Implement.

ICollection<T>

This interfaces represents a general collection class. Unfortunately this interface is meant to represent a mutable collection class and implements such methods as Add, Clear and Remove. These methods cannot be implemented on an Immutable collection given the current design. All three of these methods are void returning methods because the collection is meant to be changed in place. Immutable collections can support these operations but it involves returning a new instance of the collection.

public sealed class ImmutableCollection<T> : ICollection<T> {
    public ImmutableCollection<T> Add(T item) {
        // Actually add 
        // ...
    }

    #region ICollection<T> Members

    void ICollection<T>.Add(T item) {
        var created = this.Add(item);
        // What to do with created???
    }

    ...
}

But wait! The interface does support a property named IsReadOnly. The intention of this property is to allow an interface to programmatically declare they do not support modifications. A read only collection can just implement this interface, throw a NotSupportedException for all of the mutable methods and return true for IsReadOnly and presto we have a suitable interface for an immutable collection.

Or do we?

The design for ICollection<T> with respect to read only or immutable collections is not optimal. It attempts to combine to separate behaviors into a single interface: mutable and readonly view of a collection. Dual purpose interfaces run into problems because it’s impossible to separate out the behaviors at compile time. This is especially problematic when the behaviors are conflicting. There is no way a read only collection can prevent itself from being passed to a function that expects a mutable collection at compile time. Nor can a consumer who intends to mutate a collection prevent a read-only collection from being passed.

static void DisplayForEdit<T>(ICollection<T> col) {
    // ...
    m_clearButton.Click += (x, y) => col.Clear(); 
}

static void Example1() {
    ImmutableCollection<int> col = ImmutableCollection.Create(new int[] { 1, 2, 3, 4 });
    DisplayForEdit(col);    // Will fail as soon as Clear is clicked
}

But isn’t it the responsibility of the user of ICollection<T> to verify that IsReadOnly is false before mutating a instance? Given the current design of ICollection<T> it is indeed both the responsibility of the consumer to verify this and the implementer to ensure they are not called incorrectly. The problem with putting responsibility on the consumer though is they have to 1) know about read only uses of ICollection<T> and 2) actually care about it.

A quick search of the BCL with reflector can give us evidence of how much consumers actually check for the read only scenario. For the search I used mscorlib, System, System.Xml, System.Data, System.Drawing and System.Core and System.Windows.Forms. And the number of classes which actually take into account ICollection<T>.IsReadOnly is … 1. System.Collections.ObjectModel.Collection<T>. That’s it.

So even if an immutable collection implements this interface in a read-only fashion there’s little chance anyone will be checking for it.

IList<T>

IList<T> inherits from ICollection<T> and hence suffers from all of the same problems

Decision Time

In order to facilitate usability with existing frameworks immutable collections are forced to implement interfaces for which they have no possible way of implementing properly. If collections implement these interfaces they will be trading usability for compile time validation. Even worse is the conversion is implicit which prevents even basic searches for places this conversion occurs. This is a heavy price to pay for compatibility.

After debating this for awhile I decided that loss of compile time validation was a too heavy of a price to pay for the default scenario. But trading away usability was also unacceptable. As a compromise I opted for adding a compatibility layer to the collections. Instead of implementing the ICollection<T> and IList<T> collections directly I created a set of proxy objects that implement the interfaces on behalf of the immutable collections.

In order to centralize this effort I created a factory class, CollectionUtility, which contains appropriate overloads for all of my immutable collection classes [1].

public static class CollectionUtility {
    public static IEnumerable<T> CreateEmptyEnumerable<T>();
    public static IEnumerable<T> CreateEnumerable<T>(T value);
    public static ICollection<T> CreateICollection<T>(IReadOnlyCollection<T> col);
    public static IDictionary<TKey, TValue> CreateIDictionary<TKey, TValue>(IReadOnlyMap<TKey, TValue> map);
    public static IList<T> CreateIList<T>(IReadOnlyList<T> list);
    public static ICollection CreateObjectICollection<T>(IReadOnlyCollection<T> col);
    public static IDictionary CreateObjectIDictionary<TKey, TValue>(IReadOnlyMap<TKey, TValue> map);
    public static IList CreateObjectIList<T>(IReadOnlyList<T> list);
    public static IEnumerable<int> GetRangeCount(int start, int count);
}

The proxy objects live as private inner classes inside CollectionUtility. They implement the collection interfaces in the most read-only way possible. When confronted with mutating calls, the proxies throw NotSupportedException.

So at the end of the day I have compile time validation for immutable collections. If a developer wants to use them in a less than safe scenario it requires an explicit conversion.

static void Example2() { 
  var col = ImmutableCollection.Create(new int[] { 1, 2, 3, 4 }); 
  // Still fails, but explicit conversion required 
  DisplayForEdit(CollectionUtility.CreateICollection(col)); 
  }

I feel like this as an appropriate tradeoff. In the worst case scenario, a developer can search for all accesses of the CollectionUtility class and find places where a proxy is being created.

Next time, lets take a look at a different way of approaching an interface hierarchy for a set of collections. One that will allow us to avoid this problem altogether going forward.

[1] It actually contains overloads for a set of truly read only collection interfaces that I wrote for my library but we’ll get to that another time.

Edit: Updated the exception to be NotSupportedException

RantPack Update 2.0.0.2

Jared Parsons — Wed, 23 Jul 2008 15:00:13 GMT

Version 2.0.0.2 released.

Summary: RantPack is a utility library I maintain and actively use. The main themes of this library are functional programming, patterns, immutable/pressitent collections, future and other threading primitives. I've placed the code and binaries on http://code.msdn.microsoft.com/RantPack for educational and sharing purposes.

Release Link: https://code.msdn.microsoft.com/Release/ProjectReleases.aspx?ProjectName=RantPack&ReleaseId=1306

Release Notes:

Renamed Tuple properties to be more consistent

Item0, Item1 instead of A,B

Added a Match operator
Made Enumerable more compatible
Bug fixes

To name a tuple value: A or Item0?

Jared Parsons — Mon, 23 Jun 2008 15:00:17 GMT

One of the parts of a Tuple implementation for RantPack I struggled the most with was naming. I tend to struggle with names quite a bit and there's really no reason for it. It's a combination of pickiness and ... well there's really no good reason. Pieces which can bother me range from

Don't like the way it types. If I can't type something easily it drives me nuts and I secretly desire to change it to a word that is easier to type. I have sadly spent a large amount of time on www.thesaurus.com looking for better words to type.
Looks funny. No real solid criteria here, just what I don't like. Words sometimes look good the first time you type them but over time lose their appeal
Doesn't match conventions. Ah, now we're onto solid reasons. Names should match conventions.
Name is ambiguous. If the name doesn't give a clear intention of the operation I do my best to change it.
FxCop violation. This bothers me less than others but if FxCop doesn't like it I think a bit harder about naming it.

Now comes Tuples. Many implementations of tuples don't even have names but instead rely on language constructs such as pattern matching to access the values. That's not really the DotNet way of doing things though so I chose to give Tuple values names.

Originally I chose A for item 0, B for item 1, etc ... This made sense at the time because it kept the syntax short. However I've grown to dislike this naming convention. Primarily because it violates #3 and #4 above. While A-E is perfectly clear for me I've talked with others who this confuses.

In addition, to keep the nameless syntax option I provided an indexer property. So values are available with a 0 based index. Since several languages expose this with the name "Item", there are now really two names for a value. For example the first value is both tuple.A and tuple.Item[0].

After some thought I decided to name the values Item0->Item4 depending on the size of the tuple.

RantPack Update

Jared Parsons — Tue, 03 Jun 2008 03:26:08 GMT

I released a new version of RantPack today. Mostly this is a bug fix release with a couple of minor new features.

https://code.msdn.microsoft.com/Release/ProjectReleases.aspx?ProjectName=RantPack&ReleaseId=1119

Features

Added a way to shim Immutable collections to non-immutable interfaces to increase the level of interoperability. The collections are still immutable and throw an exception whenever one of the mutable APIs are called. However if the collection is used in an immutable fashion, such as data binding, CollectionUtility.Create* can be used to quickly create a wrapper

var q = ImmutableStack<int>.Empty;
CollectionUtility.GetRangeCount(1, 10).ForEach(x => q = q.Push(x));
ICollection<int> list = CollectionUtility.CreateICollection(q);

var m = ImmutableAvlTree<int, string>.Empty;
CollectionUtility.GetRangeCount(1, 10).ForEach(x => m = m.Add(x, x.ToString()));
IDictionary<int, string> map = CollectionUtility.CreateIDictionary(m);

Added an immutable queue named ImmutableQueue
Added more overloads to Immutable*.Create to allow more interoperability with BCL data structures.

Bugs

MutableTuple.GetHashCode() violated the contract for Object.GetHashCode() by allowing updates to mutate the hash code

RantPack - A utility library

Jared Parsons — Thu, 01 May 2008 06:47:00 GMT

I often post code examples, samples and snippets on this blog. Many of these samples are a part of a utility library I've been writing and maintaining for many years now. Essentially since I got involved in DotNet programming.

I write a lot of code for internal apps, demos and hobby projects. Having a utility library to tote around with me greatly increases my impact in a project because it gives me a place to put common code that I would otherwise have to repeatedly cut and paste into projects.

Whenever I work on something I think the community at large would benefit from I try to blog about it. Unfortunately for the more complex samples blogging is not always easy. Complex structures don't really fit on a blog and often utilities depend on utilities and it forces users to cut and paste between multiple blog entries. Not good.

Now with http://code.msdn.com I have a forum to fully share the items I've been working on. This is the perfect place to share code for the community and helps avoid the messiness of cutting and pasting blog posts. As such I've released my Library, RantPack, on Code Gallery. This includes both binaries and sources and is covered under the Microsoft Public License.

RantPack is available at http://code.msdn.microsoft.com/RantPack

A couple of notes on the sample. This is mainly for education and sharing purposes. The goal of this library is to make developing software faster and more reliable. It includes, but is not limited to

Functional Programming Patterns

Tuples
Immutable/Persistent collections

Threading

Futures
Active Objects
Various other primitives

General Patterns
General Utilities

Another item you will find browsing the sources are tests. Lots and lots of tests. At the time of this writing there are around 400 some odd unit tests on the library. There is nothing more frustrating than a utility library that doesn't work. As such I rigorously test the code I write.

Although I performance test much of my code and it performs great in the applications I use it for I'm sure there are a few cases I missed. Please feel free to point this out to me and I'll look into them.

Why the name RantPack? My blog is Rantings of a programmer and it's a pack of code :)