Krzysztof Cwalina : General API Design

Extensible Framework Design Studio Released

kcwalina — Sat, 30 Aug 2008 20:19:00 GMT

This summer we had a high school intern, Nick Moloney, who worked on incorporating MEF into FDS. The fruits of his labor are now on code gallery. You can download the extensible FDS here. Congratulations to Nick!

The current release has just a few extensibility points, but you should expect more in the future.

API Design Myth: Exceptions are for "Exceptional Errors"

kcwalina — Thu, 17 Jul 2008 23:09:00 GMT

I was updating FDG section on exceptions. I added one anntation that I thought I would post here as well:

KRZYSZTOF CWALINA

One of the biggest misconceptions about exceptions is that they are for “exceptional conditions.” The reality is that they are for communicating error conditions. From a framework design perspective, there is no such thing as an “exceptional condition”. Whether a condition is exceptional or not depends on the context of usage, --- but reusable libraries rarely know how they will be used. For example, OutOfMemoryException might be exceptional for a simple data entry application; it’s not so exceptional for applications doing their own memory management (e.g. SQL server). In other words, one man’s exceptional condition is another man’s chronic condition.

Nullable Usage Guidelines

kcwalina — Wed, 16 Jul 2008 23:47:00 GMT

These guidelines were just added as part of an update to the Framework Design Guidelines book (upcomming 2nd edition). Hope you find them useful.

Nullable<T> is a simple type added to the .NET Framework 2.0. The type was designed to be able to represent Value Types with “null” values.

Nullable<int> x = null;

Nullable<int> y = 5;

Console.WriteLine(x == null); // prints true

Console.WriteLine(y == null); // prints false

Note that C# provides special support for Nullable<T> in the form of language aliases for Nullable types, lifted operators, and the new coalescing operator.

int? x = null; //

long? d = x; // calls cast operator from Int32 to Int64

Console.WriteLine(d??10); // coalescing; prints 10 because d == null

þ CONSIDER using Nullable<T> to represent values that might not be present (i.e. optional values). For example, use it when returning a strongly typed record from a database with a property representing an optional table column.

ý Do NOT use Nullable<T> unless you would use a reference type in a similar manner, taking advantage of the fact that reference type values can be null.

For example, you would not use null to represent optional parameters.

// bad design

public class Foo {

public Foo(string name, int? id);

}

// good design

public class Foo {

public Foo(string name, int id);

public Foo(string name);

}

ý AVOID using Nullable<bool> to represent a general three-state value.

Nullable<bool> should only be used to represent truly optional Boolean values: true, false, and not available. If you simply want to represent three states (e.g. yes, no, cancel), consider using an enum.

ý AVOID using System.DBNull. Prefer Nullable<T> instead.

Nullable<T> is in general a better representation of optional database values. One thing to consider though it that while Nullable<T> gives you the ability to represent null values, you don’t get database null operational semantics. Specifically, you don’t get null propagation through operators and functions. If you deeply care about the propagation semantics, consider sticking with DBNull.

Framework Design Guidelines Digest v2

kcwalina — Thu, 10 Apr 2008 00:19:00 GMT

Almost 4 years ago, I blogged about Framework Design Guidelines Digest. At that time, my blog engine did not support attaching files and I did not have a convenient online storage to put the document on, and so I asked people to email me if they want an offline copy.

Believe it or not, I still receive 1-2 emails a week with requests for the offline copy. Now that I have a convenient way to put the document online, and the fact that I wanted to make some small updates, I would like to repost the digest. The abstract is below and the full document can be downloaded here.

This document is a distillation and a simplification of the most basic guidelines described in detail in a book titled Framework Design Guidelines by Krzysztof Cwalina and Brad Abrams. Framework Design Guidelines were created in the early days of .NET Framework development. They started as a small set of naming and design conventions but have been enhanced, scrutinized, and refined to a point where they are generally considered the canonical way to design frameworks at Microsoft. They carry the experience and cumulative wisdom of thousands of developer hours over several versions of the .NET Framework.

Framework Design Studio Released

kcwalina — Fri, 04 Apr 2008 21:10:00 GMT

When I was coming back from Mix 2007, I was bored on the plane and so started to write a dev tool. What a geeky thing to do on a plane. :-)

The tool allows comparing two versions of an assembly to identify API differences: API additions and removals. Comparing versions of APIs comes very handy during API design process. Often you want to ensure that things did not get removed accidentally (which can cause incompatibilities), and as APIs grow, you want to review the addition without having to re-review APIs that were already reviewed. The tool, called Framework Design Studio (FDS) supports these scenarios.

Later on, I got lots of help from Hongping Lim (a developer on our team), and David Fowler (our 2007 summer intern). David ported the application to WPF, and Hongping basically took it from an early prototype stage to what it is today and made it possible to ship it externally.

Anyway, you can get the tool at the MSDN Code Gallery, the user guide is attached to this post, and lastly, here is the API diff output that the tool generates. Hope you find it useful.

Simulated Covariance for .NET Generics

kcwalina — Wed, 02 Apr 2008 18:00:00 GMT

I just wrote this pattern, but I am not sure if I should add it officially to the Framework Design Guidelines. It seems like a bit of a corner case scenario, though I do get questions about it from time to time. Anyway, let me know what you think.

Different constructed types don’t have a common root type. For example, there would not be a common representation of IEnumerable<string> and IEnumerable<object> if not for a pattern implemented by IEnumerable<T> called Simulated Covariance. This post describes the details of the pattern.

Generics is a very powerful type system feature added to the .NET Framework 2.0. It allows creation of so called parameterized types. For example, List<T> is such a type and it represents a list of objects of type T. The T is specified at the time when the instance of the list is created.

List<string> names = new List<string>();

names.Add(“John Smith”);

names.Add(“Mary Johnson”);

Such Generic data structures have many benefits over their non-Generic counterparts. But they also have some, sometimes surprising, limitations. For example, some users expect that a List<string> can be cast to List<object>, just as a String can be cast to Object. But unfortunately, the following code won’t even compile.

List<string> names = new List<string>();

List<object> objects = names; // this won’t compile

There is a very good reason for this limitation, and that is to allow for full strong typing. For example, if you could cast List<string> to a List<object> the following incorrect code would compile, but the program would fail at runtime.

static void Main(){

List<string> names = new List<string>();

// this of course does not compile, but if it did

// the whole program would compile, but would be incorrect as it

// attempts to add arbitrary objects to a list of strings.

AddObjects((List<object>)names);

string name = names[0]; // how could this work?

}

// this would (and does) compile just fine.

static void AddObjects(List<object> list){

list.Add(new object()); // it’s a list of strings, really. Should we throw?

list.Add(new Button());

}

Unfortunately this limitation can also be undesired in some scenarios. For example, there is nothing wrong with casting a List<string> to IEnumerable<object>, like in the following example.

List<string> names = new List<string>();

IEnumerable<object> objects = names; // this won’t compile

foreach(object obj in objects){

Console.WriteLine(obj.ToString());

}

In general, having a way to represent “any list” (or in general “any instance of this generic type”) is very useful.

// what type should ??? be?

static void PrintItems(??? anyList){

foreach(object obj in anyList){

Console.WriteLine(obj.ToString());

}

Unfortunately, unless List<T> implemented a pattern that will be described in a moment, the only common representation of all List<T> instances would be System.Object. But System.Object is too limiting and would not allow PrintItems method to enumerate items in the list.

The reason that casting to IEnumerable<object> is just fine, but casting to List<object> can cause all sorts of problems is that in case of IEnumerable<object>, the object appears only in the output position (the return type of GetEnumerator is IEnumerator<object>). In case of List<object>, the object represents both output and input types. For example, object is the type of the input to the Add method.

// T does not appear as input to any members or dependencies of this interface

public interface IEnumerable<T> {

IEnumerator<T> GetEnumerator();

}

public interface IEnumerator<T> {

T Current { get; }

bool MoveNext();

}

// T does appear as input to members of List<T>

public class List<T> {

public void Add(T item); // T is an input here

public T this[int index]{

get;

set; // T is actually an input here

}

In other words, we say that in IEnumerable<T>, the T is at covariant positions (outputs). In List<T>, the T is at covariant and contravariant (inputs) positions.

To solve the problem of not having a common type representing the root of all constructions of a generic type, you can implement what’s called the Simulated Covariance Pattern.

Given a generic type (class or interface) and its dependencies

public class Foo<T> {

public T Property1 { get; }

public T Property2 { set; }

public T Property3 { get; set; }

public void Method1(T arg1);

public T Method2();

public T Method3(T arg);

public Type1<T> GetMethod1();

public Type2<T> GetMethod2();

}

public class Type1<T> {

public T Property { get; }

}

public class Type2<T> {

public T Property { get; set; }

}

Create a new interface (Root Type) with all members containing Ts at contravariant positions removed. In addition, feel free to remove all members that might not make sense in the context of the trimmed down type.

public interface IFoo<T> {

T Property1 { get; }

T Property3 { get; } // setter removed

T Method2();

Type1<T> GetMethod1();

IType2<T> GetMethod2(); // note that the return type changed

}

public interface IType2<T> {

T Property { get; } // setter removed

}

The generic type should then implement the interface explicitly and “add back” the strongly typed members (using T instead of object) to its public API surface.

public class Foo<T> : IFoo<object> {

public T Property1 { get; }

public T Property2 { set; }

public T Property3 { get; set;}

public void Method1(T arg1);

public T Method2();

public T Method3(T arg);

public Type1<T> GetMethod1();

public Type2<T> GetMethod2();

object IFoo<object>.Property1 { get; }

object IFoo<object>.Property3 { get; }

object IFoo<object>.Method2() { return null; }

Type1<object> IFoo<object>.GetMethod1();

IType2<object> IFoo<object>.GetMethod2();

}

public class Type2<T> : IType2<object> {

public T Property { get; set; }

object IType2<object>.Property { get; }

}

Now, all constructed instantiation of Foo<T> have a common root type IFoo<object>.

var foos = new List<IFoo<object>>();

foos.Add(new Foo<int>());

foos.Add(new Foo<string>());

…

foreach(IFoo<object> foo in foos){

Console.WriteLine(foo.Property1);

Console.WriteLine(foo.GetMethod2().Property);

}

þ CONSIDER using the Simulated Covariance Pattern, if there is a need to have a representation for all instantiations of a generic type.

The pattern should not be used frivolously as it results in additional types in the library and can makes the existing types more complex.

þ DO ensure that the implementation of the root’s members is equivalent to the implementation of the corresponding generic type members.

There should not be an observable difference between calling a member on the root type and calling the corresponding member on the generic type. In many cases the members of the root are implemented by calling members on the generic type.

public class Foo<T> : IFoo<object> {

public T Property3 { get { ... } set { ... } }

object IFoo<object>.Property3 { get { return Property3; } }

…

}

þ CONSIDER using an abstract class instead of an interface to represent the root.

This might sometimes be a better option as interfaces are more difficult to evolve (see section X). On the other hand there are some problem with using abstract classes for the root. Abstract class members cannot be implemented explicitly and the subtypes need to use the new modifier. This makes it tricky to implement the root’s members by delegating to the generic type members.

þ CONSIDER using a non-generic root type, if such type is already available.

For example, List<T> implements IEnumerable for the purpose of simulating covariance.

Job Openings on the .NET Framework Core Team

kcwalina — Fri, 14 Mar 2008 22:49:00 GMT

We have been incubating ideas about building a simple extensibility framework for some time. Now, as plans for the next version of the .NET Framework crystallize a bit more, we decided to productize the project. As a result, we have opened a job position (and most probably will be opening more) on the .NET Framework team. If you are interested, please see details here and send me an email at “kcwalina at microsoft.com.”

So, what is this extensibility framework? Initially, it will be a low level core .NET Framework feature to make it easy for applications to expose extensibility points and consume extensions. Think about what for example FxCop has to do define rule contracts and load rule implemented by the community. These are the basics, and we can talk about the broader and longer term vision when you come to Redmond for an interview :-)

This is a technical Program Manager position in Redmond, WA, and it’s basically exactly the job I did when I joined Microsoft. Besides working on the Framework features, all Program Managers on the core team have opportunities to work on API design and architecture projects.

Video Recording of "Framework Engineering: Architecting, Designing, and Developing Reusable Libraries"

kcwalina — Tue, 08 Jan 2008 21:42:00 GMT

I just received a video recording of a talk I did at the last TechEd. You can find the abstract below, and the WMV file can be downloaded from here. Hope you find it useful.

[UPDATE: I attched the slides in xps format. The ppt file is 10x larger]

Framework Engineering: Architecting, Designing, and Developing Reusable Libraries

This session covers the main aspects of reusable library design: API design, architecture, and general framework engineering processes. Well-designed APIs are critical to the success of reusable libraries, but there are other aspects of framework development that are equally important, yet not widely covered in literature. Organizations creating reusable libraries often struggle with the process of managing dependencies, compatibility, and other design processes so critical to the success of modern frameworks. Come to this session and learn about how Microsoft creates its frameworks. The session is based on experiences from the development of the .NET Framework and Silverlight, and will cover processes Microsoft uses in the development of managed frameworks.

Framework Design Guidelines 2nd Edition

kcwalina — Fri, 04 Jan 2008 01:39:00 GMT

My blog was relatively silent for several weeks. First, I was traveling to Europe for the TechEd, then was busy at work, then the holiday break. It's time to go back to more regular posting.

I will start with an announcement (or at least a more formal and broader announcement): after my TechEd presentation, the first question I got from the audience was whether we are working on a new edition of the Framework Design Guidelines. The answer is "yes", which I am super excited about. Right before the conference, we signed formal documents with the publisher and started wortking on the book. It's going to cover the new features in the .NET Framework 3.0, 3.5, and new advances in languages (e.g. LINQ) that are relevant to Framework design. BTW, I would appreciate feedback on what specifically you would like to get covered or clarified.

We are shooting to have the book ready around the end of 2008, but the publisher already sent me a draft of the cover art:

What Do Swimmers Have to Say About Framework Design?

kcwalina — Fri, 05 Oct 2007 03:09:00 GMT

I am starting to feel pressure to finish up slides for my presentation at the upcoming TechEd in Barcelona. I will be talking about framework architecture and design. Here is the abstract I took from the conference’s site:

WIN304 Framework Engineering: Architecting, Designing, and Developing Reusable Libraries

I am super excited about the talk, which I want to make into a continuation to the API design pre-con I did with Brad Abrams at the last PDC (BTW, the same content is available here). The PDC talk was about designing the API surface, but there is so much more to framework design!

I am also excited coming back to the wonderful city of Barcelona. Last time I was there, believe it or not, I was on the Polish swimming team competing in the Olympics (see #13). I hope this trip will bring back some memories from the other and completely different “career” I had in the past.

China Trip

kcwalina — Tue, 02 Oct 2007 21:42:00 GMT

Sorry for not blogging for such a long time. First I was on a combined business/vacations trip to China and when I came back, I got involved in some intensive planning for the future releases of the platform. I will try to post something about software in the next couple of days.

The trip to China was great. First day I taught a class on API design at the Microsoft center in Beijing and then my wife and I stayed for several days doing sightseeing, drinking the best tea ever, and indulging ourselves in great food (complete photo documentary is here).

FxCop Designers Honored with the Chairman's Award

kcwalina — Wed, 04 Jul 2007 02:21:00 GMT

Last week, during the annual Engineering Excellence week, several Microsoft engineers and managers involved in development of engineering tools and practices were presented with Engineering Excellence Awards. In addition, the principal designers of three static analysis tools were honored with so called “Engineering Excellence Chairman’s Award,” which is given for contributions that our chairman (Bill) considers especially important. The Chairman award was given twice in the history of Microsoft.

Mike Fanning, Brad Abrams, and I received nice glass statuettes and a framed letters from Bill and Jon (our VP responsible for EE). See pictures below. The letter says “The Engineering Excellence Chairman’s Award is the Microsoft’s highest award for engineering group employees worldwide.”

It’s extremely rewarding to receive something like that for what started as a hobby project on which we worked mainly in our free time (at least, before it was officially productized). I would like to thank Brad for being the “let’s do it now” person, pushing the whole effort of design guidelines and fxcop development, and Mike for his passion for quality and being the super-developer on the team.

Of course, many others contributed to the success of fxcop. Once we gather all the names, we will post a new entry or update this one to thank everybody and record all the contributions for posterity. [UPDATE: Brad just posted on this here.]

Generic Methods as Casts

kcwalina — Thu, 07 Jun 2007 20:56:00 GMT

Somebody just asked me which of the following API design alternatives is better. They both do what we could call “casting.”

// Generic Method “Cast”

var foo = someInstance.SomeMethod<IFoo>();

// Simple Cast

var foo = (IFoo)someInstance.SomeMethod();

The Generic method approach can potentially have two benefits. A constraint can tell users what the subset of valid casts is. For example, the following API will allow casts to a collection (a subtype of IEnumerable):

public TCollection SomeMethod<TCollection>() where TCollection:IEnumerable {

…

}

If the user calls this method and tries to “cast” the return value to let’s say an Int32, the compiler with complain. For example,

var int someInstance.SomeMethod<int>();

… will generate the following error:

The type 'int' must be convertible to 'System.Collections.IEnumerable' in order to use it as parameter 'TCollection' in the generic type or method 'SomeClass.SomeMethod<TCollection>()'

Secondly, the Generic method might do different things based on the type parameter. The method relying on a simple cast has now knowledge about the type the user will cast it’s return value to.

public TCollection SomeMethod< TCollection >()

where TCollection:IEnumerable,new() {

TCollection newCollection = new TCollection ();

ArrayList newArray = newCollection as ArrayList;

if (newArray != null) newArray.Capacity = 0;

return newCollection;

}

If neither of these benefits applies in your case, I would use a simple cast. It’s more transparent in terms of what’s going on. The users of a generic method might wonder what kind of “magic” is being done with the generic type parameter.

Framework design lecture avaliable for download

kcwalina — Fri, 01 Jun 2007 23:38:00 GMT

I have good news for all who asked about an off line version of the API design lecture I blogged about a few weeks ago. After literally a hundred emails and comments on the blog (thanks for pushing me to get this done), weeks of trying to get this setup, including the difficulty of finding a server to host a 500MB file, and -- ultimately getting some help from Scott Guthrie (thanks!), the zip file is finally available for download here. Enjoy!

Nullable usage guideline

kcwalina — Fri, 27 Apr 2007 18:00:00 GMT

I am starting to work on some updates to the framework design guidelines related to new framework and language features added in .NET 2.0, 3.0, and 3.5. One of such additions are the nullable types (Nullable<T>).

Coincidentally, somebody asked me today whether they should use Nullable<bool>. Here is the question:

Q: Do the guidelines have anything to say about the use of nullable types in public API? Subject came up here in context of a proposal to return Nullable<bool> from a method to retrieve an attribute value (null means attribute not found, true means value retrieved, false means no value for existing attribute).

… and in the absence of formal guidelines, here is what I recommended:

A: I would say that the rule of thumb is that you should use Nullable<bool>, if it’s obvious what all three values mean. For example, it there is an optional column in a database of type Boolean, it’s clear that true means true, false means false, and null means the value is not there. In your example, you had to explain what the values mean, so I would not recommend using nullable types.