Neal's Blog

Overrides and overload resolution, revisited

nealho — Tue, 15 Aug 2006 00:58:00 GMT

I've got some fun LINQ-related discussion to post soon, but I thought it'd be good to add a bit of a wrap-up to the last discussion here about overload resolution. I was happy at first having verified that our behavior matched the spec, so I moved on to various other things. However, the customer who brought the issue to me in the first place was unsatisfied with this response. He wanted to know the reasoning behind the behavior, not just that we had documented our behavior correctly. Thanks to Jeffrey Sax for raising this issue, and hounding me about it until I delved deeper and really thought about C# language design. Since much of my job involves verifying the compiler's behavior against the spec, it's easy to forget that I also need to be continually evaluating that spec and the decisions we've made in the past. His blog post on the subject is a great read and highlights both the design principles behind the decision to prefer more derived methods and some of the problems with it, so I won't go into too much detail, but if you read my previous article on overload resolution, I highly recommend it.

Overrides and Overload Resolution

nealho — Fri, 20 Jan 2006 05:11:00 GMT

I got an e-mail earlier this week asking a fairly common question regarding C#’s overload resolution. Before I launch into a discussion of the actual behavior of the compiler, think about what the output of the following program will be:

using System;

public class Program {
    public static void Main() {
        Derived d = new Derived();
        Console.WriteLine(d.Method("My String"));
    }
}

public class Base {
    public virtual string Method(string s) {
        return "Base.Method(string)";
    }
}

public class Derived : Base {
    public override string Method(string s) {
        return "Derived.Method(string)";
    }
    public virtual string Method(object o) {
        return "Derived.Method(object)";
    }
}

My first thought, as well as the first thought of most people I’ve talked to about this, is that this code would call Derived.Method(string) because it is the best overload. However, as you either know from having tried it or have already guessed from the very fact that I’m writing this post, the method that actually gets called is Derived.Method(object). In tracking this issue down, I started where I usually do—with the C# language spec. As I skimmed through overload resolution, it seemed to confirm what I had thought originally—a method that requires no conversion is a better choice than a method that requires a conversion. Puzzled, I moved on, and read the part about member access, where I concluded that the result of looking up Method on an instance of Derived should be the method group containing both overloads. Well, that was no help, so I read deeper. Looking at the next section, about invocation expressions, I read the following: “The set of candidate methods is reduced to contain only methods from the most derived types.” Some normative text followed, but this is the detail I had been missing.

The methods in a derived class are preferred to inherited methods. Add to that the fact that the compiler doesn’t consider overrides when performing overload resolution—only methods introduced by a given class are considered to be on that class—and, surprisingly enough, the compiler is making the correct choice.

C#, Java, and CS Education

nealho — Fri, 06 Jan 2006 03:42:00 GMT

I read an interesting article the other day about how CS is taught in school now. The article basically argues that you can’t learn real computer science skills in Java. Without debating his attitude towards the Java language itself (and by extension, the C# language), I thought I’d take a moment to write some thoughts regarding the article.

I went to Rice University, and while I don’t think it is exactly what he describes so disparagingly as a “Java School,” Java was the primary language that I used while there. At Rice, the introductory programming course is taught in Scheme; computer organization is taught in some combination of C and SPARC assembly; programming languages and compilers are taught in some combination of C, Java, and Scheme; of course, operating systems is taught in C; however, intermediate programming, algorithms and data structures, and most software engineering courses are taught in Java (some of the newer ones in C#, but the two are effectively the same for the purposes of this discussion). Oddly enough, I knew almost nothing about C++ until my summer internship after junior year, where I spent the entire summer getting a painful crash course in how C++ programming is not like Java programming. But I digress.

You’ll note a preponderance of non-Java languages in that laundry list of courses and languages, and I think that’s important to have. It is important for computer scientists to be able to write real code in C, using pointers and hacking bits for all they’re worth, because there is a lot of real-world code that needs that level of control. It’s equally important for computer scientists to be able to think about programs at a functional level: functional programs teach you the important lesson that code and data are really the same thing. So what, then, are we even doing teaching people silly object-oriented languages like C# and Java?

In my opinion, we are teaching those languages precisely because they are “easier.” It’s easier to express your intent in a good OO language than it is in C. You gain the powerful abstraction of classes, which, in addition to the usual yadda yadda about polymorphism and various OO patterns, come with nearly everything you wanted from functional closures (though I won’t claim that you get the same elegance and power as Lisp/Scheme macros—but OO vs. Scheme as a language of choice is not the topic today). Thanks to the managed environment, you don’t have to worry about corrupting some random part of memory or blowing your stack to smithereens, nor do you have to allocate memory with careful precision so that you can remember to free it later. You have all the tools you need to write your code while decreasing the overhead involved.

I think he’s right that Java isn’t the be-all and end-all of computer science (neither is C#, much as I’d love to claim that it is). He’s absolutely right that the sort of “mental flexibility you get from learning about” pointers and functional programming makes you a better programmer in any language. Still, he has a certain elitism about his article that rubs me the wrong way. I think I understand pointers, but I wouldn’t want the fact that I hate working with C-style linked lists to keep me out of a job that itself required no C programming. Even if CS departments at universities should be training their students for real programming, they shouldn’t have a responsibility to “weed out” students who can’t grasp the entirety of Scheme in a single day.

In the end, I suppose that you do have to understand pointers because all of your programs run on real computers with code that, somewhere underneath, is using those pointers. Despite having to understand them, I don’t think anyone should be required to like them.

Enumerated Types: Enumerating the Values

nealho — Thu, 29 Dec 2005 07:21:00 GMT

One of the things you lose when doing class-based enums is the support of the System.Enum class and its static methods for all enum types. Take a look at the documentation on MSDN—the only methods that really remain applicable are GetValues, which returns an array of all values in the enum, and GetNames, which returns an array of all the names of the values in the enum. (Because it’s trivial to write GetNames from GetValues, I’ll omit that in my discussion here.) Other methods are either not applicable (GetUnderlyingType, IsDefined, ToObject) or really belong as instance methods on the enums themselves (GetName, Format).

Indeed, it seems silly that you would be unable to enumerate the values in your enumerated type programatically. As a side note before I begin, I would prefer to have my GetValues method return a set, but the base class library in the .NET framework lacks a such a concept, so I’ll muddle through with lists and arrays. The first solution I’ve used is to add a static set to the enum class itself, then have the constructor of the enum add the new instance to that set:

public abstract class EnumType {
    public static readonly EnumType One = new TypeOne();
    public static readonly EnumType Two = new TypeTwo();
    // ...

    private static readonly List<EnumType> allValues = new List<EnumType>();

    private EnumType() {
        allValues.Add(this);
    }

    public static EnumType[] GetValues() {
        return allValues.ToArray();
    }

    // ...
}

This seems to me to be a pretty good solution for a single enum type. The main problem is that you end up having to duplicate this code in every single enum type you create, which could be a lot of extra work if you have a number of enum types. It might be a good time to take a step back and think about the best way to do this.

What the .NET framework’s GetValues method does is take the System.Type it’s given as a parameter, use reflection on the type to get the values, and assign them to a System.Array, since System.Enum doesn’t know the runtime type of the Type you give at compile-time. Well, even if we use classes, we still will have to use reflection to get the values—there’s no way to know them at the time you write the base enum type—but we can avoid requiring you to pass in the Type as a parameter and return a correctly-typed array type rather than just the general Array supertype with a clever application of generics. Here’s the solution I propose:

public abstract class EnumBase<E> where E : EnumBase<E> {
    public static E[] GetValues() {
        FieldInfo[] fields = typeof(E).GetFields(BindingFlags.Public | BindingFlags.Static);
        return Array.ConvertAll<FieldInfo, E>(fields, delegate(FieldInfo f) { return (E) f.GetValue(null); });
    }
}

public abstract class EnumType : EnumBase<EnumType> {
    // ...
}

Before I say anything else, I’ll admit to borrowing part of this idea from Java: in Java 5.0, java.lang.Enum is parameterized in the same way as my EnumBase type. The constraint on the type parameter is a very difficult one to get your head around, and thinking about it too hard is likely to cause your head to spin into an infinite loop. I think of it as “the type parameter to the EnumBase type has to be the enum type itself.” If it still doesn’t make sense to you, I wouldn’t worry about it too much; try the code and convince yourself that it does, in fact, compile. However, even though I got the idea from Java, this method couldn’t be written in Java in the same way because of how they implemented generics.

I like this solution because now, you can simply call EnumType.GetValues, or if you make any other enum type, it will inherit the static GetValues method, which will do the right thing for that enum. Best of all, all enums can now be related in the class hierarchy, just as System.Enum is a supertype of all built-in enum types. Thus, if you come up with any more functionality for your enums to share, you can just add it to the base type, instead of trying to change every enum class itself.

One final note before I go: because the GetValues method returns an array, it isn’t safe to return the same array to all consumers, since any consumer could modify the array. Ideally, it would return a read-only set, which could be computed once, then cached for future calls, which could all return that instance. However, with the arrays, you could cache a single copy of the array, then just copy that array upon each call to GetValues to avoid the cost of performing the reflection on each call.

Enumerated Types: Dynamic Differentiation

nealho — Tue, 20 Dec 2005 02:04:00 GMT

So class-based enumerated types allow you to define dynamic behaviors, where the consumer of the enum doesn't have to know specifically which instance of the enum he has; he need only invoke the appropriate method, and the type of the instance will determine the runtime behavior. This use of dynamic dispatch is familiar to all of us object-oriented programming nerds, and imo, this is the cornerstone of OO's power.

However, we're left with a problem: what if, when designing the code, we don't know all the dynamic behaviors that someone might want to use on our enum? Well, if we're writing code for any kind of public API, then this is obviously the case. Using our model from before, we'd need code like:

    EnumType myEnumValue;

    // do something to get your value

    if (myEnumValue == EnumType.One) {
    }
    else if (myEnumValue == EnumType.Two) {
    }
    // ... rest of cases
    else {
        // this should not be possible, but it's still prudent to cover this case
        throw new UnreachableCodeException();
    }

While this is not horrible, it has two issues. The first is its inefficiency—in order to get to any given element of the enum, a comparison must be performed with each element preceding it. The second is that, just as with C#'s built-in enums, you are left with that final else block. It should be unreachable, but the compiler can't tell you that for sure.

The most obvious solution is to restore the functionality given by built-in enums by adding an integer index to each enum value. Because case labels must be constant, this is a bit awkward and requires adding constants for each enum value so that these indices can be known at compile-time:

public abstract class EnumType {
    public static readonly EnumType One = new TypeOne();
    public const int OneValue = 1;
    public static readonly EnumType Two = new TypeTwo();
    public const int TwoValue = 2;
    // ...

    private int _value;
    private string _name;

    private EnumType(int value, string name) {
        _value = value;
        _name = name;
    }

    public abstract void DoSomething();

    public int Value {
        get {
            return _value;
        }
    }

    public string Name {
        get {
            return _name;
        }
    }

    private class TypeOne : EnumType {
        public TypeOne() : base(OneValue, "One") {
        }

        public override void DoSomething() {
            // Do whatever One does
        }
    }
    
    private class TypeTwo : EnumType {
        public TypeTwo() : base(TwoValue, "Two") {
        }

        public override void DoSomething() {
            // Do whatever Two does
        }
    }
    
    // ...
}

We are now able to use a switch statement on this enum:

   switch (myEnumValue.Value) {
        case EnumType.OneValue:
            // myEnumValue == EnumType.One
            break;
        case EnumType.TwoValue:
            // myEnumValue == EnumType.Two
            break;
        // ...
        default:
            throw new UnreachableCodeException();
            break;
    }

Well, I think we've improved our perf and code readability, but we've introduced something pretty ugly: every enum value now requires two declarations instead of just one. This may not be too bad at first, but it could be a maintenance issue, and if the constants get out of sync with their corresponding enum instances, it could also be a source of some pretty insidious bugs. Moreover, we still have that default block, because the compiler doesn't know for sure that we've covered all our enum cases.

For serious OOP junkies like me, the solution is probably screaming at you: use the visitor pattern. For those of you not familiar with it, a “visitor” is a class that implements an algorithm on the type being visited. The class then has a “visitor hook,” which is a method that calls the appropriate case in the visitor class. So your new code would look like this:

public interface IEnumTypeVisitor<T> {
    T ForOne();
    T ForTwo();
    // ...
}

public abstract class EnumType {
    public static readonly EnumType One = new TypeOne();
    public static readonly EnumType Two = new TypeTwo();
    // ...

    private string _name;

    private EnumType(string name) {
        _name = name;
    }

    public abstract T Accept<T>(IEnumTypeVisitor<T> visitor);

    public string Name {
        get {
            return _name;
        }
    }

    private class TypeOne : EnumType {
        public TypeOne() : base("One") {
        }

        public override T Accept<T>(IEnumTypeVisitor<T> visitor) {
            return visitor.ForOne();
        }
    }

    private class TypeTwo : EnumType {
        public TypeTwo() : base("Two") {
        }

        public override T Accept<T>(IEnumTypeVisitor<T> visitor) {
            return visitor.ForTwo();
        }
    }

    // ...
}

public class MyEnumVisitor : IEnumTypeVisitor<string> {
    public string ForOne() {
        return "This is MyEnumVisitor, and it was called on an EnumType.One!";
    }

    public string ForTwo() {
        return "This is MyEnumVisitor, and it was called on an EnumType.Two!";
    }

    // ...
}

Now, any consumer can write their own IEnumTypeVisitor to define their own dynamic behaviors, and simply call the Accept method of their enum instance to have the correct case automatically determined by the runtime. We've even removed the extra default case that we know we should never reach: because the possible enum values are determined by the visitor interface, the compiler knows that we will never hit another case.

I'm not going to claim this enum design is perfect, but I think it is much better than the integer-based alternative. By using a truly object-oriented design, I think it’s a lot easier to write clean, elegant code.

That pretty much covers the basics of how I think about enum design. I’ll post some of my favorite tricks later on.

Enumerated Types: Using Classes

nealho — Sat, 17 Dec 2005 01:13:00 GMT

I said I’d blog about how exactly I use enums, but there are a lot of interesting issues here, so I think I’ll develop my approach over the course of a few posts. Here’s my simplest form of class-based enum:

public class EnumType {
    public static readonly EnumType One = new EnumType("One");
    public static readonly EnumType Two = new EnumType("Two");
    // You can add as many instances as you want, but I’ll stick to two for the sake of brevity

    private string _name;

    private EnumType(string name) {
        _name = name;
    }
    
    public string Name {
        get {
            return _name;
        }
    }
}

This approach solves one problem C#’s built-in enum types: because the EnumType constructor is private, no one else can ever create another instance. If EnumType.One and EnumType.Two are all you create in the class, then you know that there will be no other instances of EnumType.

This style of enum is good for simple enumerations where all you’re doing is differentiating between a several options and maybe displaying a value depending on which option you pick. To check which value of the enum you have, you can just use reference equality and chained if/else blocks, which are admittedly less performant than a switch statement, but not notably more difficult to code.

The next step is a simple composite pattern, which I’m sure you can all guess, so I’ll go on with that right now:

public abstract class EnumType {
    public static readonly EnumType One = new TypeOne();
    public static readonly EnumType Two = new TypeTwo();
    // ...

    private EnumType() {
    }

    public abstract void DoSomething();

    public abstract string Name { get; }

    private class TypeOne : EnumType {
        public override void DoSomething() {
            Console.WriteLine("I am #1!");
        }
        
        public override string Name {
            get {
                return "One";
            }
        }
    }
    
    private class TypeTwo : EnumType {
        public override void DoSomething() {
            Console.WriteLine("I am #2!");
        }
        
        public override string Name {
            get {
                return "Two";
            }
        }
    }
    
    // ...
}

Now, each instance of EnumType is actually a singleton instance of its own class. Now, your enums are much more powerful: in addition to fields, each value in the enum can have its own behaviors in the form of concrete implementations of the parent’s abstract method. There is still no way for another user to add values to the enum—because EnumType’s constructor is private, it can’t be subclassed outside of the class itself.

One minor implementation detail here—the only reason I chose to change the Name property from referencing a field to being abstract and implemented by each subclass was to show that it was an option. It could just as easily have remained a field of EnumType, and TypeOne and TypeTwo would have called the base construtor with the appropriate strings instead of overriding the Name property. I am not convinced that one way is better than the other; it’s just a stylistic preference.

We’re making progress away from the C-style need to treat everything as an int, but there’s still a lot of distance to cover. Notably, we still can’t switch on the enum types (long, chained if-else blocks are so inelegant!), and we've lost the support of methods from System.Enum like GetValues and Parse. I’ll explore how to resolve these issues and try to think about how C# or a language like it could provide better support for this sort of enum in upcoming posts.

Enumerated Types

nealho — Fri, 16 Dec 2005 05:20:00 GMT

I had an interesting discussion earlier regarding enums and how they work in .NET, and since I haven't I blogged in a while, I thought I'd post some thoughts.

Conceptually, an enumerated type is a type that has a set of values that can be (surprise surprise) enumerated. This means, most significantly, that there is a finite number of possible values you can assign an enum. That's a fairly abstract and, frankly, not very useful definition in the context of a programming language.

Let's move on to an (admittedly impractical) example. The classic example of enum usage is as a primitive type identifier, such as:

public enum FruitKind {
    Banana, Cherry, Orange
}

Then, you write some methods that deal with these fruits:

public class FruitConsumer {
    ...
    public string GetColor(FruitKind kind) {

    switch (kind) {
        case FruitKind.Banana:
            return "Yellow";
        case FruitKind.Cherry:
            return "Red";
        case FruitKind.Orange:
            return "Orange";
        }
    }
}

But wait, the compiler is complaining that this code doesn't compile. "'FruitConsumer.GetColor(FruitKind)': not all code paths return a value," it tells me. But I covered every possible value of the enum, right? What other code path is there? The answer, of course, lies with something that is intimately familiar to those of us with C/C++ background: enums are really integers under the hood. FruitKinds are really ints under the hood, wrapped in a pretty box with a little language support for using the named values.

This dichotomy has some advantages. For example, there are a lot of flag-type enums, where you can have any number of a series of values. For example, the System.Reflection.BindingFlags type allows you to specify any number of binding options with the | (bitwise or) operator, and then the consumers of these can use & (bitwise and) to see if your flags fit. This has good perf without overly obfuscating the meaning of those operations.

Additionally, you gain the convenience is the ability to switch on enums. One other "advantage" many people will cite is the ability to do regular arithmetic on enums, though I have yet to see a convincing example of this being a good idea.

There are some downsides, too: in the FruitKind example, you weren't certain that the code could never be called with a value other than those you defined. Mostly, I am just bothered by the "weirdness" factor, that your enums are not guaranteed to be one of the values you set forth. The OO solution is to use an abstract base class as the enum type with a private constructor, and have nested classes with singleton instances as your enum values (good post on the subject). I think that's by far the best solution: in return for arithmetic operations, you get methods; in return for switch/case, you can add a visitor hook. Overall, it gives much greater flexibility (which, like most things, comes at the cost of some complexity).

One final note, which is completely off this train of thought, but it is the last thing that bugs me about enums in C#. The literal zero is always implicitly convertible to any enum type. Any integral value can be converted to any enum type explicitly, but only zero can be converted implicitly. The reasoning, as I understand it, has to do with the fact that the memory that your enum instances are occupying is going to start out as zero anyway, so saying "FruitKind kind = 0" is the same as saying "FruitKind kind = new FruitKind()"; this is logical enough from a computer organization point of view. At a high level, though, it doesn't mean I have to like it. ;)

Breaking Changes

nealho — Wed, 23 Nov 2005 06:06:00 GMT

Something we in the C# Compiler team have been discussing recently is the issue of including "breaking changes" in our code. This is a big deal, especially for a compiler, which people expect to be pretty rock-solid. There are a lot of issues here, most of which remain unresolved.

The first issue is the question of what, exactly, a breaking change is. One could probably make the case that any code change, no matter how minor, could be a "breaking" change because someone could be relying on that particular functionality. Thus, we're required to make a few assumptions about our users. A good starting point is to assume that no one relies on crashes of any kind. Any time we see "Internal Compiler Error" we know that fixing the issue is the right thing to do—a code change that causes the compiler not to crash where it crashed before is not a breaking change. All right, that one's a no-brainer, and while that's an important class of errors, it hardly covers the majority of issues users might encounter, so let's break down code changes further into two major categories, excluding crashing issues: there is code that did not compile under the previous version, and code that did compile under the previous version.

If the code didn't compile in the first place, there are two changes we can make. One is to change the error messages that are printed out, probably with the intention of making them clearer or more relevant to the problem at hand. It seems reasonable to assume that people wouldn't rely on code not compiling in a certain way—the code didn't compile before, and it still doesn't compile. This class of errors does not contain breaking changes. The other possibility is that the code that didn't compile is now accepted. This also isn't likely to break anyone. If we add a new feature that reasonably accepts code that would have been illegal before, people aren't likely to complain that we are accepting programs they were expecting to fail (I can't imagine hearing a complaint that class Foo<T> { ... } is accepted by the Whidbey compiler, since it's not valid code in Everett!). We also don't consider these breaking changes.

All right, so we've determined that it's okay to change things about code that doesn't compile anyway. I'll take the category of code that did compile and further divide it into three subcategories: first, changes in warnings generated; second, changes in actual code generated; and third, changes that raise new errors. Changing the warnings is generally considered acceptable, and is often a prelude to more serious changes. For example, it is common practice to add a warning on a newly-deprecated construct. These are not breaking changes.

The next two categories are a bit more serious, and the astute reader will probably think that this is the interesting part. First, if the generated code is different, that is likely to be a problem for some people who depended on an old behavior. For example, in Whidbey, we fixed an issue with enum types and integer math, where an enum whose underlying type was byte could be subtracted from another enum of the same type, assigned to an int, and the result would be -1. Now, if you were to do the same math with the underlying byte type, you would end up with 255, because the byte arithmetic would overflow (byte is unsigned, and operations are unchecked by default). We're giving the right answer here, but it's still a breaking change because we had real customers who relied on the old behavior, even though it was incorrect.

Finally, let's look at the case where code that was accepted simply fails on a new version. These are the clearest breaking changes; anyone who had the (previously valid) code would be broken by this sort of change. Take, for example, our handling of [Conditional(DEBUG)] attribute. In Everett, we allowed methods marked with [Conditional(DEBUG)] to have out parameters, but in Whidbey, it was disallowed because it cannot be guaranteed that the parameter will be assigned in a release build. However, this is also a real customer scenario where we broke someone who had [Conditional(DEBUG)] methods that had out parameters. What seemed to be a sensible design revision caused some customers a lot of trouble.

All this thought has been part of a long discussion among some of us on the C# team on how, exactly, to deal with breaking changes. This is a long process, starting with devs considering if they might be adding breaking changes, going through QA noticing breaking changes involved in bug fixes, and finally moving on to PMs and other customer-focused team members educating users about what changes have been made. It's no single person's responsibility, which means that it has not always gotten as much attention as it should.

What's the right solution? I'll let you know when we come up with it, but for now, we'll be encouraging devs to think about users they may be breaking when they make checkins, asking QA to consider each bug they sign off on carefully, and getting everyone to keep their PMs informed about these changes as soon as they are made and discovered. That way, we'll have an easier time knowing and documenting what changes exactly may adversely affect our users between versions, and help them transition more smoothly from each version to the next.

Expressions vs. Statements

nealho — Tue, 08 Nov 2005 06:38:00 GMT

This morning, I was discussing issues around my previous post with Mads Torgersen, the PM who owns the C# compiler, and it reminded me of the next issue I wanted to talk about. Something I'm pushing for in the Orcas timeframe is to add not only the Func<> types, but Proc<> types as well, which specify delegates that return void. This and related types complete the set of standard delegate types for various APIs. However, the very need for these extra times brings me to another issue with C#, which is a holdover from being a C-like language (and so, of course, this is a problem in C++ and Java as well). That is the difference between a statement and an expression.

Now, this difference is probably a lot more intuitive for most programmers than the ones I was discussing last time, because nearly everyone who's ever done any kind of programming has done at least some coding in C, C++, Java, and C#. All of these languages have these two constructs in their grammars, and the difference is woven into the very fabric of these languages. We are taught to think of programs conceptually as a series of statements, where each statement modifies the state of the program's state in some way before the next statement executes (I'll limit the discussion to single-threaded applications for simplicity). The reasoning for this is clear—this is what happens when our programs run. The (classical) processor executes an instruction, which changes some register or writes some value to memory, then executes the next instruction. The C language, which is essentially portable assembly, should obviously be a series of statements.

Unfortunately, even in C, we want to be able to use higher-level constructs than can be encoded in individual or even a small group of machine instructions. This leads to the concept of an expression, which can be a part of a statement. Now, subroutines can can "return" a value to be used by the caller instead of simply modifying explicitly declared state. Obviously, this is a powerful concept, but it leads to an unfortunate inconsistency in the design of the language. For exaple, compare (condition ? trueCase : falseCase) with (if (condition) { trueCase; } else { falseCase; }). What is the difference here? The expression "condition" is evaluated in both cases, exactly the same way. If trueCase and falseCase are expressions, then there is no difference at all! If condition is true, then trueCase is executed, else falseCase is executed. Why are there two language constructs to express exactly the same thing? I ask rhetorically, of course, because if-else expects "statements" while the ternary conditional operator expects "expressions." Honestly, though, it seems silly that ? : can't contain blocks, and that if-else can't return a value.

Still, this leaves a problem. Conceptually, a statement is simply an expression that returns no value. Okay, but if every statement becomes an expression, how do you differentiate between a method with a return value and a method that returns void? Even in this pretty, expression-based world, we may still want a method that simply modifies state and doesn't need to return a value, so why not make void an actual part of the type system? A method can still return void, and you can still call it and ignore its return value. Moreover, we wouldn't need separate Proc types; one could simply say Func<T, void> to get that Proc<T> (or Action<T>, if we're looking at Whidbey APIs). Of course, this would make some silly things possible, like declaring a variable of type void, or having a List<void>, but I can't imagine why anyone would want to do that. :)

The very syntax of C#, Java, and other C-like languages makes this difficult, since statements are baked into the grammar, so we're stuck with void being its own special non-type. So if you find yourself annoyed at the inconsistency of statements and expressions, take a break and write some code in Scheme or OCaml. The exercise in functional thinking will do you good, and imho, it'll make you write better code in C#.

Func<> and delegate types

nealho — Fri, 04 Nov 2005 07:47:00 GMT

I'm a bit new both to the C# team and to blogging, so bear with me as I muddle through my first post about something that interests me. :)

Let me start at the beginning. Since C# 1.0, one of the things that I liked about it (compared to Java, since I'm an OO nerd) was the fact that it had a way to pass functions from one place to another. However, delegate types hardly provided the ease of code passing that my functional background would like, and with Java, anonymous inner classes at least provided a simple way of writing a bit of code in a method, including closing over some of the local state, and passing that to another method. Of course, the downside was that I always kept an IFunction interface around, which specified a single method from object to object (and, once I started using the Sun's JSR-14 comiler, from T to U) that I could instantiate as needed for my code passing requirements.

C# 2.0's anonymous methods made that IFunction type look clunky with all of Java's extra syntax required to create an anonymous inner class. As with the IFunction type in Java, however, I needed a delegate type for the signature of any method that needed to accept code as an argument. No big deal, just requires writing a line declaring that delegate type above the function, and thanks to implicit conversion from anonymous methods to an appropriate delegate type, I never needed to think about that type name again, unless it came up in error messages (usually due to the limitations of the type inference that goes on when generic delegate types are combined with polymorphic methods--but this is an issue for another blog entry). The convenience of not having to write a new method every time I wanted to create a delegate combined with the intuitive power of closures (well, almost closures--but this is a well-covered issue that I don't want to reiterate here) made me that much more productive when coding.

Now to the main event. Since the PDC, I've heard a lot of buzz surrounding all of the various "function" delegate types that are included in the LINQ preview. I'm sure those of you with any experience with functional programming are breathing sighs of relief that such obvious types are finally being included in the .NET framework--I know I am! Still, along with this relief is a healthy dose of caution. I find myself wary of how these types will integrate into the existing framework, with all of its predefined delegate types. For example, suppose I write a method using the LINQ framework that accepts Func<int, bool>, and am interested in using this function to filter a List<int> that I already have. Looking down the IntelliSense-provided options suggests that the FindAll method will perform the action that I want. So I call FindAll on my list and pass it my Func<int, bool>. But what's this? The compiler is telling me that Func<int, bool> cannot be converted to Predicate<int>! That's ridiculous: I have a predicate, at least conceptually. Really, what's the difference between a Predicate<int> and a Func<int, bool>? Both represent a function that takes an int and returns a bool, so why aren't they compatible?

I had mistakenly convinced myself that methods really were just as powerful in C# as they are in Lisp. All the ease of using delegates that came with anonymous methods and implicit conversions to delegate types had blinded me to their original flaw--that despite their ease of use, methods are still not first-class entities. Because the language doesn't consider a method an actual type, despite the fact that the same method could be converted to either a Func<int, bool> or a Predicate<int>, you can't simply convert one to the other. Of course, thanks to anonymous methods, and more recently, lambda expressions, this isn't hard to work around. Given a Func<int, bool> func, you can create a Predicate<int> simply by saying arg => func(arg). But having to write the identity function still feels awkward, and reduces the functional appeal of your program.

Where am I going with all this? Well, even without fully first-class method support, it would have been nice if the Func<> types had been included in the framework since v1.0. Of course, this wasn't possible before 2.0 (hooray for generics!), and I guess no one thought they'd be important enough to add for Whidbey. Maybe they're right on that point; it does seem like very few C# programmers use delegates in their APIs.

I suppose the real bottom line is that I need to bear in mind the distinction between methods, which are the actual code that take and return arguments, and delegates, which merely encapsulate a method (or several methods--but once again, this is not an issue for this post). This distinction is subtle, but anyone who programs in C# should understand it.