Sam Ng's Blog : Optional arguments

Named and Optional arguments – ties and philosophies

samng — Sat, 18 Apr 2009 01:19:46 GMT

Okay, my attempt at a clever title failed… Ties and Philosophers? I oughtta stick with technical writing. :)

We’re almost done with our chat about named and optional arguments. We’ve covered what the feature is about, and covered overload resolution in more detail. This time I want to do a quick wrap up of our discussion by talking about the tie breaker rules, and then I want to give a bit of background and philosophy behind why we’re electing to do this feature now, instead of several releases ago when it was first considered.

Tie breakers

During overload resolution, the compiler may find that there are two or more candidates that are perfectly legal candidates given the arguments specified. When that happens, we apply our tie breaker rules to figure out which of the methods is the best one.

In the past, breaking ties was simple (well, not really, but simpler than it will be now!). Because every candidate had the same number of arguments, (again, not really – param arrays! I’ll explain that in a sec) we simply needed to apply our betterness rules to the conversions to see if we could find a best method. This check amounted to taking each pair of candidates, and checking if either of them has equal-or-better conversions for each parameter. If so, that method was considered the better choice, and was chosen as the best method to bind to. Optional arguments throw a wrench in the mix – now we can have two candidates that are both perfectly valid, but have a different number of parameters.

First, a quick example:

class C
{
    public void Foo(object o) { }
    public void Foo(int x) { }

    static void Main()
    {
        C c = new C();
        c.Foo(10);
    }
}

In this example, both overloads of Foo are applicable – the integer 10 is convertible to object, and is certainly convertible to int. The traditional tie breaker rule simply says that since 10 is an integer, the conversion to int is better than the conversion to object. Therefore Foo(int x) wins.

Param arrays

Well what about parameter arrays? Lets tweak our example a little bit:

class C
{
    public void Foo(int x, int y) { }
    public void Foo(params int[] x) { }

    static void Main()
    {
        C c = new C();
        c.Foo(10, 20);
    }
}

In this example, the compiler takes the params array, and expands it so that the signature looks like Foo(int x_1, int x_2), and uses that signature as the candidate signature. However, it also notes that the signature came from a params array, and param arrays are treated as second-class compared to real parameters. Since everything else is equal, the second-class-ness of the params array loses the tie to the first class parameters.

Optional arguments

So how do optionals play into the mix? Well lets consider this third example:

class C
{
    public void Foo(int x) { }
    public void Foo(int x, int y = 0, int z = 10) { }

    static void Main()
    {
        C c = new C();
        c.Foo(10);
    }
}

In this example, both candidates are applicable, since they both match for the first argument, and the second candidate has optional arguments for the rest. The conversion for the first argument is the same for both candidates, and there are no param arrays. How do we pick which one is better?

Much like param arrays, the thing to do here is to treat optional arguments like param arrays – as second class arguments. Because the first method has no optional arguments and the second one does, we pick the first one.

Notice that if both candidates had optional arguments, at that point we’ve got nothing more to go on, and so the call is ambiguous.

Philosophy time!

Okay! Time for some philosophy. Why are we doing this feature now? Why didn’t we do it earlier?

Let me tackle the second question first, as it’s got a shorter answer. The reason we didn’t do it earlier is that we really didn’t want this feature in our language. We’ve pushed back on it for this long because it’s not the paradigm that we want.

So that brings us to the first question – why now?

Well, the quick answer is because of COM. It just won’t go away! Try as we might, people still use it (and are still going to continue using it). What does COM have to do with C#? Office. Office PIAs. The Office PIAs are designed such that many of the methods have about 30 parameters, and all of them are optional. Most of the time, what you’d want to do is specify one argument, and use the defaults for the rest.

Enter named and optional arguments. Because we allow you now to call methods without specifying their optional arguments, you can now call these Office methods without passing Type.Missing in as every other argument. And because we allow you to use names to specify exactly which parameter you’re passing this argument for, you can call the method by only passing what you want, and omitting the rest.

Combine this with our COM-no-ref feature in which the compiler takes the optional ref parameter and generates a local temporary variable for you and passes that as the parameter, your COM code will look much cleaner and will be much less tedious to write.

As I mentioned way back when, one of the big themes for C# 4.0 is interop with other “runtimes” – COM, dynamic languages, other user-defined type systems. That theme made this feature set a must-have.

As always, I would love all the feedback you guys have! And of course, happy coding!

Named arguments and overload resolution

samng — Wed, 01 Apr 2009 21:16:42 GMT

Last time we talked about the basics of named arguments, optional arguments, and default values. From here on out, I’m just going to refer to the whole feature group as “named and optional arguments” – it’s just too much typing otherwise (we actually just refer to the feature as N&O internally). Let’s now dive a little deeper into how overload resolution works for the feature.

Where do we get those wonderful names?

The first thing we need to figure out, is where we get the names from. Because the CLR does not consider parameter names as part of the method signature, it is perfectly legal to override a method and specify different parameter names than the base method that one is overriding. Lets consider the following:

public class Animal
{
    public virtual void Eat(string foodType = "Grub") { }
}

public class Monkey : Animal
{
    public override void Eat(string bananaType = "Green banana") { }
}

This is perfectly legal C# code. So then which names do we pick? Do we pick the ones from the base method? Or the most derived one? What about interfaces? Let’s consider a usage of named arguments with this example.

public class Program
{
    static void Main()
    {
        Monkey m = new Monkey();
        Animal a = m;

        m.Eat(bananaType:"Ripe banana");
        a.Eat(foodType:"Yummy grub");
    }
}

If we consider the receiver (aka the calling object/type) as the anchor in this whole scheme, and use its statically determined type to figure out the names, then we’ve got ourselves a nice little scheme that is deterministic and quite sensible.

In our example then, because m is statically typed to be of type Monkey, m.Eat gets the names of the parameters from the Monkey class, and so bananaType is the correct name to be using. Similarly, a is typed Animal, and so the call to a.Eat gets the name foodType as its parameter. Notice that even though their runtime types will be identical (that is, we’ve taken an instance of Monkey and assigned it into an Animal local variable), that doesn’t matter – the named parameter feature is simply a syntactic sugar for a compile time rewrite.

So what happens under the covers?

Lets take a look at the example that we used last time:

public class ContactList
{
    List<Contact> SearchForContacts(
        string name = "any",
        int age = -1,
        string address = "any") { ... }

    static void Main()
    {
        ContactList list = new ContactList();
        var x = list.SearchForContacts(age:26);
    }
}

What actually happens under the covers here? When the compiler sees the call to list.SearchForContacts, it first performs a few quick validations to make sure that any positional arguments (arguments not specified by name) occur before any named arguments, and that no names are specified twice. Then it generates a set of all applicable candidates. In our example, there is but one candidate to consider. Then for each candidate, the compiler looks at the arguments, and performs a few verifications.

First it checks to make sure that the names specified in the call (in our case, “age”) is valid for the candidate (ie the candidate has a parameter named “age”). Next, the compiler moves past all positional arguments attempts to match each named argument up with its corresponding parameter. Of course, all named arguments must match parameters who do not have a positional argument specified for it (ie you cannot specify an argument for the same parameter twice).

For each parameter that does not have a corresponding positional or named argument, the compiler checks to make sure it is optional. It then uses the default parameter value for each of those arguments.

Once the compiler has generated this augmented argument list, it performs argument convertibility on it as usual. The resulting augmented list in our example then, is: “any”, 26, “any”.

Just compiler magic

I love magic. I love the concept of magic. It amazes me. I went to Disneyland recently for PDC, and it was magical. I know it isn’t real, and that something else is happening, but I love it anyway. That’s what named arguments are like. Underneath the covers, there is no trace of named (or optional) things – it all looks like straight IL, as if you called the methods with the augmented argument list.

That means that after the compiler has generated the augmented list for you and has found the best candidate, it treats the call as if you had called it with the augmented list, and everything else behaves as it used to.

That being said, it means that this feature is totally a compile time syntactic sugar, and so it doesn’t introduce any new dependencies, and doesn’t introduce any new compatibility issues or anything like that. Programs that you compile against one set of names and default values will absolutely continue to keep working even if the library changes and has a new set of names and default values.

One more piece of magic

Turns out there’s one more crucial thing that the compiler does for you, which is really worth mentioning. Order of evaluation. Lets consider the following example:

public class C
{
    static void Main()
    {
        C c = new C();

        c.M(z:Foo(), x:Bar(), y:Baz());
    }

    void M(int x, int y, int z) { ... }
}

What happens here? Well, the compiler will reorder the arguments so that the result of Bar() gets passed as the first argument, the result of Baz() gets passed as the second, and the result of Foo() gets passed as the third. However, what order should these sub-expressions be evaluated? You’d expect them to be evaluated as written – Foo first, then Bar, then Baz.

And that’s exactly what we do. We essentially create temporaries which store the value of evaluating each of those sub-expressions (which means that all side effects of evaluating each expression happen in syntax order as you’d expect), and reorder those temporaries to match their respective named positions. See? Magic!

So hopefully that gives you a good feel for how the feature works, and how overload resolution for it works. Have fun with it, and definitely give us your feedback!

Named arguments, optional arguments, and default values

samng — Wed, 04 Feb 2009 09:33:01 GMT

C# 4.0 introduces the concept of optional parameter values into the language. Now, this has been a controversial subject in the past, and we have had many requests for the feature, but have traditionally stayed away from it. So, why now?

Well, before we get into the philosophy of why we decided to add it this time (which we will! I promise!), first lets discuss the feature itself.

I'll first let's talk about the features themselves, and will follow that with a brief discussion about the focus of C# 4.0 and how these features align with that focus. I'll expand in depth as to how each feature works, and what the compiler does behind the scenes in future posts.

Default parameter values

We've all had the experience of writing (or at least using) methods whose parameters are optional in nature. Often, those methods will contain overloads which will have the optional parameters removed, and will simply be a wrapper for the actual method, passing the default values that the library writer wants to provide for the method.

Default parameter values give us a way to explicitly declare the default value that we would like to be used for the parameter in the method signature itself, instead of in the forwarding method. This gives the added benefit of allowing the IDE to help out and inform the consumer of the default value for the given parameter.

These guys are specified in one of two ways:

public class C
{
    static void Foo([DefaultParameterValueAttribute(10)] int x, int y = 20) { }
}

The first mechanism is purely a backwards compatibility issue and an interop issue with COM and VB. We highly recommend against using this method, however, because it specifies that there is a default parameter value for the parameter, but does not specify that the parameter is in fact optional. This means that the default value given will never be used.

The second mechanism really says two things - first, it tells the compiler that this parameter is optional, meaning the user does not have to specify an argument in this position. Second, it gives the compiler the value to use when the user does not specify an argument for the parameter.

How default parameter values are encoded

Because the new syntax really does two things, it is encoded in the metadata as two things. We encode both the optional flag in the metadata signature (which is equivalent to using the OptionalAttribute attribute), and we encode the default value as the DefaultParmeterValueAttribute attribute.

These two mechanisms have already existed in the CLR, and are in fact what the VB compiler produces today, giving us an easy choice for how to encode the feature.

Note that the compiler will give you an error if you try to specify both a DefaultParameterValueAttribute attribute as well as a default value using the new syntax.

Also note that the compiler enforces that all optional parameters using the new syntax are specified after all required parameters.

Lastly, the compiler will not allow you to specify default parameter values for ref or out parameters. This is because there is no valid constant expression that is convertible to the ref or out type.

Optional arguments - how default parameter values are used

In order to make use of the default parameter values, we've added a feature which allows the caller to omit arguments for parameters which have default parameter values specified.

Notice that because we enforce that optional parameter values are specified at the end of the parameter list, you cannot omit an argument but specify an argument for a later position.

This means that even though existing libraries may have specified the DefaultParameterValueAttribute and the OptionalAttribute for a parameter in the middle of the list, the C# compiler will not allow you to call that method without specifying a value for that parameter.

You can kind of think of optional arguments as a params array - the compiler will allow you to call the method without specifying the arguments, but they must be at the end of the parameter list.

What gets code gen'ed?

First we need to note that the use of optional arguments is really just syntactic sugar. The compiler cannot generate a call without actually providing all the arguments - it simply provides an argument for you and allows you to omit specifying it.

Once the compiler realizes that you're calling a method and omitting an argument because it is optional, it takes the value specified in the DefaultParameterValueAttribute and encodes that as a constant value for the argument that you've omitted. Note that if you're calling a library that doesn't have a default value specified but is still optional, the compiler will use default(T) as the value, where T is the type of the parameter. Also note that for COM calls, we also allow you to omit arguments to ref parameters by generating the temporaries for you.

Putting it all together - Named arguments

The ability to omit specifying arguments for parameters that have a default value is really taken advantage of by the Named arguments feature. This feature allows you to specify by name the parameter for which you are providing a value. This means that you can now omit specifying arguments for all parameters, and only specify the ones which you actually care about.

For COM programmers, this is like heaven! I recently "got" to play with Office interop a bit, and found that the average method had 30 parameters! Most of the parameters are wonderfully optional, and with the ability to now omit them, code looks much cleaner and is much more readable.

Sorry, I'm getting ahead of myself. First let me describe the feature.

Named argument usage

This feature introduces new syntax at the call site of a method. Consider the following example:

public class ContactList
{
    List<Contact> SearchForContacts(
        string name = "any",
        int age = -1,
        string address = "any") { ... }

    static void Main()
    {
        ContactList list = new ContactList();
        var x = list.SearchForContacts(age:26);
    }
}

We've got some library method that searches through some contacts and finds the ones that match our criteria (I know, its a horrible API, but bear with me). The library is great! ;) It specifies the default wildcards for us so that we can omit them as we wish. Now suppose we want to find all contacts aged 26.

Well, since we now have optional arguments, we can omit the arguments after the age parameter, but that gets us half way there. Named arguments get us the rest of the way.

By using the new syntax, we can specify by name the argument that we want to specify a value for, and omit the rest. The syntax is quite simple (well, not really, but I'll get into the details of its complexities later): simply specify the name of the parameter that you want to specify an argument for, add a colon, then add the value that you want.

Note that the value can be any expression type that you normally could have specified. Note also that named arguments are not restricted to parameters that are optional or have default values. Lastly, note that you don't even have to specify the arguments in order! We could call our method with the following:

list.SearchForContacts(address:"home", name:"sam", age:30);

The arguments don't have to be in any particular order. The only rule is that the named arguments must be specified at the end of the argument list. This means that all positional arguments (the "normal" ones that aren't specified by name) must be given first. Once the compiler encounters a named specification, it will produce an error upon encountering any further unnamed arguments.

So that's the feature in a nutshell. There are more details that I'll get to later, but in the mean time I'd love to get your feedback on the feature, on its uses, and on how we've chosen to design it.

And as always, happy coding!