Covariance and Contravariance in C#, Part Nine: Breaking Changes

Covariance and Contravariance in C#, Part Nine: Breaking Changes

Rate This
  • Comments 34

Today in the last entry in my ongoing saga of covariance and contravariance I’ll discuss what breaking changes adding this feature might cause.

Simply adding variance awareness to the conversion rules should never cause any breaking change. However, the combination of adding variance to the conversion rules and making some types have variant parameters causes potential breaking changes.

People are generally smart enough to not write:

if (x is Animal)
  DoSomething();
else if (x is Giraffe)
  DoSomethingElse(); // never runs

because the second condition is entirely subsumed by the first. But today in C# 3.0 it is entirely sensible to write

if (x is IEnumerable<Animal>)
  DoSomething();
else if (x is IEnumerable<Giraffe>)
  DoSomethingElse();

because there did not used to be any conversion between IEnumerable<Animal> and IEnumerable<Giraffe>. If we turn on covariance in IEnumerable<T> and the compiled program containing the fragment uses the new library then its behaviour when given an IEnumerable<Giraffe> will change. The object will be assignable to IEnumerable<Animal>, and therefore the “is” will report “true”.

There is also the issue of existing source code changing semantics or turning compiling programs into erroneous programs. For example, overload resolution may now fail where it used to succeed. If we have:

interface IBar<T>{} // From some other assembly
...
void M(IBar<Tiger> x){}
void M(IBar<Giraffe> x){}
void M(object x) {}
...
IBar<Animal> y = whatever;
M(y);

Then overload resolution picks the object version today because it is the sole applicable choice. If we change the definition of IBar to

interface IBar<-T>{}

and recompile then we get an ambiguity error because now all three are applicable and there is no unique best choice.

We always want to avoid breaking changes if possible, but sometimes new features are sufficiently compelling and the breaks are sufficiently rare that it’s worth it. My intuition is that by turning on interface and delegate variance we would enable many more interesting scenarios than we would break.

What are your thoughts? Keep in mind that we expect that the vast majority of developers will never have to define the variance of a given type argument, but they may take advantage of variance frequently. Is it worth our while to invest time and energy in this sort of thing for a hypothetical future version of the language?

  • Re: Your plan is much more ambitious. There will be huge hurdles even if it does make it past the -100 point mark

    Actually, its just the opposite.  Variance on interfaces and delegates will be easy to implement because the CLR already supports it natively and has since generics were introduced. C# is just not taking advantage of it yet.  

    The CLR does NOT support variance on virtual overrides natively, so implementing that would require a lot more work on both the design and implementation side.

  • The lack of variance support is one of my top two complaints of C#.

    It is so lacking that I regularly need to drop down to the IL level to do work.

    Whidbey's introduction of generics was great; however, it only went part way. Variance is absolutely needed, even if it may break some code in very rare cases.

    PS My number two complaint is the inability to declare generic overloaded operators because there is no way to know what is addable, subtractable etc.

  • Welcome to the thirty-fifth edition of Community Convergence. This week we have an interview with C#

  • How about create a set of new assemblies version 3.0 (for CLR 3.0) that run side by side with the CRL 2.0  

    Or another option (this might seem crazy) but couldn't CLR 3.0 just ignore variance info for ver. 2.0 assemblies and enable it for 3.0 assemblies and above ?

    Also variance has got to come sometime and I say better early than late. Variance it's a feature that brings closer my other long desired feature for .Net: parametric polymorphisms using generics ( class Foo<T> : T {...} // :D ).

  • Separation of concerns. You have two changes you'd like to make:

    1) A language change, to allow variance to be defined for generics. Correct me if I'm wrong, but no current C# code is broken by this (if you're using an assembly from another language which takes advantage of variance, I'm assuming you already get the variance behaviour in C#).

    2) A library change. In this post, you talk about a breaking change to IEnumerable, and a breaking change to a hypothetical IBar. Obviously this is badness. Breaking changes are badness almost by definition.

    Now, as far as I (a non-C#-using guy) am concerned, (1) is a good thing. As far as I can see, the only downsides are a) the -100 points, and b) the added cost to C# developers of understanding variance (and usually they won't have to).

    But (2) is nowhere near as clear-cut. Contrary to what others have said, assuming my assumption above is valid, you can't just say "this problem is trivially solvable by ruling that variance is off if it's a CLR2 assembly" since this may break C# code which already uses generics with variance from other languages (including hand-coded MSIL, I guess). (2) has -100 points of its own, and I don't see it getting the necessary +100 to justify it.

  • Richard,

    IEnumerable is the foundation of LINQ. If it does not get covariance, we might as well have no variance for interfaces at all. Now, that's an exaggeration, but just a small one. Introducing an new interface would conflict with C# features that support IEnumerable -> confusion.

    Having to cast between IEnumerable<T>'s of related types is a major PITA. We will start to feel the pain when we really use LINQ. Making it covariant would get my +100 and then some!

    BTW, C# currently ignores the CLR's variance bits says Eric.

    Stefan

  • 1) Make the change. The effects of the breaking change will be one time for code that's migration. Without the change we'll be forced to continue to fight the type system in some of the examples illustrated in this series and more.

    2) Go with the most intention revealing syntax possible. +/- is nice and it work well for me, but I suspect as you do that it would cause confusion among many. The more verbose options seem more universally clear.

  • I notice that you seem to mix 2 things when discussing the breaking change:

    1. Adding co- & contravariance itself does <b>not</b> break anything

    2. Changing existing classes / interface to include variance <b>is</b> a breaking change.

    So what to do? Add variance and restrain from changing existing interfaces. Just as the introduction of generics delivered a generic version of IENumerable, we now would get additionally a variance version of IENumerable...

  • mbuzina,

    how would you call this new interface? IEnumerable2<T>? (recalling the horror of COM...)

    IEnumerable and IEnumerable<T> are easily separable (in fact, they have different names under the hood). IEnumerable<T> and IEnumerable<+T> are not. Creating two interfaces would also mean that you have to understand the difference going forward (i.e., everyone must understand covariance). It would make the language a great deal uglier.

    All this just to prevent a few breaking lines of code that can easily be fixed by bringing the tests in an order that would have been more logical in the first place? I don't know...

    I say let's have those bugs and fix them. Hopefully, they are very rare anyway!

  • Now here's the killer:

    if (x is IEnumerable<Animal>)

     DoSomething();

    else if (x is IEnumerable<Giraffe>)

     DoSomethingElse();

    This code is broken already!

    It is NOT, like Eric said, entirely sensible to write something like this in C# now. This will work as expected on generic collection classes, but not on arrays. So, if you write code like that, and test it only using collection classes, it will break the minute you feed it arrays. Because an Giraffe[] IS an Animal[], and therefore an IEnumerable<Animal> too, the Giraffe branch will never see execution for arrays.

    This way of testing IEnumerable is broken already. Any code depending on it is broken, unless it explicitly tests for arrays first! It might not result in visible errors in a certain application if it only gets collection classes. But this is a time bomb waiting to explode.

    Let's get rid of this fast. In fact, I'd vote for a switch from IEnumerable<T> to IEnumerable<+T> with C# 3.0 (Although unfortunately, I assume it's probably too late for this. Even if the change would just take minutes to implement, there's always the thing about the light bulb...)

  • Yeah, way too late considering they've committed to releasing C# 3.0 this month.

  • Hey, so what, _I_ would do it ;-)

    Let's go find that build server, ildasm, hack, ilasm, and sneak out of here. I'm sure Stuart Scott would have let me in :-D

  • * I actually do not know what our exact release schedule is. Our lead time for releases is so long that I stopped thinking about C# 3.0 weeks ago and have been thinking only about hypothetical future service packs and hypothetical future releases. C# 3.0 is done as far as I am concerned; I understand that, you know, _customers_ may not see it that way yet. :-)

    * We strongly considered flipping the variance bits on for IEnumerable/IEnumerator/IComparer/etc for the upcoming base class library release but ultimately decided that it would be better to update the libraries and compiler in lockstep to implement this feature. Were we to do so. Hypothetically.

    *  "if you're using an assembly from another language which takes advantage of variance, I'm assuming you already get the variance behaviour in C#"

    That assumption is incorrect. In that case you will get variance behaviour everywhere that the C# compiler defers to the CLR, but never in situations where the C# compiler itself needs to make a decision about convertibility.

    Let me characterize the difference. If you have a class Foo which implements a covariant IFoo:

    object x = new Foo<Giraffe>();

    bool b = x is IFoo<Animal>;

    here the C# compiler simply generates a runtime check, and the CLR says sure, that thing is an IFoo<Animal>.

    But if you have

    void M(object f) {}

    void M(IFoo<Animal> f) {}

    ...

    M(new Foo<Giraffe>());

    then the C# compiler needs to decide at compile time which overload to call. Since the C# compiler does not know about variance, it will pick the object version today, even if Foo implements covariant IFoo.

  • Eric, I never seriously considered it possible to do this with .NET 3.5. But what do you think about the variance problem we already have when we mix arrays with IEnumerable<T>? (As opposed to generic collections.) Would a breaking change really make this any worse? There are probably less than 5 people in the world who produced such code _and_ are aware that this behavess completely differently for List<T> and T[]...

  • I think there's a bigger reason not to include variance than the fact that it might, in very rare cases, break some existing code:

    It makes the language more complicated.

    Even if I only *consume* variant code (e.g. things which publish IEnumerable<something which extends Foo> I still need to understand that. Now, at the moment we do already get people confused about why they can't return a List<string> in a method returning IEnumerable<object>, but that's going to be the case even when variance becomes an option.

    I agree it would be useful - but frankly I think even C# 3 is a complicated language to learn from scratch. I'd really like to see a post from Eric dedicated to this topic: "when do we stop?". I'd really welcome a gap of at least 3 years before C# 4 is released, just so everyone can get their heads round C# 3. It's worth understanding that many, many developers don't understand C# 2 yet, let alone C# 3.

    There are features I'd like to see in C# 4, certainly. Yeah, I'd love to have them *right now* in many ways. But we need to understand that developers already have a vast array of libraries to learn about (WPF, WCF, Silverlight, AJAX, LINQ etc). The balance between innovating and just overwhelming developers is a fine one, and my gut feeling is that MS have been on the "overwhelming" side for the last year or so.

    What's the best way of being part of the discussion of such concerns, beyond comments in blogs?

Page 2 of 3 (34 items) 123