Iterator blocks Part Four: Why no yield in catch?

re: Iterator blocks Part Four: Why no yield in catch?

James — Wed, 29 Jul 2009 07:03:19 GMT

I just want to say I really appreciate the comment discussions. I feel enlightened.

re: Iterator blocks Part Four: Why no yield in catch?

holatom — Sun, 26 Jul 2009 18:41:19 GMT

Some comments mentiond awesome "returning anonymous types" proposed feature. Sadly it would be now even much harder to ever actualy see it in some version of C# because of including tuples in .net 4.0. Many people will be now satisfied and simply instead of (proposed):

class Foo

{

public { int Min; int Max; } GetSomeResults()

{

return new { Min = 1, Max = 10 };

}

static void Test()

{

var results = GetSomeResults();

Console.WriteLine("Min: (O), Max: {1}", results.Min, results.Max);

}

will write this horrible code:

class Foo

{

public Tuple<int, int> GetSomeResults()

{

return new Tuple<int, int>(1, 10); //Min, Max

}

static void Test()

{

var results = GetSomeResults(); //Note if you incidentally don't know: First is min, Second is max

Console.WriteLine("Min: (O), Max: {1}", results.First, results.Second);

}

re: Iterator blocks Part Four: Why no yield in catch?

Rüdiger Klaehn — Sat, 25 Jul 2009 14:05:40 GMT

"I would love to have such a thing, but as you correctly note, I'm the wrong person to ask."

That's a shame. You had no problems getting lots of new classes included into the framework class library for LINQ. So it should be possible to get an immutable List, Map and Set included as well if there is a desire for it.

By the way: you will find that writing and using alternative collection libraries is kind of annoying since you either have to include System.Collections.Generic to get IEnumerable<T>, or always use the full name. It is currently not possible to import just a single generic class or interface.

IEnumerable<T> is so central to the language that it should have been in the System namespace.

"LINQ is not domain-specific. Sorting, searching, projecting, grouping and joining are abstractions which are generally useful in almost any conceivable domain, not narrowly tailored to a specific domain."

There is a scala library for building parsers. Parsing obviously also applies to many different domains, yet Martin Odersky describes it as an "internal domain specific language" in programming in scala, page 619.

According to wikipedia, "a domain-specific language (DSL) is a programming language or specification language dedicated to a particular problem domain, a particular problem representation technique, and/or a particular solution technique."

LINQ is a internal mini language for querying. Of course querying is a generic concept, so it applies to many problem domains. Yet according to the above definition it is a DSL since it is dedicated to a particular problem representation and solution technique.

I guess I just don't like the approach where microsoft decides for me which problem domains deserve their own language extensions and which don't. But as you mentioned this is "by design" in C#. That is why I find the inlining limitations of the CLR so frustrating: they prevent me from writing high performance numerical code in any CLR based language.

"I encourage you to pass on that feedback to the people who produces those APIs."

I already did so. But since there is such a big mismatch between what I find important and what the people making decisions at microsoft find important, I am seriously thinking about abandoning the .NET platform in the long term. And in the short term there is no way we are going to use Direct2D anyway, since we are working in a very conservative industry (space operations) and Direct2D is not available for windows XP.

"This feature is called return-type covariance; it is one of the most frequent feature requests we get. We have not implemented it because (1) the CLR does not support it, and (2) Anders believes that it is a bad language feature. I don't think I could succinctly and accurately summarize Anders' reasons, so I shan't attempt to do so."

If Anders does not like it, it won't happen. Which is a shame, since it is one way to make writing inheritance hierarchies of immutable objects bearable in C#.

The other way (which we are currently using) is to let abstract base classes have a type parameter TSelf so that "clone with changes" methods can declare the proper return type. An abstract base class with a property A would have a method TSelf WithA(TA newValue). But the problem then is that there is no longer an "is a" relationship between abstract base classes and concrete classes.

Of course you could solve this by introducing self types as a language construct, but this is such an abstract feature that there won't be any MVPs and focus groups asking for it...

re: Iterator blocks Part Four: Why no yield in catch?

TheCPUWizard — Sat, 25 Jul 2009 01:28:42 GMT

Eric,

Your in-line feedback on the comments is one of the things I *REALLY* appreciate, as it provides tremendous value.

You reply about returning anonymous types (without a common declared bast type) makes sense. .....just thinking "out loud"...While I hate adding keywords (even contextually), I do wonder if it makes sense to differentiate "a named type which I choose not to explicirly name" from and anonymous type. My thinking is that if this were done, it *might* be possible to constrain the performance impact, thus allowing it to be a deliberate decision.

I would like to discuss the issue of using aritmetic symbols on generics with you sometime. I briefly talks with Anders about it at the last MVP summit, and I think that I do have a means (basically syntactic sugar) that would be viable without changing anything under the covers...anyone interested, feel free to ping me: david "dot" corbin "at" dynconcepts.com.

re: Iterator blocks Part Four: Why no yield in catch?

Rudiger Klaehn — Fri, 24 Jul 2009 18:57:19 GMT

"As you know if you've read this blog for any time, I am a big fan of immutable functional-style programming. We discuss all the time in the language design meeting how to make it easier to program in an immutable style. We also try our best to write our libraries in an immutable style; you'll notice that expression trees are immutable, for example."

Yes, I noticed that expression trees are immutable. I also saw your posts about immutable collections. I realize that the collections library is probably another department, but what are the chances of getting an immutable collections library into the base class libraries? I have written my own immutable collections library for one project, but I would prefer to use a standard library.

Nobody except pedants and masochists like me will ever write C# in a functional style if there is no standard immutable collections library.

I would love to have such a thing, but as you correctly note, I'm the wrong person to ask. It's bad enough that I comment on the inchoate future of my own product; I'm certainly not going to speculate about what other teams might or might not produce in the future. -- Eric

"Variance is limited to interfaces and delegates because the CLR supports variance on interfaces and delegates. The CLR does not support variance on classes because its almost never typesafe. A class of T almost always has a field of type T, which means T can be used in both input and output positions, which means that T must be invariant. Since 99.9% of the time a class would have to be always either invariant or fragile, it would be foolish to add the feature."

I think I did not express precisely what I meant: If I have a class A with a method public virtual object Bla(), I should be able to override this method in a derived class with a more specific return type because it does not violate the contract. Likewise, if I have a method void Blub(string text), I should be able to override the method in a derived class with a more generic parameter like object since again it does not violate the contract.

Especially refining return values in derived classes is extremely helpful when writing immutable class hierarchies with "clone with changes" methods. I always write stuff in that style.

y=x.Default

.WithName("Bla")

.WithAge(12);

This is generally a good way to design immutable record-like objects: instead of having a million different constructor overloads, provide a default value, and "clone with modifications" methods for each (immutable ) field..

This feature is called return-type covariance; it is one of the most frequent feature requests we get. We have not implemented it because (1) the CLR does not support it, and (2) Anders believes that it is a bad language feature. I don't think I could succinctly and accurately summarize Anders' reasons, so I shan't attempt to do so. -- Eric

"I'm not following your point here."

The point is that we desperately need a fast 2D drawing API for one of our applications. WPF is much too slow because it is a retained mode framework. GDI+ is too slow since it is not hardware accelerated. And managed Direct2D is a pain since all it can draw is triangles and we don't have the time for writing our own 2d api on top of a fast triangle renderer.

We are a .NET shop, and since .NET has been sold by MS as "the future of windows programming" I find it reasonable to assume that there will be at least a managed wrapper for important new APIs.

I encourage you to pass on that feedback to the people who produces those APIs. -- Eric

"I appreciate that you want to use our language and framework. It's unfortunate that apparently it doesn't meet your personal standards for design, features, performance or adoption. Sadly, we are not going to delight every single one of our customers with our offerings. Thanks for your comments."

You are welcome. Please don't take this personally. I am just very disappointed about how C# has changed since 2.0. If the roadmap for C# had been clear in 2004 I would have simply chosen another language.

No worries. I only take it personally when people make it personal. I frequently read about what my and my coworkers allegedly believe or what factors we allegedly did or did not consider in coming up with a particular design. These speculations are usually presented as facts, which, frankly, I resent. I particularly resent it when the alleged beliefs are in fact the opposite of what I actually believe. Part of the point of this blog is to get accurate information out there about what at least one member of the language design team actually thinks. -- Eric

re: Iterator blocks Part Four: Why no yield in catch?

Rudiger Klaehn — Fri, 24 Jul 2009 17:00:22 GMT

Re: TheCPUWizard

Well, then I guess I don't like LINQ. I think a language should either provide the tools for users of the language to define their own DSL (domain specific languages)/syntax candy in libraries like scala does, or not do DSLs at all. I don't like being treated like a second class citizen by a language :-)

LINQ is not domain-specific. Sorting, searching, projecting, grouping and joining are abstractions which are generally useful in almost any conceivable domain, not narrowly tailored to a specific domain. -- Eric

re: Iterator blocks Part Four: Why no yield in catch?

TheCPUWizard — Fri, 24 Jul 2009 15:06:24 GMT

@Rüdiger Klaehn - You raise some interesting points. But I have to take exception with:

"But I really dislike the special SQL like syntax extension"

This is the "N"atural portion of the functionallity. The ability to NOT use methid/propert like system. With out this being "I"ntegrated into the "L"anguage, it would not be LINQ, it would be just another library.

This means that your really *dont* like LINQ, but would rather mainain a more conventional systax, which you are certainly free to do by directly invoking the class propert/methods that back LINQ....

re: Iterator blocks Part Four: Why no yield in catch?

Rabeeh Abla — Fri, 24 Jul 2009 14:54:00 GMT

"The computer will always understand the code, but not humans" I prefer maintainable simple code, that can be easily documented and read, rather than syntax that dissolves and disappears with languages, because the percentage of its use is very low.

Why enter into such complexity, while what are of higher priority are the algorithm and its efficiency and code maintenance?

Regards

Rabeeh

re: Iterator blocks Part Four: Why no yield in catch?

Rudiger Klaehn — Fri, 24 Jul 2009 12:36:30 GMT

"if you disagree with these choices, I encourage you to propose alternate designs, or, for that matter, invent a language that does it better. I'm pro more languages."

Here is a proposal for an alternative design for C#:

http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=412848

But apparently structural type constraints are another thing that only microsoft may use (foreach, property initializers, collection initializers), so it got closed as "wontfix".

As you can see on the page you reference, Mads -- the C# language PM -- provided an extensive explanation of why we do not see the proposed feature as being generally useful. Apparently you disagree. We get proposed features every single day, most of which we've seen before, and most of which we reject. Believe me, you would like our product less if we took everyone's suggestions. -- Eric

But that is still much better than acknowledging that it is a major issue, leaving it open, yet doing nothing whatsoever about it like with the arithmetic operations on generic types. If MS had closed the IArithmetic feedback item in 2004 as "wontfix", I would just have moved on to a different language (or back to C++).

I have also been thinking about writing my own language. But I have serious doubts that I could come up with something better than scala.

"I disagree. We work hard to ensure that language features are general-purpose. For example, take LINQ. Sorting, searching, filtering, joining, grouping and projecting data are operations used in all information domains, whether you're writing a raytracer or building a three-tier business web site."

I have nothing against LINQ. In fact I use it all the time. But I really dislike the special SQL like syntax extension. It feels completely foreign in a C# program. It makes the language much more complex.

The question is whether the costs of additional complexity are paid for by the increase in representational power. LINQ makes my C# programs shorter and clearer, so for me, it pays off. Apparently you disagree. -- Eric

LINQ would have been just as useful as a library with the only language change being extension methods. Extension methods are an example of a language feature that is generally useful. But again, why are there no static extension methods or extension properties?

We designed, implemented and cut extension properties for C# 4. We did not do a good enough job of reviewing the feature in the design phase with the constituents who were most likely to use it to solve real-world problems. When we did do so, we discovered that we had designed a feature which did not do a good job of solving their primary problem. Rather than ship a feature with a broken design, we cut it, thereby allowing us to design it properly in the future if we choose to. -- Eric

By the way: in scala you could write something like LINQ as a library without changing anything about the language. _That_ is a genuinely general purpose language feature.

It is a bit of a two class situation. If microsoft thinks that X is a nice language feature, they extend the grammar of the language. Examples I could live without include the special syntax for Nullable<T>, the SQL like language extensions for LINQ, and the new dynamic keyword in C# 4.0. Every single one of those language extensions could be written as a library in scala. A large part of scala that looks like special syntax is in fact just a library.

C# was not a language designed for end-user extensions to the syntax. If that's your requirement for a language then C# does not meet your needs. Consider scala, nemerle, maybe haskell. We are considering adding metaprogramming extensions in the future, but it will take a long time to retrofit metaprogramming into an existing language implementation.

A design in which only the vendor has the ability to add major new syntax but third parties can ship their own class libraries has both pros and cons. You've identified the cons. The pros are that the language remains standardizable and mutually comprehensible across organizational boundaries; you can get some C# code from a colleague on a different team and have a reasonable chance of understanding its control flow. Languages which allow users to introduce their own control flow make it difficult to understand what is going on. This is particularly bad in C++. Many teams at Microsoft have written their own macro libraries that add new control flow to C++, and that code is practically incomprehensible if you don't know whats a macro and what exactly the macro does. By putting LINQ into the language, we standardize it so that everyone only has to learn it once, rather than there being a dozen competing mutually incomprehensible syntax extensions. -- Eric

"I disagree that I lack attention to detail; in fact, I am noted around here as a somewhat detail-obsessed guy. I also disagree that this has anything to do with the language. I'm sorry that the CLR is imperfect, believe me. I have to work with it every day too. But none of your concerns are anything that the language team can do anything about; try asking the CLR team."

C# is the main language using the CLR. So please don't tell me that the C# team does not have a major influence on the CLR team. But there are also various design decisions in C# that have nothing to do with the CLR. For example the new constructor and collection initializer syntax: it strongly encourages making pure data objects mutable. That is a really bad idea for many reasons, last but not least concurrency. And why exactly is covariance limited to interfaces?

I agree that many new features of C# 3 encourage programming in a mutable style. We were very aware of this when we designed them. For better or for worse, OO programmers like mutating data. As you know if you've read this blog for any time, I am a big fan of immutable functional-style programming. We discuss all the time in the language design meeting how to make it easier to program in an immutable style. We also try our best to write our libraries in an immutable style; you'll notice that expression trees are immutable, for example.

Variance is limited to interfaces and delegates because the CLR supports variance on interfaces and delegates. The CLR does not support variance on classes because its almost never typesafe. A class of T almost always has a field of type T, which means T can be used in both input and output positions, which means that T must be invariant. Since 99.9% of the time a class would have to be always either invariant or fragile, it would be foolish to add the feature. -- Eric

"In the meanwhile, you should not think of "connect" as a democracy; we certainly take connect data into consideration when prioritizing new language and runtime features, but its not like a high number of connect votes constitutes a binding referrendum. We have many priorities for the language and runtime, priorities which always come from users, but do not necessarily come from "connect". We talk to MVPs and other industry insiders, we conduct focus groups, we have many other ways of getting real-world feedback on what people are doing."

I exchanged email with various people at microsoft, including kevin frei and rico mariani, about the inlining limitations of the CLR. They all agreed that this is _the_ most important performance problem in the CLR. So it is not just connect feedback that points to that issue. But nothing has been done about it.

Let's suppose you're right, and the (former) performance architect of the CLR thinks that this is the most important issue. Then I speculate that either your assertion that nothing has been done about it is incorrect, or nothing has been done about it because the performance architect of the CLR has good reasons to believe that doing something about it would cost more than the benefit accrued. But rather than you grousing and me speculating about it, why don't you ask Rico what the status of the issue is? He's not the perf architect anymore, but I'm sure he can point you at the right people. -- Eric

About the generics limitations: every time you try to do something more than collections with generics you hit this limitation. I am certainly not the only one who noticed that you can not use operators or static methods from generics. But apparently you do not want generics to be used for more than collections/containers. Otherwise you would have fixed this issue. Another example of a language feature that is useful just for one particular application.

I disagree. I see many usages of generics that have nothing to do with collections or containers. That said, of course containers are the primary motivating scenario for generics. -- Eric

Anyway, it seems that MS just don't see C# as a core language for writing large applications, but rather as a language for writing glue code, database frontends and for scripting applications written in C++.

We design C# as a core language for large application programming. We might not have succeeded in achieving that goal by the reckoning of some people. Or, we might have succeeded, but failed to provide a compelling alternative to other offerings. That's life in a competitive world; we're doing our best with the resources at our disposal, and continually trying to do better.

Also, you might consider not trying to figure out "the opinion of Microsoft". Microsoft is composed of 90 thousand people, all of whom have their own opinions. Sussing out the "opinion of the corporation" from the behaviour of individual teams is seldom productive. -- Eric

New APIs like Direct2D and DirectWrite are released as pure unmanaged code. There is not even a managed wrapper. There is no managed wrapper for DirectX except for the XNA game development toolkit. And as far as I could see, there are almost no managed applications in windows 7.

I'm not following your point here.

I appreciate that you want to use our language and framework. It's unfortunate that apparently it doesn't meet your personal standards for design, features, performance or adoption. Sadly, we are not going to delight every single one of our customers with our offerings. Thanks for your comments.

-- Eric

re: Iterator blocks Part Four: Why no yield in catch?

TheCPUWizard — Fri, 24 Jul 2009 01:58:57 GMT

@Michael, I see most of the points in your post, and agree with a fair number of them. Especially having a good set of interfaces for dealing with Office applications. Because I develop a fair amount of applications in this space, I have invested the time (signnificant time) to isolate "application" code from the details of the COM interfaces and it has definately paid off.

However, one part I fail to dsee is "They can't return the var type. Are you saying nobody wants to return a var?"

"var" is a syntactic shortcut for a strong type where the compiler can determin the actual type from the context. In all cases, you can replace "var" with a single specific type [and there are plenty of tools that do just this].

*IF* one could pass or return something declared as "var" then it would REQUIRE implicit "leaking" of implementation back out through the signature. Not only would this be difficult to achieve, I dont see where it would provide any value.

The value is that you could use "extract method" on a query expression to make a helper method that returns a sequence of anonymous type. You cannot do that today because there is no way to represent the anonymous type in the return value of a method. You're forced to refactor the query to produce a nominal type first, and then extract the helper method. The down side is that doing so can impose the burden of whole-program analysis on the compiler. The language has been cleverly designed so that you can analyze all the method bodies in parallel if you choose to; var on returns would prevent that. -- Eric