Iterator Blocks, Part Three: Why no yield in finally?

re: Iterator Blocks, Part Three: Why no yield in finally?

Matthew Jones — Thu, 16 Jul 2009 18:43:57 GMT

Whoa, what an amazing post! This kind of stuff really opens your eyes to the kind of problems you guys encounter when designing features. The scenarios you describe in your post sound like a real nightmare.

I'd love to be a fly on the wall when you guys discuss these things in your design meetings. A podcast would be cool ;-)

There's a point in the post where you mention having to convince the CLR guys to allow spaghetti code. Do you run into situations where you have new requirements of the CLR often? Or is this one of the many signs that a feature should be cut?

We try to not drive features into the CLR, though it does happen occasionally. For example, the "Silverlight" CLR and the "desktop" CLR have slightly different security models for generated code, which makes dynamic code spitting via compilation of expression trees tricky in Silverlight; that language feature drove a change into the Silverlight CLR that was not anticipated in the design. Or, as we expose existing CLR features like generic variance, suddenly features that have been not heavily tested historically are suddenly in more widespread usage, which can trigger CLR changes to fix design or implementation warts. And sometimes a feature clearly requires CLR changes, like the "no PIA" feature.

And its a two-way street. Sometimes the CLR guys come up with a great new CLR feature and then ask us if we'd like to provide a "surface" for that feature in our language. -- Eric

Thanks for the excellent series of posts Eric, I look forward to the rest.

re: Iterator Blocks, Part Three: Why no yield in finally?

Abhijeet Patel — Thu, 16 Jul 2009 23:09:32 GMT

Wow, I never really paid attention to the "why not" for yield returns in iterator blocks, typically all you find in books are a straight copy/paste from the language spec/MSDN about where a construct cannot be used legally.

# An iterator member's signature cannot contain any out or ref parameters.

# An iterator cannot contain a return statement .

# An iterator may not contain any unsafe code.

# A finally block may not contain a yield return or a yield break statement.

# A yield return statement may not appear in a try block that has a catch block.

# A yield return statement may not appear in a catch block (yield break is OK)

Thanks for providing a very meaningful explanation about the "why" and "why not".

Just a side note, most of the material in your blog is something one can never find in books,it's always insightful and stimulating reading your stuff. I always end up walking away with an "a-aha". :)

re: Iterator Blocks, Part Three: Why no yield in finally?

omario — Fri, 17 Jul 2009 01:18:38 GMT

Only "A yield return statement may not appear in a try block that has a catch block" left unexplaned.

That's part five. Be patient. -- Eric

re: Iterator Blocks, Part Three: Why no yield in finally?

configurator — Fri, 17 Jul 2009 05:03:00 GMT

I'd go with "A finally block may not contain a yield return or a yield break statement because it makes no sense whatsoever", but I suppose your explanation is a bit more verbose :)

Seriously though, can you think of an example that makes sense where someone would actually want to do that?

Also, it seems obvious to me that it wouldn't be possible, since a yield return or break is basically a return, and that is not possible in a finally block.

Well, hold on a minute before you dismiss it. It only fails to make sense if you conceive of a yield return/break as a return. The generalized feature's semantics do not obligate us to conceive of yields in that manner.

The language spec allows us to conceive of yields in that manner, and allows an implementation in which yields actually are returns of the MoveNext method, but as I said in the first couple posts, we could have designed iterators completely differently.

If iterators are coroutines implemented by fibers then a yield in a finally makes perfect sense; a yield anywhere makes perfect sense. (Well, in a lock is probably a bad idea...) It just means "stop running this fiber and switch back to the main fiber". The fiber has its own stack, so there's no ambiguity about where the exception is going to go; up the stack of the fiber.

Similarly, in a couple of episodes we'll see that if we'd conceived of iterators as "pushing data" instead data of being pulled, then a "yield" isn't logically a return, it is logically a function call. And there's no problem at all with function calls in a finally.

Or, if we choose to implement iterators as continuations, then we'd probably be in a system where exception handling is also structured out of continuations; again, it makes sense. It just becomes a new choice about what the structure of the next continuation looks like.

There are lots of ways to conceive of iterators, and the way we chose is actually pretty goofy. By choosing to have an easier time implementing the feature, we ended up in a world where you have to have crazy restrictions on where yields can appear.

-- Eric

re: Iterator Blocks, Part Three: Why no yield in finally?

fred — Fri, 17 Jul 2009 14:49:57 GMT

// BUG?

// While examining yield in a try block that has a catch block,

// I found that generated iterator code leaks implementation detail, namely:

// 1) If iterator has finally block, Dispose() resets current pointer.

// 2) If iterator has not finally block, Dispose() does not reset current pointer.

// I.e., call site can reflect "Does iterator block uses finally block?".

// Method TestIterator() intentionally does not use foreach

// and does not use break or continue, but it is still perfectly legal.

// My expectation was, that Dispose() will reset current pointer in any case.

// Tested with 3.5.30729.1 and 4.0.20506.1 C# compilers.

// csc -target:exe -out:TestDisposeIterator.exe -codepage:1257 Program.cs

using System;

using System.Collections.Generic;

namespace TestDisposeIterator

{

public class Program

{

public static void Main(string[] args)

{

int sentry = 2;

Program.TestIterator(Program.IteratorWithFinally, sentry);

Program.TestIterator(Program.IteratorWithoutFinally, sentry);

}

private static void TestIterator<T>(IEnumerable<T> iterator, T sentry)

where T : struct, IEquatable<T>

{

Console.WriteLine("TestIterator: {0}", iterator);

using (IEnumerator<T> mover = iterator.GetEnumerator())

{

while (mover.MoveNext())

{

Console.WriteLine("Current: {0}", mover.Current);

if (mover.Current.Equals(sentry))

{

mover.Dispose(); // Instead of break.

}

private static IEnumerable<int> IteratorWithFinally

{

get

{

try

{

yield return 1;

yield return 2;

yield return 3;

}

finally

{

}

private static IEnumerable<int> IteratorWithoutFinally

{

get

{

yield return 1;

yield return 2;

yield return 3;

}

// TEST RESULTS:

// TestIterator: TestDisposeIterator.Program+<get_IteratorWithFinally>d__0

// Current: 1

// Current: 2

// TestIterator: TestDisposeIterator.Program+<get_IteratorWithoutFinally>d__4

// Current: 1

// Current: 2

// Current: 3

re: Iterator Blocks, Part Three: Why no yield in finally?

TheCPUWizard — Fri, 17 Jul 2009 17:54:48 GMT

I think we should all yield to this explanation, and finally return to other aspect of the patters involving iterators.... <duck & run>

re: Iterator Blocks, Part Three: Why no yield in finally?

configurator — Mon, 20 Jul 2009 05:03:18 GMT

"Well, hold on a minute before you dismiss it." - I didn't dismiss it! You did, after much forethought. I just gave my first-minute impression of this restriction.

I never really understood continuations.

I mean, I know exactly how they work. I've worked with and understood CPS style programing for a while. But I never mastered it, I haven't (yet, hopefully) had that "aha" moment. Maybe for that reason, even if yield were done using continuations, I'd still think of them as a return from a function. So it would still seem obvious to me that it shouldn't work - even if it were an implementation detail of an implementation that doesn't exist!

That said, I'm still having a hard time coming up with a real-world enough example where someone would want to yield in a finally, and the code would still make sense.

re: Iterator Blocks, Part Three: Why no yield in finally?

Scott Kuhn — Mon, 20 Jul 2009 23:51:45 GMT

It's information like this that leads me back to your blog every week. It's one thing to understand a language and a whole other thing to understand why the language is designed and works the way it does. Reading and understanding information like this series is providing gives us users of C# a depth of knowledge that improves our coding skills. Thank you!

re: Iterator Blocks, Part Three: Why no yield in finally?

Mark Sowul — Wed, 05 Aug 2009 20:38:46 GMT

"Leaving a finally block via "return" is illegal because it is weird. Suppose the try block returns, causing the finally to execute. What happens when the finally block returns? The original return value would be lost and some other return value would be substituted for it. This seems bizarre and bug-prone, so its not legal; you cannot exit a finally via return."

This curious perversion is legal in Java, of course.

try {

return "5";

}

finally {

return "WTF";

}

cf.

while(true)

{

try {

break;

}

finally {

continue;

}

System.out.println("Java is awesome"); //needless to say, never executes

re: Iterator Blocks, Part Three: Why no yield in finally?

Ben Voigt [C++ MVP] — Mon, 26 Oct 2009 15:55:16 GMT

Eric, since you've (in all the various Iterator Blocks posts, this is just the most relevant) explained how a lot of features have messy interactions with yield return and these interactions have been purposefully forbidden by the C# spec, why is yield return allowed inside a lock block? Could you do a post on lock-yield interactions?