Fabulous Adventures In Coding : Iterators

Iterator Blocks Part Seven: Why no anonymous iterators?

Eric Lippert — Mon, 24 Aug 2009 16:23:00 GMT

This annotation to a comment in part five I think deserves to be promoted to a post of its own.

Why do we disallow anonymous iterators? I would love to have anonymous iterator blocks. I want to say something like:

IEnumerable<int> twoints = ()=>{ yield return x; yield return x*10; };
foreach(int i in twoints) ...

It would be totally awesome to be able to build yourself a little sequence generator in-place that closed over local variables. The reason why not is straightforward: the benefits don't outweigh the costs. The awesomeness of making sequence generators in-place is actually pretty small in the grand scheme of things and nominal methods do the job well enough in most scenarios. So the benefits are not that compelling.

The costs are large. Iterator rewriting is the most complicated transformation in the compiler, and anonymous method rewriting is the second most complicated. Anonymous methods can be inside other anonymous methods, and anonymous methods can be inside iterator blocks. Therefore, what we do is first we rewrite all anonymous methods so that they become methods of a closure class. This is the second-last thing the compiler does before emitting IL for a method. Once that step is done, the iterator rewriter can assume that there are no anonymous methods in the iterator block; they've all be rewritten already. Therefore the iterator rewriter can just concentrate on rewriting the iterator, without worrying that there might be an unrealized anonymous method in there.

Also, iterator blocks never "nest", unlike anonymous methods. The iterator rewriter can assume that all iterator blocks are "top level".

If anonymous methods are allowed to contain iterator blocks, then both those assumptions go out the window. You can have an iterator block that contains an anonymous method that contains an anonymous method that contains an iterator block that contains an anonymous method, and... yuck. Now we have to write a rewriting pass that can handle nested iterator blocks and nested anonymous methods at the same time, merging our two most complicated algorithms into one far more complicated algorithm. It would be really hard to design, implement, and test. We are smart enough to do so, I'm sure. We've got a smart team here. But we don't want to take on that large burden for a "nice to have but not necessary" feature.

Iterator Blocks, Part Six: Why no unsafe code?

Eric Lippert — Mon, 27 Jul 2009 17:07:00 GMT

There are three good reasons to disallow unsafe blocks inside an iterator block.

First, it is an incredibly unlikely scenario. The purpose of iterator blocks is to make it easy to write an iterator that walks over some abstract data type. This is highly likely to be fully managed code; it's simply not a by-design scenario.

Second, the scenario is a violent mixing of "levels." You think of the level of abstraction of a programming language feature as how "far from the machine" the feature is. Unsafe pointer manipulation is as close to the machine as it gets in C#. You are running without a safety net, poking and peeking at the raw bytes in the virtual memory space of the process. All the protections of verifiable type safety are completely gone and you have total responsibility over every bit in the user-mode address space.

Iterator blocks, by contrast, are the highest level of abstraction in C# 2.0. There is no immediately obvious mapping from the source code to the generated code; it's a magic box that does the right thing because the compiler is doing some pretty massive transformations on the code, generating classes and implementing interfaces for you.

Mixing those two levels of abstraction in one method seems like a really bad idea.

Third, we would have to do some extra work to make "fixed" blocks work correctly, work that would be directly opposed to our guidelines for the correct and efficient use of the "fixed" block.

I suppose I should briefly explain what the "fixed" block does. When you call unmanaged code that needs to, say, write into the unmanaged address that is the beginning of an array of 16 bit chars, somehow you need to tell the garbage collector "hey, you, GC, running on that other thread over there -- if you get the urge to reorganize memory to be more efficient, I need you to please not move this array for a moment, because unmanaged code on this thread is going to be writing into that address". This operation is called "fixing" or "pinning" the object.

Clearly it is a bad idea to pin lots of stuff and leave it pinned for a long time. The GC works as well as it does because it is able to reorganize memory to be more efficient; pinning blocks of memory across collections impedes the GC's ability to get work done. We therefore recommend that you pin as little as possible for as little time as possible.

The CLR provides us two ways to pin something. The hard, flexible, expensive way is to explicitly get a GCHandle on the object, tell the garbage collector "this object is pinned", take the address of the object, do what you need to do, and then unpin it.

The easy way is to mark a local variable as a "pinned local" using the "fixed" statement. The C# compiler will generate special code that marks the local as "pinned". When the garbage collector does a collection, clearly the GC needs to look at all the locals on the current stack frame, because those locals are all "alive". When the GC sees that a local is pinned, it not only notes that it is alive, it also makes a note to itself that the referent of the contents of this variable are not to be moved during the collection. This is a cheap and easy alternative to explicitly getting a GC handle, but it depends on the variable in question being a local.

In an iterator block, locals are all realized by hoisting them to fields. Because we cannot pin fields, we'd have to change the code generation for the fixed block so that it took out a handle to the object, pinned it, and ensured that the object was unpinned at some point.

That's bad enough; now consider what happens when you yield while an object is pinned. Arbitrarily much time could pass between the yield and the unpin! This directly violates our guidance that things should be pinned for as little time as possible.

For all those reasons, no unsafe code is allowed in iterator blocks. In the unlikely event that you need to dereference pointers in an iterator block, you can always extract the unsafe code to its own helper method and call it from the iterator block.

***********

I hope you've enjoyed this little trip into the weird corner cases of iterator blocks. This is a complicated feature with lots of interesting design choice points.

I'm flying south to Canada to spend some time with my family on the beaver-shark infested shores of the great inland sea known as Lake Huron, so the next few fabulous adventures will be pre-recorded. See you when I'm back.

Iterator Blocks, Part Five: Push vs Pull

Eric Lippert — Thu, 23 Jul 2009 17:05:00 GMT

A while back I posted some commentary I did for the Scripting Summer Games where I noted that there is an isomorphism between "pull" sequences and "push" events. Normally you think of events as something that "calls you", pushing the event arguments at you. And normally you think of a sequence as something that you "pull from", asking it for the next value until you're done. But you can treat a stream of event firings as being a sequence of event argument objects. And similarly, you could implement a sequence iterator so that it called a method for every object in the collection. Heck, you could implement all of LINQ on this model if you really wanted.

The implementation of iterator blocks is clearly on the "pull" paradigm. It didn't have to be. We could have gone with a sort of "inversion of control" approach. "Pull" iterators have to simulate coroutines with a little state machine that knows how to get back to the state where the code left off. But we already have a mechanism for code to "get back to the state where it left off" -- that's what you do every time you call a method! You remember what you were doing, call the method, and then pick up where you left off. We could have done the same thing with iterators. We could say that this:

void Integers(int length, IObserver<int> observer)
{
for (int i = 0; i < length; ++i) yield return i;
}

is a syntactic sugar for

void Integers(int length, IObserver<int> observer)
{
for (int i = 0; i < length; ++i) observer.Yield(i);
observer.Break();
}

That is, we "raise an event" by calling the observer every time something yields, and raise another event when we're done.

This would be a much more trivial transformation to make than our current state machine approach, but since most people want sequence iteration on the "pull" model, we do the harder work to make that happen.

I said this had something to do with exception handling. What's the connection?

Did you notice something weird about how we handle finally blocks? Consider what happens with the "push model":

TextReader reader = File.Open(file);
try
{
blah blah blah yield the lines
}
finally
{
reader.Close();
}

If something in the "blah blah blah" is realized as a call to observer.Yield(line), what happens if the code consuming the result throws? Easy. It is just a method call like any other. The call stack unwinds, we find the finally, the file gets closed, everything is good.

Now suppose this is realized as the MoveNext of a "pull" iterator. What happens when the code that is consuming the result throws? If we're consuming the result then we've returned from the call to MoveNext! There is no "try", there is no "finally". And yet usually the finally gets executed anyways! If the code that is consuming the results throws, odds are good that it was in a foreach; when foreach throws, the enumerator is disposed, and when the enumerator is disposed, we figure out which "finallys" were around when we last left MoveNext, and we run a special method that does whatever was in the pending finally blocks.

This is pretty weird. What's going on here is that for finally blocks, "pull" iterators have the same semantics as "push" iterators. When you yield from a try containing a finally, it looks like the finally is still "on the stack" and runs when the consumer throws.

So what if the try block has a catch?

The original designers of C# 2.0 -- and remember, this is long before my time on the team -- had a huge debate over this. A debate which was repeated in miniature when I sent them all an email asking for the justification for this decision. There are three basic positions:

1) Do not allow yield returns in try blocks at all. (Or blocks that are sugars for try blocks, like using blocks.) Use some other mechanism other than "finally" to express the idea of "this is the code that you run when the caller shuts down the enumeration."

2) Allow yield returns in all try blocks.

3) Allow yield returns in try blocks that have finally blocks, but not if they have catch blocks.

The down sides of (1) are that you have to come up with some syntax for representing the "finally" logic that isn't actually a "finally", and that it becomes more difficult to iterate over stuff that requires cleanup, like iterating over the contents of a file. This makes the feature both confusing and weak; we should avoid this option if at all possible.

The down side of (2) is that the exception handling behaviour beomes deeply, weirdly inconsistent between finally blocks and catch blocks. If you have a yield in a try block with a finally, and the consumer of the iteration throws, then the finally runs, as though you were on the "push" model. But if you have a yield in a try block with a catch, and the consumer of the iteration throws, then the catch does not run, because its not on the call stack. Users have a reasonable expectation that finally and catch work more or less the same way when an exception happens; that is, that control will be consistently transferred to a matching catch or the finally.

The down side of (3) is that the rule seems arbitrary and weird -- until you read five unnecessarily prolix blog entries that explain what on earth the design team was thinking.

Obviously, they picked (3), and now you know why.

Next time, we'll finish up this series with a look at unsafe code. Now that you know all this stuff, seeing why there's no unsafe code allowed in an iterator block is pretty straightforward by comparison.

Iterator blocks Part Four: Why no yield in catch?

Eric Lippert — Mon, 20 Jul 2009 16:44:00 GMT

Now that you know why we disallow yielding in a finally, it’s pretty straightforward to see why we also disallow yielding in a catch.

First off, we still have the problem that it is illegal to “goto” into the middle of the handler of a try-protected region. The only way to enter a catch block is via the “non-local goto” that is catching an exception. So once you yielded out of the catch block, the next time MoveNext was called, we’d have no way to get back into the catch block where we left off.

Second, the exception that was thrown and caught is an intrinsic part of the execution of the catch block because it can be re-thrown using the “empty throw” syntax. We have no way of preserving that state across calls to MoveNext.

I suppose that we could in theory allow yields inside catch blocks that do not have an “empty throw”. We could turn this:

try
{
M();
}
catch(Exception x)
{
yield return x.ToString().Length;
yield return 123;
}

into

switch(this.state)
{
case 0: goto LABEL0;
case 1: goto LABEL1;
case 2: goto LABEL2;
}
LABEL0:
try
{
M();
this.state = 2;
goto LABEL2;
}
catch(Exception x)
{
this.x = x;
}
this.current = this.x.ToString().Length;
this.state = 1;
return true;
LABEL1:
this.current = 123;
this.state = 2;
return true;
LABEL2:
return false;

But frankly, that would be pretty weird. It would feel like a very odd restriction to disallow a fundamental part of the “catch” syntax.

And really, what’s the usage case that motivates this situation in the first place? Do people really want to try something and then yield a bunch of results if it fails? I suppose you could do something like:

IEnumerable<string> GetFooStrings(Foo foo)
{
try
{
yield return Foo.Bar();
}
catch(InvalidFooException x)
{
yield return "";
}
… etc.

but there are lots of ways to easily rewrite this so that you don’t have to yield in a catch:

IEnumerable<string> GetFooStrings(Foo foo)
{
string result;
try
{
result = Foo.Bar();
}
catch(InvalidFooException x)
{
result = "";
}
yield return result;
… etc.

Since there are usually easy workarounds, and we would have to put crazy-seeming restrictions on the use of re-throws, it’s easier to simply say “no yields inside a catch”.

We still haven’t explained why it is illegal to yield inside a try that has a catch, nor why it is legal to yield inside a try that has a finally. To understand why, next time I’ll need to muse about the duality between sequences and events.

Iterator Blocks, Part Three: Why no yield in finally?

Eric Lippert — Thu, 16 Jul 2009 16:18:00 GMT

There are three scenarios in which code could be executing in a finally in an iterator block. In none of them is it a good idea to yield a value from inside the finally, so this is illegal across the board. The three scenarios are (1) normal cleanup, (2) exception cleanup, and (3) iterator disposal.

For the first scenario, suppose we have something like

try
{
Setup();
yield return M();
}
finally
{
yield return N();
Cleanup();
}

How should we transform this into an iterator state machine? Naively, we want to do something like:

switch (this.state)
{
case 0: goto LABEL0;
case 1: goto LABEL1:
case 2: goto LABEL2:
case 3: goto LABEL3:
}
LABEL0:
try
{
Setup();
this.current = M();
this.state = 1;
return true; // BUT DON'T RUN THE FINALLY!
LABEL1:
}
finally
{
this.current = N();
this.state = 2;
return true;
LABEL2:
Cleanup();
}
LABEL3:
this.state = 3;
return false;

There's an immediate problem with this: we both "goto" into a finally block and "return" out of one. Neither is legal.

Leaving a finally block via "return" is illegal because it is weird. Suppose the try block returns, causing the finally to execute. What happens when the finally block returns? The original return value would be lost and some other return value would be substituted for it. This seems bizarre and bug-prone, so its not legal; you cannot exit a finally via return. And of course, you don't want the finally to run after the return in the try in this case!

Furthermore, in the CLR model of exception handling it is illegal to branch via a goto into a try block or its "handler" (that is, the catch or finally clause). Nor can you branch out of the handler. These special regions have special code that needs to be run when the region is entered and exited; you cannot skip either with a goto.

So we have no immediately obvious way to generate verifiable code for this scenario. Right off the bat, we have a huge number of points against this feature; we'd have to either convince the CLR guys to allow spaghetti code involving protected regions, or come up with some clever technique for generating this code.

And that's just the scenario where nothing is going wrong yet! Suppose a miracle happens and we manage to successfully generate code for scenario one. Now consider our second scenario, where M throws exception X. Remember, the finally block "catches" the exception and processes the cleanup code. If the cleanup code throws, then the original exception is discarded and the new processing of the new exception takes over. If the cleanup code succeeds, then the original exception continues to be "thrown" up the call stack, looking for more finally blocks or catch blocks.

Suppose the cleanup code does not throw. What should the control flow look like? The caller calls MoveNext for the first time. M() throws X. The finally block takes over. The finally block calls N(), and returns control to the caller with the results of N() !? What happened to X, the exception? Is it just waiting there, in limbo? When the second call to MoveNext happens, should the cleanup code get run then, and X suddenly pops back into existence and continues to be thrown up the stack? That makes no sense at all; the two calls to MoveNext could have had completely different stacks! This is crazy, plus we have no mechanism at all in the CLR for these kind of shenanigans.

Third, suppose we manage to solve all these problems. Now consider what happens when the caller calls MoveNext, M() succeeds, control returns to the caller, and the caller calls an early Dispose() on the enumerator. (I guess they wanted only one item.) We generate a Dispose method that checks the current state and executes any pending finally blocks. What the heck is the Dispose method supposed to do when it encounters the yield return in the pending finally block? We're not even in a call to MoveNext anymore! Should we call N() and ignore the result? Should we return from the Dispose() after the call to N(), or do the cleanup and then return? What exactly should the control flow do here? We're in a context where we might not even be iterating anymore, and yet we're still yielding.

It really doesn't make any logical sense to do a yield return in this scenario; we're possibly not in a position where the caller is expecting things to continue to be yielded to it.

So in short, we'd have to do at least two impossible things in order to enable a scenario that makes no sense in the first place. If ever a feature called out to be cut at the design phase, this is it. Therefore: no yields inside finally blocks. Thank goodness for that.

Next time: now that you know all that, figuring out "why no yields inside catch blocks" is pretty straightforward.

Iterator Blocks, Part Two: Why no ref or out parameters?

Eric Lippert — Mon, 13 Jul 2009 16:35:00 GMT

A long and detailed discussion of how exactly we implement iterator blocks would take me quite a while, and would duplicate work that has been done well by others already. I encourage you to start with Raymond’s series, which is a pretty gentle introduction: part 1, part 2, part 3. If you want a more detailed look at how this particular sausage is made, Jon’s article is quite in-depth.

To make a long story short, we implement iterators by:

Spitting a class that implements the relevant interfaces.
Hoisting locals of the iterator block to become fields of the class.
Rewriting the iterator block as the “MoveNext” method of the class.
Tracking the point where the last yield happened in the MoveNext method, and branching to that point directly using a switched “goto” when MoveNext is called.
Moving the logic in “finally” blocks (or equivalents, like the auto-generated finally blocks associated with lock statements and using statements) into the Dispose method, to ensure that stuff gets cleaned up when the iteration is completed or abandoned.

This is not the only implementation strategy we could have pursued. As I said last time, we could have implemented full-blown continuations and built the whole thing out of them; any conceivable control flow can be built out of continuations. But getting continuations right requires a lot of support from the framework, and would be very expensive. We can get by with less, so we do.

Similarly, we could have built this out of “coroutine fibers”. A fiber is like a thread in that it has its own stack, but the code on the fiber itself decides when the fiber is done running. Again, this is a strategy – implement coroutines – that I mentioned last time which we rejected as too costly and difficult to get right in the CLR. Again, we can get by with less, so we do.

But this choice of a feasible, cost-effective implementation strategy then drives restrictions back into the design space; we have to lose some generality in order to make this strategy work.

The first thing that we lose is the ability to have iterator blocks be in methods which take ref or out parameters. In order to preserve the values of local variables and formal parameters across calls to MoveNext, we hoist the locals and parameters into fields of a compiler-generated class.

As I discussed earlier, the CLR designers had an implementation goal of being able to take advantage of the performance characteristics of the system stack. What happens when you store a reference in a field? The reference could be to something on the stack, but the field could live longer than the thing on the stack. The designers of the CLR were faced with four choices:

Build a system which is as fragile and horrid as unmanaged C code, a language in which accidentally storing a reference to something with too-short lifetime is a frequent source of awful crashing bugs and data corruptions.
Disallow (*) taking references to locals; any local taken as a reference must be hoisted to become a field on a garbage-collected reference type. Safe, but slow.
Disallow storing references in fields.
Invent something else that solves this problem.

The CLR designers chose the third option. Which means that our choice of hoisting parameters to fields immediately runs into a problem; we have no safe way of storing references to other variables in a field. The reference could be to something on the stack, but the iterator could live longer than the variable that is on the stack. A confluence of implementation choices has now driven an unfortunate and seemingly arbitrary restriction into the design. We lose a bit of generality in order to gain simplicity of implementation and higher performance in common cases.

Now think about that decision from the perspective of the larger goal of the feature. Remember, the feature is not “make any possible method into an iterator block”. The feature is “make it easier to write code that enumerates the members of a collection”. An “out” or “ref” parameter exists solely to enable a method to mutate an outside variable and therefore pass back information to the caller when the method terminates.

Why would you want to do that in a method whose sole purpose is to produce the sequence of values stored in a collection? What would the meaning of mutations to the variable even be if the iterator block mutates the variable multiple times as the iterator is running? Normally that kind of thing is not visible (modulo goofy multithreading scenarios) until the method is done, but an iterator block returns control to its caller many times before its done.

This seems like a scenario which is uncommon, strange and hard to implement. And furthermore, there are ways to achieve mutation of outer variables during iteration without passing in a ref to a variable; you could pass in an Action delegate. Instead of this illegal program:

IEnumerable<int> Frob(out string s)
{
yield return 1;
s = "goodbye";
}
void Grob()
{
string s = "hello";
foreach(int i in Frob(out s)) …
}

you can always do the equivalent thing with this legal program:

IEnumerable<int> Frob(Action<string> setter)
{
yield return 1;
setter("goodbye");
}
void Grob()
{
string s = "hello";
foreach(int i in Frob(x=>{s=x;})) …
}

Since making the more general feature work given our implementation constraints is (1) hard or impossible, (2) clearly a corner case, not a mainline scenario, and (3) has a possible workaround, it’s a no-brainer to put a restriction on the design and disallow ref and out parameters.

Next time, why no yield in a finally block?

***********

(*) Or, weaker, make it allowed but unverifiable. That is pretty much the same thing from the compiler implementers perspective; we try to never generate unverifiable code unless it is specifically marked “unsafe”.

Iterator Blocks, Part One

Eric Lippert — Thu, 09 Jul 2009 17:11:00 GMT

There is a constant tension in language design between solving general problems and solving specific problems; finding the right spot on the general-to-specific spectrum can be quite tricky.

The design of iterator blocks yields (ha ha) a germane example. At almost every step along the way, there are opportunities for making choices that determine whether a more general or a more specific problem is being solved.

Let’s start with the high-level design of the feature. Iterator blocks are, as the name implies, all about making it easier to write code that iterates over some collection of items in a natural way. This is a pretty darn general scenario; there are lots of potential collections (stacks, queues, lists, dozens of different kinds of trees…) containing arbitrarily many different types of items, and lots of ways to iterate over them (in order, post order, pre order, filtered, projected, grouped, sorted…)

We could have made it much more general. Our iterator blocks can be seen as a weak kind of coroutine. We could have chosen to implement full coroutines and just made iterator blocks a special case of coroutines. And of course, coroutines are in turn less general than first-class continuations; we could have implemented continuations, implemented coroutines in terms of continuations, and iterators in terms of coroutines.

But we didn’t. We decided that the sweet spot for the high-level scenario-driven design of the feature was iterators over collections, and so we concentrated on that. Some adventurous people use the fact that iterators are “poor-man’s coroutines” as a shortcut to building coroutine-like systems that have only a tenuous connection to the semantics of iterating over the items in a collection, but those are the exception, not the rule. Their scenarios certainly did not drive the design of the feature.

We want to balance the generality of the feature against the high cost of that generality. Premature optimization is often cited as the root of all evil, but I don’t think that’s quite true. Premature generality is responsible for a lot of evil too! As we’ll see, a lot of the time when faced with a design decision we take the YAGNI position; we choose to implement a little bit less generality to get a large cost savings, with the assumption that hardly anyone would benefit from that spending.

Over our next few fabulous adventures in coding I’ll discuss some of the reasons for the seemingly odd restrictions on the generality of iterator blocks – things like, why can there be no unsafe code in iterator blocks? Why can an iterator block not take a ref or out parameter? Why can’t you put a yield in a finally? And so on.

Iterators at the Summer Games

Eric Lippert — Fri, 26 Jun 2009 17:09:00 GMT

Ed "Scripting Guy" Wilson was kind enough to ask me to be a guest commentator at this years Summer Scripting Games, which have just completed.

I've been working on a series for this blog about some unusual cases in the design of the "iterator block" feature in C# 2.0; this bit from my commentary is going to be germane to that series. I thought I'd post it here as an appetizer before we get into the main courses of the series. The series on iterators will probably run throughout July.

The problem for event ten of the scripting games was to write a script which changes the priority of every new process that has a particular name.

***********

There’s an odd thing that you learn when working on developer tools: The people who design and build the tools are often not the experts on the actual real-world use of those tools. I could tell you anything you want to know about the VBScript parser or the code generator or the runtime library, but I’m no expert on writing actual scripts that solve real problems. This is why I was both intrigued and a bit worried when the Scripting Guys approached me and asked if I’d like to be a guest commentator for the 2009 Summer Scripting Games.

I wrote this script the same way most scripters approach a technical problem that they don’t immediately know how to solve; I searched the Internet for keywords from the problem domain to see what I could come up with. Of course, I already knew about our MSDN documentation, I had a (very) basic understanding of WMI, and I knew that the Scripting Guys had a massive repository of handy scripts.

My initial naïve thought was that I would have to go with a polling solution; sit there in a loop, querying the process table every couple of seconds, waiting for new processes to pop up. Fortunately, my Internet searches quickly led me to discover that process startup events can be treated as an endless collection of objects returned by a WMI query.

That got me thinking about the powerful isomorphism between events and collections.

A collection typically uses a “pull” model — the consumer asks for each item in the collection one at a time as needed, and the call returns when the item is available. Events typically work on a “push” model — the consumer registers a method that gets called every time the event fires. But not necessarily; the WMI provider implements events on a “pull” model. The event is realized as a collection of “event objects.” It can be queried like any other collection. Asking for the next event object that matches the query simply blocks until it is available.

Similarly, collection iterators could be implemented on a “push” model. They could call a method whenever the next item in the collection becomes available. The next version of the CLR framework is likely to have standard interfaces that represent “observable collections”, that is, collections that “push” data to you, like events do. The ability to treat events as collections and collections as events can lead to some interesting and powerful coding patterns.

***********

The rest of the solution and analysis is here.

Have a good weekend!