Iterator Blocks Part Seven: Why no anonymous iterators?

Iterator Blocks Part Seven: Why no anonymous iterators?

Rate This
  • Comments 28

This annotation to a comment in part five I think deserves to be promoted to a post of its own.

Why do we disallow anonymous iterators? I would love to have anonymous iterator blocks.  I want to say something like:

IEnumerable<int> twoints = ()=>{ yield return x; yield return x*10; };
foreach(int i in twoints) ...

It would be totally awesome to be able to build yourself a little sequence generator in-place that closed over local variables. The reason why not is straightforward: the benefits don't outweigh the costs. The awesomeness of making sequence generators in-place is actually pretty small in the grand scheme of things and nominal methods do the job well enough in most scenarios. So the benefits are not that compelling.

The costs are large. Iterator rewriting is the most complicated transformation in the compiler, and anonymous method rewriting is the second most complicated. Anonymous methods can be inside other anonymous methods, and anonymous methods can be inside iterator blocks. Therefore, what we do is first we rewrite all anonymous methods so that they become methods of a closure class. This is the second-last thing the compiler does before emitting IL for a method. Once that step is done, the iterator rewriter can assume that there are no anonymous methods in the iterator block; they've all be rewritten already. Therefore the iterator rewriter can just concentrate on rewriting the iterator, without worrying that there might be an unrealized anonymous method in there.

Also, iterator blocks never "nest", unlike anonymous methods. The iterator rewriter can assume that all iterator blocks are "top level".

If anonymous methods are allowed to contain iterator blocks, then both those assumptions go out the window. You can have an iterator block that contains an anonymous method that contains an anonymous method that contains an iterator block that contains an anonymous method, and... yuck. Now we have to write a rewriting pass that can handle nested iterator blocks and nested anonymous methods at the same time, merging our two most complicated algorithms into one far more complicated algorithm. It would be really hard to design, implement, and test. We are smart enough to do so, I'm sure. We've got a smart team here. But we don't want to take on that large burden for a "nice to have but not necessary" feature.

  • That just sounds scary.

    Seems perfectly reasonable why it wasn't done.

  • What would be really cool would be for the compiler error messages to link to Eric's blog posts.  They already link to MSDN, so adding "This error is explained on Eric Lippert's blog" hyperlinks automatically to topics matching specially formatted blog keywords ("keyword: CSnnnn" anyone) would be really neat.  Even someone adding the links by hand would be really helpful.

  • Eric:Thanks for devoting an entire post to my question(comment) about Anonymous iterators from Part5 :-)

  • First, I believe that Microsoft should commit itself to achieving the goal, before this decade is out, of allowing anonymous iterator blocks and having them nest safely. No single feature in this period will be more awesome to mankind, or more important for the future of c#; and none will be so difficult or expensive to accomplish. We propose to accelerate the development of the appropriate compiler. We propose to develop alternate iterator anonymous method rewriter, much smarter than any now being developed, until certain which is superior.

  • @Eric: One more approach to get this done, you could have some Sequence expression pattern as in F#(using Monads), that allows you to wrap a IEnumerable on the return values. Of course, this is not totally applicable for C#, but that could be one way to get this done for anonymous methods? What do you say?

    Seq(()=>{yield return x;})

    Adding up, this would be a great benefit for LINQ too,

  • @Eric: "If anonymous methods are allowed to contain iterator blocks..."

    Is the cost of just discovering nested iterators and nested anonymous methods within iterators any lower?  If a restriction were to be placed on anonymous yields to prevent these compiler complexities, would this feature still be worth implementing?

    I'm thinking that in the phase when anonymous methods are rewritten, the compiler can issue an error if it detects an anonymous method within an anonymous iterator.  Likewise for the iterator rewriter phase, but detecting nested anonymous iterators instead.

    The cost of each may be smaller if nesting could be detected after the top-level closures have already been created, assuming that each phase already works recursively.  (Full disclosure: I don't know diddly about compiler theory, so this is all probably just rubbish anyway :p)

    Sure, it would suck to not have the ability to nest anonymous iterators and methods.  And it would also be inconsistent in C# (barring the existing strangeness of iterators, as you've pointed out in other posts).  But I think it might still be useful.

    Either that or perhaps put something into the BCL that's like the following?

    static class Enumerable {

     // NOTE: this is not meant to be an extension method

     public static IEnumerable<T> Iterate<T>(params T[] items)

     {

       return items.ToList().AsReadOnly();

     }

    }

  • Any thoughts on allowing anonymous types to implement interfaces? That would be a really useful feature.

  • C# is a great language and it seems to get significantly better with each version. For the most part, I really like the direction your team has taken the language.

    If there is one criticism I would level, it is that I feel that I often bump into what seem like arbitrary limitations of a given language feature. To put it more precisely, I feel that a lot of the language features are not very isomorphic.

    The word you're looking for is "orthogonal", not "isomorphic". "Orthogonal" means that two things can vary independently of each other. By analogy, "orthogonal" features are those where there are no weird interactions with other features. "Isomorphic" means "having the same shape". -- Eric

    I guess I'm trying to say that when you folks are doing your cost/benefit analysis, I hope that a lot of consideration is given to trying to make the language natural and consistent.

    Perhaps I was not sufficiently clear. The whole purpose of this series of articles was to justify why it is that sometimes we choose to make language features nonorthogonal: orthogonality is a goal, but sometimes it is too expensive. Sometimes it is better to get a feature that works well with 95% of the language. -- Eric

  • Anonymous iterators is simply a *must* do for the compiler team.  More and more async frameworks are discovering that iterators can be used to implement coroutine-like functions.  However, in the absence of anonymous iterators, it puts additional effort and complexity on the developer.  Given that we're in the age of concurrency, there will be more need for async/concurrent/coordinated code.  Do we really want to leave this responsibility to each and every developer instead of the compiler and/or a framework?

    It may be a complex endeavor, but it's also one that will yield many rewards!  No pun intended.

  • I would like to add my voice to the plea for anonymous iterators. We've built a coroutine framework on top of yield and do a lot of iterator work. But we are basically stuck in .NET 1.0 land in many ways, since we cannot take advantage of closure capturing to simplify our code.

    In short, adding anonymous iterators would bring us folks into .NET 2.0 land and get all the benefits of anonymous methods and I'm sure that nobody is challenging that anonymous method weren't hugely beneficial!

  • Well I'll add my voice to the chorus of folks saying that anonymous iterators would be nice to have, but given a finite budget I would much rather see the brains at Microsoft implement deep const and isolation semantics into the CLR so that languages targeting it can then expose them.

  • @Tom, I agree with you! [eric: I also know how difficult "const" is to properly implement]

  • @Eric:

    >> The word you're looking for is "orthogonal", not "isomorphic".

    Well, that's embarassing, but not at all out of character for the day I had. For what it's worth, I do know what those words mean, but they're certainly not part of my daily vernacular.

    >> The whole purpose of this series of articles was to justify...

    I have not read the whole series of posts. In any case, I imagine that much of the feedback your team recieves is asking you to focus on the pragmatic aspects of the language. I just wanted to let it be known that even though this specific feature is not one I (yet) feel is absolutely necessary, I would still love to see it implemented someday for the sake of language consistency.

  • Of course you can do anonymous iterators in F#:

    let y = 10

    seq { yield y; yield 10*y } |> Seq.iter (fun x -> printfn "%A" x )

    (and you can even have "inner iterators" via "yield!".)

  • unacceptable!

    This isn't a "would be nice to have" feature. This feature is absolutely required for writing the highest readable code possible. Code readability and thus maintainability are the number one concerns when writing. Having to hunt down some generator method somewhere because I could not write it inline reduced readability and increases maintenance cost.

    Well then, if C# does not meet your absolute requirements then you shouldn't use it. Though it is disappointing to not delight every customer, I do not think it reasonable to expect that we'll meet all the requirements of every person. Try F# instead. -- Eric

Page 1 of 2 (28 items) 12