GrantRi's WebLog [MS] : Iterators

Iterator Usage

grantri — Tue, 17 Aug 2004 19:52:00 GMT

OK, a few days ago I asked about how you used anonymous methods. Thanks for the very few who responded. I'm hoping that more people have used them than chose to respond. Now it's time to ask about my other pet feature: Iterators.

How an possibly more importantly why do you use iterators? I'm sure that a good programmer could always sit down and write a better implementation of IEnumerable/IEnumerator for their given data. I wonder if that time is really well spent. Ideally iterators would allow you write comparable performing code in significantly less time. Especially if you count the time spent to debug and fix the inevitable bugs and correctness issues.

There are valid reasons to not use iterators, just like sometimes you need to use a for loop instead of a foreach loop. I've bumped into these a few times, but usually I've changed other things so I can use an iterator, rather than trying to write the IEnumerable/IEnumerator code myself. I want to know how often these situations come up in real world code, and how you resolve them. Do you change your code so you can use an iterator, or do you hand-write the enumerator?

I use iterators because they save me time and I have fewer bugs. Why do you use them, or why don't you use them?

--Grant

More thoughts on Iterators

grantri — Fri, 07 May 2004 07:35:00 GMT

So I had a short email exchange with a collegue about iterators. He mentioned that I should put in in my blog. Seemed reasonable, so here it is:

Subject: Iterator generated code

...
I'm looking at porting a hand-coded enumerator to use iterators (this is for a sample and maybe an article I have in mind to write about moving a production-quality collection to Whidbey)

I'm decompiling the generated code with the latest .net reflector, as my IL isn't really up to much.

My hand-written code tries hard to implement the full contract from the MSDN docs for IEnumerator, i.e. stuff like throwing InvalidOperationException if the underlying collection changes in both Reset and MoveNext and throwing in Current if it is past the last element.

I was wondering a couple of things

a) Will the past last element code get added to Current (i.e. state == -1)

b) Will a Reset implementation be added?
I can imagine that in many cases, the conditionals required to get to the yield return would be sufficient as a test in MoveNext and Reset, but I'm sure there are cases when it isn't.

However I guess this would require that any initialiser code for user state would have to move into the constructor for this to work, as docs for Reset specify that it throws if the collection has changed between creation and Reset
...

Subject: RE: Iterator generated code
a) No. The language designers specifically wanted the Current property to be inlineable, and NEVER to throw, even if it is accessed in an invalid state (before calling MoveNext, after MoveNext has returned false, when the collection has changed, etc.). The general reason is that most people use foreach (or some equivalent) and there should be no reason to punish them performance-wise fo rhte few people who try to call Current some other way.

b) Nope. Again Anders mainly thought that enumerables are meant to be throw-away object and should never be reset. He even pushed the generic form of IEnumerator to not have a Reset method (I think he would have removed it from the non-generic IEnumerator if it was possible).

I've often wished we had some nice syntax to say "run this code xyz for each iteration". The classic example is checking so see if the underlying collection has changed. Without such syntax the user has to write boiler plate code (hopefully just calling a single method) before or after every yield.

--Grant

Subject: RE: Iterator generated code
...
Would be good to position iterators as a performance-optimized foreach-server rather than a super-strict implementation of the IEnumerable contract.
...

This got me thinking some more about iterators. The frameworks often use an int as a version field to detect when a collection has changed. This seems like a bit of overkill and isn't 100% correct because it does suffer from wrap-around (although that is almost entirely impossible short of malicious code). So I was wondering how you could write 100% correct code and still use an iterator, and have it be clean and maintainable code.

My first thought was that you shouldn't need to increment the version all the time, only when there are live enumerator objects. My next thought was that to have maintainable code the version checking and throwing should be encapsulated into a single method. In C/C++ I would have used a macro to wrap up the version checking with the yield statement, but in C# I'm forced to just remeber to call my checker every place I yield.

Well, for the method that checks the version and throws, I have 2 choices: on the outer collection class, or as an anonymous method inside the iterator. Somehow the anonymous methods seems cleaner (and cooler) because it fits with the notion that an iterator effectively creates an anonymous class.

Now back to detecting changes. Instead of having an int, why not have an event that fires when the collection changes. The iterator could subscribe to it with an anonymous method as well. Then when there are no active iterators, there is little overhead (which is faster: a null check or an interlocked increment?).

So here's some quick slapped together pseudo-code:

public class SomeCollection { private event EventHandler CollectionChanged; private void OnCollectionChanged() { EventHandler h = CollectionChanged; if (h != null) h(this, EventArgs.Empty); } delegate void SimpleDelegate(); public IEnumerable<SomeType> MyIterator() { bool changed = false; EventHandler changedHandler = null; SimpleDelegate preCondition = delegate { if (changed) throw new InvalidOperationException(...); // Possibly some other validation code here??? }; changedHandler = delegate { changed = true; // Once it's changed, we don't need to know about future changes CollectionChanged -= changedHandler; changed Handler = null; }; try { CollectionChanged += changedHandler; ... yield return someValue; preCondition(); ... yield return someValue; preCondition(); ... } finally { if (changedHandler != null) { CollectionChanged -= changedHandler; changedHandler = null; } } } }

So what do you think? It's still not perfect because the event isn't subscribed to until the first call into MoveNext, which may happen significantly after the call to GetEnumerator. The only way around that would be to wrap the iterator itself with another method that created the changedHandler, subscribed it and then passed in the changedHandler/preCondition pair to the real iterator method. The down-side to that would be that if somebody called GetEnumerator twice on the same IEnumerable, both IEnumerators would share the same changedHandler/preCondition.

It still does seem like an awful lot of boiler-plate code. I might keep the notion of using an anonymous method as a way to pre and post conditions in an iterator, but using the int as a version for collections does seem to have fewer draw-backs (and a lot less boiler-plate code)...

--Grant

Remoting...

grantri — Sun, 18 Apr 2004 05:32:00 GMT

First off, I don't even pretend to have a good comprehension of how remoting works.

What I do know is that right now the compiler generated classes derive directly from Object, and somehow this is bad for remoting, instead they should derive from MarshalByReferenceObject. On the surface that is a very simple change, but it has some possibly serious performance implications.

Memory-wise, MBRO adds only a little bit. Certainly not enough to outweight the benefits of better remoting. The penalty comes when accessing fields of MBRO objects. The penalty is that every field access gets indirected through a method call. The method call is not needed when accessing fields off of the this pointer.

With iterators, this is a non-issue because the fields are only accessed internally off of the this pointer. Anonymous methods have the potential to be greatly impacted. It's bad enough that a local is now heap based, but making a local access into a method call would just supprise too many people.

So is there a way to have the best of both worlds: fast access and easy remoting? Here are the two ideas I've heard tossed around (well I guess one is mine):

Some keyword or syntax to specify whether a given anonymous method should be MBRO. Then you, the programmer, can clearly decide if remoting is more important than accessing locals quickly. The problem here is what syntax?
Base it on the containing object. This seems like a way of allowing you to decide, but without any extra syntax. The problem is that many classes derive from frameworks classes that have already made that decision (and probably for different reasons).

So here's your chance! Voice your thoughts and opinions. Do you have a better idea? Do you think I'm making a big deal over nothing?

--Grant

Recursive Iterators made Iterative

grantri — Fri, 09 Apr 2004 01:41:00 GMT

So I've been thinking about this, and although it seems like CS101 to transform a recursive algorithm into an iterative one, and example might be worth while. The natural one to attack would be the generic binary tree traversal iterator. So here goes!

My first thought would be to use the new Queue class to keep track of parent nodes:

public class BinaryTree<DataType> { public DataType Data; public BinaryTree LeftChild; public BinaryTree RightChild; public IEnumerable<DataType> InOrder { get { // if we were smart and had the data // we could set the initial capacity to our max depth Stack<BinaryTree<DataType>> stack = new Stack(); for (BinaryTree<DataType> current = this; current != null || stack.Count > 0; current = current.RightChild) { while (current != null) { stack.Push(current); current = current.LeftChild; } current = stack.Pop(); yield return current.Data; } } } }

If I did everything right, this will only allocate the 1 stack object and enough storage within that for log N references. And no recursion!

--Grant

A good author writes better than a good developer

grantri — Tue, 30 Mar 2004 18:39:00 GMT

So here's a recent MSDN article that I actually had the chance of reviewing several months ago. They've done a much better job at capturing the good new stuff in C# than I could ever do.

http://msdn.microsoft.com/vcsharp/default.aspx?pull=/msdnmag/issues/04/05/c20/default.aspx

I do still have a few personal corrections:

I commented that the internal names that the compiler generates have changed and will probably change again, and so the article should probably just mention that the name is compiler generated and not give the actual name (those that care can run ILDASM and see for themselves), but they didn't listen. At least they did mention that is might change. For those of you who have the community drop you'll notice that the compiler generated names have changed from what's in the article.

I also commented that using the binary tree wasn't exactly the best example of iterators because of it's miserable performance. Yes it is clean and neat code, but if you have a binary tree with 5000 nodes and you foreach it a few times, you can watch your machine crawl as it is forced to spend more and more time in the GC rather than running your code. At least they mentioned this potential problem (see the last paragraph before the Partial Types section). Too bad my brain wasn't thinking and I got the numbers wrong. It should state, “approximately n/2 iterator instantiations, where n is the number of nodes in the tree.”

Also note the small paragraph on #pragma warning that now gives you line-level control over individual warnings!

--Grant

Some Cool New Features

grantri — Fri, 26 Mar 2004 22:03:00 GMT

I've been looking over some of my last few posts and they could easily come across as if I thought iterators and anonymous methods were the worst language features every added, and will never serve a good purpose. If anything I think the opposite (or else I wouldn't have spent so much time on them).

Second only to generics I think iterators and anonymous methods are the coolest features added to the C# compiler.

So then why have I been writing all this negative stuff? Well mainly because these guys are so powerful it's easy to write bad code and then blame it on the feature. My goal is to point out their weaknesses so that you as educated developers can write clean, neat, and efficient code, and not get burned. I believe in general these are both very powerful features, but like most powerful things they can be quite deadly if used improperly. Somehow I just don't feel like the MSDN docs are going to capture some of these gotchas in their 1/2 page topic on anonymous method or iterator syntax. Also I'm already starting to see bad habits and styles floating around and the features haven't even been officially released yet!

Well, if you've been itching to try them, we finally have an answer for you: Community Drops! If you attend VSLive! or have a MSDN Universal subscription, we're sending out or first community drop. One word of caution: this first release was definitely date driven, so there are known bugs, but it's a lot newer than the Alpha bits a few of you already have. I would personally recommend installing it on a 'throw away machine' (i.e. a machine you don't mind re-installing the OS on just to clean things up), but go out and try it. Hopefully the higher-ups will come up with a sane schedule for us releasing more frequent drops like this. Enjoy!

http://msdn.microsoft.com/vs2005/

--Grant

Recursive Iterators (aka Perf killers)

grantri — Thu, 25 Mar 2004 06:25:00 GMT

Technically this isn't specific to iterators, but its so much easier to do with iterators that I thought I'd mention it. Some languages are smart enough to 'flatten' iterators, such that if an iterator is called insde of another iterator, only one iteration object exists. C# is not one of those languages (at least not yet). Generally if you see any sort of recursion inside an iterator, you're asking for performance problems, simply because each time it recurses, it's going to create a whole new iteration object on the GC heap.

Consider this binary tree:

public class BinaryTree<DataType> {
    public DataType Data;
    public BinaryTree LeftChild;
    public BinaryTree RightChild;

    public IEnumerable<DataType> InOrder {
        get {
            if (LeftChild != null)
                foreach (DataType d in LeftChild.InOrder)
                    yield return d;
            yield return Data;
            if (RightChild != null)


                foreach (DataType d in RightChild.InOrder)
                    yield return d;
        }
    }
}

Now assume the class is properly done up with properties, and accessors and is always kept balanced. How many objects are created when walking different sized trees? With just 1 element, the answer is 1. With 2 you get 2, 3 -> 3, 4 -> 4, 5 -> 5, etc. You can keep doing the math, but allocation wise, this neat, clean, and simple code is O(n). That's a pretty heavy burden on the CG if you ask me. If that's a tax you're willing to bear, then by all means keep this clean and clear code.

--Grant

Anonymous method (and Iterator) perf

grantri — Sat, 20 Mar 2004 05:09:00 GMT

probably a few of you hav asked how to make iterators and anonymous methods fast. Well hopefully if you understand how the compiler translates them, it should be relatively easy to optimize them. The first thing to remember is that for all practical purposes the C# compiler does not have an optimizer. Thus the code that you write is fairly directly translated into IL. The best way to optimize the IL is to optimize your code. In the case of iterators and anonymous methods, I've tried (and I think this was the language designers' intent as well) to make the transformations farily transparent, so that the programmer can easily write educated and efficient code while taking advantage of these great features. So here's my rules for making effecient anonymous methods and iterators (in no particular order):

Think carefully about which locals you use (a.k.a. capture). As soon as you touch a local in an anonyous method, that local becomes heap based and goes through an extra level of indirection (and prevent enregistering).
Think carefully about where locals are declared. In order to maintain proper scoping semantics the compiler creates a new heap object on entry of each scope that has captured locals. If you can move all the captured locals into one scope you'll probably get better perf. If possible move that scope 'inward' to avoid an GC object creation on a hot path (assuming the anonymous method is not on the hot path). If possible move that scope out of loops to avoid GC object creations on each iteration of the loop.
The captured locals do not have to be in the same scope as the anonymous method. The delegate that holds the anonymous method will be cached at the scope of the innermost captured local. Thus if the locals are outside of a loop, but the anonymous method is inside, that's OK.
Add extra scopes as hints to the compiler about lifetimes. For iterators, any scopes without a 'yield return' statement will be mostly left intact, including using real locals for everytihg in that scope instead of heap-based fields.
For iterators, if a try/finally block contains a yield statement, the finally block must be cloned from the MoveNext method into the Dispose method (so the same code is in two places).
For iterators, calling GetEnumerator twice is just as expensive as calling the original iterator method/property/indexer twice.
Iterators are designed to help amortize the cost of enumerations, so if you only need the first few elements, you don't need to pay the price for enumerating everything. Follow that pattern in your own iterators where possible.
Don't enumerate something twice when once will do. Most enumerations have another method to obtain the count of elements without iterating. So use that instead of iterating once just to count the elements, and then again to extract the values.
Anonymous methods make some algorithms clearer, and iterators tend to make more correct IEnumerable/IEnumerator implementations, but neither of them will never be as efficient as hand written code (or hand written assembler, but who has time for that anymore). So write well documented, clear, clean code, and then use a profiler to see if you need a speed improvement. And remember that a faster algorithm will probably trump any micro-optimizations in your code (like counting down instead of up to avoid that one extra opcode on th eloop test)

I'm sure I think of more later, but that's most of them for now. Enjoy, and feel free to correct me. I can't wait until you guys can get a newer build and see the stuff we've been working on for almost the last year.

--Grant