jaredpar's WebLog : Testing

A Lesson in Serialization

Jared Parsons — Tue, 02 Sep 2008 15:00:46 GMT

A few days ago, I recklessly added a [Serialization] attribute to a few of my immutable collection types. I needed to pass data between AppDomain's and adding [Serialization] was the quick and dirty fix. Compiled, ran and I didn't think much about it.

Luckily I was updating some unit tests last night and I remembered this and added a couple of serialization sanity tests. Most of the tests passed first time but for my ImmutableStack class[1] was throwing an exception. Well, it was actually my ImmutableQueue but it was failing in one of the inner ImmutableStack instances. The test code was fairly straight forward

var stack = ImmutableStack.Create(new int[] { 1, 2, 3 });
using (var stream = new MemoryStream()) {
    var f = new BinaryFormatter();
    f.Serialize(stream,stack);
    stream.Position = 0;
    var obj = f.Deserialize(stream);
    var stack2 = (ImmutableStack<int>)obj;
    var stack3 = stack2.Reverse();
}

I did a bit of digging and discovered the exception was coming from the stack2.Reverse() call. Jumped through the code and didn't see much wrong. I had several existing tests around ImmutableStack.Reverse() and I couldn't see why Serialization would make any difference.

public ImmutableStack<T> Reverse() {
    var r = ImmutableStack<T>.Empty;
    var current = this;
    while (current != ImmutableStack<T>.Empty) {
        r = r.Push(current.Peek());
        current = current.Pop();
    }

    return r;
}

Can you see what's wrong with the code?

It took me a few minutes of debugging and frustration. The bug is in the while loop conditional. Until you introduce serialization this code is just fine. ImmutableStack<T>.Empty is a static readonly declaration. The code implementation only allows for one to be created so it a singleton and equality can be done with a quick reference check.

private static readonly EmptyImmutableStack s_empty = new EmptyImmutableStack();

public static ImmutableStack<T> Empty {
    get { return s_empty; }
}

Unfortunately serialization breaks the assumption that EmptyImmutableStack is a singleton. The EmptyImmutableStack class is a singleton by convention only. It's a private nested class that's only instantiated once per AppDomain. There is nothing stopping the CLR or Serialization for that matter from creating a second instance. In the case of deserialization that's exactly what happens. The serializer isn't built to recognize this pattern and instead simply creates a new instance of EmptyImmutableStack upon deserialization.

This essentially prevents you from safely using a functional style Empty pattern inside a serializable collection.

The fix is simple enough, alter the conditional to be (!current.IsEmpty).

[1] The version of ImmutableStack I'm using is heavily based off of Eric Lippert's implementation.

Simple class for tests involving a SynchronizationContext

Jared Parsons — Mon, 25 Aug 2008 15:00:34 GMT

Recently I had to test a class which heavily depended upon a SynchronizationContext. This threw me off for about half an hour as I didn't want to write multi-threaded unit tests. Multi-threaded code is difficult enough without adding needless threads.

The solution I came up with is simple and gives the unit test a large degree of control over the execution of posted delegates. The resulting tests were much easier to code and understand.

public sealed class TestSynchronizationContext : SynchronizationContext {
    private List<Tuple<SendOrPostCallback, object>> m_pending 
        = new List<Tuple<SendOrPostCallback, object>>();

    public override void Send(SendOrPostCallback d, object state) {
        d(state);
    }

    public override void Post(SendOrPostCallback d, object state) {
        m_pending.Add(Tuple.Create(d, state));
    }

    public void RunAllPosted() {
        m_pending.ForEach(x => x.First(x.Second));
    }
}

ISynchronizeInvoke ... now

Jared Parsons — Tue, 08 Jan 2008 05:25:30 GMT

ISynchronizeInvoke is an interface which allows you to execute a delegate synchronously or asynchronously. The implementer of the interface can control how the delegate is executed. In particular the implementer controls on which thread the delegate is executed.

It's common for thread sensitive objects to implement this interface. It allows consumers to execute long lived actions on a background thread and then use the ISynchronizeInvoke interface to get back onto the original thread. System.Windows.Forms.Control is a great example of this usage pattern.

Several API's that I own are very thread aware and will often defer to a background thread for the completion of their operation. It makes it easier to build UI on top of it. They take an ISynchronizedInvoke as a parameter in order to properly signal the caller than an operation is completed.

The difficulty can come in testing it. Many implementers of ISynchronizeInvoke use message pumping to implement the behavior. As a result it's not always easy to use for testing (unit testing in particular). To work around this I designed an implementation of ISynchronizeInvoke that does not rely on the message pumping but provides a completely compliant ISynchronizeInvoke implementation.

The idea is to just do it ... now. I call the class ImmediateInvoke. The basic methods are straight forward.

        object ISynchronizeInvoke.Invoke(Delegate method, object[] args)
        {
            return method.DynamicInvoke(args);
        }

        bool ISynchronizeInvoke.InvokeRequired
        {
            get { return false; }
        }

The other two methods are a little more tricky. The require an implementation of IAsyncResult. The basic usage pattern is the consumer will call BeginInvoke, peform some operations and finally call EndInvoke when it wants to join with the asynchronous operation. I will use the thread pool to perform this operation and define a private nested class for the IAsnycResult implementation called AsyncResult

The implementation needs a few variables to implement the contract.

m_handle - An implementation of ManualResetEvent to satisfy the AsyncWaitHandle property
m_completed - A simple int to capture whether or not we have completed
m_return - Return of the delegate
m_exception - Exception thrown by calling the delegate.

Most of the properties are straight forward.

            public object AsyncState
            {
                get { return this; }
            }

            public System.Threading.WaitHandle AsyncWaitHandle
            {
                get { return m_handle; }
            }

            public bool CompletedSynchronously
            {
                get { return false; }
            }

            public bool IsCompleted
            {
                get { return m_completed == 1; }
            }

Now we need to define a method to run the delegate passed to BeginInvoke in the thread pool and update the state as we go along. I call this method directly from the constructor.

            private void RunDelegateAsync(Delegate method, object[] args)
            {
                WaitCallback del = delegate(object unused)
                {
                    try
                    {
                        object temp = method.DynamicInvoke(args);
                        Interlocked.Exchange(ref m_return, temp);
                    }
                    catch (Exception ex)
                    {
                        Interlocked.Exchange(ref m_exception, ex);
                    }

                    Interlocked.Exchange(ref m_completed, 1);
                    m_handle.Set();
                };

                ThreadPool.QueueUserWorkItem(del);
            }

Notice I've avoided using a lock in this implementation. This is safe for this case. All of the members are set atomically. Only m_completed can be accessed before the operation is completed and it is simply checked for the value 1. Since the value is set atomically this is safe. In the implementation of EndInvoke I will not access any of the other variables until the WaitHandle is signaled and then I will not make any decision based on the contents of their values (rather the abscence or presence).

Also notice that I did not explicitly dispose of the WaitHandle. This is a quirk of the ISynchronizeInvoke interface. It specifies that callers of BeginInvoke must call EndInvoke and that the IAsyncResult must be valid until EndInvoke is called. As such you can't really free a resource inside of the IAsyncResult implementation. In fact if you implement IDisposable, who will see it (generally not possible since C# and VB don't support covariant return types). Instead you should free it as part of your EndInvoke implementation.

Now BeginInvoke and EndInvoke.

        IAsyncResult ISynchronizeInvoke.BeginInvoke(Delegate method, object[] args)
        {
            return new AsyncResult(method, args);
        }

        object ISynchronizeInvoke.EndInvoke(IAsyncResult result)
        {
            var r = (AsyncResult)result;
            try
            {
                r.AsyncWaitHandle.WaitOne();
            }
            finally
            {
                r.AsyncWaitHandle.Close();
            }

            if (r.m_exception != null)
            {
                throw new Exception("Error during BeginInvoke", r.m_exception);
            }
            return r.m_return;
        }

Unfortunately EndInvoke has to take care of two cases. The first is the delegate completed successfully and produced a value which can now be returned as a part of the interface contract. The other case is when the delegate throws and exception and it's a bit more tricky. The exception cannot be simply re-thrown because you will loose all of the original call stack and generally speaking most of the data which would help track down the problem. The better option is to throw a new exception and make this exception the inner exception.

This sample could be improved in a few ways (delay create the WaitHandle, rethrow with something other than System.Exception). But it's a compliant version that gets the job done.

Testing your code

Jared Parsons — Thu, 27 Sep 2007 00:29:44 GMT

This is a little off topic from my typical purely technical discussion.

I'm a huge advocate of automated code verification. Unit testing is preferred but really any method is acceptable as long as it gets the job done of testing your code. IMHO, testing is the most important part of a developers job.

Besides the 3 or 4 projects I am doing as part of my job at Microsoft, I work on numerous side projects. Most of these come in the form of tools, libraries or little fun gadgets. These are written in a variety of languages; C#, VB, C++, Powershell Script, etc ...

This is pretty common amongst many devs I know. We love our job and code before, during and after work. Some of the devs I know don't bother automating testing of any of their tools. When asked they generally respond that it's not worth it.

I disagree strongly for certain types of tools. For tools and libraries I own that I expect long term usage on, I test methodically. On any given day I may update a couple of the tools. The only item that adds sanity to the situation is testing. This allows me to respond quickly to customer requests for features, bug fixes, etc and get quick verification that I haven't introduced a regression.