jaredpar's WebLog : Active Object

ActiveObject

Jared Parsons — Sun, 30 Mar 2008 19:39:28 GMT

I've been busy lately and neglected my series on Active Objects. It's been a fairly busy time for me both in and out of work. Enough excuses, back to the fun.

With the basic PipeSingleReader class, we now have the last piece necessary to create an ActiveObject. This article will focus on building the base ActiveObject which will take care of scheduling, construction, destruction and error handling. The goal is to make implementing an ActiveObject that actually does work easy.

Lets break down the implementation of an ActiveObject into the three phases of any object; construction, destruction and running behavior.

Construction

Active Objects are associated with and have an affinity to a particular thread. Constructing an ActiveObject mainly consists of creating a thread and initializing the member variables of the object.

There are a couple of requirements that need to be met when initializing the object. The first is getting the thread into a known state before returning out of the constructor. It's possible for coders to create and then immediately destroy an object. Part of destructing an object is understanding the state you are destructing. Returning from an ActiveObject constructor before the thread is up and running means that we can be destructed while in an inconsistent state. Normally this isn't much an issue with objects because they are single threaded. We will fix this by doing a simple wait until the thread is finished initializing.

        protected ActiveObject() {
            m_thread = new Thread(() => InitializeAndRunBackgroundThread());
            m_thread.Start();
            while (0 == m_backgroundInitialized) { Thread.Sleep(0); }
        }

Next is providing implementers with a way to initialize member variables on the new thread. There are many reasons for wanting to initialize members on the ActiveObject thread. Besides general consistency concerns, there is also the issue that objects can have affinity to a particular thread and including forcing initialization to occur on that thread. To make this simple part of the thread initialization code will call a virtual method allowing base classes to initialize variables.

        private void InitializeAndRunBackgroundThread() {
            Interlocked.Exchange(ref m_affinity, new ThreadAffinity());
            Interlocked.Exchange(ref m_pipe, new PipeSingleReader<Future>());
            InitializeMembersInBackground();
            Interlocked.Exchange(ref m_backgroundInitialized, 1);
            RunBackgroundActions();
        }

        protected virtual void InitializeMembersInBackground() {
        }

Running Behavior

Active Objects exist for one reason, to run Futures. The main behavior is to loop over the set of Futures and run them. The PipeSingleReader class takes care of most of the scheduling and threading work. This leaves the ActiveObject free to make policy decisions.

One question that comes up is how to handle the case where a Future throws an exception? If we run the Future with no protection it will simple cause an unhandled exception and likely a process crash. We could catch and try to filter them but based on what criteria? IMHO there is no way to properly handle an exception in the Active Object base because we don't know what the purpose of that object is. Only the actual object implementer knows. Therefore we will make it their problem by passing unhandled exceptions into an abstract method.

        private void RunBackgroundActions() {
            do {
                RunFuture(m_pipe.GetNextOutput());
            } while (0 == m_backgroundFinished);
            Future future;
            while (m_pipe.TryGetOutput(out future)) {
                RunFuture(future);
            }
        }
        private void RunFuture(Future future) {
            try {
                m_affinity.Check();
                future.Run();
            }
            catch (Exception ex) {
                OnBackgroundUnhandledException(ex);
            }
        }

If the second loop looks a bit out of place, hopefully the destruction section will explain it's purpose.

All that is left is to provide helper methods to let base classes queue up Futures to run.

        protected Future RunInBackground(Action action) {
            var f = Future.CreateNoRun(action);
            m_pipe.AddInput(f);
            return f;
        }
        protected Future<T> RunInBackground<T>(Func<T> func) {
            var f = Future.CreateNoRun(func);
            m_pipe.AddInput(f);
            return f;
        }

Destruction

Destruction of an ActiveObject can be tricky with respect to handling pending actions. Should they be executed, aborted or just completely ignored? What happens if more input is added once we start the dispose process? If we don't allow more input where should we error?

IMHO, the simplest user and programming model is the following.

Dispose is synchronous. It will block until the background thread is destroyed. Dispose is the equivalent of destruction so it follows that all resources including the thread will be destroyed when destruction completes.
Once dispose starts input will be stopped. This prevents live-lock scenarios where one thread is disposing the ActiveObject and another thread is constantly adding data.
If another thread tries to add an operation during the middle of disposing they will be given an exception at that time.

In future posts, we'll explore how to create ActiveObjects with differing dispose semantics.

Now how can we signal the background thread that we are done processing? Just add a future to the queue to be running. Because this will run on the only thread reading the int there is no need for an Interlocked operation.

        private void Dispose(bool disposing) {
            if (disposing) {
                m_pipe.AddInput(Future.CreateNoRun(() => { m_backgroundFinished = 1; }));
                m_pipe.CloseInput();
                m_thread.Join();
            }
        }

Now that we've gone over the dispose code, hopefully the reason for the second loop in RunBackgroundActions is a little more apparent. Between the two calls to m_pipe in Dispose another thread can post a Future. Without the second loop the user will get no exception and the future will never run. Likely they would hopelessly deadlock. The second loop will run all Futures which get caught it this gap.

The Code

Here is the full version of the code.

    public abstract class ActiveObject :  IDisposable {
        private PipeSingleReader<Future> m_pipe;
        private ThreadAffinity m_affinity;
        private Thread m_thread;
        private int m_backgroundInitialized;
        private int m_backgroundFinished;

        protected ActiveObject() {
            m_thread = new Thread(() => InitializeAndRunBackgroundThread());
            m_thread.Start();
            while (0 == m_backgroundInitialized) { Thread.Sleep(0); }
        }
        #region Dispose
        public void Dispose() {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
        ~ActiveObject() {
            Dispose(false);
        }
        private void Dispose(bool disposing) {
            if (disposing) {
                m_pipe.AddInput(Future.CreateNoRun(() => { m_backgroundFinished = 1; }));
                m_pipe.CloseInput();
                m_thread.Join();
            }
        }
        #endregion
        private void InitializeAndRunBackgroundThread() {
            Interlocked.Exchange(ref m_affinity, new ThreadAffinity());
            Interlocked.Exchange(ref m_pipe, new PipeSingleReader<Future>());
            InitializeMembersInBackground();
            Interlocked.Exchange(ref m_backgroundInitialized, 1);
            RunBackgroundActions();
        }
        private void RunBackgroundActions() {
            do {
                RunFuture(m_pipe.GetNextOutput());
            } while (0 == m_backgroundFinished);
            Future future;
            while (m_pipe.TryGetOutput(out future)) {
                RunFuture(future);
            }
        }
        private void RunFuture(Future future) {
            try {
                m_affinity.Check();
                future.Run();
            }
            catch (Exception ex) {
                OnBackgroundUnhandledException(ex);
            }
        }
        protected Future RunInBackground(Action action) {
            var f = Future.CreateNoRun(action);
            m_pipe.AddInput(f);
            return f;
        }
        protected Future<T> RunInBackground<T>(Func<T> func) {
            var f = Future.CreateNoRun(func);
            m_pipe.AddInput(f);
            return f;
        }
        protected virtual void InitializeMembersInBackground() {
        }
        protected abstract void OnBackgroundUnhandledException(Exception ex);
    }

PipeSingleReader

Jared Parsons — Sun, 02 Mar 2008 20:53:00 GMT

Before we can get to building an Active Object implementation, there are some more primitive structures we need to define. Active Objects live on a separate thread where every call is executed in a serialized fashion on that thread. The next primitive will allow us to easily pass messages in the form of delegates from the caller to the background thread.

The structure must support adding messages from N callers but only needs to support 1 reader (the active object). Successive writes from the same thread should add the items in their respective order. The input end of the pipe can be closed while leaving the output end active. This allows for the consumer to deterministically reach an end state while not ignoring any any input.

The name of the structure is PipeSingleReader.

Designing a thread safe mutable structure is significantly different from a non-thread safe or immutable data structure. There is a significant temptation to apply existing patterns. We should question every pattern we apply to these collections. Many of these patterns lead us to design API's which encourage bad programming practices.

Our existing collection patterns are designed around structures which behave deterministically on any given thread because they are 1) Immutable or 2) designed for single thread use only. Many of these concepts do not apply to mutable thread safe collections because they are constantly being accessed and mutated by multiple threads. This gives them the appearance of behaving non-deterministically with respect to a given thread.

Take the member Count for instance. This is found on virtually every collection class in the BCL. Yet having it on a mutable thread safe collection class only leads to programming errors. The only reason to have a member such is Count is to use it to make a decision. However in a mutable thread-safe collection making a decision off of this value is wrong. The value can and will change between any two instructions in code.

Take the example below. Just because the Count is >0 in the if block has no dependable relevance to what the value will be inside the if block.

            var ThreadSafeList<int> col = GetList();
            if( col.Count > 0 )
            {
                //...
            }

To guard against this and have users of the collections avoid the pit of failure members such as Count should not appear on mutable thread-safe collections.

Instead we'll design API's around the functionality of this structure. The input end of the structure is straight forward. All writers want is to add data.

The output end is more interesting. The end goal is to read output from the pipe but how to deal with cases where there is no data. Some programs will want to check for data, others block until data is available. We can define three methods to satisfy most scenarios

WaitForOutput - void method which blocks until input is available
GetNextOutput - Blocks until input is available and returns it
TryGetOutput - Returns immediately. If output is available it will be returned.

    public class PipeSingleReader<T> : IDisposable {
        private readonly ThreadAffinity m_affinity = new ThreadAffinity();
        private readonly Queue<T> m_queue = new Queue<T>();
        private readonly AutoResetEvent m_event = new AutoResetEvent(false);
        private readonly object m_lock = new object();
        private bool m_inputClosed;

        public PipeSingleReader() {
        }

        #region Dispose
        public void Dispose() {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
        ~PipeSingleReader() {
            Dispose(false);
        }
        private void Dispose(bool disposing) {
            if (disposing) {
                m_event.Close();
            }
        }
        #endregion
        public void WaitForOutput() {
            m_affinity.Check();
            do {
                lock (m_lock) {
                    if (m_queue.Count > 0) {
                        return;
                    }
                }
                m_event.WaitOne();
            } while (true);
        }
        public T GetNextOutput() {
            m_affinity.Check();
            T data;
            while (!TryGetOutput(out data))
            {
                m_event.WaitOne();
            } 
            return data;
        }
        public bool TryGetOutput(out T value) {
            m_affinity.Check();
            lock (m_lock) {
                if (m_queue.Count == 0) {
                    value = default(T);
                    return false;
                }

                value = m_queue.Dequeue();
                return true;
            }
        }
        public void AddInput(T value) {
            lock (m_lock) {
                if (m_inputClosed) {
                    throw new InvalidOperationException("Input end of pipe is closed");
                }
                m_queue.Enqueue(value);
            }
            m_event.Set();
        }
        public void CloseInput() {
            lock (m_lock) {
                m_inputClosed = true;
            }
        }
    }

At a quick glance it may seem odd in GetNextOutput that I loop around m_event being set and TryGetOutput. Why loop? Shouldn't a single check for the settness of m_event be enough? In this case no. The reason why is TryGetOutput will remove output from the queue without resetting the settness of m_event. Thus m_event can be set without actually having any data in m_queue. In general the implementation must treat m_event being set as the possibility of data rather than a guarantee.

This implementation uses locks to synchronize access to the data. In general I'm a fan of avoiding locks when possible since it's very easy to miss trivial cases. Next time we'll look at an implementation of PipeSingleReader which avoids the use of locks.

Active Objects and Futures

Jared Parsons — Tue, 29 Jan 2008 07:57:08 GMT

Herb Sutter gave one of my favorite and inspiring presentations. It is called "The Free Lunch is Over". The original article can be found here. My first encounter though came from his PDC presentation and highly recommend viewing that as well.

The part that interested me the most about the talk was two new threading abstractions I hadn't encountered before. Future's and ActiveObjects. One of the basic premise is that concurrency should be grep`able and somewhat declarative. The act of calling a method on a background thread and later waiting for it to complete should be simple, not complicated.

Asynchronous programming is one of my favorite aspects of computing. What interests me the most is how asynchronous programming can be useful for UI. My biggest pet peeve is when UI hangs because an operation call, or network operation takes too long. Why not make multi-threading easy and give users a way to cancel out of these operations??? Or start loading on the background thread instead of waiting for the user to perform a specific. Hopefully over the next month or so I'll lay out some utilities and classes building on Futures and Active Objects that will do precisely this.

Future's are actions where work can be done now, but the result is not needed until a future time. Work occurs on a separate thread and the results can be easily joined once work is complete.

            var f = Future.Create(() => LongCalculation());
            // ...
            var result = f.Wait();

Future's are now exposed via the Parallel Extension team's work. You can download the CTP off of their web site and get to work.

ActiveObjects are objects which only expose Asynchronous functions where the return value is exposed as a Future. So instead of

        string GetName()

You would have

        Future<string> GetName()

An ActiveObject essentially lives on or owns a thread. All operations are queued up and processed one at a time. Since only one action at a time can be executing the object internals don't have to use locks or consider many types of race conditions. In fact if your return types are immutable a great many threading concerns go out the window. Yet all of the calls are inherently asynchronous so callers can get the result only when they are needed. The best of both worlds.

Both of these provide significant advantages over the "lock before use" patterns. In my experience I find these to be hard to maintain and lead to difficult to track down bugs. I can't tell you how many times I've gone through someone else's code, or even my own, and wondered ...

Did they forget to lock here or is this an optimization?
Is a join needed here or can these terminate at separate times?
OK I need to touch that variable, can it be accessed in multiple threads or is it safe?

Futures/Active Objects on the other hand are a bit more declarative and straight forward to understand than plain old locks. They allow you to do away with many uses of plain old locking. Don't confuse this with me saying they are a cure all for threading. They're not. But in my experiences I've found them to be a significant upgrade.

Over the next month or so I'll be laying out the design for a basic ActiveObject implementation. We will likely have to deviate off of the Parrallel Extension work to get certain behaviors but the concepts map well.