• mwinkle.blog

    Different Execution Patterns with WF (or, Going beyond Sequential and State Machine)

    • 7 Comments

    How do I do this?

    image

    A lot of times people get stuck with the impression that there are only two workflow models available: sequential and state machine. True, out of the box these are the two that are built in, but only because there are is a set of common problems that map nicely into their execution semantics. As a result of these two being "in the box," I often see people doing a lot of very unnatural things in order to fit their problem into a certain model.

    The drawing above illustrates the flow of one such pattern. In this case, the customer wanted parallel execution with two branches ((1,3) and (2,5)). But, they had an additional factor that played in here, 4 could execute, but it could only execute when both 1 and 2 had completed. 4 didn't need to wait for 3 and 5 to finish, 3 and 5 could take a long period of time, so 4 could at least start once 1 and 2 were completed. Before we dive into a more simple solution, let's look at some of the ways they tried to solve the problem, in an attempt to use "what's in the box."

    The "While-polling" approach

    image

     

    The basic idea behind this approach is that we will use a parallel activity, and in the third branch we will place a while loop that loops on the condition of "if activity x is done" with a brief delay activity in there so that we are not busy polling. What's the downside to this approach:

    • The model is unnatural, and gets more awkward given the complexity of the process (what do we do if activity 7 has a dependency on 4 and 5)
    • The polling and waiting is just not an efficient way to solve the problem
    • This Is a lot to ask a developer to do in order to translate the representation she has in her head (first diagram, with the model we are forcing into).

    The SynchScope approach

    WF V1 does have the idea of synchronizing some execution by using the SynchronizationScope activity. The basic idea behind the SynchronizationScope is that one can specify a set of handles that the activity must be able to acquire before allowing it's contained activities to execute. This let's us serialize access and execution. We could use this to mimic some of the behavior that the polling is doing above. We will use sigma(x, y, z) to indicate the synchronization scope and it's handles (just because I don't get to use nearly as many Greek letters as I used to).

    image

    This should work, provided the synchronization scopes can obtain the handles in the "correct" or "intended" order. Again, the downside here is that this gets to be pretty complex, how do we model 4 having a dependency on 3 and 2? Well, our first synchronization scope now needs to extend to cover the whole left branch, and then it should work. For the simple case like the process map I drew at the beginning, this will probably work, but as the dependency map gets deeper, we are going to run into more problems trying to make this work.

    Creating a New Execution Pattern

    WF is intended to be a general purpose process engine, not just a sequential or state machine process engine. We can write our own process execution patterns by writing our own custom composite activity. Let's first describe what this needs to do:

    • Allow activities to be scheduled based on all of their dependent activities having executed.

      • We will start by writing a custom activity that has a property for expressing dependencies. A more robust implementation would use attached properties to push those down to any contained activity
    • Analyze the list of dependencies to determine which activities we can start executing (perhaps in parallel)
    • When any activity completes, check where we are at and if any dependencies are now satisfied. If they are, schedule those for execution.

    So, how do we go about doing this?

    Create a simple activity with a "Preconditions" property

    In the future, this will be any activity using an attached property, but I want to start small and focus on the execution logic. This one is a simple Activity with a "Preconditions" array of strings where the strings will be the names of the activities which must execute first:

    public partial class SampleWithPreconProperty: Activity
    {
        public SampleWithPreconProperty()
        {
            InitializeComponent();
        }
    
        private string[] preconditions = new string[0];
    
        public string[] Preconditions
        {
            get { return preconditions; }
            set { preconditions = value; }
        }
    

    Create the PreConditionExecutor Activity

    Let's first look at the declaration and the members:

    [Designer(typeof(SequentialActivityDesigner),typeof(IDesigner))]
    public partial class PreConditionExecutor : CompositeActivity
    {
        // this is a dictionary of the executed activities to be indexed via
        // activity name
        private Dictionary<string, bool> executedActivities = new Dictionary<string, bool>();
    
        // this is a dictionary of activities marked to execute (so we don't 
        // try to schedule the same activity twice)
        private Dictionary<string, bool> markedToExecuteActivities = new Dictionary<string, bool>();
    
        // dependency maps
        // currently dictionary<string, list<string>> that can be read as 
        // activity x has dependencies in list a, b, c
        // A more sophisticated implementation will use a graph object to track
        // execution paths and be able to check for completeness, loops, all
        // that fun graph theory stuff I haven't thought about in a while
        private Dictionary<string, List<string>> dependencyMap = new Dictionary<string, List<string>>();
    

    We have three dictionaries, one to track which have completed, one for which ones are scheduled for execution, and one to map the dependencies. As noted in the comments, a directed graph would be a better representation of this so that we could do some more sophisticated analysis on it.

    Now, let's look at the Execute method, the one that does all the work.

    protected override ActivityExecutionStatus Execute(ActivityExecutionContext executionContext)
    {
        if (0 == Activities.Count)
        {
            return ActivityExecutionStatus.Closed;
        }
        // loop through the activities and mark those that have no preconditions
        // as ok to execute and put those in the queue
        // also generate the graph which will determine future activity execution.
        foreach (Activity a in this.Activities)
        {
            // start with our basic activity
            SampleWithPreconProperty preconActivity = a as SampleWithPreconProperty;
            if (null == preconActivity)
            {
                throw new Exception("Not right now, we're not that fancy");
            }
            // construct the execution dictionary
            executedActivities.Add(a.Name, false);
            markedToExecuteActivities.Add(a.Name, false);
            List<string> actDependencies = new List<string>();
            if (null != preconActivity.Preconditions)
            {
                foreach (string s in preconActivity.Preconditions)
                {
                    actDependencies.Add(s);
                }
            }
            dependencyMap.Add(a.Name, actDependencies);
        }
        // now we have constructed our execution map and our dependency map
        // let's do something with those, like find those activities with
        // no dependencies and schedule those for execution.
        foreach (Activity a in this.Activities)
        {
            if (0 == dependencyMap[a.Name].Count)
            {
                Activity executeThis = this.Activities[a.Name];
                executeThis.Closed += currentlyExecutingActivity_Closed;
                markedToExecuteActivities[a.Name] = true;
                executionContext.ExecuteActivity(this.Activities[a.Name]);
                Console.WriteLine("Scheduled: {0}", a.Name);
            }
        }
        return ActivityExecutionStatus.Executing;
    }

    Basically, we first construct the execution tracking dictionaries, initializing those to false. We then create the dictionary of dependencies. We then loop through the activities and see if there are any that have no dependencies (there has to be one, this would be a good point to raise an exception if there isn't. We record in the dictionary that this one has been marked to execute and then we schedule it for execution (after hooking the Closed event so that we can do some more work later). So what happens when we close?

    void currentlyExecutingActivity_Closed(object sender, ActivityExecutionStatusChangedEventArgs e)
    {
        e.Activity.Closed -= this.currentlyExecutingActivity_Closed;
        if (this.ExecutionStatus == ActivityExecutionStatus.Canceling)
        {
            ActivityExecutionContext context = sender as ActivityExecutionContext;
            context.CloseActivity();
        }
        else if (this.ExecutionStatus == ActivityExecutionStatus.Executing)
        {
            // set the Executed Dictionary
            executedActivities[e.Activity.Name] = true;
            if (executedActivities.ContainsValue(false) /* keep going */)
            {
                // find all the activities that contain the precondition 
                // and remove them, and then cycle through any that have 0 
                // preconditions (and have not already executed or are marked
                // to execute
                // who contains this precondition? 
                foreach (Activity a in this.Activities)
                {
                    // filter out those activities executed or executing
                    if (!(executedActivities[a.Name] || markedToExecuteActivities[a.Name]))
                    {
                        if (dependencyMap[a.Name].Contains(e.Activity.Name))
                        {
                            // we found it, remove it
                            dependencyMap[a.Name].Remove(e.Activity.Name);
                            // if we now have no dependencies, let's schedule it
                            if (0 == dependencyMap[a.Name].Count)
                            {
                                a.Closed += currentlyExecutingActivity_Closed;
                                ActivityExecutionContext context = sender as ActivityExecutionContext;
                                markedToExecuteActivities[a.Name] = true;
                                context.ExecuteActivity(a);
                                Console.WriteLine("Scheduled: {0}", a.Name);
                            }
                        }
                    }
                }
            }
            else //close activity
            {
                ActivityExecutionContext context = sender as ActivityExecutionContext;
                context.CloseActivity();
            }
        }
    }

     

    There are a few lines of code here, but it's pretty simple what's going on.

    • We remove the event handler
    • If we're still executing, mark the list of activities appropriately
    • Loop through and see if any of them have dependencies on the activity which just now completed being done
    • If they do, remove that entry from the dependency list and check if we can run it (if the count == 0). If we can, schedule it, otherwise keep looping.
    • If all the activities have completed (there is no false in the Executed list) then we will close out this activity.

    To actually use this activity, we place it in the workflow, place a number of the child activity types within it (again, with the attached property, you could put nearly any activity in there) and specify the activities that it depends on. Since I haven't put a designer on it, I just use the SequenceDesigner. Here's what it looks like (this is like the graph I drew above but kicks off with the "one" activity executing first:

    image

     

    Where can we go from here

    • Validation, remember all that fun graph theory stuff checking for cycles and completeness and no gaps. Yeah, we should probably wire some of that stuff up here to make sure we can actually execute this thing
    • Analysis of this might be interesting, especially as the process gets more complex (identifying complex dependencies, places for optimization, capacity stuff)
    • A designer to actually do all of this automatically. Right now, it is left as an exercise to the developer to express the dependencies by way of the properties. It would be nice to have a designer that would figure that out for you, and also validate so you don't try to do the impossible.
    • Make this much more dynamic and pull in the preconditions and generate the context for the activities on the fly.  This would be cool if you had a standard "approval" activity that you wanted to have a more configurable execution pattern.  You could build the graph through the designer and then use that to drive the execution

    I'm going to hold off on posting the code, as I've got a few of these and I'd like to come up with some way to put them out there that would make it easy to get to them and use them. You should be able to pretty easily construct your own activity based on the code presented here.

  • mwinkle.blog

    Paste XML as Serializable Type

    • 3 Comments

    Every now and then, there's a really cool feature that's buried somewhere that just hits you and makes you say "Wow, that's insanely helpful, why didn't somebody think of this sooner." 

    I was playing around with the BizTalk Services SDK, specifically the different web programming bits and pieces and stumbled up on the Paste As Serializable visual studio add in (navigate to BizTalk Services SDK\Samples\Web\Tooling). 

    Let's say you're interested in programming against some services that return normal, plain XML. Well, nobody likes writing code to query XML, Linq to XML is fun, but I'd just really like to mess around with some objects and not really deal with how it looks over the wire or under the hood.

    This great little utility lets you select some snippet of XML (say I get it from here, because I am working on a Twitter mashup).  Copy the XML to the clipboard (here's a snippet of what the XML looks like).

    <statuses>
      <status>
        <created_at>Wed May 02 16:54:45 +0000 2007</created_at>
        <id>47434042</id>
        <text>has advil &amp; beineigts in the back of Steve and Don's talk in 4101B.  Anybody needing either is welcome to some, the pastries will go fast.</text>
        <user>
          <id>5440022</id>
          <name>Matt Winkler</name>
          <screen_name>mwinkle</screen_name>
          <location>Seattle</location>
          <description>workflow technical evangelist</description>
          <profile_image_url>http://assets2.twitter.com/system/user/profile_image/5440022/normal/Matt_Winkler-72dpi.jpg?1177350091</profile_image_url>
          <url>http://blogs.msdn.com/mwinkle</url>
          <protected>false</protected>
        </user>
      </status>

    Go into Visual Studio into a code file and open up the edit menu.

    image

    There, clearly indicated by the "this is just an alpha icon," is the option to "Paste XML as Serializable Type".  Here's what it outputs:

       [System.Xml.Serialization.XmlRootAttribute(Namespace = "", ElementName = "statuses")]
        public class StatusesClass
        {
    
            private System.Collections.Generic.List<StatusClass> statusField;
    
            [System.Xml.Serialization.XmlElementAttribute(Namespace = "", ElementName = "status")]
            public virtual System.Collections.Generic.List<StatusClass> Status
            {
                get
                {
                    return this.statusField;
                }
                set
                {
                    this.statusField = value;
                }
            }
    
            [System.Xml.Serialization.XmlRootAttribute(Namespace = "", ElementName = "status")]
            public class StatusClass
            {
    
                private string created_atField;
                private string idField;
                private string textField;
                private UserClass userField;
                [System.Xml.Serialization.XmlElementAttribute(Namespace = "", ElementName = "created_at")]
                public virtual string Created_at
                {
                    get
                    {
                        return this.created_atField;
                    }
                    set
                    {
                        this.created_atField = value;
                    }
                }
    

    ...

     

    Remainder of code truncated for the sake of the readers. 

    This lets me do some cool stuff like this:

    static void Main(string[] args)
    {
        WebHttpClient twc = new WebHttpClient();
        twc.UriTemplate = "http://twitter.com/statuses/user_timeline/mwinkle.xml";
        StatusesClass sc = twc.Get().GetBody<StatusesClass>();
        Console.WriteLine("Got {0}", sc.Status.Count);
        
    }

    And get my result back in a nice typed object that I can then use elsewhere in my code (or at least debug)

     

    image

    Steve, thanks, this thing rocks!

Page 1 of 1 (2 items)