mwinkle.blog

Microsoft HDInsight Installation & Dependency Management

Matt Winkler -- MSFT — Sun, 16 Dec 2012 21:58:11 GMT

It’s a rainy Saturday afternoon here in Seattle, and the kids are keeping themselves busy running around the Christmas tree, so I’ve got a little time to put together a post that addresses some questions that have come up a few times in the forums as well as in our internal discussion aliases for our on-premises install of HDInisght.

We currently use the Web Platform Installer to take care of dependency management of the installation.

It’s important to point out that there are actually two key pieces that get installed

The installation of the Hortonworks Data Platform for Windows
The installation for Microsoft HDInsight

The second has a dependency on the first. The WebPI feed also contains a number of pre-requisites required to set up IIS and a few other things for the HDInsight dashboard. Here’s what it looks like on a completely fresh Windows Server 2012 machine. Most developer machines likely have some or most of the IIS pre-reqs installed. We’re also working to clean up some of this to minimize installation & setup.

Let’s talk a little bit about what’s in each one.

Hortonworks Data Platform installer

This msi includes the core Hadoop bits (Map/Reduce, HDFS), as well as a number of other Apache projects in the Hadoop ecosystem. The full list included in the current installer are:

Map Reduce
HDFS
Hive
Pig
HCatalog

Each of these projects is packaged into a zip file that contains a PowerShell script that automates the installation and setup of the component. There are more components in Hortonworks Data Platform, and the teams are working to get these packaged and included.

Microsoft HDInisght Installer

This msi contains bits that are Microsoft specific, and may also contain additional Hadoop projects. The current install (as of today) contains:

HDInsight dashboard
Sqoop
Isotope.js
Getting Started content

These are packaged the same way as the Hadoop projects above. Additionally, there is an installation PowerShell script here which will do some initialization of the single node installer, such as starting the services for the Hadoop components.

Alternate Approaches

The team discussed a number of potential factorings, and we very much welcome feedback here. A few ideas that we’ve thought about:

Stable and Experimental packages. This would allow us to set expectations around quality and stability of the bits.
Decomposing every project into an individual msi
Integrating and building a Chocolatey package for these

What Does This Mean For Me?

What this means is that when you install HDInsight out of WebPI, you are installing two different msi’s. We are revving the Microsoft msi every two weeks to pull in bug fixes (and very shortly include some experimental features). The Hortonworks msi will be revved on a different schedule, as the team there decides to release an update. We are partnering closely with the team there and so we will coordinate releases so that the combined installation will always work.

More directly, this implies that if you want to uninstall completely, you will need to uninstall both packages from Add/Remove Programs:

This also means that when we issue an update for the Microsoft HDInsight package, you don’t have to “lose” your cluster by uninstalling both products. You should be able to simply uninstall & update the Microsoft HDInsight package.

The team would love to get more feedback on this approach, so, let us know what you think!

Updating HDInsight Preview

Matt Winkler -- MSFT — Tue, 04 Dec 2012 16:50:24 GMT

Today we’ve shipped an update to the single node HDInsight Server Preview that is installable via Web Platform Installer.

What Are We Doing?

Every two weeks, we’re going to take a snapshot of the work in progress and update the public installer with this (provided it passes a basic level of validation). This gives us the opportunity to rapidly get new bits in front of customers, experiment with new features, and address bugs in a timely fashion.

We’ve also snapped versions such that releases of the SDK as well as the One-Box installer will all share the same version. As such, we’ve versioned the msi as 0.2.0.0.

How do I Get This?

Simply by installing HDInisght out of WebPI onto a new machine after uninstalling (in place upgrade is not currently supported). To completely uninstall previous versions, you will want first uninstall “Microsoft HDInsight Community Technology Preview” and then the “Hortonworks Data Platform”. We will not preserve data in HDFS using uninstall/reinstall, so archive that prior to uninstalling.

What’s New?

There are no new features in this release, we’ve worked to improve some of the install and setup experience to address some bugs that have been reported on this alias as well as the forums.

IIS / Dashboard setup issues experienced on certain OS SKU’s
Addition of HDInsight Dashboard link into Start Menu
Fix to Hive console for multiple line input errors
Clean up some uninstall issues
Minor changes to getting started content

Hortonworks has also shipped an update to the HDP installation to address the bug encountered on Friday (documented here). By removing both installations above and re-installing, you will get updated bits here.

What if Something Doesn’t Work?

Please report issues on the forums. We’re also tracking a set of known issues via the release notes.

When Will These Bits Be Updated In Azure?

We’re using this as an opportunity to ship these bits early and often, and they will find their way into future service updates to Azure HDInsight, typically in the next monthly update.

What’s Next?

We’ll continue to address common issues as they are reported. From a feature perspective, we hope the next update will include updates to the dashboard to include a new and improved Hive console.

Getting Up and Running with Piggybank on HDInsight

Matt Winkler -- MSFT — Sat, 03 Nov 2012 00:17:41 GMT

It’s been a fun couple of weeks launching HDInsight, and I’m going to be getting back into doing some more technical blogging. There are a few easy topics off the bat that we’ve heard requested from customers. The first one involves Piggybank, which is a user contributed collection of useful Pig user defined functions (UDF’s).

This assumes your machine is set up with:

HDInsight (grab from WebPI here)
Java build tools (Ant and Ivy on your command path)

Next, let’s build Piggybank by grabbing the Pig source code and checking out the 0.9 branch

git clone https://github.com/apache/pig.git
git checkout -b branch-0.9 remotes/origin/branch-0.9

At this point, you should now have a pig directory, move to that and type ant in order to build.

Next, navigate to .\contrib\piggybank\java, and again, type ant in order to build. This will produce piggybank.jar.

Next, open your HDInsight console window and type pig. This brings up Grunt, the interactive pig shell.

At this point, you can now use the following in your script:

REGISTER C:\Your\Path\To\piggybank.jar ;

foo = FOREACH entry GENERATE org.apache.pig.piggybank.evaluation.string.UPPER(item_name);

At this point, you can now take advantage of all of the functions in piggybank, and if you’re interested in contributing your own, details are here.

Every Day is a Good Day When You Paint

Matt Winkler -- MSFT — Mon, 30 Jul 2012 16:37:57 GMT

Slightly off-topic, but a friend of mine posted a video on Facebook that struck me as being very relevant to our space.

At the same time I saw this, another friend and co-worker was having a fairly crummy day with lots of crazy meetings and requests from co-workers, but that day was made a lot better by sitting down and solving a fun little deployment script problem. The deployment script is neither here nor there, it was more that so many of the folks in this industry love solving a problem, whether that is getting a query right, chasing down a pesky bug, or getting that a-ha moment when you are trying to design something. That’s “painting” for a lot of us.

Every day is a good day when you paint.

[and working auto-tuned Bob Ross into a blog post was just too good to pass up]

Writing a Map/Reduce Job for Hadoop using Windows PowerShell

Matt Winkler -- MSFT — Sat, 21 Jul 2012 01:25:21 GMT

I had a little bit of time on my hand and wanted to whip up a quick sample using PowerShell for a Hadoop job.

This uses the Hadoop streaming capability, which essentially allows for mappers and reducers to be written as arbitrary executables that operate on standard input and output.

The .ps1 scripts are pretty simple, these operate over a set of airline data that looks like this:

0,EV,18,0,MIA,2011-12-18T00:00:00,12,TPA,1227859,2011    
0,EV,17,0,MIA,2011-12-17T00:00:00,12,TPA,1227860,2011    
6,EV,17,4,CLE,2011-12-17T00:00:00,12,ATL,1227861,2011

The schema here is a comma separated set of US flights with delays.

The goal of my job is to pull out the airlines, the number of flights, and then some very basic (min and max) statistics on the arrival and departure delays.

My mapper:

   1:  function Map-Airline

   2:  {

   3:      [Console]::Error.WriteLine( "reporter:counter:powershell,invocations,1")

   4:      $line = [Console]::ReadLine()

   5:      while ($line -ne $null)

   6:      {

   7:          [Console]::WriteLine($line.Split(",")[1] + "`t" + $line)

   8:          [Console]::Error.WriteLine("reporter:counter:powershell,record,1")

   9:          $line = [Console]::ReadLine()

  10:      }

  11:  }

12:

13:

  14:  Map-Airline

My reducer:

   1:  function Reduce-Airlines

   2:  {

   3:      $line = ""

   4:      $oldLine = "<initial invalid row value>"

   5:      $count = 0

   6:      $minArrDelay = 10000

   7:      $maxArrDelay = 0

   8:      $minDepDelay = 10000

   9:      $maxDepDelay = 0

10:

  11:      $line = [Console]::ReadLine()

12:

  13:      while  ($line -ne $null)

  14:      {

  15:          if (($oldLine -eq $line.Split("`t")[0]) -or ($oldLine -eq "<initial invalid row value>"))

  16:          {

  17:              $flightRecord = $line.Split("`t")[1].Split(',')

  18:              if ([Int32]::Parse($flightRecord[0]) -ne 0)

  19:              {

  20:                  $minArrDelay = [Math]::Min($minArrDelay, $flightRecord[0])

  21:              }

  22:              if ([Int32]::Parse($flightRecord[3]) -ne 0)

  23:              {

  24:                  $minDepDelay = [Math]::Min($minDepDelay, [Int32]::Parse($flightRecord[3]))

  25:              }

  26:              $maxArrDelay = [Math]::Max($maxArrDelay, $flightRecord[0])

  27:              $maxDepDelay = [Math]::Max($maxDepDelay, $flightRecord[3])

  28:              $count = $count+ 1

  29:              [Console]::Error.WriteLine( "reporter:counter:powershell,"+$oldLine + ",1")

  30:          }

  31:          else

  32:          {

  33:              [Console]::WriteLine($oldLine + "`t" + $count + "," + $minArrDelay + "," +$maxArrDelay + "," + $minDepDelay +","+ $maxDepDelay)

  34:              $count = 1

  35:              $minArrDelay = 10000

  36:              $maxArrDelay = 0

  37:              $minDepDelay = 10000

  38:              $maxDepDelay = 0

  39:              [Console]::Error.WriteLine("reporter:counter:powershell,"+$oldLine + ",1")

  40:          }

  41:          $oldLine = $line.Split("`t")[0]

  42:          $line = [Console]::ReadLine()

  43:      }

  44:      [Console]::WriteLine($oldLine + "`t" + $count + "," + $minArrDelay + "," +$maxArrDelay + "," + $minDepDelay +","+ $maxDepDelay)

  45:  }

46:

  47:  Reduce-Airlines

One thing to note on the reducer is that we use the $oldLine variable in order to keep tabs on when our group of results is moving to the next one. When using Java, your reduce function will be invoked once per group and so you can reset the state at the beginning of each of those. With streaming, you will never have groups that split reducers, but your executable will only be spun up once per reducer (which, in the sample here, is one). You can also see that I’m writing out to STDERR in order to get a few counters recorded as well.

The next trick is to get these to execute. The process spawned by the Streaming job does not know about .ps1 files, it’s basically just cmd.exe. To get around that we will also create a small driver .cmd file and upload the file with the –file directive from the command line.

AirlineMapperPs.cmd

@call c:\windows\system32\WindowsPowerShell\v1.0\powershell -file ..\..\jars\AirlineMapper.ps1

AirlineReducerPs.cmd

@call c:\windows\system32\WindowsPowerShell\v1.0\powershell -file ..\..\jars\AirlineReducer.ps1

The ..\..\jars directory is where the –file directive will place the files when they execute

And now we execute:

hadoop jar %HADOOP_HOME%\lib\hadoop-streaming.jar 
    -mapper d:\dev\_full_path_\AirlineMapperPs.cmd 
    -reducer d:\dev\_full_path_\AirlineReducerPs.cmd 
    -input fixed_flights 
    -output psMapReduce3 
    -file d:\dev\_full_path_\AirlineMapper.ps1 
    -file d:\dev\_full_path_\AirlineReducer.ps1

And we get our results.

There is still some work to be done here, I’d like to make it a little easier to get these running (so possibly wrapping submission in a script which takes care of the wrapping for me). Also, on Azure, we either need to sign the scripts, or log into each of the machines and allow script execution. As I get that wrapped up, I’ll drop it along side our other samples. We’ll also work to make this easier to get up and running on Azure if that’s interesting for folks.

Talking About Hadoop on Windows

Matt Winkler -- MSFT — Sat, 09 Jun 2012 17:20:00 GMT

A few folks have asked, so I decided to put the data in one place. We'll be talking more about Hadoop at TechEd North America and Hadoop Summit next week, and then later in the month at TechEd Europe.

Here are the sessions we're presenting:

TechEd (NA and Europe, links are to NA sessions)

Learn Big Data Application Development on Windows Azure -- Wenming Ye

Web 2.0 companies have been fully taking advantage of Hadoop based open source tools to tackle Big Data needs. Microsoft now offers the best of both worlds with its own Hadoop solution on Windows Azure with full compatibility and additional rich toolsets. This session is a "getting-started" tutorial on developing Big Data applications on Windows Azure. We cover application scenarios, Hadoop on Azure, tools, and applied data analytics. More importantly, we show you how to put everything together with a couple of sample applications

Big Data, Big Deal? -- Gert Drapers

Are you ready for the exploding world of big data? Do you know the difference between Hive and Pig? Do you know why MapReduce is being taught in many universities rather than SQL? If not, pay attention because this talk will help get you started in understanding this new world. While sometimes the Hadoop toolkit (which includes HDFS, MapReduce, Hive, Pig, and Sqoop) is used as an alternative to relational database systems such as SQL Server, more frequently customers are using it as a complementary tool. Sometimes it may be used as an ETL tool or to perform an initial analysis of a freshly acquired data set to determine whether or not it is worth loading into the data warehouse, and sometimes to process massive data sets that are too big to even contemplate loading into all but the very largest data warehouses. In addition to covering the basics of the various parts of the Hadoop stack, this talk discusses the strengths and weakness of the Hadoop approach compared to that provided by relational database systems and explores how the two technologies can be used productively in conjunction with one another.

Harnessing Big Data With Hadoop

Attend this session to learn about the Hadoop Big Data solution from Microsoft that unlocks insights on all your data, including structured and unstructured data of any size. Accelerate your analytics with a Hadoop service that offers integration with Microsoft BI and the ability to enrich your models with publicly available data. Finally, learn about our roadmap for Hadoop on Windows Server and Windows Azure and for broadening access to Hadoop through simplified deployment, management and programming, including JavaScript integration

Hadoop Summit

Unleash Insights On All Data With Microsoft Big Data -- Tim Mallalieu

Do you plan to extract insights from mountains of data, including unstructured data that is growing faster than ever? Attend this session to learn about Microsoft’s Big Data solution that unlocks insights on all your data, including structured and unstructured data of any size. Accelerate your analytics with a Hadoop service that offers deep integration with Microsoft BI and the ability to enrich your models with publicly available data from outside your firewall. Come and see how Microsoft is broadening access to Hadoop through dramatically simplified deployment, management and programming, including full support for JavaScript.

How Klout is changing the landscape of social media with Hadoop and BI -- Denny Lee, David Mariani (VP of Engineering, Klout)

In this age of Big Data, data volumes grow exceedingly larger while the technical problems and business scenarios become more complex. Compounding these complexities, data consumers are demanding faster analysis to common business questions asked of their Big Data. This session provides concrete examples of how to address this challenge. We will highlight the use of Big Data technologies—including Hadoop and Hive —with classic BI systems such as SQL Server Analysis Services.

Session takeaways:
• Understand the architectural components surrounding Hadoop, Hive, Classic BI, and the Tier-1 BI ecosystem
• Get strategies for addressing the technical issues when working with extremely large cubes
• See how to address the technical issues when working with Big Data systems from the DBA perspective

I think that there is a pretty nice mix of Hadoop, Microsoft plans, and applications across these sessions. Hope you get a chance to see them (or watch them after the events!)

Tap, tap, tap, is this thing on?

Matt Winkler -- MSFT — Tue, 08 May 2012 21:25:53 GMT

I'll be back to bloging here in a bit. Since you last visited, I've had a fun journey working further on Workflow, Azure and now I've landed in the SQL team working on Hadoop, in particular, the developer story.

Not a lot to say right now, but make sure to check out hadoopOnAzure.com !

Thoughts on Flowchart

Matt Winkler -- MSFT — Fri, 13 Aug 2010 22:45:26 GMT

Last night I saw that Maurice had a few tweets that caught my attention about flowchart, but this is the one that I want to talk about:

Come to think of it I also really mis a parallel execution in a flowchart. But other than that flowcharts rock! #wf4 #dmabout 13 hours ago via web

I thought about replying on twitter, but it’s a bit of a longer topic. When we were building the flowchart for VS2010, we considered a number of possible capabilities, but ultimately the capabilities customers were looking for fell into two categories:

Arbitrary rework patterns
Coordination patterns

Arbitrary Rework

We can describe the model of execution in the current flowchart to be one that supports arbitrary rework. You could also refer to this as GOTO. One benefit (and downside) of breaking out of the tightly scoped, hierarchical execution model that we see with the Sequential style of workflows (and in most procedural languages) is the fact there exists more freedom in terms of defining the “next” step to be taken. The simplest way to think about this is the following flowchart:

Now, this isn’t particularly interesting, and most folks who look at this will simply ask “why not a loop?” which in this case is a valid question. As a process gets more sophisticated (or if humans tend to be involved), the number of possible loops required gets rather tricky to manage in a sequential flow (consider the following rework scenario which includes “next steps” across conditional boundaries, and from some branches but not others.

Now, mathematically, we can describe the machinery in both of these machines as a set of valid transitions, and it is likely possible that there exists a complete mapping from any flowchart into procedural.

Coordination

The second pattern we consistently saw was the use of a free form style of execution in order to build complex coordination patterns. The example I consistently return to (pointing back to a blog post I made back in the WF3 days)

Here I want to be able to coordinate a few different things, namely that 3 executes when 1 completes, 5 when 2, 4 when 1 AND 2, 6 when 3 AND 4 AND 5 complete. Here we’ve created a dependency graph that can’t be handled with the procedural constructs that we have. How could this happen? Imagine we’re trying to model the ranking process for a series of horse races. Task 3 can happen when Race 1 completes, as Task 5 can happen when Race 2 completes. Task 4 represents some work that requires the data from both Races. When those 3 tasks (3, 4, and 5) complete, I can move forward and take some action (like bet on the next race).

This type of pattern can be very powerful, and is often described by petri-nets. There exists a multitude of join semantics from simple AND joins to conditional joins (ranging from a simple “count” to a condition based on data).

How’d we get to where we are today?

Today, the flowchart in the .NET framework only supports the former pattern, that of arbitrary rework. How’d we get there. While we found a lot of value in both patterns what we found when we tried to compose them, we often got into places that became very hard to have predictable execution semantics. The basic crux of the design issue gets to the “re-arming” of a join. If I decide to jump back to task 3 at some point, what does it mean for 3 to complete? Do I execute 6 again right away, do I wait for 4 and 5 to complete a second time? What happens if I jump to 3 a second time? Now, there certainly exist formal models for this, and things like Petri-net define a set of semantics for these things. What we found though was that any expression of the model that took these things into account required a pretty deep understanding of the process model, and we lost one of the key benefits we see from a flowchart, which is that it is a simple, approachable model. This is not to say that we don’t think that the Coordination pattern is useful, or that petri-nets don’t ultimately hold the unified approach, we just did not have the time to implement a coordination pattern in VS 2010, and creating an approachable petri-net backed flowchart modeling style was something we’d need more time to get correct.

Where does this leave us?

If you’re looking at this and saying, “but I want coordination (or a full blown petri-net)”, there are a couple of options:

Roll up your sleeves and write that activity. In the blog post above, I outline how you could go about building the coordination pattern, and with the new activity model, this would be simpler. The harder part is still writing a designer, and there is now some good guidance on free form designers in the source of the state machine on codeplex.
Provide feedback on connect, when you do this it opens a bug in our bug database that our team is looking at daily. Describe your scenario, what you’re trying to do, and what you’d like to see. This type of customer feedback is vital to helping us plan the right features for the next release.

Improving ModelItem with Dynamic

Matt Winkler -- MSFT — Wed, 11 Aug 2010 17:22:40 GMT

Hi all, I’m back, and apologize for the delay, things have been crazy here and I’ve taken a little time off following shipping VS2010, so I’m getting back into it. I’d like to talk about one of my favorite late additions to the WF4 designer programming model. We added this change sometime in the RC/RTM timeframe in response to some feedback that we had gotten about the verbose-ness of programming against ModelItem.

If you recall from this post, ModelItem exists to serve as a INotifyPropertyChanged (INPC) aware proxy to the underlying instance being edited within the designer. In exchange for this fairly flexible proxy layer, you get in return a pretty verbose way in which one has to interact to actually get to the data. Often times there are fairly long incantations of myModelItem.Properties[“foo”].Value.Properties[“bar”].Value.GetCurrentValue() in order to actually return the data (or be able to set the value). From the beginning, we had decided we wanted to optimize for the WPF XAML consumer of the model item, and so we chose to implement ICustomTypeDescriptor so that WPF binding would be pretty simple (the above would result in a binding syntax of ModelItem.foo.bar).

One quick note, looking at the definition of ModelItem itself will show that only the INPC interface is implemented. However, ModelItem is an abstract type, and the implementation subclass ModelItemImpl contains the definition and implementation of the following interfaces ModelItem, IModelTreeItem, ICustomTypeDescriptor, IDynamicMetaObjectProvider

As we worked with some of our customers, it became clear that they were frequently using ModelItem in code as well, and so the programming model was becoming more of a problem (we had somewhat hoped that the majority of customers would be using XAML). We initially went with the idea that we would create a series of Design time peers to all of the common types in WF (making a SequenceDesign, ParallelDesign, etc) which provided an identical programming surface to its peer, but under the covers in the property getters and setters would redirect to an underlying ModelItem. This has the advantage that we could create the appearance of a strongly typed API surface, but has the downside that it introduces an extra type for every activity we ship, and it requires customers to create two types as well, and not just for activities, you would want one for customer too. The other approach we kicked around was using the Dynamic capabilities introduced in .NET 4.

It’s important to note that these are ultimately complimentary approaches, that is, it’s possible in a future framework release (or in a codeplex project if anyone is looking to for a little fun on the side) you could release the strongly typed design time peers. From a timing perspective, it was not likely that we could introduce 30+ new types in the RC/RTM release, and we had questions about the long term sustainability of requiring an additional type for activities. So, we went ahead with the plan for dynamic support. In order to do this, we had to add an implementation of IDynamicMetaObjectProvider. Bill Wagner has a good article here on the basics of doing this. This requires us to implement one method, GetMetaObject. In our case, we have an internal, nested ModelItemMetaObject type which inherits from System.Dynamic.DynamicMetaObject.

In our case, there are basically two important methods that we care about, BindGetMember and BindSetMember. These wire up to two methods on ModelItemImpl to get and set based on the property name. The net result of this is that the following code

root.Properties["Residence"].
        Value.
        Properties["StreetAddress"].
        Value.GetCurrentValue()

Can now be written as

dynamic mi = root;
Console.WriteLine(root.Residence.StreetAddress)

Now, with anything there are some tradeoffs, the biggest one is that this is dynamic, I don’t get compile time checking, and I could have a type that will result in an exception being thrown (that said, that problem exists in the verbose form as well). This does let me write much more succinct code when programming the model item tree though for both gets and sets. As an added bonus, we found there is actually a decent perf improvement from the WPF XAML data binding that now leverages the IDynamicMetaObjectProvider (mentioned in ScottGu’s post here) in the way that the runtime will only compute the way to resolve the property once (as opposed to every time when it goes through the ICustomTypeDescriptor code path).

Enjoy!

VS 2010 RC Is Available

Matt Winkler -- MSFT — Tue, 09 Feb 2010 22:09:36 GMT

We’ve been hard at work here in Redmond (and with our team in Shanghai) working on getting WF ready for release. We’ve made a ton of progress in the RC build that was made available yesterday, please download it and check it out. Also, and in important bold text, if you have feedback, please, please, please file an issue on Connect so that the team can look at it right away. One thing that I want to point out about Connect is that it is not a vacuum, the entries there go directly into our bug tracking system, which we look at daily in our various shiproom meetings. If something isn’t working right, please let us know!

We’ve fixed a number of issues that came directly from Connect in this last milestone, including some areas that revealed some gaps in our testing. Your feedback is making this better. For bugs that we’ve marked as postponed, as we start planning for vNext, we will start going through those to figure out the areas we need to improve on.

It’s cool the home stretch, this morning I presented to our support team to get them prepared to handle any PSS incidents that occur. It’s great to see the release coming together.

Making Swiss Cheese Look Good, or Designers for ActivityAction in WF4

Matt Winkler -- MSFT — Fri, 15 Jan 2010 00:06:54 GMT

In my last post, I covered using ActivityAction in order to provide a schematized callback, or hole, for the consumers of your activity to supply. What I didn’t cover, and what I intend to here, is how to create a designer for that.

If you’ve been following along, or have written a few designers using WorkflowItemPresenter, you may have a good idea how we might go about solving this. There are a few gotcha’s along the way that we’ll cover as we go through this.

First, let’s familiarize ourselves with the Timer example in the previous post:

using System;

using System.Activities;

using System.Diagnostics;

namespace WorkflowActivitiesAndHost

    public sealed class TimerWithAction : NativeActivity<TimeSpan>

        public Activity Body { get; set; }

        public Variable<Stopwatch> Stopwatch { get; set; }

        public ActivityAction<TimeSpan> OnCompletion { get; set; }

        public TimerWithAction()

            Stopwatch = new Variable<Stopwatch>();

        protected override void CacheMetadata(NativeActivityMetadata metadata)

            metadata.AddImplementationVariable(Stopwatch);

            metadata.AddChild(Body);

            metadata.AddDelegate(OnCompletion);

        protected override void Execute(NativeActivityContext context)

            Stopwatch sw = new Stopwatch();

            Stopwatch.Set(context, sw);

            sw.Start();

            // schedule body and completion callback

            context.ScheduleActivity(Body, Completed);

        private void Completed(NativeActivityContext context, ActivityInstance instance)

            if (!context.IsCancellationRequested)

                Stopwatch sw = Stopwatch.Get(context);

                sw.Stop();

                Result.Set(context, sw.Elapsed);

                if (OnCompletion != null)

                    context.ScheduleAction<TimeSpan>(OnCompletion, Result.Get(context));

        protected override void Cancel(NativeActivityContext context)

            context.CancelChildren();

            if (OnCompletion != null)

                context.ScheduleAction<TimeSpan>(OnCompletion, TimeSpan.MinValue);

So, let’s build a designer for this. First we have to provide a WorkflowItemPresenter bound to the .Body property. This is pretty simple. Let’s show the “simple” XAML that will let us easily drop something on the Body property

<sap:ActivityDesigner x:Class="actionDesigners.ActivityDesigner1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
    <StackPanel>
        <sap:WorkflowItemPresenter 
                 HintText="Drop the body here" 
                 BorderBrush="Black" 
                 BorderThickness="2" 
                 Item="{Binding Path=ModelItem.Body, Mode=TwoWay}"/>
        <Rectangle Width="80" Height="6" Fill="Black" Margin="10"/>
    </StackPanel>
</sap:ActivityDesigner>

Not a whole lot of magic here yet. What we want to do is add another WorkflowItemPresenter, but what do I bind it to? Well, let’s look at how ActivityDelegate is defined [the root class for ActivityAction and ActivityFunc (which I’ll get to in my next post).:

hmmm, Handler is an Activity, that looks kind of useful. Let’s try that:

[warning, this XAML won’t work, you will get an exception, this is by design :-) ]

<sap:ActivityDesigner x:Class="actionDesigners.ActivityDesigner1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
    <StackPanel>
        <sap:WorkflowItemPresenter HintText="Drop the body here" BorderBrush="Black" BorderThickness="2" Item="{Binding Path=ModelItem.Body, Mode=TwoWay}"/>
        <Rectangle Width="80" Height="6" Fill="Black" Margin="10"/>
<!-- this next line will not work like you think it might --> 
        <sap:WorkflowItemPresenter HintText="Drop the completion here" BorderBrush="Black" BorderThickness="2" Item="{Binding Path=ModelItem.OnCompletion.Handler, Mode=TwoWay}"/>

    </StackPanel>
</sap:ActivityDesigner>

While this gives us what we want visually, there is a problem with the second WorkflowItemPresenter (just try dropping something on it):

Now, if you look at the XAML after dropping, the activity you dropped is not present. What’s happened here:

The OnCompletion property is null, so binding to OnCompletion.Handler will fail
We (and WPF) are generally very forgiving of binding errors, so things appear to have succeeded.
The instance was created fine, the ModelItem was created fine, and the it was put in the right place in the ModelItem tree, but there is no link in the underlying object graph, basically, the activity that you dropped is not connected
Thus, on serialization, there is no reference to the new activity in the actual object, and so it does not get serialized.

How can we fix this?

Well, we need to patch things up in the designer, so we will need to write a little bit of code, using the OnModelItemChanged event. This code is pretty simple, it just means that if something is assigned to ModelItem, if the value of “OnCompletion” is null, initialize it. If it is already set, we don’t need to do anything (for instance, if you used an IActivityTemplateFactory to initialize). One important thing here (putting on the bold glasses) YOU MUST GIVE THE DELEGATEINARGUMENT A NAME. VB expressions require a named token to reference, so, please put a name in there (or bind it, more on that below).

using System;
using System.Activities;

namespace actionDesigners
{
    // Interaction logic for ActivityDesigner1.xaml
    public partial class ActivityDesigner1
    {
        public ActivityDesigner1()
        {
            InitializeComponent();
        }

        protected override void OnModelItemChanged(object newItem)
        {
            if (this.ModelItem.Properties["OnCompletion"].Value == null)
            {
                this.ModelItem.Properties["OnCompletion"].SetValue(
                    new ActivityAction<TimeSpan>
                    {
                        Argument = new DelegateInArgument<TimeSpan>
                        {
                            Name = "duration"
                        }
                    });
            }
            base.OnModelItemChanged(newItem);

        }
    }
}

Well, this works :-) Note that you can see the duration DelegateInArgument that was added.

Now, you might say something like the following “Gosh, I’d really like to not give it a name and have someone type that in” (this is what we do in our ForEach designer, for instance). In that case, you would need to create a text box bound to OnCompletion.Argument.Name, which is left as an exercise for the reader.

Alright, now you can get out there and build activities with ActivityActions, and have design time support for them!

One question brought up in the comments on the last post was “what if I want to not let everyone see this” which is sort of the “I want an expert mode” view. You have two options. Either build two different designers and have the right one associated via metadata (useful in rehosting), or you could build one activity designer that switches between basic and expert mode and only surfaces these in expert mode.

Swiss Cheese and WF4, or, An Introduction to ActivityAction

Matt Winkler -- MSFT — Thu, 24 Dec 2009 01:46:49 GMT

One common scenario that was often requested by customers of WF 3 was the ability to have templated or “grey box” or “activities with holes” in them (hence the Swiss cheese photo above). In WF4 we’ve done this in a way that way we call ActivityAction

Motivation

First I’d like to do a little bit more to motivate the scenario.

Consider an activity that you have created for your ERP system called CheckInventory. You’ve gone ahead and encapsulated all of the logic of your inventory system, maybe you have some different paths of logic, maybe you have interactions with some third party systems, but you want your customers to use this activity in their workflows when they need to get the level of inventory for an item.

Consider more generally an activity where you have a bunch of work you want to get done, but at various, and specific, points throughout that work, you want to allow the consumer of that activity to receive a callback and provide their own logic to handle that. The mental model here is one of delegates.

Finally, consider providing the ability for a user to specify the work that they want to have happen, but also make sure that you can strongly type the data that is passed to it. In the first case above, you want to make sure that the Item in question is passed to the action that the consumer supplies.

In wf3, we had a lot of folks want to be able to do something like this. It’s a very natural extension to wanting to model things as activities and composing into higher level activities. We like being able to string together 10 items as a black box for reuse, but we really want the user to specify exactly what should happen between steps 7 and 8.

A slide that I showed at PDC showed it this way (the Approval and Payment boxes represent the places I want a consumer to supply additional logic):

Introducing ActivityAction

Very early on the release, we knew this was one of the problems that we needed to tackle. The mental model that we are most aligned with is that of a delegate/ callback in C#. If you think about a delegate, what are you doing, you are giving an object the implementation of some bit of logic that the object will subsequently call. That’s the same thing that’s going on with an ActivityAction. there are three important parts to an ActivityAction

The Handler (this is the logic of the ActivityAction)
The Shape (this determines the data that will be passed to the handler)
The way that we invoke it from our activity

Let’s start with some simple code (this is from a demo that I showed in my PDC talk). This is a timer activity which allows us to time the execution for the contained activity and then uses an activity action to act on the result.

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.Activities;

using System.Diagnostics;

namespace CustomActivities.ActivityTypes

    public sealed class Timer : NativeActivity<TimeSpan>

        public Activity Body { get; set; }

        public Variable<Stopwatch> Stopwatch { get; set; }

        public ActivityAction<TimeSpan> OnCompletion { get; set; }

        public Timer()

            Stopwatch = new Variable<Stopwatch>();

        protected override void CacheMetadata(NativeActivityMetadata metadata)

            metadata.AddImplementationVariable(Stopwatch);

            metadata.AddChild(Body);

            metadata.AddDelegate(OnCompletion);

        protected override void Execute(NativeActivityContext context)

            Stopwatch sw = new Stopwatch();

            Stopwatch.Set(context, sw);

            sw.Start();

            // schedule body and completion callback

            context.ScheduleActivity(Body, Completed);

        private void Completed(NativeActivityContext context, ActivityInstance instance)

            if (!context.IsCancellationRequested)

                Stopwatch sw = Stopwatch.Get(context);

                sw.Stop();

                Result.Set(context, sw.Elapsed);

                if (OnCompletion != null)

                    context.ScheduleAction<TimeSpan>(OnCompletion, Result.Get(context));

        protected override void Cancel(NativeActivityContext context)

            context.CancelChildren();

            if (OnCompletion != null)

                context.ScheduleAction<TimeSpan>(OnCompletion, TimeSpan.MinValue);

A few things to note about this code sample:

The declaration of an ActivityAction<TimeSpan> as a member of the activity. You’ll note we use the OnXXX convention often for activity actions.
The usage of the ActivityAction<T> with on type argument. The way to read this, or any of the 15 other types is that the T is the type of the data that will be passed to the activity action’s handler.
- Think about this like an Activity<Foo> corresponding to a void DoSomething(Foo argument1) method
The call to NativeActivityMetadata.AddDelegate() which lets the runtime know that it needs to worry about the delegate
The code in the Completed( ) method which checks to see if OnCompletion is set and then schedules it using ScheduleAction. I want to call out that line of code.

if (OnCompletion != null)
{
    context.ScheduleAction<TimeSpan>(OnCompletion, Result.Get(context));
}

It is important to note that I use the second parameter (and the third through 16th if that version is provided) in order to provide the data. This way, the activity determines what data will be passed to the handler, allowing the activity to determine what data is visible where. This is a much better way than allowing an invoked child to access any and all data from the parent. This lets us be very specific about what data goes to the ActivityAction. Also, you could make it so that OnCompletion must be provided, that is, the only way to use the activity is to supply an implementation. If you have something like “ProcessPayment” you likely want that to be a required thing. You can use the CacheMetadata method in order to check and validate this.

Now, let’s look at the code required to consume this time activity:

DelegateInArgument<TimeSpan> time = new DelegateInArgument<TimeSpan>();
a = new Timer
{
    Body = new HttpGet { Url = "http://www.microsoft.com" },
    OnCompletion = new ActivityAction<TimeSpan>
    {
        Argument = time,
        Handler = new WriteLine { 
            Text = new InArgument<string>(
                ctx => 
                    "Time input from timer " + time.Get(ctx).TotalMilliseconds)
        } 

    }
};

There are a couple of interesting things here:

Creation of DelegateInArgument<TimeSpan> : This is used to represent the data passed by the ActivityAction to the handler
Creation of the ActivityAction to pass in. You’ll note that the Argument property is set to the DelegateInArgument, which we can then use in the handler
The Handler is the “implementation” that we want to invoke. here’s it’s pretty simple, it’s a WriteLine and when we construct the argument we construct if from a lambda that uses the passed in context to resolve the DelegateInArgument when that executes.

At runtime, when we get to the point in the execution of the Timer activity, the WriteLine that the hosting app provided will be scheduled when the ScheduleAction is called. This means we will output the timing information that the Timer observed. A different implementation could have an IfThen activity and use that to determine if an SLA was enforced or not, and if not, send a nasty email to the WF author. The possibilities are endless, and they open up scenarios for you to provide specific extension points for your activities.

That wraps up a very brief tour of ActivityAction. ActivityAction provides a easy way to create an empty place in activity that the consumer can use to supply the logic that they want executed. In the second part of this post, we’ll dive into how to create a designer for one of these, how to represent this in XAML, and a few other interesting topics.

It’s that time of year that I’ll be taking a little bit of time off for the holidays, so I will see y’all in 2010!

Photo Credit

http://www.flickr.com/photos/myklroventine/ / CC BY 2.0

Inspection, Default Services and Items (WF4 EditingContext Intro Part 6)

Matt Winkler -- MSFT — Wed, 23 Dec 2009 01:59:00 GMT

This part 6 of my 6 part series on the EditingContext.

Introduction

Sharing Functionality between Designers

Host provided capabilities

Providing callbacks for the host

Subscription/Notification engine

Inspection, Default Services and Items (you are here)

I want to wrap up this series of posts by posting some code for an activity designer that functions more as a diagnostic tool, and will display all of the Items and services of the EditingContext within the designer. This will be useful from an investigation perspective, and hopefully as a diagnostic tool. We will use this to help us understand what are the services that are available out of the box in VS, as well as in a rehosted application.

We first need to create an empty activity to attach a designer to.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Activities;
using System.ComponentModel;

namespace blogEditingContext
{
    [Designer(typeof(DiagnosticDesigner))]
    public sealed class Diagnosticator : CodeActivity
    {
        // Define an activity input argument of type string
        public InArgument<string> Text { get; set; }

        // If your activity returns a value, derive from CodeActivity<TResult>
        // and return the value from the Execute method.
        protected override void Execute(CodeActivityContext context)
        {
            // Obtain the runtime value of the Text input argument
            string text = context.GetValue(this.Text);
        }
    }
}

Now, let’s create our designer. We could do fancy treeviews or object browser style UI’s, but as this is a blog post, I want to provide you with the basics, and then let you figure out how that is most useful to you. So, we will just create a designer that writes out to debug output the relevant information.

<sap:ActivityDesigner x:Class="blogEditingContext.DiagnosticDesigner"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
  <Grid>
        <Button Click="Button_Click">Debug.WriteLine Context Data</Button>
    </Grid>
</sap:ActivityDesigner>

And now the code

using System.Diagnostics;
using System.Linq;
using System.Windows;

namespace blogEditingContext
{
    // Interaction logic for DiagnosticDesigner.xaml
    public partial class DiagnosticDesigner
    {
        public DiagnosticDesigner()
        {
            InitializeComponent();
        }

        private void Button_Click(object sender, RoutedEventArgs e)
        {
            // the goal here is to output meaningful and useful information about 
            // the contents of the editing context here. 
            int level = Debug.IndentLevel;
            Debug.WriteLine("Items in the EditingContext");
            Debug.IndentLevel++;
            foreach (var item in Context.Items.OrderBy(x => x.ItemType.ToString()))
            {
                Debug.WriteLine(item.ItemType);
            }

            Debug.IndentLevel = level;
            Debug.WriteLine("Services in the EditingContext");
            foreach (var service in Context.Services.OrderBy(x => x.ToString()))
            {
                Debug.WriteLine(service);
            }
        }
    }
}

Let’s break this down. The work here happens in the button click where we simply order by types’ string representations and output them to the debug writer (a more robust implementation might use a trace writer that could be configured in the app, but for this purpose, this will be sufficient.

So, what output do we get?

VS Standard Services and Items

We determine this by using the activity in a freshly opened WF project

Items

System.Activities.Presentation.Hosting.AssemblyContextControlItem

System.Activities.Presentation.Hosting.ReadOnlyState

System.Activities.Presentation.Hosting.WorkflowCommandExtensionItem

System.Activities.Presentation.View.Selection

System.Activities.Presentation.WorkflowFileItem

Services

System.Activities.Presentation.Debug.IDesignerDebugView

System.Activities.Presentation.DesignerPerfEventProvider

System.Activities.Presentation.FeatureManager

System.Activities.Presentation.Hosting.ICommandService

System.Activities.Presentation.Hosting.IMultiTargetingSupportService

System.Activities.Presentation.Hosting.WindowHelperService

System.Activities.Presentation.IActivityToolboxService

System.Activities.Presentation.IIntegratedHelpService

System.Activities.Presentation.IWorkflowDesignerStorageService

System.Activities.Presentation.IXamlLoadErrorService

System.Activities.Presentation.Model.AttachedPropertiesService

System.Activities.Presentation.Model.ModelTreeManager

System.Activities.Presentation.Services.ModelService

System.Activities.Presentation.Services.ViewService

System.Activities.Presentation.UndoEngine

System.Activities.Presentation.Validation.IValidationErrorService

System.Activities.Presentation.Validation.ValidationService

System.Activities.Presentation.View.ActivityTypeDesigner+DisplayNameUpdater

System.Activities.Presentation.View.DesignerView

System.Activities.Presentation.View.IExpressionEditorService

System.Activities.Presentation.View.ViewStateService

System.Activities.Presentation.View.VirtualizedContainerService

Basic Rehosted Application Standard Services and Items

Items

System.Activities.Presentation.Hosting.ReadOnlyState

System.Activities.Presentation.Hosting.WorkflowCommandExtensionItem

System.Activities.Presentation.View.Selection

Services

System.Activities.Presentation.DesignerPerfEventProvider

System.Activities.Presentation.FeatureManager

System.Activities.Presentation.Hosting.WindowHelperService

System.Activities.Presentation.Model.AttachedPropertiesService

System.Activities.Presentation.Model.ModelTreeManager

System.Activities.Presentation.Services.ModelService

System.Activities.Presentation.Services.ViewService

System.Activities.Presentation.UndoEngine

System.Activities.Presentation.Validation.ValidationService

System.Activities.Presentation.View.DesignerView

System.Activities.Presentation.View.ViewStateService

System.Activities.Presentation.View.VirtualizedContainerService

Comparison Table View

Items	VS	Rehosted
System.Activities.Presentation.Hosting.AssemblyContextControlItem	Yes	No
System.Activities.Presentation.Hosting.ReadOnlyState	Yes	Yes
System.Activities.Presentation.Hosting.WorkflowCommandExtensionItem	Yes	Yes
System.Activities.Presentation.View.Selection	Yes	Yes
System.Activities.Presentation.WorkflowFileItem	Yes	No

Services	VS	Rehosted
System.Activities.Presentation.Debug.IDesignerDebugView	Yes	No
System.Activities.Presentation.DesignerPerfEventProvider	Yes	Yes
System.Activities.Presentation.FeatureManager	Yes	Yes
System.Activities.Presentation.Hosting.ICommandService	Yes	No
System.Activities.Presentation.Hosting.IMultiTargetingSupportService	Yes	No
System.Activities.Presentation.Hosting.WindowHelperService	Yes	Yes
System.Activities.Presentation.IActivityToolboxService	Yes	No
System.Activities.Presentation.IIntegratedHelpService	Yes	No
System.Activities.Presentation.IWorkflowDesignerStorageService	Yes	No
System.Activities.Presentation.IXamlLoadErrorService	Yes	No
System.Activities.Presentation.Model.AttachedPropertiesService	Yes	Yes
System.Activities.Presentation.Model.ModelTreeManager	Yes	Yes
System.Activities.Presentation.Services.ModelService	Yes	Yes
System.Activities.Presentation.Services.ViewService	Yes	Yes
System.Activities.Presentation.UndoEngine	Yes	Yes
System.Activities.Presentation.Validation.IValidationErrorService	Yes	No
System.Activities.Presentation.Validation.ValidationService	Yes	Yes
System.Activities.Presentation.View.ActivityTypeDesigner+DisplayNameUpdater	Yes	No
System.Activities.Presentation.View.DesignerView	Yes	Yes
System.Activities.Presentation.View.IExpressionEditorService	Yes	No
System.Activities.Presentation.View.ViewStateService	Yes	Yes
System.Activities.Presentation.View.VirtualizedContainerService	Yes	Yes

Conclusion

This wraps up our series on the editing context. We’ve gone through the basics of why we need it, what we can do with it, and then we moved how to use it, from both the very simple to the very complex. We’ve finished with a diagnostic tool to help understand what all items I can bind to.

What’s Next From Here?

A few ideas for the readers who have read all of these:

Wire up a few attached properties to reflect back through to some interesting data (like if it is selected). These attached properties could then be used directly by your UI (via the binding in XAML) to let your designers display and react to changes in the data
Think about ideas for services you might want to add in VS without depending on an activity to inject it (and send me mail, I am trying to compile a list of interesting things)
Are there service/item implementations you want to override in VS?
Is there a service/item you expect to see that is not there?

Thanks for now!

Subscription / Notification Engine (WF4 EditingContext Intro Part 5)

Matt Winkler -- MSFT — Tue, 22 Dec 2009 21:20:08 GMT

This part 5 of my 6 part series on the EditingContext.

In this post, we’re going to tie together a few of the things we’ve seen in the last few posts and show how we can wire up parts of the designer (or the hosting application) to changes made to the Items collection of the EditingContext to do some interesting things.

You will note that both the ServiceManager and ContextItemManager have Subscribe methods, and I’ve talked in previous posts about how the publish mechanism is a little different. I want to dive a little deeper into how these work, the different overloads, and what you can expect have happen on the subscribe side of things.

Services

On service, there are two different publish methods. I will list all four, and then talk about how there are really only two :-)

Method	Description
Publish(Type, PublishServiceCallback)	Publishes the specified service type, but does not declare an instance. When the service is requested, the Publish service callback will be invoked to create the instance. The callback is invoked only once. After that, the instance it returned is cached.
Publish(Type, Object)	Publishes the given service. After it is published, the service instance remains in the service manager until the editing context is disposed of.
Publish<(Of <(TServiceType>)>)(TServiceType)	Publishes the given service. After it is published, the service instance remains in the service manager until the editing context is disposed of.
Publish<(Of <(TServiceType>)>)(PublishServiceCallback<(Of <(TServiceType>)>))	Publishes the given service type, but does not declare an instance yet. When the service is requested, the PublishServiceCallback will be invoked to create the instance. The callback is invoked only once. After that, the instance it returned is cached.

There are really only two methods here, and some generic sugar for the other two. They are Publish(Type, PublishServiceCallback), and Publish(Type, Object). If you were to look at the implementation of the generic versions, they simply turn around and call the un-generic form.

The difference between the basic one (Publish(Type, Object)) and the version with the callback is that the callback enables us to be a little more lazy and wait to actually create the instance of the object until it is first requested. Let’s look at how PublishServiceCallback is defined:

public delegate Object PublishServiceCallback(
    Type serviceType
)

What this lets us do is that the first time someone calls GetService, this method will be called with the responsibility of returning an instance of the service type. Subsequent calls to GetService will simply return the instance returned by the method provided fro the PublishServiceCallback. It is important to note that on Publish, any subscribers will be notified. As the Subscribe callback takes an instance, we will internally call GetService on the notification, which will in turn call the PublishServiceCallback to instantiate an object. If we have subscribers, our publish is less lazy (but that’s by design, we have consumers who are ready and waiting to consume).

Let’s now look at the subscribe methods. Again, here there are two methods (generic and non-generic), but they both do the same thing:

Method	Description
Subscribe	Invokes the provided callback when someone has published the requested service. If the service was already available, this method invokes the callback immediately.
Subscribe<(Of <(TServiceType>)>)	Invokes the provided callback when someone has published the requested service. If the service was already available, this method invokes the callback immediately.

Both of these use a SubscribeServiceCallback defined as the following

public delegate void SubscribeServiceCallback(
    Type serviceType,
    Object serviceInstance
)

This allows any consumer to be notified when a service is initially published. This is an important distinction we will call out versus items which provide a more advanced subscription method (namely, to changes).

Why is this Useful?

Generally we find this useful for a few reasons:

Services may not be available at designer initialization, or the order in which they are created may not be fixed (it is ultimately up to the host to determine this).
Services may be provided by the host. It is possible your activity designer may be used in a host that does not provide that service. You may want to be flexible in handling that
Services can be injected at any time. A publish – subscribe model lets us have a little more flexibility to react to new services as they are added. You could imagine a spell checking service that a host only provides on the first time someone hits “spell check.” When this service comes online, then our designers can decide to consume this.
Flipping things around, you may want to use a service to callout to a host, and you would like the host to subscribe for when an certain activity designer will publish a service.

Now let’s talk about Items:

Items

Items do not have a publish method, per se, but they have the SetValue method which basically publishes an instance to the context. The semantics of SetValue are that it will first attempt to store the new value. Provided that succeeds, we then call OnItemChanged on the ContextItem itself. This is basically notifies the object itself (giving it a chance to react, clean up, or throw if something is really wrong). If this throws, the old value is preserved. If this succeeds, we then notify anyone who has subscribed to the changes.

GetValue allows me to retrieve the ContextItem. There are two GetValue’s, one generic, the other non-generic, but with a type as its parameter. It is important to note the point that is also present in the docs. If there is not an item present when this is called, the default value will be instantiated and returned.

Provided items are written using SetValue, all of the subscribers will be subsequently notified. If I just do an arbitrary GetValue and then make a few changes without calling SetValue, by default nothing interesting is going to happen (that is, no subscribers will be notified, subsequent calls to GetValue will get the updated object however). Subscribe (and it’s generic counterpart) allow me to provide a SubscribeContextCallback which will be invoked whenever SetValue is called. This functions basically in the same way that it does for Services.

An interesting pattern for Items that we use in a few places throughout the designer is to create an AttachedProperty on the modelItem (similar to this post) which in the implementation of the Getter and Setter will call out to the editing context to get or set the value from a ContextItem. This gives me a WPF friendly binding surface (foo.Bar binding syntax) that we can wire up to be change aware. We do this for a number of our triggers within our style implementation for things like is selected, etc. Future post note for me is that I should go through all of the attached properties present on a ModelItem that you could use to bind to :-)

This wraps up a tour of the Subscription / Notification engine present within the EditingContext.

Providing Callbacks for the Host (WF4 EditingContext Intro Part 4)

Matt Winkler -- MSFT — Tue, 22 Dec 2009 17:57:01 GMT

This part 4 of my 6 part series on the EditingContext.

In addition to having a host provide an instance of a type to be used within the designer, it can also be used to pass an instance that will route callbacks to the host. I covered this briefly in a previous post (Displaying Validation Errors in a Rehosted WF4 Designer). In that case, we provide an implementation of IValidationErrorService, which the designer infrastructure will call, if available, towards the end of a completion of a validation episode. In the sample application in that post, we use that instance to route, and display the validation errors in the system.

Rather than duplicate the code, I will simply encourage you to check out that post and think about the way you could publish a service that your activity designers know about, and use it as a mechanism for calling methods within the hosting application.

Host Provided Capabilities (WF4 EditingContext Intro Part 3)

Matt Winkler -- MSFT — Tue, 22 Dec 2009 17:03:21 GMT

This part 3 of my 6 part series on the EditingContext.

Introduction

Sharing Functionality between Designers

Host provided capabilities (you are here)

Providing callbacks for the host

Subscription/Notification engine

Inspection, Default Services and Items

EditingContext is used by our primary hosting application, Visual Studio, to provide concrete implementations of certain services. The example that we will talk about here is the IExpressionEditorService. Now, one thing that we would have really liked to have done in vs2010 is to provide a way to use the intellisense enabled VB editor that you see within VS inside a rehosted app. For various reasons, we were not able to ship with that dependency in the .NET framework. However, we needed a mechanism for our primary host to have the intellisense experience (and similarly, you could build an experience, or maybe we’ll ship one in the future for rehosted apps).

Let’s look at the design of IExpressionEditorService:

Name	Description
CloseExpressionEditors	Closes all the active expression editors.
CreateExpressionEditor	Overloaded. Creates a new expression editor.
UpdateContext	Updates the context for the editing session.

Inside the ExpressionTextBox control, when the control needs to render the editor, it has code that looks like the following (note, if it can’t find an instance, it skips and just uses a plain old text box):

if (this.Context.Services.GetService<IExpressionEditorService>() != null)
{
      return this.Context.Services.GetService<IExpressionEditorService>().CreateExpressionEditor(...)
}

Using the following overload of CreateExpressionEditor:

IExpressionEditorInstance CreateExpressionEditor(
    AssemblyContextControlItem assemblies,
    ImportedNamespaceContextItem importedNamespaces,
    List<ModelItem> variables,
    string text
)

Now, what happens is that inside the code that we ship in the Microsoft.VisualStudio.Activities.Addin.dll, there is both a concrete implementation of this type, as well as the code which will publish an instance of this to the editing context. Remember, this is the same thing that you can do in your app for a these host provided services. In a subsequent post, I will get into more details of what are the ones that the designer has built in (like IExpressionEditorService).

Sharing Functionality Between Designers (WF4 EditingContext Intro Part 2)

Matt Winkler -- MSFT — Mon, 21 Dec 2009 22:10:24 GMT

This part 2 of my 6 part series on the EditingContext.

Setup

We will need a custom activity, EmptyOne and designer called InteractWithServiceDesigner.

using System.Activities;
using System.ComponentModel;

namespace blogEditingContext
{
    [Designer(typeof(InteractWithServicesDesigner))]
    public sealed class EmptyOne : CodeActivity
    {
        // Define an activity input argument of type string
        public InArgument<string> Text { get; set; }

        // If your activity returns a value, derive from CodeActivity<TResult>
        // and return the value from the Execute method.
        protected override void Execute(CodeActivityContext context)
        {
            // Obtain the runtime value of the Text input argument
            string text = context.GetValue(this.Text);
        }
    }
}

What We Will See

The designers for Foo will leverage a new service in order to display a list of database tables. We will also need to publish this service to the editing context, and handle the fact that we don’t know who might publish it (or when it might be published). Note that in VS, there is no way to inject services except by having an activity designer do it. In a rehosted app, the hosting application could publish additional services (see part 4) that the activities can consume. In this case though, we will use the activity designer as our hook.

Publishing a Service

Let’s look at the designer for Foo (as Foo is our generic, and relatively boring activity).

<sap:ActivityDesigner x:Class="blogEditingContext.InteractWithServicesDesigner"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
    <StackPanel>
        <ListBox Height="100" Name="listBox1" Width="120" />
        <Button Name="button1" Click="Button_Click">Publish Service</Button>
    </StackPanel>
</sap:ActivityDesigner>

Not much to this, except a drop down list that is currently unbound (but a name is provided). Also note that there is a button that says to “publish the service”. Let’s first look at the code for the button click

private void Button_Click(object sender, RoutedEventArgs e)
{
    if (!this.Context.Services.Contains<ISampleService>())
    {
        this.Context.Services.Publish<ISampleService>(new SampleServiceImpl());
    }
}

What are we doing here? We first check if this service is already published using the Contains method. We can do this because ServiceManager implements IEnumerable<Type>.

[update, finishing this sentence.] One could also consume the service using GetService<TResult>. You may also note that there is a GetRequiredService<T>. This is a call that we know won’t return null, as the services we are requesting must be there for the designer to work. Rather than returning null, this will throw an exception. Within the designer, we generally think of one service as required:

ModelService

Let’s look at the definition of the service. Here you can see that we are using both an interface and then providing an implementation of that interface. You could just as easily use an abstract class, or even a concrete class, there is no constraint on the service type.

using System.Collections.Generic;

namespace blogEditingContext
{
    public interface ISampleService
    {
        IEnumerable<string> GetDropdownValues(string DisplayName);
    }

    public class SampleServiceImpl : ISampleService
    {
        public IEnumerable<string> GetDropdownValues(string DisplayName)
        {
            return new string[]  { 
                DisplayName + " Foo", 
                DisplayName + " Bar",
                "Baz " + DisplayName
            } ;
        }
    }

}

If there is not a service present, we will publish an instance of one. This becomes the singleton instance for any other designer that may request it. Right now, we have a designer that can safely publish a service. Let’s look at consuming one

Consuming a Service

Let’s look at some code to consume the service. There are two parts to this. One is simply consuming it, which we already saw above in discussing GetService and GetRequiredService . The second is hooking into the notification system to let us know when a service is made available. In this case, it’s a little contrived, as the service isn’t published until the button click, but it’s good practice to use the subscription mechanism as we make no guarantees on ordering, or timing of service availability.

Subscribing to Service

Here, using the Subscribe<TServiceType> method, we wait for the service to be available. The documentation summarizes this method nicely:

Invokes the provided callback when someone has published the requested service. If the service was already available, this method invokes the callback immediately.

In the OnModelItemChanged method, we will subscribe and hook a callback. The callback’s signature is as follows:

public delegate void SubscribeServiceCallback<TServiceType>(
    TServiceType serviceInstance
)

As you can see, in this callback, the service instance is provided, so we can query it directly. You may ask, “why not in Intialize?” well, there are no guarentees that the editing context will be available at that point. We could either subscribe to context being made available, or just use ModelItemChanged:

protected override void OnModelItemChanged(object newItem)
{
    if (!subscribed)
    {
        this.Context.Services.Subscribe<ISampleService>(
            servInstance =>
            {
                listBox1.ItemsSource = servInstance.GetDropdownValues(this.ModelItem.Properties["DisplayName"].ComputedValue.ToString());
                button1.IsEnabled = false;
            }
            );
        subscribed = true; 
    }
}

This wraps a basic introduction to the ServiceManager type and how to leverage it effectively to share functionality in designers.

Let’s look at a before and after shot in the designer:

Before & After

What about Items?

Items follow generally the same Get, Subscribe, and publish pattern, but rather than publish, there is a SetValue method. If you have “just data” that you would like to share between designers (or between the host and the designer) an Item is the way to go about that. The most commonly used item we’ve seen customers use is the Selection item in order to be able to get or set the currently selected model item.

That’s our tour of basic publish and subscribe with Services and Items.

[updated 12/22/2009 @ 10:23 am to finish an unclear sentence about GetService<>]

[updated 12/22/2009 @ 8:50pm : Link to download sample code is here]

Attachment(s): blogEditingContext.zip

Introduction to the WF4 Designer Editing Context (Part 1)

Matt Winkler -- MSFT — Mon, 21 Dec 2009 18:02:43 GMT

I want to briefly touch on the editing context and give a little introduction to its capabilities. This is part 1 of a 6 part series

The way to think about the editing context is that it is the point of contact between the hosting application, and the designer (and elements on the designer). In my PDC talk, I had the following slide which I think captures the way to think about how these elements are layered together.

Motivation

The editing context represents the a common boundary between the hosting application and the designer, and the mechanism to handle interaction with the designer (outside of the most common methods that have been promoted on WorkflowDesigner). If you were to look at the implementation of some of the more common methods on WorkflowDesigner, you would see that almost all of these use the editing context in order to get anything done. For instance, the Flush method (and Save which calls Flush) will acquire an IDocumentPersistenceService from the Services collection, and then use that in order to properly serialize the document.

The EditingContext type has two important properties

Items

The Items collection is for data that is shared between the host and the designer, or data that is available to all designers. These need to derive from ContextItem which will provide the mechanism to hook into subscription and change notification. There are a couple of interesting methods on the ContextItemManager type :

Services

Services represent functionality that is either provided by the host for the designer to use, or is used by the designer to make functionality available to all designers within the editor. Generally, these are likely defined as interfaces, but can also be a type. It is then required for an implementation or an instance to be provided. This instance will be shared as a singleton. There are a few interesting methods on the ServiceManager type:

We’ll start walking through these over the next few posts.

Emitting the mc:Ignorable Instruction In Your WF4 XAML

Matt Winkler -- MSFT — Thu, 10 Dec 2009 21:33:48 GMT

Frequent forum guest Notre posed this question to the forums the other day noting that the XAML being produced from ActivityXamlServices.CreateBuilderWriter() was slightly different than the XAML being output from WorkflowDesigner.Save(). The reason for this stems from the fact that WorkflowDesigner leverages an additional internal type (which derives from XamlXmlWriter) in order to attach the mc:Ignorable attribute.

Why use mc:Ignorable?

From the source at MSDN:

The mc XML namespace prefix is the recommended prefix convention to use when mapping the XAML compatibility namespace http://schemas.openxmlformats.org/markup-compatibility/2006.

Elements or attributes where the prefix portion of the element name are identified as mc:Ignorable will not raise errors when processed by a XAML processor. If that attribute could not be resolved to an underlying type or programming construct, then that element is ignored. Note however that ignored elements might still generate additional parsing errors for additional element requirements that are side effects of that element not being processed. For instance, a particular element content model might require exactly one child element, but if the specified child element was in an mc:Ignorable prefix, and the specified child element could not be resolved to a type, then the XAML processor might raise an error.

Basically, this lets a XAML reader gracefully ignore any content marked from that namespace if it cannot be resolved. So, imagine a runtime scenario where we don’t want to load System.Activities.Presentation every time we read a WF XAML file that may contain viewstate. As a result, we use mc:Ignorable, which means the reader will skip that content when it does not have that assembly referenced at runtime.

This is what the output from the designer usually contains:

<Sequence 
     mc:Ignorable="sap" 
     mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces"
     xmlns="http://schemas.microsoft.com/netfx/2009/xaml/activities" 
     xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
     xmlns:mva="clr-namespace:Microsoft.VisualBasic.Activities;assembly=System.Activities" 
     xmlns:sap="http://schemas.microsoft.com/netfx/2009/xaml/activities/presentation" 
     xmlns:scg="clr-namespace:System.Collections.Generic;assembly=mscorlib" 
     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
  <sap:WorkflowViewStateService.ViewState>
    <scg:Dictionary x:TypeArguments="x:String, x:Object">
      <x:Boolean x:Key="IsExpanded">True</x:Boolean>
    </scg:Dictionary>
  </sap:WorkflowViewStateService.ViewState>
  <Persist sap:VirtualizedContainerService.HintSize="211,22" />
  <Persist sap:VirtualizedContainerService.HintSize="211,22" />
  <WriteLine sap:VirtualizedContainerService.HintSize="211,61" />
</Sequence>

mc:Ignorable will cause the ViewState and HintSize to be ignored.

Why do I have to worry about this?

If you use WorkflowDesigner.Save(), you don’t. If you want to be able to serialize the ActivityBuilder and have XAML which is what the designer produces, you need will need to add a XamlXmlWriter into the XamlWriter stack in order to get the right output. You may also worry about this if you are implementing your own storage and plan on writing some additional XAML readers or writers for additional extensibility and flexibility.

How Do I Get the Same Behavior?

The code below describes the same approach you would need to take to implement an XamlXmlWriter that does the same thing our internal type does. While I can’t copy and paste code, this does the same thing. We do two things:

Override WriteNamespace() to collect all of the namespaces being emitted. We do this to specifically check for ones that we should ignore, and to also gather all of the prefixes to make sure we don’t have a collision
Override WriteStartObject() to generate and write out the Ignorable attribute within the start (first) member for any namespaces we should ignore.

What namespaces do we ignore in the designer? Just one: http://schemas.microsoft.com/netfx/2009/xaml/activities/presentation

using System.Collections.Generic;

using System.IO;

using System.Xaml;

using System.Xml;

namespace IgnorableXamlWriter

    class IgnorableXamlXmlWriter : XamlXmlWriter

        HashSet<NamespaceDeclaration> ignorableNamespaces = new HashSet<NamespaceDeclaration>();

        HashSet<NamespaceDeclaration> allNamespaces = new HashSet<NamespaceDeclaration>();

        bool objectWritten;

        bool hasDesignNamespace;

        string designNamespacePrefix;

        public IgnorableXamlXmlWriter(TextWriter tw, XamlSchemaContext context)

            : base(XmlWriter.Create(tw,

                                    new XmlWriterSettings { Indent = true, OmitXmlDeclaration = true }),

                                    context,

                                    new XamlXmlWriterSettings { AssumeValidInput = true })

        public override void WriteNamespace(NamespaceDeclaration namespaceDeclaration)

            if (!objectWritten)

                allNamespaces.Add(namespaceDeclaration);

                // if we find one, add that to ignorable namespaces

                // the goal here is to collect all of them that might point to this

                // if you had a broader set of things to ignore, you would collect

                // those here.

                if (namespaceDeclaration.Namespace == "http://schemas.microsoft.com/netfx/2009/xaml/activities/presentation")

                    hasDesignNamespace = true;

                    designNamespacePrefix = namespaceDeclaration.Prefix;

            base.WriteNamespace(namespaceDeclaration);

        public override void WriteStartObject(XamlType type)

            if (!objectWritten)

                // we should check if we should ignore

                if (hasDesignNamespace)

                    // note this is not robust as mc could naturally occur

                    string mcAlias = "mc";

                    this.WriteNamespace(

                        new NamespaceDeclaration(

                            "http://schemas.openxmlformats.org/markup-compatibility/2006",

                            mcAlias)

);

            base.WriteStartObject(type);

            if (!objectWritten)

                if (hasDesignNamespace)

                    XamlDirective ig = new XamlDirective(

                        "http://schemas.openxmlformats.org/markup-compatibility/2006",

                        "Ignorable");

                    WriteStartMember(ig);

                    WriteValue(designNamespacePrefix);

                    WriteEndMember();

                    objectWritten = true;

One note on the code above, it is noted that the generation of the namespace prefix “mc” is not robust. In the product code we will check to see if mc1, mc2, … are available up to mc1000. In that case we would then append a GUID for the ugliest XML namespace known to mankind. The chance of collision up to 1000 would be a highly extreme edge case.

How would I use this? The following code shows feeding this into a CreateBuilderWriter that is passed to XamlServices.Save()

StringBuilder sb = new StringBuilder();
XamlSchemaContext xsc = new XamlSchemaContext();
var bw = ActivityXamlServices.CreateBuilderWriter(
    new IgnorableXamlXmlWriter(new StringWriter(sb), xsc));

XamlServices.Save(bw,
                  wd.Context.Services.GetService<ModelTreeManager>().Root.GetCurrentValue());

AttachedProperty Part 2, Putting it Together

Matt Winkler -- MSFT — Mon, 07 Dec 2009 05:19:09 GMT

On my last post , Jason jumped right to the punchline in his comment here . He asks “if there is an easy way to have the properties value serialized out to the xaml.”

First, let’s look at what we need to do from the XAML side.

First, create a helper type with a getter and setter for the property that you want to attach. Here we’re going to attach comments:

public class Comment
{
    static AttachableMemberIdentifier CommentTextName = new AttachableMemberIdentifier(typeof(Comment), "CommentText");

    public static object GetCommentText(object instance)
    {
        object viewState;
        AttachablePropertyServices.TryGetProperty(instance, CommentTextName, out viewState);
        return viewState;
    }

    public static void SetCommentText(object instance, object value)
    {
        AttachablePropertyServices.SetProperty(instance, CommentTextName, value);
    }

}

Next, let’s use the AttachableMemberIdentifier and the AttachablePropertyServices to do something interesting with this:

AttachableMemberIdentifier ami = new AttachableMemberIdentifier(typeof(Comment), "CommentText");
Dog newDog = new Dog { Age = 12, Name = "Sherlock", Noise = "Hooowl" };
AttachablePropertyServices.SetProperty(newDog, ami, "A very good dog");
string s = XamlServices.Save(newDog);
Console.WriteLine("XAML");
Console.WriteLine(s);
Dog aSecondNewDog = XamlServices.Load(new StringReader(s)) as Dog;
string outter;
AttachablePropertyServices.TryGetProperty(aSecondNewDog, ami, out outter);
Console.WriteLine("read out: {0}", outter);

Let’s see the output from this:

XAML
<Dog
     Age="12"
     Comment.CommentText="A very good dog"
     Name="Sherlock"
     Noise="Hooowl" 
     xmlns="clr-namespace:AttachedPropertiesBlogPosting;assembly=AttachedPropertiesBlogPosting" />
read out: A very good dog

You’ll note that the value is contained in the XAML under Comment.CommentText.

Pulling it all Together

Let’s take what we did in the last post and combine it with the above stuff in order to have an attached property that writes through and is stored inside the XAML.

AttachedProperty<string> Comment = new AttachedProperty<string>
{
    IsBrowsable = true,
    Name = "Comment",
    Getter = (mi =>
        { string temp;
          AttachablePropertyServices.TryGetProperty<string>(mi.GetCurrentValue(), ami, out temp);
          return temp;
        }),
    Setter = ( (mi,val) => AttachablePropertyServices.SetProperty(mi.GetCurrentValue(), ami, val) )

};
aps.AddProperty(Comment);
dogMi.Properties["Comment"].SetValue("I think I like that dog");
string xaml = XamlServices.Save(dogMi.GetCurrentValue());
Console.WriteLine("XAML");
Console.WriteLine(xaml);

What are we doing here, well, we basically just use the Getter and Setter to write through to the underlying instance. You’ll note that usually, we never want to go and use GetCurrentValue(), as any changes made there are not made via the ModelItem tree which means we might miss a change notification. However, given that the only place where we can store the XAML attached property is on the instance itself, this gives us a good way to write through. The XAML output below shows that this works as expected:

<Dog 
   Age="5"
   Comment.CommentText="I think I like that dog"
   Name="Sasha" 
   Noise="Snort" 
   xmlns="clr-namespace:AttachedPropertiesBlogPosting;assembly=AttachedPropertiesBlogPosting" />

WF4 Design Time AttachedPropertiesService and Attached Properties

Matt Winkler -- MSFT — Sun, 06 Dec 2009 04:51:01 GMT

I’ve been meaning to throw together some thoughts on attached properties and how they can be used within the designer. Basically, you can think about attached properties as injecting some additional “stuff” onto an instance that you can use elsewhere in your code.

Motivation

In the designer, we want to be able to have behavior and view tied to interesting aspects of the data. For instance, we would like to have a view updated when an item becomes selected. In WPF, we bind the style based on the “isSelectionProperty.” Now, our data model doesn’t have any idea of selection, it’s something we’d like the view level to “inject” that idea on any model item so that a subsequent view could take advantage of. You can kind of view Attached Properties as a nice syntactic sugar to not have to keep a bunch of lookup lists around. As things like WPF bind to the object very well, and not so much a lookup list, this ends up being an interesting model.

To be clear, you could write a number of value converters that take the item being bound, look up in a lookup list somewhere, and return the result that will be used. The problem we found is that we were doing this in a bunch of places, and we really wanted to have clean binding statements inside our WPF XAML, rather than hiding a bunch of logic in the converters.

How Does it Work

First, some types.

Name	Description
AttachedPropertiesService	Service in editing context for managing AttachedProperties
AttachedProperty	Base attached property type (abstract)
AttachedProperty<T>	Strongly typed attached property with interesting getter/setter programmability

in diagram form:

One thing that might look a little funny to some folks who have used attached properties in other contexts (WF3, WPF, XAML), is the “IsBrowsable ” property. The documentation is a little sparse right now, but what this will do is determine how discoverable the property is. If this is set to true, the attached property will show up in the Properties collection of the ModelItem to which the AP is attached. What this means is that it can show up in the Property grid, you can bind WPF statements directly to it, as if it were a real property of the object. Attached properties by themselves have no actual storage representation, so these exist as design time only constructs.

Getter/ Setter?

One other thing that you see on the AttachedProperty<T> is the Getter and Setter properties. These are of type Func<ModelItem,T> and Action<ModelItem,T> respectively. What these allow you to do is perform some type of computation whenever the get or set is called against the AttachedProperty. Why is this interesting? Well, let’s say that you’d like to have a computed value retrieved, such as “IsPrimarySelection” checking with the Selection context item to see if an item is selected. Or, customizing the setter to either store the value somewhere more durable, or updating a few different values. The other thing that happens is that since all of these updates go through the ModelItem tree, any changes will be propagated to other listeners throughout the designer.

Looking at Some Code

Here is a very small console based app that shows how you can program against the attached properties. An interesting exercise for the reader would be to take this data structure, put it in a WPF app and experiment with some of the data binding.

First, two types:

public class Dog
{
    public string Name { get; set; }
    public string Noise { get; set; }
    public int Age { get; set; }
   
}

public class Cat
{
    public string Name { get; set; }
    public string Noise { get; set; }
    public int Age { get; set; }
}

Ignore no common base type, that actually makes this a little more interesting, as we will see.

Now, let’s write some code. First, let’s initialize and EditingContext and ModelTreeManager.

   1:       static void Main(string[] args)

   2:          {

   3:              EditingContext ec = new EditingContext();

   4:              ModelTreeManager mtm = new ModelTreeManager(ec);

   5:              mtm.Load(new object[] { new Dog { Name = "Sasha", Noise = "Snort", Age = 5 },

   6:                                      new Cat { Name="higgs", Noise="boom", Age=1 } });

   7:              dynamic root = mtm.Root;

   8:              dynamic dog = root[0];

   9:              dynamic cat = root[1];

  10:              ModelItem dogMi = root[0] as ModelItem;

  11:              ModelItem catMi = root[1] as ModelItem;

Note, lines 7-9 will not work in Beta2 (preview of coming attractions). To get lines 10-11 working in beta2, cast root to ModelItemCollection and then use the indexers to extract the values

Now, let’s build an attached property, and we will assign it only to type “dog”

   1:  // Add an attached Property

   2:  AttachedProperty<bool> ap = new AttachedProperty<bool>

   3:  {

   4:      IsBrowsable = true,

   5:      Name = "IsAnInterestingDog",

   6:      Getter = (mi => mi.Properties["Name"].ComputedValue.ToString() == "Sasha"),

   7:      OwnerType = typeof(Dog)

   8:  };

   9:  ec.Services.Publish<AttachedPropertiesService>(new AttachedPropertiesService());

  10:  AttachedPropertiesService aps = ec.Services.GetService<AttachedPropertiesService>();

  11:  aps.AddProperty(ap);

12:

  13:  Console.WriteLine("---- Enumerate properties on dog (note new property)----");

  14:  dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));

15:

  16:  Console.WriteLine("---- Enumerate properties on cat (note  no new property) ----");

  17:  catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));

Let’s break down what happened here.

Line2-8, create an AttachedProperty<bool>

We set IsBrowsable to true, we want to see it in the output
Name, that’s what it will be projected as
OwnerType, we only want this to apply to Dog’s, not Cat’s or Objects or whatever.
Finally, Getter, and look what we do here, we operate on the model item to do some computation and return a bool (in this case, we look to see if the name property equals “Sasha”

Line 9-11 create an AttachedPropertiesService and add it to the editing context.
Lines 13-17 output the properties, and let’s see what that looks like:

---- Enumerate properties on dog (note new property)----

 Property : Name

 Property : Noise

 Property : Age

 Property : IsAnInterestingDog

---- Enumerate properties on cat (note  no new property) ----

 Property : Name

 Property : Noise

 Property : Age

Ok, so that’s interesting, we’ve injected a new property, only on the dog type. If I got dogMI.Properties[“IsAnInterestingDog”], I would have a value that I could manipulate (albeit returned via the getter).

Let’s try something a little different:

   1:  AttachedProperty<bool> isYoungAnimal = new AttachedProperty<bool>

   2:  {

   3:      IsBrowsable = false,

   4:      Name = "IsYoungAnimal",

   5:      Getter = (mi => int.Parse(mi.Properties["Age"].ComputedValue.ToString()) < 2)

   6:  };

7:

   8:  aps.AddProperty(isYoungAnimal);

9:

  10:  // expect to not see isYoungAnimal show up

  11:  Console.WriteLine("---- Enumerate properties on dog  (note isYoungAnimal doesn't appear )----");

  12:  dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));

  13:  Console.WriteLine("---- Enumerate properties on cat (note isYoungAnimal doesn't appear )----");

  14:  catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));

15:

  16:  Console.WriteLine("---- get attached property via GetValue ----");

  17:  Console.WriteLine("getting non browsable attached property on dog {0}", isYoungAnimal.GetValue(dogMi));

  18:  Console.WriteLine("getting non browsable attached property on cat {0}", isYoungAnimal.GetValue(catMi));

Let’s break this down:

Lines 1-6 create a new attached property

IsBrowsable is false
No OwnerType being set
The Getter does some computation to return true or false

Lines 10-14 write out the properties (as above)
Lines 17-18 extract the value with AttachedPropertyInstance.GetValue(ModelItem)

Let’s see the output there:

---- Enumerate properties on dog  (note isYoungAnimal doesn't appear )----

 Property : Name

 Property : Noise

 Property : Age

 Property : IsAnInterestingDog

---- Enumerate properties on cat (note isYoungAnimal doesn't appear )----

 Property : Name

 Property : Noise

 Property : Age

---- get attached property via GetValue ----

getting non browsable attached property on dog False

getting non browsable attached property on cat True

As we can see, we’ve now injected this behavior, and we can extract the value.

Let’s get a little more advanced and do something with the setter. Here, if isYoungAnimal is set to true, we will change the age (it’s a bit contrived, but shows the dataflow on simple objects, we’ll see in a minute a more interesting case).

   1:  // now, let's do something clever with the setter.

   2:  Console.WriteLine("---- let's use the setter to have some side effect ----");

   3:  isYoungAnimal.Setter = ((mi, val) => { if (val) { mi.Properties["Age"].SetValue(10); } });

   4:  isYoungAnimal.SetValue(cat, true);

   5:  Console.WriteLine("cat's age now {0}", cat.Age);

Pay attention to what the Setter does now. We create the method through which subsequent SetValue’s will be pushed. Here’s that output:

---- let's use the setter to have some side effect ----
cat's age now 10

Finally, let’s show an example of how this can really function as some nice sugar to eliminate the need for a lot of value converters in WPF by using this capability as a way to store the relationship somewhere (rather than just using at a nice proxy to change a value):

   1:  // now, let's have a browesable one with a setter.

   2:  // this plus dynamics are a mini "macro language" against the model items

3:

   4:  List<Object> FavoriteAnimals = new List<object>();

5:

   6:  // we maintain state in FavoriteAnimals, and use the getter/setter func

   7:  // in order to query or edit that collection.  Thus changes to an "instance"

   8:  // are tracked elsewhere.

   9:  AttachedProperty<bool> isFavoriteAnimal = new AttachedProperty<bool>

  10:  {

  11:      IsBrowsable = false,

  12:      Name = "IsFavoriteAnimal",

  13:      Getter = (mi => FavoriteAnimals.Contains(mi)),

  14:      Setter = ((mi, val) =>

  15:          {

  16:              if (val)

  17:                  FavoriteAnimals.Add(mi);

  18:              else

  19:              {

  20:                  FavoriteAnimals.Remove(mi);

  21:              }

  22:          })

  23:  };

24:

25:

  26:  aps.AddProperty(isFavoriteAnimal);

27:

  28:  dog.IsFavoriteAnimal = true;

  29:  // remove that cat that isn't there

  30:  cat.IsFavoriteAnimal = false;

  31:  cat.IsFavoriteAnimal = true;

  32:  cat.IsFavoriteAnimal = false;

33:

  34:  Console.WriteLine("Who are my favorite animal?");

  35:  FavoriteAnimals.ForEach(o => Console.WriteLine((o as ModelItem).Properties["Name"].ComputedValue.ToString()));

Little bit of code, let’s break it down one last time:

Line 14 – Create a setter that acts upon the FavoriteAnimals collection to either add or remove the element
Line 28-32 – do a few different sets on this attached property

NOTE: you can’t do that in beta2 as the dynamic support hasn’t been turned on. Rather you would have to do isFavoriteAnimal.SetValue(dogMi, true).

Line 35 then prints the output to the console, and as expected we only see the dog there:

-- Who are my favorite animals?
Sasha

I will attach the whole code file at the bottom of this post, but this shows you how you can use the following:

Attached properties to create “computed values” on top of existing types
Attached properties to inject a new (and discoverable) property entry on top of the designer data model (in the form of a new property)
Using the Setter capability to both propagate real changes to the type, providing a nice way to give a cleaner interface, as well as use it as a mechanism to store data about the object outside of the object, but in a way that gives me access to it such that it seems like the object.

This is some really nice syntactic sugar that we sprinkle on top of things

What do I do now?

Hopefully this post gave you some ideas about how the attached property mechanisms work within the WF4 designer. These give you a nice way to complement the data model and create nice bindable targets that your WPF Views can layer right on top of.

A few ideas for these things:

Use the Setters to clean up a “messy” activity API into a single property type that you then build a custom editor for in the property grid.
Use the Getters (and the integration into the ModelProperty collection) in order to create computational properties that are used for displaying interesting information on the designer surface.
Figure out how to bridge the gap to take advantage of the XAML attached property storage mechanism, especially if you author runtime types that look for attached properties at runtime.
Use these, with a combination of custom activity designers to extract and display interesting runtime data from a tracking store

Full Code Posting

using System;
using System.Activities.Presentation;
using System.Activities.Presentation.Model;
using System.Collections.Generic;
using System.Linq;

namespace AttachedPropertiesBlogPosting
{
    class Program
    {
        static void Main(string[] args)
        {
            EditingContext ec = new EditingContext();
            ModelTreeManager mtm = new ModelTreeManager(ec);
            mtm.Load(new object[] { new Dog { Name = "Sasha", Noise = "Snort", Age = 5 },
                                    new Cat { Name="higgs", Noise="boom", Age=1 } });
            dynamic root = mtm.Root;
            dynamic dog = root[0];
            dynamic cat = root[1];
            ModelItem dogMi = root[0] as ModelItem;
            ModelItem catMi = root[1] as ModelItem;
          
            // Add an attached Property
            AttachedProperty<bool> ap = new AttachedProperty<bool>
            {
                IsBrowsable = true,
                Name = "IsAnInterestingDog",
                Getter = (mi => mi.Properties["Name"].ComputedValue.ToString() == "Sasha"),
                OwnerType = typeof(Dog)
            };
            ec.Services.Publish<AttachedPropertiesService>(new AttachedPropertiesService());
            AttachedPropertiesService aps = ec.Services.GetService<AttachedPropertiesService>();
            aps.AddProperty(ap);

            Console.WriteLine("---- Enumerate properties on dog (note new property)----");
            dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));

            Console.WriteLine("---- Enumerate properties on cat (note  no new property) ----");
            catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));


            
            AttachedProperty<bool> isYoungAnimal = new AttachedProperty<bool>
            {
                IsBrowsable = false,
                Name = "IsYoungAnimal",
                Getter = (mi => int.Parse(mi.Properties["Age"].ComputedValue.ToString()) < 2)
            };

            aps.AddProperty(isYoungAnimal);

            // expect to not see isYoungAnimal show up
            Console.WriteLine("---- Enumerate properties on dog  (note isYoungAnimal doesn't appear )----");
            dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
            Console.WriteLine("---- Enumerate properties on cat (note isYoungAnimal doesn't appear )----");
            catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));

            Console.WriteLine("---- get attached property via GetValue ----");
            Console.WriteLine("getting non browsable attached property on dog {0}", isYoungAnimal.GetValue(dogMi));
            Console.WriteLine("getting non browsable attached property on cat {0}", isYoungAnimal.GetValue(catMi));
            
            
            // now, let's do something clever with the setter. 
            Console.WriteLine("---- let's use the setter to have some side effect ----");
            isYoungAnimal.Setter = ((mi, val) => { if (val) { mi.Properties["Age"].SetValue(10); } });
            isYoungAnimal.SetValue(cat, true);
            Console.WriteLine("cat's age now {0}", cat.Age);

            // now, let's have a browesable one with a setter.
            // this plus dynamics are a mini "macro language" against the model items

            List<Object> FavoriteAnimals = new List<object>();

            // we maintain state in FavoriteAnimals, and use the getter/setter func
            // in order to query or edit that collection.  Thus changes to an "instance"
            // are tracked elsewhere.
            AttachedProperty<bool> isFavoriteAnimal = new AttachedProperty<bool>
            {
                IsBrowsable = false,
                Name = "IsFavoriteAnimal",
                Getter = (mi => FavoriteAnimals.Contains(mi)),
                Setter = ((mi, val) => 
                    {
                        if (val)
                            FavoriteAnimals.Add(mi);
                        else
                        {
                            FavoriteAnimals.Remove(mi);
                        }
                    })
            };
            aps.AddProperty(isFavoriteAnimal);
            dog.IsFavoriteAnimal = true;
            // remove that cat that isn't there
            cat.IsFavoriteAnimal = false;
            cat.IsFavoriteAnimal = true;
            cat.IsFavoriteAnimal = false;
            Console.WriteLine("Who are my favorite animals?");
            FavoriteAnimals.ForEach(o => Console.WriteLine((o as ModelItem).Properties["Name"].ComputedValue.ToString()));
            Console.ReadLine();
        }
    }

    public class Dog
    {
        public string Name { get; set; }
        public string Noise { get; set; }
        public int Age { get; set; }
    }

    public class Cat
    {
        public string Name { get; set; }
        public string Noise { get; set; }
        public int Age { get; set; }
    }
}

WF4 ViewStateService

Matt Winkler -- MSFT — Sun, 06 Dec 2009 01:33:20 GMT

A comment posted by Notre asked for some more details about view state and attached property services, so I thought I would dive into those next. I will follow-up in a subsequent post on the AttachedPropertyService, as there is a little bit more going on there.

Motivation

Why do I care about viewstate? Well, usually it is because we want to write something down and store it for later that is not required for runtime. A common example of viewstate is the position of nodes within a flowchart. While not required to execute the flowchart, they are required to effectively view the flowchart.

Where to write them down?

This was a question that caused a fair amount of debate on the team. There are basically two places to write down things like view state in a file-based world.

In the source document itself
In a document that stays close to the source (usually referred to as a sidecar file)

We had customers asking for both. The motivation for the first is that for things like flowchart, where I may always care about the visualization representation, I want to keep that metadata around and only deal with one element. For the second, it is motivated by the reason that we want a clean source document that only describes the minimal artifact to run. Now, there are certainly many stops along the spectrum (for instance, we might always want to keep annotations or source comments in the source document, and put positioning elsewhere). For VS2010, we landed with a unified API to use, and we write in the source document. This is something that is likely to change in future releases, as it does make things like textual diffs rather painful.

So, that’s why we want to use it.

How do we use it?

We are going to create a simple activity designer that lets me write down a comment.

A few simple steps:

Create a new VS Project, let’s create an Activity Library
Add a Designer to that activity library
Add an attribute to the activity pointing to the designer
Add a new WorkflowConsoleApp Project
Build

Now, let’s go and make our activity designer a little interesting.

Let’s add a text box and a button. We’ll make the text of the button something obvious like “commit comment” The XAML for the activity designer looks like this:

<sap:ActivityDesigner x:Class="simpleActivity.CommentingDesigner"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
    xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
  <Grid>
        <StackPanel Name="stackPanel1" VerticalAlignment="Top" >
            <TextBox  Name="commentBlock"   />
            <Button Content="Load View State" Name="loadViewState" Click="loadViewState_Click" />
            <Button Content="Commit View State"  Name="button1" Click="button1_Click" />
        </StackPanel>
    </Grid>
</sap:ActivityDesigner>

Now, let’s add some code to the button (and to the initialization of the form)

ViewStateService has a few useful methods on it. I want to call out a subtle difference. You will see StoreViewState and StoreViewStateWithUndo. The primary distinction as the name implies is that one will simply write the view state down and will bypass the undo/redo stack. This is for view state like an expanded/collapsed view. You don’t really want ctl-z to simply flip expanded versus collapsed for you. But for something like flowchart, where changing some of the viewstate, like position, might be such a thing that you want support for undoing the action. That’s the primary difference.

So, our code for the button looks like this:

private void button1_Click(object sender, RoutedEventArgs e)
{
    ViewStateService vss = this.Context.Services.GetService<ViewStateService>();
    vss.StoreViewStateWithUndo(this.ModelItem, "comment", commentBlock.Text);
}

Now, on load, we want to be able to populate the value, so we will use the RetrieveViewState method in order to extract this.

private void loadViewState_Click(object sender, RoutedEventArgs e)
{
    ViewStateService vss = this.Context.Services.GetService<ViewStateService>();
    commentBlock.Text = vss.RetrieveViewState(this.ModelItem, "comment") as string;
}

Now, let’s go back to our workflow project and put an instance of this activity on the surface:

Let’s add some viewstate information and commit it. Now let’s look at the XAML:

<s4:NotRealInterestingActivity Text="{x:Null}" sap:VirtualizedContainerService.HintSize="200,99">
  <sap:WorkflowViewStateService.ViewState>
    <scg3:Dictionary x:TypeArguments="x:String, x:Object">
      <x:String x:Key="comment">basic comment</x:String>
    </scg3:Dictionary>
  </sap:WorkflowViewStateService.ViewState>
</s4:NotRealInterestingActivity>

Ok, now, to show that we can pull this in, let’s change the text in the xaml and then reload our designer.

You can muck around with ctl-z to see that this does get handled correctly via the undo.

The other important thing to note is that this takes an object, so your viewstate is not limited to strings, you can have more full featured objects if you’d like. Finally, the ViewStateService also has a ViewStateChanged you can subscribe to in order to handle, dispatch, and react to view state changes in the designer.

Where is System.Activities.Design in WF4 Beta2 and Beyond?

Matt Winkler -- MSFT — Sat, 05 Dec 2009 23:40:33 GMT

I got an email over the weekend asking about this, and I realized that it’s somewhat buried in this post here. Anyway, in Beta1, you often saw System.Activities.Design. For beta2 (and RTM), one important change

System.Activities.Design => System.Activities.Presentation

The primary reason for this change is that the *.Design suffix is generally reserved for VS design extensibility. As our designer ships in the framework, *.Design was not the correct suffix. *.Presentation is where we landed.

Hoping that putting this in the title lands this high up in search queries so that this post might be useful for a few people.

WF4, WCF and AppFabric Sessions @ PDC09

Matt Winkler -- MSFT — Mon, 23 Nov 2009 23:16:44 GMT

I’m taking this week off to catch up on everything I haven’t done the last two months and to celebrate the Thanksgiving holiday here in the US.

PDC was a blast! It was incredibly awesome to meet so many folks interested in WF and talking with others about how they are using (or plan to use) WF4. I’ll be following up with a more detailed post on my talk, including demos and code, but I wanted to give a summary of the talks that came from my team at this PDC. Below is the diagram that breaks down some of the “capabilities” of AppFabric and I have color coded them for the various talks that we gave. All of the PDC Sessions are available online here.

WF Talks

Spice Up Your Applications with Windows Workflow Foundation 4 – Matt Winkler
Windows Workflow Foundation 4 from the Inside Out – Bob Schmidt

WCF Talks

What’s New for Windows Communication Foundation 4 – Ed Pinto
Developing REST Applications with the .NET Framework – Henrik Nielsen

AppFabric Talks

Microsoft Application Server Technologies: Present and Future – Anil Nori
Microsoft BizTalk Server Futures and Roadmap – Balasubramanian Sriram
Workflow Services and Windows Server AppFabric – Mark Fussell
Application Server Extensibility with Microsoft .NET 4 and Windows Server AppFabric – Nicholas Allen

There’s a ton of great content up at the PDC site, plenty to sit back and enjoy!

A Few Additional Treats @ PDC09

Matt Winkler -- MSFT — Fri, 13 Nov 2009 21:15:50 GMT

One fun thing we will be doing this year is using the theater in our lounge/booth area to have a few chalk style presentations. These will be brief 30 minute demo/ q&a / conversation sessions with somebody from the product team. These will dive a little deeper and be more interactive about topics we know you’re interested in. We’d love you to come and just ask a ton of questions here:

Thursday, 11:00-11:30 / Future Directions for State Machine Workflows / Alan Ko

Discuss options for using the State Machine modeling style on WF4, and see a demonstration of a WF4-based State Machine design experience.

Thursday, 1-1:30 / Migrating WF 3.5 Workflows to WF 4 / Bob Schmidt

Learn some tips and techniques for migrating your WF 3.5 workflows to the new WF4 runtime, and see a demonstration of a tool that helps automate this process.

Thursday, 2:00-2:30 / WF Rehosting Deep Dive / Kushal Shah

See how WF’s rehostable runtime, designer, and debugger can add powerful capabilities to your applications

I’m excited because we’re going to use this as an opportunity to show some cool stuff that we’re thinking about. Usual disclaimers apply, some of the stuff here we will show are prototypes, ideas we are looking for feedback, etc, etc. These are things that we think are important, and your feedback helps us understand what you really need to make these things useful for you

Stop on by the booth if you’re interested in any of the above topics!

mwinkle.blog

Microsoft HDInsight Installation & Dependency Management

Hortonworks Data Platform installer

Microsoft HDInisght Installer

Alternate Approaches

What Does This Mean For Me?

Updating HDInsight Preview

What Are We Doing?

How do I Get This?

What’s New?

What if Something Doesn’t Work?

When Will These Bits Be Updated In Azure?

What’s Next?

Getting Up and Running with Piggybank on HDInsight

Every Day is a Good Day When You Paint

Writing a Map/Reduce Job for Hadoop using Windows PowerShell

Talking About Hadoop on Windows

Tap, tap, tap, is this thing on?

Thoughts on Flowchart

Arbitrary Rework

Coordination

How’d we get to where we are today?

Where does this leave us?

Improving ModelItem with Dynamic

VS 2010 RC Is Available

Making Swiss Cheese Look Good, or Designers for ActivityAction in WF4

Swiss Cheese and WF4, or, An Introduction to ActivityAction

Motivation

Introducing ActivityAction

Inspection, Default Services and Items (WF4 EditingContext Intro Part 6)

VS Standard Services and Items

Items

Services

Basic Rehosted Application Standard Services and Items

Items

Services

Comparison Table View

Conclusion

Subscription / Notification Engine (WF4 EditingContext Intro Part 5)

Services

Method

Description

Method

Description

Why is this Useful?

Items

Providing Callbacks for the Host (WF4 EditingContext Intro Part 4)

Host Provided Capabilities (WF4 EditingContext Intro Part 3)

Sharing Functionality Between Designers (WF4 EditingContext Intro Part 2)

Setup

What We Will See

Publishing a Service

Consuming a Service

Subscribing to Service

Before & After

What about Items?

Introduction to the WF4 Designer Editing Context (Part 1)

Motivation

Items

Services

Emitting the mc:Ignorable Instruction In Your WF4 XAML

Why use mc:Ignorable?

Why do I have to worry about this?

How Do I Get the Same Behavior?

AttachedProperty Part 2, Putting it Together

Pulling it all Together

WF4 Design Time AttachedPropertiesService and Attached Properties

Motivation

How Does it Work

Getter/ Setter?

Looking at Some Code

What do I do now?

Full Code Posting

WF4 ViewStateService

Motivation

Where to write them down?

How do we use it?

Where is System.Activities.Design in WF4 Beta2 and Beyond?

System.Activities.Design => System.Activities.Presentation

WF4, WCF and AppFabric Sessions @ PDC09