• mwinkle.blog

    Azure HDInsight Job Logging

    • 0 Comments

    We’ve made a nice fix to the Templeton job submission service that runs on the HDInsight clusters for remote job submission.  We’ve talked with a number of customers who want to be able to get access to the logs for the jobs remotely as well.  This typically requires access directly to the cluster.  We’ve updated Templeton to support dropping the job logs directly into ASV as part of the status directory.

    The way to do this is to pass “enablelogs” as a query string parameter set to true.  Here’s the what the request looks like:

    image

    Upon job completion, the logs will be moved into the status directory, under a logs folder with the following structure:

    $log_root/list.xml (summary of jobs)
    $log_root/stderr (frontend stderr)
    $log_root/stdout (frontend stdout)
    $log_root/$job_id (directory home for a job)
    $log_root/$job_id/job.xml.html
    $log_root/$job_id/$attempt_id (directory home for a attempt)
    $log_root/$job_id/$attempt_id/stderr
    $log_root/$job_id/$attempt_id/stdout
    $log_root/$job_id/$attempt_id/syslog

    Here’s a screen shot from Storage Studio that shows the folder structure:

    image

    If you look in the syslog file here, you’ll see a bunch of goodness about your job execution.  For more complex jobs that spin off multiple map/reduce jobs (eg, Pig, hive, Cascading), you will see the set of jobs recorded there. The root directory will also contain a list.txt with the details of all the jobs, and each jbo will contain a jobs.xml.html which contains all of the details, environment variables, and configuration of the job.  All of this information is useful in debugging and tuning your jobs.

    We will be updating the SDK to support this parameter in the next release, but for now, you can submit jobs directly to the cluster and add this parameter to get job logs.

    Here’s the text from my execution of a simple Hive query:

    2013-07-11 02:48:56,632 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
    2013-07-11 02:48:56,726 INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: c:/hdfs/mapred/local/taskTracker/distcache/1562199005048822745_1328179698_1268869633/namenodehost/hive/scratch/hive_2013-07-11_02-48-38_757_5240085401495207012/-mr-10003/34a2bf31-e18b-440d-8d91-be8d0e445d2e <- c:\hdfs\mapred\local\taskTracker\admin\jobcache\job_201307110233_0003\attempt_201307110233_0003_m_000000_0\work\HIVE_PLAN34a2bf31-e18b-440d-8d91-be8d0e445d2e
    2013-07-11 02:48:56,992 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
    2013-07-11 02:48:57,289 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.WindowsResourceCalculatorPlugin@a4f5b6d
    2013-07-11 02:48:57,726 WARN org.apache.hadoop.hive.conf.HiveConf: hive-site.xml not found on CLASSPATH
    2013-07-11 02:48:57,961 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library not loaded
    2013-07-11 02:48:58,101 INFO org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader: Processing file asv://mwinkletemp37@mwinkle.blob.core.windows.net/hive/warehouse/hivesampletable/HiveSampleData.txt
    2013-07-11 02:48:58,101 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0
    2013-07-11 02:48:58,117 INFO ExecMapper: maximum memory = 954466304
    2013-07-11 02:48:58,117 INFO ExecMapper: conf classpath = [file:/C:/hdfs/mapred/local/taskTracker/admin/jobcache/job_201307110233_0003/attempt_201307110233_0003_m_000000_0/classpath-5670276484193870096.jar]
    2013-07-11 02:48:58,117 INFO ExecMapper: thread classpath = [file:/C:/hdfs/mapred/local/taskTracker/admin/jobcache/job_201307110233_0003/attempt_201307110233_0003_m_000000_0/classpath-5670276484193870096.jar]
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Adding alias hivesampletable to work list for file asv://mwinkletemp37@mwinkle.blob.core.windows.net/hive/warehouse/hivesampletable
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.MapOperator: dump TS struct<clientid:string,querytime:string,market:string,deviceplatform:string,devicemake:string,devicemodel:string,state:string,country:string,querydwelltime:double,sessionid:bigint,sessionpagevieworder:bigint>
    2013-07-11 02:48:58,164 INFO ExecMapper:
    < MAP>Id =4
      <Children>
        <TS>Id =3
           <Children>
            <FIL>Id =2
              <Children>
                 <SEL>Id =1
                  <Children>
                    <FS>Id =0
                      <Parent>Id = 1 null<\Parent>
                    < \FS>
                  <\Children>
                  <Parent>Id = 2 null<\Parent>
                <\SEL>
              <\Children>
              <Parent>Id = 3 null<\Parent>
            <\FIL>
          <\Children>
          <Parent>Id = 4 null<\Parent>
        <\TS>
      <\Children>
    < \MAP>
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initializing Self 4 MAP
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing Self 3 TS
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Operator 3 TS initialized
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing children of 3 TS
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: Initializing child 2 FIL
    2013-07-11 02:48:58,164 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: Initializing Self 2 FIL
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: Operator 2 FIL initialized
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: Initializing children of 2 FIL
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing child 1 SEL
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing Self 1 SEL
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: SELECT struct<clientid:string,querytime:string,market:string,deviceplatform:string,devicemake:string,devicemodel:string,state:string,country:string,querydwelltime:double,sessionid:bigint,sessionpagevieworder:bigint>
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Operator 1 SEL initialized
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initializing children of 1 SEL
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Initializing child 0 FS
    2013-07-11 02:48:58,179 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Initializing Self 0 FS
    2013-07-11 02:48:58,195 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Operator 0 FS initialized
    2013-07-11 02:48:58,195 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Initialization Done 0 FS
    2013-07-11 02:48:58,195 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: Initialization Done 1 SEL
    2013-07-11 02:48:58,195 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: Initialization Done 2 FIL
    2013-07-11 02:48:58,195 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: Initialization Done 3 TS
    2013-07-11 02:48:58,195 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 4 MAP
    2013-07-11 02:48:58,601 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias hivesampletable for file asv://mwinkletemp37@mwinkle.blob.core.windows.net/hive/warehouse/hivesampletable
    2013-07-11 02:48:58,601 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 forwarding 1 rows
    2013-07-11 02:48:58,601 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 forwarding 1 rows
    2013-07-11 02:48:58,617 INFO ExecMapper: ExecMapper: processing 1 rows: used memory = 87647896
    2013-07-11 02:48:58,617 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 forwarding 10 rows
    2013-07-11 02:48:58,617 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 forwarding 10 rows
    2013-07-11 02:48:58,617 INFO ExecMapper: ExecMapper: processing 10 rows: used memory = 87647896
    2013-07-11 02:48:58,617 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 forwarding 100 rows
    2013-07-11 02:48:58,617 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 forwarding 100 rows
    2013-07-11 02:48:58,617 INFO ExecMapper: ExecMapper: processing 100 rows: used memory = 87647896
    2013-07-11 02:48:58,632 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 forwarding 1000 rows
    2013-07-11 02:48:58,632 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 forwarding 1000 rows
    2013-07-11 02:48:58,632 INFO ExecMapper: ExecMapper: processing 1000 rows: used memory = 87647896
    2013-07-11 02:48:58,804 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 forwarding 10000 rows
    2013-07-11 02:48:58,804 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 forwarding 10000 rows
    2013-07-11 02:48:58,804 INFO ExecMapper: ExecMapper: processing 10000 rows: used memory = 87647896
    2013-07-11 02:48:59,211 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 finished. closing...
    2013-07-11 02:48:59,211 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 forwarded 59793 rows
    2013-07-11 02:48:59,211 INFO org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
    2013-07-11 02:48:59,211 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 finished. closing...
    2013-07-11 02:48:59,211 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 forwarded 59793 rows
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: 2 finished. closing...
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: 2 forwarded 0 rows
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: PASSED:0
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: FILTERED:59793
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 finished. closing...
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarded 0 rows
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: 0 finished. closing...
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: 0 forwarded 0 rows
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS hdfs://namenodehost:9000/hive/scratch/hive_2013-07-11_02-48-38_757_5240085401495207012/_tmp.-ext-10001/000000_0
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS hdfs://namenodehost:9000/hive/scratch/hive_2013-07-11_02-48-38_757_5240085401495207012/_task_tmp.-ext-10001/_tmp.000000_0
    2013-07-11 02:48:59,226 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS hdfs://namenodehost:9000/hive/scratch/hive_2013-07-11_02-48-38_757_5240085401495207012/_tmp.-ext-10001/000000_0
    2013-07-11 02:48:59,336 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 Close done
    2013-07-11 02:48:59,336 INFO org.apache.hadoop.hive.ql.exec.FilterOperator: 2 Close done
    2013-07-11 02:48:59,336 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 3 Close done
    2013-07-11 02:48:59,336 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 4 Close done
    2013-07-11 02:48:59,336 INFO ExecMapper: ExecMapper: processed 59793 rows: used memory = 96526440
    2013-07-11 02:48:59,336 INFO org.apache.hadoop.mapred.Task: Task:attempt_201307110233_0003_m_000000_0 is done. And is in the process of commiting
    2013-07-11 02:48:59,382 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201307110233_0003_m_000000_0' done.
    2013-07-11 02:48:59,414 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
    2013-07-11 02:48:59,476 INFO org.apache.hadoop.io.nativeio.NativeIO: Initialized cache for UID to User mapping with a cache timeout of 14400 seconds.

  • mwinkle.blog

    Microsoft HDInsight Installation & Dependency Management

    • 1 Comments

    It’s a rainy Saturday afternoon here in Seattle, and the kids are keeping themselves busy running around the Christmas tree, so I’ve got a little time to put together a post that addresses some questions that have come up a few times in the forums as well as in our internal discussion aliases for our on-premises install of HDInisght.

    We currently use the Web Platform Installer to take care of dependency management of the installation. 

    It’s important to point out that there are actually two key pieces that get installed

    The second has a dependency on the first.  The WebPI feed also contains a number of pre-requisites required to set up IIS and a few other things for the HDInsight dashboard.  Here’s what it looks like on a completely fresh Windows Server 2012 machine.  Most developer machines likely have some or most of the IIS pre-reqs installed.  We’re also working to clean up some of this to minimize installation & setup.

    image

     

    Let’s talk a little bit about what’s in each one.

    Hortonworks Data Platform installer

    This msi includes the core Hadoop bits (Map/Reduce, HDFS), as well as a number of other Apache projects in the Hadoop ecosystem.  The full list included in the current installer are:

    • Map Reduce
    • HDFS
    • Hive
    • Pig
    • HCatalog

    Each of these projects is packaged into a zip file that contains a PowerShell script that automates the installation and setup of the component.  There are more components in Hortonworks Data Platform, and the teams are working to get these packaged and included.

    Microsoft HDInisght Installer

    This msi contains bits that are Microsoft specific, and may also contain additional Hadoop projects.  The current install (as of today) contains:

    • HDInsight dashboard
    • Sqoop
    • Isotope.js
    • Getting Started content

    These are packaged the same way as the Hadoop projects above.  Additionally, there is an installation PowerShell script here which will do some initialization of the single node installer, such as starting the services for the Hadoop components.

    Alternate Approaches

    The team discussed a number of potential factorings, and we very much welcome feedback here.  A few ideas that we’ve thought about:

    • Stable and Experimental packages.  This would allow us to set expectations around quality and stability of the bits. 
    • Decomposing every project into an individual msi
    • Integrating and building a Chocolatey package for these

    What Does This Mean For Me?

    What this means is that when you install HDInsight out of WebPI, you are installing two different msi’s.  We are revving the Microsoft msi every two weeks to pull in bug fixes (and very shortly include some experimental features).  The Hortonworks msi will be revved on a different schedule, as the team there decides to release an update.  We are partnering closely with the team there and so we will coordinate releases so that the combined installation will always work.

    More directly, this implies that if you want to uninstall completely, you will need to uninstall both packages from Add/Remove Programs:

    image

    This also means that when we issue an update for the Microsoft HDInsight package, you don’t have to “lose” your cluster by uninstalling both products.  You should be able to simply uninstall & update the Microsoft HDInsight package. 

    The team would love to get more feedback on this approach, so, let us know what you think!

  • mwinkle.blog

    Updating HDInsight Preview

    • 1 Comments

    Today we’ve shipped an update to the single node HDInsight Server Preview that is installable via Web Platform Installer.  

    What Are We Doing?

    Every two weeks, we’re going to take a snapshot of the work in progress and update the public installer with this (provided it passes a basic level of validation).  This gives us the opportunity to rapidly get new bits in front of customers, experiment with new features, and address bugs in a timely fashion. 

    We’ve also snapped versions such that releases of the SDK as well as the One-Box installer will all share the same version.  As such, we’ve versioned the msi as 0.2.0.0.

    How do I Get This?

    Simply by installing HDInisght out of WebPI onto a new machine after uninstalling (in place upgrade is not currently supported).  To completely uninstall previous versions, you will want first uninstall “Microsoft HDInsight Community Technology Preview” and then the “Hortonworks Data Platform”.  We will not preserve data in HDFS using uninstall/reinstall, so archive that prior to uninstalling.

    clip_image002

    What’s New?

    There are no new features in this release, we’ve worked to improve some of the install and setup experience to address some bugs that have been reported on this alias as well as the forums.

    • IIS / Dashboard setup issues experienced on certain OS SKU’s
    • Addition of HDInsight Dashboard link into Start Menu
    • Fix to Hive console for multiple line input errors
    • Clean up some uninstall issues
    • Minor changes to getting started content

    Hortonworks has also shipped an update to the HDP installation to address the bug encountered on Friday (documented here).  By removing both installations above and re-installing, you will get updated bits here.

    What if Something Doesn’t Work?

    Please report issues on the forums.  We’re also tracking a set of known issues via the release notes.

    When Will These Bits Be Updated In Azure?

    We’re using this as an opportunity to ship these bits early and often, and they will find their way into future service updates to Azure HDInsight, typically in the next monthly update.

    What’s Next?

    We’ll continue to address common issues as they are reported.  From a feature perspective, we hope the next update will include updates to the dashboard to include a new and improved Hive console.

  • mwinkle.blog

    Getting Up and Running with Piggybank on HDInsight

    • 0 Comments

    It’s been a fun couple of weeks launching HDInsight, and I’m going to be getting back into doing some more technical blogging.  There are a few easy topics off the bat that we’ve heard requested from customers.  The first one involves Piggybank, which is a user contributed collection of useful Pig user defined functions (UDF’s).

    This assumes your machine is set up with:

    • HDInsight (grab from WebPI here)
    • Java build tools (Ant and Ivy on your command path)

    Next, let’s build Piggybank by grabbing the Pig source code and checking out the 0.9 branch

     

    At this point, you should now have a pig directory, move to that and type ant in order to build.

    Next, navigate to .\contrib\piggybank\java, and again, type ant in order to build.  This will produce piggybank.jar.

    Next, open your HDInsight console window and type pig.  This brings up Grunt, the interactive pig shell.

    At this point, you can now use the following in your script:

     

    REGISTER C:\Your\Path\To\piggybank.jar ;

    foo = FOREACH entry GENERATE org.apache.pig.piggybank.evaluation.string.UPPER(item_name);

     

    At this point, you can now take advantage of all of the functions in piggybank, and if you’re interested in contributing your own, details are here.

  • mwinkle.blog

    Every Day is a Good Day When You Paint

    • 0 Comments

    Slightly off-topic, but a friend of mine posted a video on Facebook that struck me as being very relevant to our space. 

     

    At the same time I saw this, another friend and co-worker was having a fairly crummy day with lots of crazy meetings and requests from co-workers, but that day was made a lot better by sitting down and solving a fun little deployment script problem.  The deployment script is neither here nor there, it was more that so many of the folks in this industry love solving a problem, whether that is getting a query right, chasing down a pesky bug, or getting that a-ha moment when you are trying to design something.  That’s “painting” for a lot of us. 

    Every day is a good day when you paint.

    [and working auto-tuned Bob Ross into a blog post was just too good to pass up]

  • mwinkle.blog

    Writing a Map/Reduce Job for Hadoop using Windows PowerShell

    • 0 Comments

    I had a little bit of time on my hand and wanted to whip up a quick sample using PowerShell for a Hadoop job.

    This uses the Hadoop streaming capability, which essentially allows for mappers and reducers to be written as arbitrary executables that operate on standard input and output.

    The .ps1 scripts are pretty simple, these operate over a set of airline data that looks like this:

    0,EV,18,0,MIA,2011-12-18T00:00:00,12,TPA,1227859,2011    
    0,EV,17,0,MIA,2011-12-17T00:00:00,12,TPA,1227860,2011    
    6,EV,17,4,CLE,2011-12-17T00:00:00,12,ATL,1227861,2011

    The schema here is a comma separated set of US flights with delays.

    The goal of my job is to pull out the airlines, the number of flights, and then some very basic (min and max) statistics on the arrival and departure delays. 

    My mapper:

       1:  function Map-Airline 
       2:  {
       3:      [Console]::Error.WriteLine( "reporter:counter:powershell,invocations,1")
       4:      $line = [Console]::ReadLine()
       5:      while ($line -ne $null) 
       6:      {
       7:          [Console]::WriteLine($line.Split(",")[1] + "`t" + $line)
       8:          [Console]::Error.WriteLine("reporter:counter:powershell,record,1")
       9:          $line = [Console]::ReadLine()
      10:      }  
      11:  }
      12:   
      13:   
      14:  Map-Airline 

    My reducer:

       1:  function Reduce-Airlines
       2:  {
       3:      $line = ""
       4:      $oldLine = "<initial invalid row value>"
       5:      $count = 0
       6:      $minArrDelay = 10000
       7:      $maxArrDelay = 0
       8:      $minDepDelay = 10000 
       9:      $maxDepDelay = 0 
      10:   
      11:      $line = [Console]::ReadLine()
      12:   
      13:      while  ($line -ne $null)
      14:      {
      15:          if (($oldLine -eq $line.Split("`t")[0]) -or ($oldLine -eq "<initial invalid row value>"))
      16:          {
      17:              $flightRecord = $line.Split("`t")[1].Split(',')
      18:              if ([Int32]::Parse($flightRecord[0]) -ne 0) 
      19:              {
      20:                  $minArrDelay = [Math]::Min($minArrDelay, $flightRecord[0])
      21:              }
      22:              if ([Int32]::Parse($flightRecord[3]) -ne 0) 
      23:              {
      24:                  $minDepDelay = [Math]::Min($minDepDelay, [Int32]::Parse($flightRecord[3]))
      25:              }
      26:              $maxArrDelay = [Math]::Max($maxArrDelay, $flightRecord[0])
      27:              $maxDepDelay = [Math]::Max($maxDepDelay, $flightRecord[3])
      28:              $count = $count+ 1
      29:              [Console]::Error.WriteLine( "reporter:counter:powershell,"+$oldLine + ",1")
      30:          }
      31:          else
      32:          {
      33:              [Console]::WriteLine($oldLine + "`t" + $count + "," + $minArrDelay + "," +$maxArrDelay + "," + $minDepDelay +","+ $maxDepDelay)
      34:              $count = 1
      35:              $minArrDelay = 10000
      36:              $maxArrDelay = 0
      37:              $minDepDelay = 10000 
      38:              $maxDepDelay = 0 
      39:              [Console]::Error.WriteLine("reporter:counter:powershell,"+$oldLine + ",1")           
      40:          }
      41:          $oldLine = $line.Split("`t")[0]
      42:          $line = [Console]::ReadLine()
      43:      }
      44:      [Console]::WriteLine($oldLine + "`t" + $count + "," + $minArrDelay + "," +$maxArrDelay + "," + $minDepDelay +","+ $maxDepDelay)
      45:  }
      46:   
      47:  Reduce-Airlines

    One thing to note on the reducer is that we use the $oldLine variable in order to keep tabs on when our group of results is moving to the next one.  When using Java, your reduce function will be invoked once per group and so you can reset the state at the beginning of each of those.  With streaming, you will never have groups that split reducers, but your executable will only be spun up once per reducer (which, in the sample here, is one).  You can also see that I’m writing out to STDERR in order to get a few counters recorded as well.

    The next trick is to get these to execute.  The process spawned by the Streaming job does not know about .ps1 files, it’s basically just cmd.exe.  To get around that we will also create a small driver .cmd file and upload the file with the –file directive from the command line.

     

    AirlineMapperPs.cmd

    @call c:\windows\system32\WindowsPowerShell\v1.0\powershell -file ..\..\jars\AirlineMapper.ps1

     

    AirlineReducerPs.cmd

    @call c:\windows\system32\WindowsPowerShell\v1.0\powershell -file ..\..\jars\AirlineReducer.ps1

     

    The ..\..\jars directory is where the –file directive will place the files when they execute

    And now we execute:

    hadoop jar %HADOOP_HOME%\lib\hadoop-streaming.jar 
        -mapper d:\dev\_full_path_\AirlineMapperPs.cmd 
        -reducer d:\dev\_full_path_\AirlineReducerPs.cmd 
        -input fixed_flights 
        -output psMapReduce3 
        -file d:\dev\_full_path_\AirlineMapper.ps1 
        -file d:\dev\_full_path_\AirlineReducer.ps1

    And we get our results.

    There is still some work to be done here, I’d like to make it a little easier to get these running (so possibly wrapping submission in a script which takes care of the wrapping for me).  Also, on Azure, we either need to sign the scripts, or log into each of the machines and allow script execution.  As I get that wrapped up, I’ll drop it along side our other samples.  We’ll also work to make this easier to get up and running on Azure if that’s interesting for folks.

  • mwinkle.blog

    Talking About Hadoop on Windows

    • 0 Comments

    A few folks have asked, so I decided to put the data in one place.  We'll be talking more about Hadoop at TechEd North America and Hadoop Summit next week, and then later in the month at TechEd Europe.  

    Here are the sessions we're presenting:

    • TechEd (NA and Europe, links are to NA sessions)
      • Learn Big Data Application Development on Windows Azure -- Wenming Ye
        • Web 2.0 companies have been fully taking advantage of Hadoop based open source tools to tackle Big Data needs. Microsoft now offers the best of both worlds with its own Hadoop solution on Windows Azure with full compatibility and additional rich toolsets. This session is a "getting-started" tutorial on developing Big Data applications on Windows Azure. We cover application scenarios, Hadoop on Azure, tools, and applied data analytics. More importantly, we show you how to put everything together with a couple of sample applications
      • Big Data, Big Deal? -- Gert Drapers
        • Are you ready for the exploding world of big data? Do you know the difference between Hive and Pig? Do you know why MapReduce is being taught in many universities rather than SQL? If not, pay attention because this talk will help get you started in understanding this new world. While sometimes the Hadoop toolkit (which includes HDFS, MapReduce, Hive, Pig, and Sqoop) is used as an alternative to relational database systems such as SQL Server, more frequently customers are using it as a complementary tool. Sometimes it may be used as an ETL tool or to perform an initial analysis of a freshly acquired data set to determine whether or not it is worth loading into the data warehouse, and sometimes to process massive data sets that are too big to even contemplate loading into all but the very largest data warehouses. In addition to covering the basics of the various parts of the Hadoop stack, this talk discusses the strengths and weakness of the Hadoop approach compared to that provided by relational database systems and explores how the two technologies can be used productively in conjunction with one another.
      • Harnessing Big Data With Hadoop
        • Attend this session to learn about the Hadoop Big Data solution from Microsoft that unlocks insights on all your data, including structured and unstructured data of any size. Accelerate your analytics with a Hadoop service that offers integration with Microsoft BI and the ability to enrich your models with publicly available data. Finally, learn about our roadmap for Hadoop on Windows Server and Windows Azure and for broadening access to Hadoop through simplified deployment, management and programming, including JavaScript integration
    • Hadoop Summit
      • Unleash Insights On All Data With Microsoft Big Data -- Tim Mallalieu
        • Do you plan to extract insights from mountains of data, including unstructured data that is growing faster than ever? Attend this session to learn about Microsoft’s Big Data solution that unlocks insights on all your data, including structured and unstructured data of any size. Accelerate your analytics with a Hadoop service that offers deep integration with Microsoft BI and the ability to enrich your models with publicly available data from outside your firewall. Come and see how Microsoft is broadening access to Hadoop through dramatically simplified deployment, management and programming, including full support for JavaScript.
      • How Klout is changing the landscape of social media with Hadoop and BI -- Denny Lee, David Mariani (VP of Engineering, Klout)
        • In this age of Big Data, data volumes grow exceedingly larger while the technical problems and business scenarios become more complex. Compounding these complexities, data consumers are demanding faster analysis to common business questions asked of their Big Data. This session provides concrete examples of how to address this challenge. We will highlight the use of Big Data technologies—including Hadoop and Hive —with classic BI systems such as SQL Server Analysis Services.

          Session takeaways:
          • Understand the architectural components surrounding Hadoop, Hive, Classic BI, and the Tier-1 BI ecosystem
          • Get strategies for addressing the technical issues when working with extremely large cubes
          • See how to address the technical issues when working with Big Data systems from the DBA perspective

    I think that there is a pretty nice mix of Hadoop, Microsoft plans, and applications across these sessions.  Hope you get a chance to see them (or watch them after the events!)

  • mwinkle.blog

    Tap, tap, tap, is this thing on?

    • 2 Comments

    I'll be back to bloging here in a bit.  Since you last visited, I've had a fun journey working further on Workflow, Azure and now I've landed in the SQL team working on Hadoop, in particular, the developer story.

    Not a lot to say right now, but make sure to check out hadoopOnAzure.com !

  • mwinkle.blog

    Thoughts on Flowchart

    • 0 Comments

    Last night I saw that Maurice had a few tweets that caught my attention about flowchart, but this is the one that I want to talk about:

    Come to think of it I also really mis a parallel execution in a flowchart. But other than that flowcharts rock! #wf4 #dmabout 13    hours ago via web

    I thought about replying on twitter, but it’s a bit of a longer topic.   When we were building the flowchart for VS2010, we considered a number of possible capabilities, but ultimately the capabilities customers were looking for fell into two categories:

    • Arbitrary rework patterns
    • Coordination patterns

    Arbitrary Rework

    We can describe the model of execution in the current flowchart to be one that supports arbitrary rework.  You could also refer to this as GOTO.   One benefit (and downside) of breaking out of the tightly scoped, hierarchical execution model that we see with the Sequential style of workflows (and in most procedural languages) is the fact there exists more freedom in terms of defining the “next” step to be taken.  The simplest way to think about this is the following flowchart:

    image

    Now, this isn’t particularly interesting, and most folks who look at this will simply ask “why not a loop?”  which in this case is a valid question.  As a process gets more sophisticated (or if humans tend to be involved), the number of possible loops required gets rather tricky to manage in a sequential flow (consider the following rework scenario which includes “next steps” across conditional boundaries, and from some branches but not others.

    image

    Now, mathematically, we can describe the machinery in both of these machines as a set of valid transitions, and it is likely possible that there exists a complete mapping from any flowchart into procedural.

    Coordination

    The second pattern we consistently saw was the use of a free form style of execution in order to build complex coordination patterns.  The example I consistently return to (pointing back to a blog post I made back in the WF3 days)

    image


    Here I want to be able to coordinate a few different things, namely that 3 executes when 1 completes, 5 when 2, 4 when 1 AND 2, 6 when 3 AND 4 AND 5 complete.  Here we’ve created a dependency graph that can’t be handled with the procedural constructs that we have.  How could this happen? Imagine we’re trying to model the ranking process for a series of horse races.  Task 3 can happen when Race 1 completes, as Task 5 can happen when Race 2 completes.  Task 4 represents some work that requires the data from both Races.  When those 3 tasks (3, 4, and 5) complete, I can move forward and take some action (like bet on the next race). 

    This type of pattern can be very powerful, and is often described by petri-nets.  There exists a multitude of join semantics from simple AND joins to conditional joins (ranging from a simple “count” to a condition based on data). 

    How’d we get to where we are today?

    Today, the flowchart in the .NET framework only supports the former pattern, that of arbitrary rework.  How’d we get there.  While we found a lot of value in both patterns what we found when we tried to compose them, we often got into places that became very hard to have predictable execution semantics.   The basic crux of the design issue gets to the “re-arming” of a join.  If I decide to jump back to task 3 at some point, what does it mean for 3 to complete?  Do I execute 6 again right away, do I wait for 4 and 5 to complete a second time?  What happens if I jump to 3 a second time?  Now, there certainly exist formal models for this, and things like Petri-net define a set of semantics for these things.  What we found though was that any expression of the model that took these things into account required a pretty deep understanding of the process model, and we lost one of the key benefits we see from a flowchart, which is that it is a simple, approachable model.  This is not to say that we don’t think that the Coordination pattern is useful, or that petri-nets don’t ultimately hold the unified approach,  we just did not have the time to implement a coordination pattern in VS 2010, and creating an approachable petri-net backed flowchart modeling style was something we’d need more time to get correct. 

    Where does this leave us?

    If you’re looking at this and saying, “but I want coordination (or a full blown petri-net)”, there are a couple of options:

    • Roll up your sleeves and write that activity.  In the blog post above, I outline how you could go about building the coordination pattern, and with the new activity model, this would be simpler.  The harder part is still writing a designer, and there is now some good guidance on free form designers in the source of the state machine on codeplex. 
    • Provide feedback on connect, when you do this it opens a bug in our bug database that our team is looking at daily.  Describe your scenario, what you’re trying to do, and what you’d like to see.  This type of customer feedback is vital to helping us plan the right features for the next release.  
  • mwinkle.blog

    Improving ModelItem with Dynamic

    • 0 Comments

    Hi all, I’m back, and apologize for the delay, things have been crazy here and I’ve taken a little time off following shipping VS2010, so I’m getting back into it.  I’d like to talk about one of my favorite late additions to the WF4 designer programming model.  We added this change sometime in the RC/RTM timeframe in response to some feedback that we had gotten about the verbose-ness of programming against ModelItem.

    If you recall from this post, ModelItem exists to serve as a INotifyPropertyChanged (INPC) aware proxy to the underlying instance being edited within the designer. In exchange for this fairly flexible proxy layer, you get in return a pretty verbose way in which one has to interact to actually get to the data.  Often times there are fairly long incantations of myModelItem.Properties[“foo”].Value.Properties[“bar”].Value.GetCurrentValue() in order to actually return the data (or be able to set the value).  From the beginning, we had decided we wanted to optimize for the WPF XAML consumer of the model item, and so we chose to implement ICustomTypeDescriptor so that WPF binding would be pretty simple (the above would result in a binding syntax of ModelItem.foo.bar). 

    One quick note, looking at the definition of ModelItem itself will show that only the INPC interface is implemented.  However, ModelItem is an abstract type, and the implementation subclass ModelItemImpl contains the definition and implementation of the following interfaces ModelItem, IModelTreeItem, ICustomTypeDescriptor, IDynamicMetaObjectProvider

    As we worked with some of our customers, it became clear that they were frequently using ModelItem in code as well, and so the programming model was becoming more of a problem (we had somewhat hoped that the majority of customers would be using XAML).  We initially went with the idea that we would create a series of Design time peers to all of the common types in WF (making a SequenceDesign, ParallelDesign, etc) which provided an identical programming surface to its peer, but under the covers in the property getters and setters would redirect to an underlying ModelItem.  This has the advantage that we could create the appearance of a strongly typed API surface, but has the downside that it introduces an extra type for every activity we ship, and it requires customers to create two types as well, and not just for activities, you would want one for customer too.  The other approach we kicked around was using the Dynamic capabilities introduced in .NET 4.

    It’s important to note that these are ultimately complimentary approaches, that is, it’s possible in a future framework release (or in a codeplex project if anyone is looking to for a little fun on the side) you could release the strongly typed design time peers. From a timing perspective, it was not likely that we could introduce 30+ new types in the RC/RTM release, and we had questions about the long term sustainability of requiring an additional type for activities.  So, we went ahead with the plan for dynamic support.   In order to do this, we had to add an implementation of IDynamicMetaObjectProvider.  Bill Wagner has a good article here on the basics of doing this.  This requires us to implement one method, GetMetaObject.  In our case, we have an internal, nested ModelItemMetaObject type which inherits from System.Dynamic.DynamicMetaObject. 

    In our case, there are basically two important methods that we care about, BindGetMember and BindSetMember.  These wire up to two methods on ModelItemImpl to get and set based on the property name.  The net result of this is that the following code

    root.Properties["Residence"].
            Value.
            Properties["StreetAddress"].
            Value.GetCurrentValue()

    Can now be written as

    dynamic mi = root;
    Console.WriteLine(root.Residence.StreetAddress)

    Now, with anything there are some tradeoffs, the biggest one is that this is dynamic, I don’t get compile time checking, and I could have a type that will result in an exception being thrown (that said, that problem exists in the verbose form as well).  This does let me write much more succinct code when programming the model item tree though for both gets and sets.  As an added bonus, we found there is actually a decent perf improvement from the WPF XAML data binding that now leverages the IDynamicMetaObjectProvider (mentioned in ScottGu’s post here) in the way that the runtime will only compute the way to resolve the property once (as opposed to every time when it goes through the ICustomTypeDescriptor code path).

     

    Enjoy!

  • mwinkle.blog

    VS 2010 RC Is Available

    • 0 Comments

    We’ve been hard at work here in Redmond (and with our team in Shanghai) working on getting WF ready for release.  We’ve made a ton of progress in the RC build that was made available yesterday, please download it and check it out.  Also, and in important bold text, if you have feedback, please, please, please file an issue on Connect so that the team can look at it right away.  One thing that I want to point out about Connect is that it is not a vacuum, the entries there go directly into our bug tracking system, which we look at daily in our various shiproom meetings.  If something isn’t working right, please let us know!

    image

    We’ve fixed a number of issues that came directly from Connect in this last milestone, including some areas that revealed some gaps in our testing.  Your feedback is making this better.  For bugs that we’ve marked as postponed, as we start planning for vNext, we will start going through those to figure out the areas we need to improve on. 

    It’s cool the home stretch, this morning I presented to our support team to get them prepared to handle any PSS incidents that occur.  It’s great to see the release coming together.

  • mwinkle.blog

    Making Swiss Cheese Look Good, or Designers for ActivityAction in WF4

    • 4 Comments

    In my last post, I covered using ActivityAction in order to provide a schematized callback, or hole, for the consumers of your activity to supply.  What I didn’t cover, and what I intend to here, is how to create a designer for that.

    If you’ve been following along, or have written a few designers using WorkflowItemPresenter, you may have a good idea how we might go about solving this.  There are a few gotcha’s along the way that we’ll cover as we go through this.

    First, let’s familiarize ourselves with the Timer example in the previous post:

    using System;
    using System.Activities;
    using System.Diagnostics;
     
    namespace WorkflowActivitiesAndHost
    {
        public sealed class TimerWithAction : NativeActivity<TimeSpan>
        {
            public Activity Body { get; set; }
            public Variable<Stopwatch> Stopwatch { get; set; }
            public ActivityAction<TimeSpan> OnCompletion { get; set; }
     
            public TimerWithAction()
            {
                Stopwatch = new Variable<Stopwatch>();
            }
     
            protected override void CacheMetadata(NativeActivityMetadata metadata)
            {
                metadata.AddImplementationVariable(Stopwatch);
                metadata.AddChild(Body);
                metadata.AddDelegate(OnCompletion);
            }
     
            protected override void Execute(NativeActivityContext context)
            {
                Stopwatch sw = new Stopwatch();
                Stopwatch.Set(context, sw);
                sw.Start();
                // schedule body and completion callback
                context.ScheduleActivity(Body, Completed);
     
            }
     
            private void Completed(NativeActivityContext context, ActivityInstance instance)
            {
                if (!context.IsCancellationRequested)
                {
                    Stopwatch sw = Stopwatch.Get(context);
                    sw.Stop();
                    Result.Set(context, sw.Elapsed);
                    if (OnCompletion != null)
                    {
                        context.ScheduleAction<TimeSpan>(OnCompletion, Result.Get(context));
                    }
                }
            }
     
            protected override void Cancel(NativeActivityContext context)
            {
                context.CancelChildren();
                if (OnCompletion != null)
                {
                    context.ScheduleAction<TimeSpan>(OnCompletion, TimeSpan.MinValue);
                }
            }
        }
    }
     

     

    So, let’s build a designer for this.  First we have to provide a WorkflowItemPresenter bound to the .Body property.  This is pretty simple.  Let’s show the “simple” XAML that will let us easily drop something on the Body property

    <sap:ActivityDesigner x:Class="actionDesigners.ActivityDesigner1"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
        xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
        <StackPanel>
            <sap:WorkflowItemPresenter 
                     HintText="Drop the body here" 
                     BorderBrush="Black" 
                     BorderThickness="2" 
                     Item="{Binding Path=ModelItem.Body, Mode=TwoWay}"/>
            <Rectangle Width="80" Height="6" Fill="Black" Margin="10"/>
        </StackPanel>
    </sap:ActivityDesigner>

    Not a whole lot of magic here yet.  What we want to do is add another WorkflowItemPresenter, but what do I bind it to? Well, let’s look at how ActivityDelegate is defined [the root class for ActivityAction and ActivityFunc (which I’ll get to in my next post).:

    image

    hmmm, Handler is an Activity, that looks kind of useful.    Let’s try that:

    [warning, this XAML won’t work, you will get an exception, this is by design :-) ]

    <sap:ActivityDesigner x:Class="actionDesigners.ActivityDesigner1"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
        xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
        <StackPanel>
            <sap:WorkflowItemPresenter HintText="Drop the body here" BorderBrush="Black" BorderThickness="2" Item="{Binding Path=ModelItem.Body, Mode=TwoWay}"/>
            <Rectangle Width="80" Height="6" Fill="Black" Margin="10"/>
    <!-- this next line will not work like you think it might --> 
            <sap:WorkflowItemPresenter HintText="Drop the completion here" BorderBrush="Black" BorderThickness="2" Item="{Binding Path=ModelItem.OnCompletion.Handler, Mode=TwoWay}"/>
    
        </StackPanel>
    </sap:ActivityDesigner>

    While this gives us what we want visually, there is a problem with the second WorkflowItemPresenter (just try dropping something on it):

    image

    Now, if you look at the XAML after dropping, the activity you dropped is not present.  What’s happened here:

    • The OnCompletion property is null, so binding to OnCompletion.Handler will fail
    • We (and WPF) are generally very forgiving of binding errors, so things appear to have succeeded. 
    • The instance was created fine, the ModelItem was created fine, and the it was put in the right place in the ModelItem tree, but there is no link in the underlying object graph, basically, the activity that you dropped is not connected
    • Thus, on serialization, there is no reference to the new activity in the actual object, and so it does not get serialized.

    How can we fix this?

    Well, we need to patch things up in the designer, so we will need to write a little bit of code, using the OnModelItemChanged event.  This code is pretty simple, it just means that if something is assigned to ModelItem, if the value of “OnCompletion” is null, initialize it.  If it is already set, we don’t need to do anything (for instance, if you used an IActivityTemplateFactory to initialize).  One important thing here (putting on the bold glasses) YOU MUST GIVE THE DELEGATEINARGUMENT A NAME.  VB expressions require a named token to reference, so, please put a name in there (or bind it, more on that below).

    using System;
    using System.Activities;
    
    namespace actionDesigners
    {
        // Interaction logic for ActivityDesigner1.xaml
        public partial class ActivityDesigner1
        {
            public ActivityDesigner1()
            {
                InitializeComponent();
            }
    
            protected override void OnModelItemChanged(object newItem)
            {
                if (this.ModelItem.Properties["OnCompletion"].Value == null)
                {
                    this.ModelItem.Properties["OnCompletion"].SetValue(
                        new ActivityAction<TimeSpan>
                        {
                            Argument = new DelegateInArgument<TimeSpan>
                            {
                                Name = "duration"
                            }
                        });
                }
                base.OnModelItemChanged(newItem);
    
            }
        }
    }

    Well, this works :-)  Note that you can see the duration DelegateInArgument that was added.

    image

    Now, you might say something like the following “Gosh, I’d really like to not give it a name and have someone type that in” (this is what we do in our ForEach designer, for instance).  In that case, you would need to create a text box bound to OnCompletion.Argument.Name, which is left as an exercise for the reader.

    Alright, now you can get out there and build activities with ActivityActions, and have design time support for them!

    One question brought up in the comments on the last post was “what if I want to not let everyone see this” which is sort of the “I want an expert mode” view.  You have two options.  Either build two different designers and have the right one associated via metadata (useful in rehosting), or you could build one activity designer that switches between basic and expert mode and only surfaces these in expert mode.

  • mwinkle.blog

    Swiss Cheese and WF4, or, An Introduction to ActivityAction

    • 8 Comments

    swiss cheese

    One common scenario that was often requested by customers of WF 3 was the ability to have templated or “grey box” or “activities with holes” in them (hence the Swiss cheese photo above).  In WF4 we’ve done this in a way that way we call ActivityAction

    Motivation

    First I’d like to do a little bit more to motivate the scenario. 

    Consider an activity that you have created for your ERP system called CheckInventory.  You’ve gone ahead and encapsulated all of the logic of your inventory system, maybe you have some different paths of logic, maybe you have interactions with some third party systems, but you want your customers to use this activity in their workflows when they need to get the level of inventory for an item. 

    Consider more generally an activity where you have a bunch of work you want to get done, but at various, and specific, points throughout that work, you want to allow the consumer of that activity to receive a callback and provide their own logic to handle that.  The mental model here is one of delegates.

    Finally, consider providing the ability for a user to specify the work that they want to have happen, but also make sure that you can strongly type the data that is passed to it.  In the first case above, you want to make sure that the Item in question is passed to the action that the consumer supplies. 

    In wf3, we had a lot of folks want to be able to do something like this. It’s a very natural extension to wanting to model things as activities and composing into higher level activities.  We like being able to string together 10 items as a black box for reuse, but we really want the user to specify exactly what should happen between steps 7 and 8. 

    A slide that I showed at PDC showed it this way (the Approval and Payment boxes represent the places I want a consumer to supply additional logic):

    image 

     

     

    Introducing ActivityAction

    Very early on the release, we knew this was one of the problems that we needed to tackle. The mental model that we are most aligned with is that of a delegate/ callback in C#.  If you think about a delegate, what are you doing, you are giving an object the implementation of some bit of logic that the object will subsequently call.  That’s the same thing that’s going on with an ActivityAction.  there are three important parts to an ActivityAction

    • The Handler (this is the logic of the ActivityAction)
    • The Shape (this determines the data that will be passed to the handler)
    • The way that we invoke it from our activity

    Let’s start with some simple code (this is from a demo that I showed in my PDC talk).  This is a timer activity which allows us to time the execution for the contained activity and then uses an activity action to act on the result.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Activities;
    using System.Diagnostics;
     
    namespace CustomActivities.ActivityTypes
    {
     
        public sealed class Timer : NativeActivity<TimeSpan>
        {
            public Activity Body { get; set; }
            public Variable<Stopwatch> Stopwatch { get; set; }
            public ActivityAction<TimeSpan> OnCompletion { get; set; }
     
            public Timer()
            {
                Stopwatch = new Variable<Stopwatch>();
            }
     
            protected override void CacheMetadata(NativeActivityMetadata metadata)
            {
                metadata.AddImplementationVariable(Stopwatch);
                metadata.AddChild(Body);
                metadata.AddDelegate(OnCompletion);
            }
     
            protected override void Execute(NativeActivityContext context)
            {
                Stopwatch sw = new Stopwatch();
                Stopwatch.Set(context, sw);
                sw.Start();
                // schedule body and completion callback
                context.ScheduleActivity(Body, Completed);
     
            }
     
            private void Completed(NativeActivityContext context, ActivityInstance instance)
            {
                if (!context.IsCancellationRequested)
                {
                    Stopwatch sw = Stopwatch.Get(context);
                    sw.Stop();
                    Result.Set(context, sw.Elapsed);
                    if (OnCompletion != null)
                    {
                        context.ScheduleAction<TimeSpan>(OnCompletion, Result.Get(context));
                    }
                }
            }
     
            protected override void Cancel(NativeActivityContext context)
            {
                context.CancelChildren();
                if (OnCompletion != null)
                {
                    context.ScheduleAction<TimeSpan>(OnCompletion, TimeSpan.MinValue);
                }
            }
        }
    }

    A few things to note about this code sample:

    • The declaration of an ActivityAction<TimeSpan> as a member of the activity.  You’ll note we use the OnXXX convention often for activity actions.
    • The usage of the ActivityAction<T> with on type argument.  The way to read this, or any of the 15 other types is that the T is the type of the data that will be passed to the activity action’s handler.
      • Think about this like an Activity<Foo> corresponding to a void DoSomething(Foo argument1) method
    • The call to NativeActivityMetadata.AddDelegate() which lets the runtime know that it needs to worry about the delegate
    • The code in the Completed( ) method which checks to see if OnCompletion is set and then schedules it using ScheduleAction.  I want to call out that line of code.
    if (OnCompletion != null)
    {
        context.ScheduleAction<TimeSpan>(OnCompletion, Result.Get(context));
    }

    It is important to note that I use the second parameter (and the third through 16th if that version is provided) in order to provide the data.  This way, the activity determines what data will be passed to the handler, allowing the activity to determine what data is visible where.  This is a much better way than allowing an invoked child to access any and all data from the parent.  This lets us be very specific about what data goes to the ActivityAction.   Also, you could make it so that OnCompletion must be provided, that is, the only way to use the activity is to supply an implementation.  If you have something like “ProcessPayment” you likely want that to be a required thing.  You can use the CacheMetadata method in order to check and validate this.

    Now, let’s look at the code required to consume this time activity:

    DelegateInArgument<TimeSpan> time = new DelegateInArgument<TimeSpan>();
    a = new Timer
    {
        Body = new HttpGet { Url = "http://www.microsoft.com" },
        OnCompletion = new ActivityAction<TimeSpan>
        {
            Argument = time,
            Handler = new WriteLine { 
                Text = new InArgument<string>(
                    ctx => 
                        "Time input from timer " + time.Get(ctx).TotalMilliseconds)
            } 
    
        }
    };

    There are a couple of interesting things here:

    • Creation of DelegateInArgument<TimeSpan> : This is used to represent the data passed by the ActivityAction to the handler
    • Creation of the ActivityAction to pass in.  You’ll note that the Argument property is set to the DelegateInArgument, which we can then use in the handler
    • The Handler is the “implementation” that we want to invoke.  here’s it’s pretty simple, it’s a WriteLine and when we construct the argument we construct if from a lambda that uses the passed in context to resolve the DelegateInArgument when that executes.

    At runtime, when we get to the point in the execution of the Timer activity, the WriteLine that the hosting app provided will be scheduled when the ScheduleAction is called.  This means we will output the timing information that the Timer observed.  A different implementation could have an IfThen activity and use that to determine if an SLA was enforced or not, and if not, send a nasty email to the WF author.  The possibilities are endless, and they open up scenarios for you to provide specific extension points for your activities.

    That wraps up a very brief tour of ActivityAction.  ActivityAction provides a easy way to create an empty place in activity that the consumer can use to supply the logic that they want executed.  In the second part of this post, we’ll dive into how to create a designer for one of these, how to represent this in XAML, and a few other interesting topics.

    It’s that time of year that I’ll be taking a little bit of time off for the holidays, so I will see y’all in 2010!

    Photo Credit

  • mwinkle.blog

    Inspection, Default Services and Items (WF4 EditingContext Intro Part 6)

    • 0 Comments

    This part 6 of my 6  part series on the EditingContext.

  • Introduction
  • Sharing Functionality between Designers 
  • Host provided capabilities  
  • Providing callbacks for the host 
  • Subscription/Notification engine
  • Inspection, Default Services and Items (you are here)

    I want to wrap up this series of posts by posting some code for an activity designer that functions more as a diagnostic tool, and will display all of the Items and services of the EditingContext within the designer.  This will be useful from an investigation perspective, and hopefully as a diagnostic tool.  We will use this to help us understand what are the services that are available out of the box in VS, as well as in a rehosted application. 

    We first need to create an empty activity to attach a designer to.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Activities;
    using System.ComponentModel;
    
    namespace blogEditingContext
    {
        [Designer(typeof(DiagnosticDesigner))]
        public sealed class Diagnosticator : CodeActivity
        {
            // Define an activity input argument of type string
            public InArgument<string> Text { get; set; }
    
            // If your activity returns a value, derive from CodeActivity<TResult>
            // and return the value from the Execute method.
            protected override void Execute(CodeActivityContext context)
            {
                // Obtain the runtime value of the Text input argument
                string text = context.GetValue(this.Text);
            }
        }
    }

    Now, let’s create our designer.  We could do fancy treeviews or object browser style UI’s, but as this is a blog post, I want to provide you with the basics, and then let you figure out how that is most useful to you.  So, we will just create a designer that writes out to debug output the relevant information. 

    <sap:ActivityDesigner x:Class="blogEditingContext.DiagnosticDesigner"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
        xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
      <Grid>
            <Button Click="Button_Click">Debug.WriteLine Context Data</Button>
        </Grid>
    </sap:ActivityDesigner>

    And now the code

    using System.Diagnostics;
    using System.Linq;
    using System.Windows;
    
    namespace blogEditingContext
    {
        // Interaction logic for DiagnosticDesigner.xaml
        public partial class DiagnosticDesigner
        {
            public DiagnosticDesigner()
            {
                InitializeComponent();
            }
    
            private void Button_Click(object sender, RoutedEventArgs e)
            {
                // the goal here is to output meaningful and useful information about 
                // the contents of the editing context here. 
                int level = Debug.IndentLevel;
                Debug.WriteLine("Items in the EditingContext");
                Debug.IndentLevel++;
                foreach (var item in Context.Items.OrderBy(x => x.ItemType.ToString()))
                {
                    Debug.WriteLine(item.ItemType);
                }
    
                Debug.IndentLevel = level;
                Debug.WriteLine("Services in the EditingContext");
                foreach (var service in Context.Services.OrderBy(x => x.ToString()))
                {
                    Debug.WriteLine(service);
                }
            }
        }
    }

    Let’s break this down.  The work here happens in the button click where we simply order by types’ string representations and output them to the debug writer (a more robust implementation might use a trace writer that could be configured in the app, but for this purpose, this will be sufficient.

    So, what output do we get?

    VS Standard Services and Items

    We determine this by using the activity in a freshly opened WF project

    Items

     

    System.Activities.Presentation.Hosting.AssemblyContextControlItem
    System.Activities.Presentation.Hosting.ReadOnlyState
    System.Activities.Presentation.Hosting.WorkflowCommandExtensionItem
    System.Activities.Presentation.View.Selection
    System.Activities.Presentation.WorkflowFileItem

    Services

    System.Activities.Presentation.Debug.IDesignerDebugView
    System.Activities.Presentation.DesignerPerfEventProvider
    System.Activities.Presentation.FeatureManager
    System.Activities.Presentation.Hosting.ICommandService
    System.Activities.Presentation.Hosting.IMultiTargetingSupportService
    System.Activities.Presentation.Hosting.WindowHelperService
    System.Activities.Presentation.IActivityToolboxService
    System.Activities.Presentation.IIntegratedHelpService
    System.Activities.Presentation.IWorkflowDesignerStorageService
    System.Activities.Presentation.IXamlLoadErrorService
    System.Activities.Presentation.Model.AttachedPropertiesService
    System.Activities.Presentation.Model.ModelTreeManager
    System.Activities.Presentation.Services.ModelService
    System.Activities.Presentation.Services.ViewService
    System.Activities.Presentation.UndoEngine
    System.Activities.Presentation.Validation.IValidationErrorService
    System.Activities.Presentation.Validation.ValidationService
    System.Activities.Presentation.View.ActivityTypeDesigner+DisplayNameUpdater
    System.Activities.Presentation.View.DesignerView
    System.Activities.Presentation.View.IExpressionEditorService
    System.Activities.Presentation.View.ViewStateService
    System.Activities.Presentation.View.VirtualizedContainerService

     

    Basic Rehosted Application Standard Services and Items

    Items

    System.Activities.Presentation.Hosting.ReadOnlyState
    System.Activities.Presentation.Hosting.WorkflowCommandExtensionItem
    System.Activities.Presentation.View.Selection

    Services

    System.Activities.Presentation.DesignerPerfEventProvider
    System.Activities.Presentation.FeatureManager
    System.Activities.Presentation.Hosting.WindowHelperService
    System.Activities.Presentation.Model.AttachedPropertiesService
    System.Activities.Presentation.Model.ModelTreeManager
    System.Activities.Presentation.Services.ModelService
    System.Activities.Presentation.Services.ViewService
    System.Activities.Presentation.UndoEngine
    System.Activities.Presentation.Validation.ValidationService
    System.Activities.Presentation.View.DesignerView
    System.Activities.Presentation.View.ViewStateService
    System.Activities.Presentation.View.VirtualizedContainerService

    Comparison Table View

    Items VS Rehosted

    System.Activities.Presentation.Hosting.AssemblyContextControlItem 

    Yes

    No

    System.Activities.Presentation.Hosting.ReadOnlyState 

    Yes

    Yes

    System.Activities.Presentation.Hosting.WorkflowCommandExtensionItem 

    Yes

    Yes

    System.Activities.Presentation.View.Selection 

    Yes

    Yes

    System.Activities.Presentation.WorkflowFileItem 

    Yes

    No

    Services VS Rehosted 

    System.Activities.Presentation.Debug.IDesignerDebugView 

    Yes

    No

    System.Activities.Presentation.DesignerPerfEventProvider 

    Yes

    Yes

    System.Activities.Presentation.FeatureManager 

    Yes

    Yes

    System.Activities.Presentation.Hosting.ICommandService 

    Yes

    No

    System.Activities.Presentation.Hosting.IMultiTargetingSupportService 

    Yes

    No

    System.Activities.Presentation.Hosting.WindowHelperService 

    Yes

    Yes

    System.Activities.Presentation.IActivityToolboxService 

    Yes

    No

    System.Activities.Presentation.IIntegratedHelpService 

    Yes

    No

    System.Activities.Presentation.IWorkflowDesignerStorageService 

    Yes

    No

    System.Activities.Presentation.IXamlLoadErrorService 

    Yes

    No

    System.Activities.Presentation.Model.AttachedPropertiesService 

    Yes

    Yes

    System.Activities.Presentation.Model.ModelTreeManager 

    Yes

    Yes

    System.Activities.Presentation.Services.ModelService 

    Yes

    Yes

    System.Activities.Presentation.Services.ViewService 

    Yes

    Yes

    System.Activities.Presentation.UndoEngine 

    Yes

    Yes

    System.Activities.Presentation.Validation.IValidationErrorService 

    Yes

    No

    System.Activities.Presentation.Validation.ValidationService 

    Yes

    Yes

    System.Activities.Presentation.View.ActivityTypeDesigner+DisplayNameUpdater 

    Yes

    No

    System.Activities.Presentation.View.DesignerView 

    Yes

    Yes

    System.Activities.Presentation.View.IExpressionEditorService 

    Yes

    No

    System.Activities.Presentation.View.ViewStateService 

    Yes

    Yes

    System.Activities.Presentation.View.VirtualizedContainerService 

    Yes

    Yes

     

     

     

     

    Conclusion

    This wraps up our series on the editing context.  We’ve gone through the basics of why we need it, what we can do with it, and then we moved how to use it, from both the very simple to the very complex.  We’ve finished with a diagnostic tool to help understand what all items I can bind to.

    What’s Next From Here?

    A few ideas for the readers who have read all of these:

    • Wire up a few attached properties to reflect back through to some interesting data (like if it is selected).  These attached properties could then be used directly by your UI (via the binding in XAML) to let your designers display and react to changes in the data
    • Think about ideas for services you might want to add in VS without depending on an activity to inject it (and send me mail, I am trying to compile a list of interesting things)
    • Are there service/item implementations you want to override in VS?
    • Is there a service/item you expect to see that is not there?

    Thanks for now!

     

     

     

  • mwinkle.blog

    Subscription / Notification Engine (WF4 EditingContext Intro Part 5)

    • 2 Comments

    This part 5 of my 6  part series on the EditingContext.

    In this post, we’re going to tie together a few of the things we’ve seen in the last few posts and show how we can wire up parts of the designer (or the hosting application) to changes made to the Items collection of the EditingContext to do some interesting things.

    You will note that both the ServiceManager and ContextItemManager have Subscribe methods, and I’ve talked in previous posts about how the publish mechanism is a little different.  I want to dive a little deeper into how these work, the different overloads, and what you can expect have happen on the subscribe side of things.

    Services

    On service, there are two different publish methods.  I will list all four, and then talk about how there are really only two :-)

    Method

    Description

    Publish(Type, PublishServiceCallback) Publishes the specified service type, but does not declare an instance. When the service is requested, the Publish service callback will be invoked to create the instance. The callback is invoked only once. After that, the instance it returned is cached.
    Publish(Type, Object) Publishes the given service. After it is published, the service instance remains in the service manager until the editing context is disposed of.
    Publish<(Of <(TServiceType>)>)(TServiceType) Publishes the given service. After it is published, the service instance remains in the service manager until the editing context is disposed of.
    Publish<(Of <(TServiceType>)>)(PublishServiceCallback<(Of <(TServiceType>)>)) Publishes the given service type, but does not declare an instance yet. When the service is requested, the PublishServiceCallback will be invoked to create the instance. The callback is invoked only once. After that, the instance it returned is cached.

    There are really only two methods here, and some generic sugar for the other two.  They are Publish(Type, PublishServiceCallback), and Publish(Type, Object).  If you were to look at the implementation of the generic versions, they simply turn around and call the un-generic form.

    The difference between the basic one (Publish(Type, Object)) and the version with the callback is that the callback enables us to be a little more lazy and wait to actually create the instance of the object until it is first requested.  Let’s look at how PublishServiceCallback is defined:

    public delegate Object PublishServiceCallback(
        Type serviceType
    )

    What this lets us do is that the first time someone calls GetService, this method will be called with the responsibility of returning an instance of the service type.  Subsequent calls to GetService will simply return the instance returned by the method provided fro the PublishServiceCallback.  It is important to note that on Publish, any subscribers will be notified.  As the Subscribe callback takes an instance, we will internally call GetService on the notification, which will in turn call the PublishServiceCallback to instantiate an object.  If we have subscribers, our publish is less lazy (but that’s by design, we have consumers who are ready and waiting to consume).

    Let’s now look at the subscribe methods.  Again, here there are two methods (generic and non-generic), but they both do the same thing:

    Method

    Description

    Subscribe Invokes the provided callback when someone has published the requested service. If the service was already available, this method invokes the callback immediately.
    Subscribe<(Of <(TServiceType>)>) Invokes the provided callback when someone has published the requested service. If the service was already available, this method invokes the callback immediately.

    Both of these use a SubscribeServiceCallback defined as the following

    public delegate void SubscribeServiceCallback(
        Type serviceType,
        Object serviceInstance
    )

    This allows any consumer to be notified when a service is initially published.  This is an important distinction we will call out versus items which provide a more advanced subscription method (namely, to changes).

    Why is this Useful?

    Generally we find this useful for a few reasons:

    • Services may not be available at designer initialization, or the order in which they are created may not be fixed (it is ultimately up to the host to determine this). 
    • Services may be provided by the host.  It is possible your activity designer may be used in a host that does not provide that service.  You may want to be flexible in handling that
    • Services can be injected at any time.  A publish – subscribe model lets us have a little more flexibility to react to new services as they are added.  You could imagine a spell checking service that a host only provides on the first time someone hits “spell check.”  When this service comes online, then our designers can decide to consume this.
    • Flipping things around, you may want to use a service to callout to a host, and you would like the host to subscribe for when an certain activity designer will publish a service.

    Now let’s talk about Items:

    Items

    Items do not have a publish method, per se, but they have the SetValue method which basically publishes an instance to the context.  The semantics of SetValue are that it will first attempt to store the new value.  Provided that succeeds, we then call OnItemChanged on the ContextItem itself.  This is basically notifies the object itself (giving it a chance to react, clean up, or throw if something is really wrong).  If this throws, the old value is preserved.  If this succeeds, we then notify anyone who has subscribed to the changes. 

    GetValue allows me to retrieve the ContextItem.  There are two GetValue’s, one generic, the other non-generic, but with a type as its parameter.  It is important to note the point that is also present in the docs.  If there is not an item present when this is called, the default value will be instantiated and returned. 

    Provided items are written using SetValue, all of the subscribers will be subsequently notified.  If I just do an arbitrary GetValue and then make a few changes without calling SetValue, by default nothing interesting is going to happen (that is, no subscribers will be notified, subsequent calls to GetValue will get the updated object however).  Subscribe (and it’s generic counterpart) allow me to provide a SubscribeContextCallback which will be invoked whenever SetValue is called.   This functions basically in the same way that it does for Services.

    An interesting pattern for Items that we use in a few places throughout the designer is to create an AttachedProperty on the modelItem (similar to this post) which in the implementation of the Getter and Setter will call out to the editing context to get or set the value from a ContextItem.  This gives me a WPF friendly binding surface (foo.Bar binding syntax) that we can wire up to be change aware.  We do this for a number of our triggers within our style implementation for things like is selected, etc.  Future post note for me is that I should go through all of the attached properties present on a ModelItem that you could use to bind to :-)

     

    This wraps up a tour of the Subscription / Notification engine present within the EditingContext. 

  • mwinkle.blog

    Providing Callbacks for the Host (WF4 EditingContext Intro Part 4)

    • 0 Comments

    This part 4 of my 6  part series on the EditingContext.

     

    In addition to having a host provide an instance of a type to be used within the designer, it can also be used to pass an instance that will route callbacks to the host.  I covered this briefly in a previous post (Displaying Validation Errors in a Rehosted WF4 Designer).  In that case, we provide an implementation of IValidationErrorService, which the designer infrastructure will call, if available, towards the end of a completion of a validation episode.  In the sample application in that post, we use that instance to route, and display the validation errors in the system.

    Rather than duplicate the code, I will simply encourage you to check out that post and think about the way you could publish a service that your activity designers know about, and use it as a mechanism for calling methods within the hosting application. 

  • mwinkle.blog

    Host Provided Capabilities (WF4 EditingContext Intro Part 3)

    • 0 Comments

    This part 3 of my 6  part series on the EditingContext.

  • Introduction
  • Sharing Functionality between Designers 
  • Host provided capabilities  (you are here)
  • Providing callbacks for the host
  • Subscription/Notification engine
  • Inspection, Default Services and Items

    EditingContext is used by our primary hosting application, Visual Studio, to provide concrete implementations of certain services.  The example that we will talk about here is the IExpressionEditorService.  Now, one thing that we would have really liked to have done in vs2010 is to provide a way to use the intellisense enabled VB editor that you see within VS inside a rehosted app.  For various reasons, we were not able to ship with that dependency in the .NET framework.  However, we needed a mechanism for our primary host to have the intellisense experience (and similarly, you could build an experience, or maybe we’ll ship one in the future for rehosted apps). 

    Let’s look at the design of IExpressionEditorService:

     

    Name Description
    CloseExpressionEditors Closes all the active expression editors.
    CreateExpressionEditor Overloaded. Creates a new expression editor.
    UpdateContext Updates the context for the editing session.

    Inside the ExpressionTextBox control, when the control needs to render the editor, it has code that looks like the following (note, if it can’t find an instance, it skips and just uses a plain old text box):

    if (this.Context.Services.GetService<IExpressionEditorService>() != null)
    {
          return this.Context.Services.GetService<IExpressionEditorService>().CreateExpressionEditor(...)
    }

    Using the following overload of CreateExpressionEditor:

    IExpressionEditorInstance CreateExpressionEditor(
        AssemblyContextControlItem assemblies,
        ImportedNamespaceContextItem importedNamespaces,
        List<ModelItem> variables,
        string text
    )

    Now, what happens is that inside the code that we ship in the Microsoft.VisualStudio.Activities.Addin.dll, there is both a concrete implementation of this type, as well as the code which will publish an instance of this to the editing context.  Remember, this is the same thing that you can do in your app for a these host provided services.  In a subsequent post, I will get into more details of what are the ones that the designer has built in (like IExpressionEditorService). 

  • mwinkle.blog

    Sharing Functionality Between Designers (WF4 EditingContext Intro Part 2)

    • 10 Comments

    This part 2 of my 6 part series on the EditingContext.

     

    Setup

    We will need a custom activity, EmptyOne and designer called InteractWithServiceDesigner. 

    using System.Activities;
    using System.ComponentModel;
    
    namespace blogEditingContext
    {
        [Designer(typeof(InteractWithServicesDesigner))]
        public sealed class EmptyOne : CodeActivity
        {
            // Define an activity input argument of type string
            public InArgument<string> Text { get; set; }
    
            // If your activity returns a value, derive from CodeActivity<TResult>
            // and return the value from the Execute method.
            protected override void Execute(CodeActivityContext context)
            {
                // Obtain the runtime value of the Text input argument
                string text = context.GetValue(this.Text);
            }
        }
    }

    What We Will See

    The designers for Foo will leverage a new service in order to display a list of database tables.  We will also need to publish this service to the editing context, and handle the fact that we don’t know who might publish it (or when it might be published).  Note that in VS, there is no way to inject services except by having an activity designer do it.   In a rehosted app, the hosting application could publish additional services (see part 4) that the activities can consume.  In this case though, we will use the activity designer as our hook.

    Publishing a Service

    Let’s look at the designer for Foo (as Foo is our generic, and relatively boring activity).

    <sap:ActivityDesigner x:Class="blogEditingContext.InteractWithServicesDesigner"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
        xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
        <StackPanel>
            <ListBox Height="100" Name="listBox1" Width="120" />
            <Button Name="button1" Click="Button_Click">Publish Service</Button>
        </StackPanel>
    </sap:ActivityDesigner>

    Not much to this, except a drop down list that is currently unbound (but a name is provided).  Also note that there is a button that says to “publish the service”.  Let’s first look at the code for the button click

    private void Button_Click(object sender, RoutedEventArgs e)
    {
        if (!this.Context.Services.Contains<ISampleService>())
        {
            this.Context.Services.Publish<ISampleService>(new SampleServiceImpl());
        }
    }

    What are we doing here?  We first check if this service is already published using the Contains method.  We can do this because ServiceManager implements IEnumerable<Type>.

    [update, finishing this sentence.]  One could also consume the service using  GetService<TResult>.  You may also note that there is a GetRequiredService<T>.  This is a call that we know won’t return null, as the services we are requesting must be there for the designer to work.  Rather than returning null, this will throw an exception. Within the designer, we generally think of one service as required:

    Let’s look at the definition of the service.  Here you can see that we are using both an interface and then providing an implementation of that interface.  You could just as easily use an abstract class, or even a concrete class, there is no constraint on the service type.

    using System.Collections.Generic;
    
    namespace blogEditingContext
    {
        public interface ISampleService
        {
            IEnumerable<string> GetDropdownValues(string DisplayName);
        }
    
        public class SampleServiceImpl : ISampleService
        {
            public IEnumerable<string> GetDropdownValues(string DisplayName)
            {
                return new string[]  { 
                    DisplayName + " Foo", 
                    DisplayName + " Bar",
                    "Baz " + DisplayName
                } ;
            }
        }
    
    }

    If there is not a service present, we will publish an instance of one.  This becomes the singleton instance for any other designer that may request it.  Right now, we have a designer that can safely publish a service.  Let’s look at consuming one

    Consuming a Service

    Let’s look at some code to consume the service.  There are two parts to this.  One is simply consuming it, which we already saw above in discussing GetService and GetRequiredService .  The second is hooking into the notification system to let us know when a service is made available.  In this case, it’s a little contrived, as the service isn’t published until the button click, but it’s good practice to use the subscription mechanism as we make no guarantees on ordering, or timing of service availability.

    Subscribing to Service

    Here, using the Subscribe<TServiceType> method, we wait for the service to be available.  The documentation summarizes this method nicely:

    Invokes the provided callback when someone has published the requested service. If the service was already available, this method invokes the callback immediately.

    In the OnModelItemChanged method, we will subscribe and hook a callback.  The callback’s signature is as follows:

    public delegate void SubscribeServiceCallback<TServiceType>(
        TServiceType serviceInstance
    )

    As you can see, in this callback, the service instance is provided, so we can query it directly. You may ask, “why not in Intialize?”  well, there are no guarentees that the editing context will be available at that point. We could either subscribe to context being made available, or just use ModelItemChanged:

    protected override void OnModelItemChanged(object newItem)
    {
        if (!subscribed)
        {
            this.Context.Services.Subscribe<ISampleService>(
                servInstance =>
                {
                    listBox1.ItemsSource = servInstance.GetDropdownValues(this.ModelItem.Properties["DisplayName"].ComputedValue.ToString());
                    button1.IsEnabled = false;
                }
                );
            subscribed = true; 
        }
    }

    This wraps a basic introduction to the ServiceManager type and how to leverage it effectively to share functionality in designers.

    Let’s look at a before and after shot in the designer:

    Before & After

    before after

     

     

    What about Items?

    Items follow generally the same Get, Subscribe, and publish pattern, but rather than publish, there is a SetValue method.  If you have “just data” that you would like to share between designers (or between the host and the designer) an Item is the way to go about that.  The most commonly used item we’ve seen customers use is the Selection item in order to be able to get or set the currently selected model item.

    That’s our tour of basic publish and subscribe with Services and Items.

     

     

    [updated 12/22/2009 @ 10:23 am to finish an unclear sentence about GetService<>]

    [updated 12/22/2009 @ 8:50pm : Link to download sample code is here]

    Attachment(s): blogEditingContext.zip

  • mwinkle.blog

    Introduction to the WF4 Designer Editing Context (Part 1)

    • 2 Comments

    I want to briefly touch on the editing context and give a little introduction to its capabilities.  This is part 1 of a 6 part series

    The way to think about the editing context is that it is the point of contact between the hosting application, and the designer (and elements on the designer).  In my PDC talk, I had the following slide which I think captures the way to think about how these elements are layered together. 

    image

    Motivation

    The editing context represents the a common boundary between the hosting application and the designer, and the mechanism to handle interaction with the designer (outside of the most common methods that have been promoted on WorkflowDesigner).  If you were to look at the implementation of some of the more common methods on WorkflowDesigner, you would see that almost all of these use the editing context in order to get anything done.  For instance, the Flush method (and Save which calls Flush) will acquire an IDocumentPersistenceService from the Services collection, and then use that in order to properly serialize the document. 

    The EditingContext type has two important properties

    Items

    The Items collection is for data that is shared between the host and the designer, or data that is available to all designers.  These need to derive from ContextItem which will provide the mechanism to hook into subscription and change notification. There are a couple of interesting methods on the ContextItemManager type  :

    Services

    Services represent functionality that is either provided by the host for the designer to use, or is used by the designer to make functionality available to all designers within the editor.  Generally, these are likely defined as interfaces, but can also be a type.  It is then required for an implementation or an instance to be provided.  This instance will be shared as a singleton.  There are  a few interesting methods on the ServiceManager type:

    We’ll start walking through these over the next few posts.

  • mwinkle.blog

    Emitting the mc:Ignorable Instruction In Your WF4 XAML

    • 1 Comments

    Frequent forum guest Notre posed this question to the forums the other day noting that the XAML being produced from ActivityXamlServices.CreateBuilderWriter() was slightly different than the XAML being output from WorkflowDesigner.Save().  The reason for this stems from the fact that WorkflowDesigner leverages an additional internal type (which derives from XamlXmlWriter) in order to attach the mc:Ignorable attribute. 

    Why use mc:Ignorable?

    From the source at MSDN:

    The mc XML namespace prefix is the recommended prefix convention to use when mapping the XAML compatibility namespace http://schemas.openxmlformats.org/markup-compatibility/2006.

    Elements or attributes where the prefix portion of the element name are identified as mc:Ignorable will not raise errors when processed by a XAML processor. If that attribute could not be resolved to an underlying type or programming construct, then that element is ignored. Note however that ignored elements might still generate additional parsing errors for additional element requirements that are side effects of that element not being processed. For instance, a particular element content model might require exactly one child element, but if the specified child element was in an mc:Ignorable prefix, and the specified child element could not be resolved to a type, then the XAML processor might raise an error.

    Basically, this lets a XAML reader gracefully ignore any content marked from that namespace if it cannot be resolved.  So, imagine a runtime scenario where we don’t want to load System.Activities.Presentation every time we read a WF XAML file that may contain viewstate.  As a result, we use mc:Ignorable, which means the reader will skip that content when it does not have that assembly referenced at runtime. 

    This is what the output from the designer usually contains:

    <Sequence 
         mc:Ignorable="sap" 
         mva:VisualBasic.Settings="Assembly references and imported namespaces serialized as XML namespaces"
         xmlns="http://schemas.microsoft.com/netfx/2009/xaml/activities" 
         xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
         xmlns:mva="clr-namespace:Microsoft.VisualBasic.Activities;assembly=System.Activities" 
         xmlns:sap="http://schemas.microsoft.com/netfx/2009/xaml/activities/presentation" 
         xmlns:scg="clr-namespace:System.Collections.Generic;assembly=mscorlib" 
         xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
      <sap:WorkflowViewStateService.ViewState>
        <scg:Dictionary x:TypeArguments="x:String, x:Object">
          <x:Boolean x:Key="IsExpanded">True</x:Boolean>
        </scg:Dictionary>
      </sap:WorkflowViewStateService.ViewState>
      <Persist sap:VirtualizedContainerService.HintSize="211,22" />
      <Persist sap:VirtualizedContainerService.HintSize="211,22" />
      <WriteLine sap:VirtualizedContainerService.HintSize="211,61" />
    </Sequence>

    mc:Ignorable will cause the ViewState and HintSize to be ignored.

     

    Why do I have to worry about this?

    If you use WorkflowDesigner.Save(), you don’t.  If you want to be able to serialize the ActivityBuilder and have XAML which is what the designer produces, you need will need to add a XamlXmlWriter into the XamlWriter stack in order to get the right output. You may also worry about this if you are implementing your own storage and plan on writing some additional XAML readers or writers for additional extensibility and flexibility.

    How Do I Get the Same Behavior?

    The code below describes the same approach you would need to take to implement an XamlXmlWriter that does the same thing our internal type does.  While I can’t copy and paste code, this does the same thing.  We do two things:

    • Override WriteNamespace() to collect all of the namespaces being emitted.  We do this to specifically check for ones that we should ignore, and to also gather all of the prefixes to make sure we don’t have a collision
    • Override WriteStartObject() to generate and write out the Ignorable attribute within the start (first) member for any namespaces we should ignore.

    What namespaces do we ignore in the designer?  Just one: http://schemas.microsoft.com/netfx/2009/xaml/activities/presentation

    using System.Collections.Generic;
    using System.IO;
    using System.Xaml;
    using System.Xml;
     
    namespace IgnorableXamlWriter
    {
        class IgnorableXamlXmlWriter : XamlXmlWriter
        {
     
            HashSet<NamespaceDeclaration> ignorableNamespaces = new HashSet<NamespaceDeclaration>();
            HashSet<NamespaceDeclaration> allNamespaces = new HashSet<NamespaceDeclaration>();
            bool objectWritten;
            bool hasDesignNamespace;
            string designNamespacePrefix;
     
            public IgnorableXamlXmlWriter(TextWriter tw, XamlSchemaContext context)
                : base(XmlWriter.Create(tw,
                                        new XmlWriterSettings { Indent = true, OmitXmlDeclaration = true }),
                                        context,
                                        new XamlXmlWriterSettings { AssumeValidInput = true })
            {
     
            }
     
            public override void WriteNamespace(NamespaceDeclaration namespaceDeclaration)
            {
                if (!objectWritten)
                {
                    allNamespaces.Add(namespaceDeclaration);
                    // if we find one, add that to ignorable namespaces
                    // the goal here is to collect all of them that might point to this
                    // if you had a broader set of things to ignore, you would collect 
                    // those here.
                    if (namespaceDeclaration.Namespace == "http://schemas.microsoft.com/netfx/2009/xaml/activities/presentation")
                    {
                        hasDesignNamespace = true;
                        designNamespacePrefix = namespaceDeclaration.Prefix;
                    }
                }
                base.WriteNamespace(namespaceDeclaration);
            }
     
            public override void WriteStartObject(XamlType type)
            {
                if (!objectWritten)
                {
                    // we should check if we should ignore 
                    if (hasDesignNamespace)
                    {
                        // note this is not robust as mc could naturally occur
                        string mcAlias = "mc";
                        this.WriteNamespace(
                            new NamespaceDeclaration(
                                "http://schemas.openxmlformats.org/markup-compatibility/2006",
                                mcAlias)
                                );
     
                    }
                }
                base.WriteStartObject(type);
                if (!objectWritten)
                {
                    if (hasDesignNamespace)
                    {
                        XamlDirective ig = new XamlDirective(
                            "http://schemas.openxmlformats.org/markup-compatibility/2006",
                            "Ignorable");
                        WriteStartMember(ig);
                        WriteValue(designNamespacePrefix);
                        WriteEndMember();
                        objectWritten = true;
                    }
                }
            }
     
        }
    }

    One note on the code above, it is noted that the generation of the namespace prefix “mc” is not robust.  In the product code we will check to see if mc1, mc2, … are available up to mc1000.  In that case we would then append a GUID for the ugliest XML namespace known to mankind.  The chance of collision up to 1000 would be a highly extreme edge case.

    How would I use this? The following code shows feeding this into a CreateBuilderWriter that is passed to XamlServices.Save()

    StringBuilder sb = new StringBuilder();
    XamlSchemaContext xsc = new XamlSchemaContext();
    var bw = ActivityXamlServices.CreateBuilderWriter(
        new IgnorableXamlXmlWriter(new StringWriter(sb), xsc));
    
    XamlServices.Save(bw,
                      wd.Context.Services.GetService<ModelTreeManager>().Root.GetCurrentValue());
  • mwinkle.blog

    AttachedProperty Part 2, Putting it Together

    • 0 Comments

    On my last post, Jason jumped right to the punchline in his comment here. He asks “if there is an easy way to have the properties value serialized out to the xaml.”

    First, let’s look at what we need to do from the XAML side.

    First, create a helper type with a getter and setter for the property that you want to attach.  Here we’re going to attach comments:

    public class Comment
    {
        static AttachableMemberIdentifier CommentTextName = new AttachableMemberIdentifier(typeof(Comment), "CommentText");
    
        public static object GetCommentText(object instance)
        {
            object viewState;
            AttachablePropertyServices.TryGetProperty(instance, CommentTextName, out viewState);
            return viewState;
        }
    
        public static void SetCommentText(object instance, object value)
        {
            AttachablePropertyServices.SetProperty(instance, CommentTextName, value);
        }
    
    }

    Next, let’s use the AttachableMemberIdentifier and the AttachablePropertyServices to do something interesting with this:

    AttachableMemberIdentifier ami = new AttachableMemberIdentifier(typeof(Comment), "CommentText");
    Dog newDog = new Dog { Age = 12, Name = "Sherlock", Noise = "Hooowl" };
    AttachablePropertyServices.SetProperty(newDog, ami, "A very good dog");
    string s = XamlServices.Save(newDog);
    Console.WriteLine("XAML");
    Console.WriteLine(s);
    Dog aSecondNewDog = XamlServices.Load(new StringReader(s)) as Dog;
    string outter;
    AttachablePropertyServices.TryGetProperty(aSecondNewDog, ami, out outter);
    Console.WriteLine("read out: {0}", outter);

    Let’s see the output from this:

    XAML
    <Dog
         Age="12"
         Comment.CommentText="A very good dog"
         Name="Sherlock"
         Noise="Hooowl" 
         xmlns="clr-namespace:AttachedPropertiesBlogPosting;assembly=AttachedPropertiesBlogPosting" />
    read out: A very good dog

    You’ll note that the value is contained in the XAML under Comment.CommentText.

    Pulling it all Together

    Let’s take what we did in the last post and combine it with the above stuff in order to have an attached property that writes through and is stored inside the XAML.

    AttachedProperty<string> Comment = new AttachedProperty<string>
    {
        IsBrowsable = true,
        Name = "Comment",
        Getter = (mi =>
            { string temp;
              AttachablePropertyServices.TryGetProperty<string>(mi.GetCurrentValue(), ami, out temp);
              return temp;
            }),
        Setter = ( (mi,val) => AttachablePropertyServices.SetProperty(mi.GetCurrentValue(), ami, val) )
    
    };
    aps.AddProperty(Comment);
    dogMi.Properties["Comment"].SetValue("I think I like that dog");
    string xaml = XamlServices.Save(dogMi.GetCurrentValue());
    Console.WriteLine("XAML");
    Console.WriteLine(xaml);

    What are we doing here, well, we basically just use the Getter and Setter to write through to the underlying instance.  You’ll note that usually, we never want to go and use GetCurrentValue(), as any changes made there are not made via the ModelItem tree which means we might miss a change notification.  However, given that the only place where we can store the XAML attached property is on the instance itself, this gives us a good way to write through.  The XAML output below shows that this works as expected:

    <Dog 
       Age="5"
       Comment.CommentText="I think I like that dog"
       Name="Sasha" 
       Noise="Snort" 
       xmlns="clr-namespace:AttachedPropertiesBlogPosting;assembly=AttachedPropertiesBlogPosting" />
  • mwinkle.blog

    WF4 Design Time AttachedPropertiesService and Attached Properties

    • 3 Comments

    I’ve been meaning to throw together some thoughts on attached properties and how they can be used within the designer.  Basically, you can think about attached properties as injecting some additional “stuff” onto an instance that you can use elsewhere in your code.

    Motivation

    In the designer, we want to be able to have behavior and view tied to interesting aspects of the data.  For instance, we would like to have a view updated when an item becomes selected.  In WPF, we bind the style based on the “isSelectionProperty.”  Now, our data model doesn’t have any idea of selection, it’s something we’d like the view level to “inject” that idea on any model item so that a subsequent view could take advantage of.  You can kind of view Attached Properties as a nice syntactic sugar to not have to keep a bunch of lookup lists around.  As things like WPF bind to the object very well, and not so much a lookup list, this ends up being an interesting model.

    To be clear, you could write a number of value converters that take the item being bound, look up in a lookup list somewhere, and return the result that will be used.  The problem we found is that we were doing this in a bunch of places, and we really wanted to have clean binding statements inside our WPF XAML, rather than hiding a bunch of logic in the converters.

    How Does it Work

    First, some types.

    Name Description
    AttachedPropertiesService Service in editing context for managing AttachedProperties
    AttachedProperty Base attached property type (abstract)
    AttachedProperty<T> Strongly typed attached property with interesting getter/setter programmability

    in diagram form:

    image

    One thing that might look a little funny to some folks who have used attached properties in other contexts (WF3, WPF, XAML), is the “IsBrowsable” property.  The documentation is a little sparse right now, but what this will do is determine how discoverable the property is.  If this is set to true, the attached property will show up in the Properties collection of the ModelItem to which the AP is attached.  What this means is that it can show up in the Property grid, you can bind WPF statements directly to it, as if it were a real property of the object.  Attached properties by themselves have no actual storage representation, so these exist as design time only constructs.

    Getter/ Setter?

    One other thing that you see on the AttachedProperty<T> is the Getter and Setter properties.  These are of type Func<ModelItem,T> and Action<ModelItem,T> respectively.  What these allow you to do is perform some type of computation whenever the get or set is called against the AttachedProperty.  Why is this interesting?  Well, let’s say that you’d like to have a computed value retrieved, such as “IsPrimarySelection” checking with the Selection context item to see if an item is selected.  Or, customizing the setter to either store the value somewhere more durable, or updating a few different values.  The other thing that happens is that since all of these updates go through the ModelItem tree, any changes will be propagated to other listeners throughout the designer.

    Looking at Some Code

    Here is a very small console based app that shows how you can program against the attached properties.  An interesting exercise for the reader would be to take this data structure, put it in a WPF app and experiment with some of the data binding.

    First, two types:

    public class Dog
    {
        public string Name { get; set; }
        public string Noise { get; set; }
        public int Age { get; set; }
       
    }
    
    public class Cat
    {
        public string Name { get; set; }
        public string Noise { get; set; }
        public int Age { get; set; }
    }

    Ignore no common base type, that actually makes this a little more interesting, as we will see.

    Now, let’s write some code.  First, let’s initialize and EditingContext and ModelTreeManager

       1:       static void Main(string[] args)
       2:          {
       3:              EditingContext ec = new EditingContext();
       4:              ModelTreeManager mtm = new ModelTreeManager(ec);
       5:              mtm.Load(new object[] { new Dog { Name = "Sasha", Noise = "Snort", Age = 5 },
       6:                                      new Cat { Name="higgs", Noise="boom", Age=1 } });
       7:              dynamic root = mtm.Root;
       8:              dynamic dog = root[0];
       9:              dynamic cat = root[1];
      10:              ModelItem dogMi = root[0] as ModelItem;
      11:              ModelItem catMi = root[1] as ModelItem;

    Note, lines 7-9 will not work in Beta2 (preview of coming attractions).  To get lines 10-11 working in beta2, cast root to ModelItemCollection and then use the indexers to extract the values

    Now, let’s build an attached property, and we will assign it only to type “dog”

       1:  // Add an attached Property
       2:  AttachedProperty<bool> ap = new AttachedProperty<bool>
       3:  {
       4:      IsBrowsable = true,
       5:      Name = "IsAnInterestingDog",
       6:      Getter = (mi => mi.Properties["Name"].ComputedValue.ToString() == "Sasha"),
       7:      OwnerType = typeof(Dog)
       8:  };
       9:  ec.Services.Publish<AttachedPropertiesService>(new AttachedPropertiesService());
      10:  AttachedPropertiesService aps = ec.Services.GetService<AttachedPropertiesService>();
      11:  aps.AddProperty(ap);
      12:   
      13:  Console.WriteLine("---- Enumerate properties on dog (note new property)----");
      14:  dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
      15:   
      16:  Console.WriteLine("---- Enumerate properties on cat (note  no new property) ----");
      17:  catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));

    Let’s break down what happened here.

    • Line2-8, create an AttachedProperty<bool>
      • We set IsBrowsable to true, we want to see it in the output
      • Name, that’s what it will be projected as
      • OwnerType, we only want this to apply to Dog’s, not Cat’s or Objects or whatever.
      • Finally, Getter, and look what we do here, we operate on the model item to do some computation and return a bool (in this case, we look to see if the name property equals “Sasha”
    • Line 9-11 create an AttachedPropertiesService and add it to the editing context.
    • Lines 13-17 output the properties, and let’s see what that looks like:
    ---- Enumerate properties on dog (note new property)----
     Property : Name
     Property : Noise
     Property : Age
     Property : IsAnInterestingDog
    ---- Enumerate properties on cat (note  no new property) ----
     Property : Name
     Property : Noise
     Property : Age

    Ok, so that’s interesting, we’ve injected a new property, only on the dog type.  If I got dogMI.Properties[“IsAnInterestingDog”], I would have a value that I could manipulate (albeit returned via the getter).

    Let’s try something a little different:

       1:  AttachedProperty<bool> isYoungAnimal = new AttachedProperty<bool>
       2:  {
       3:      IsBrowsable = false,
       4:      Name = "IsYoungAnimal",
       5:      Getter = (mi => int.Parse(mi.Properties["Age"].ComputedValue.ToString()) < 2)
       6:  };
       7:   
       8:  aps.AddProperty(isYoungAnimal);
       9:   
      10:  // expect to not see isYoungAnimal show up
      11:  Console.WriteLine("---- Enumerate properties on dog  (note isYoungAnimal doesn't appear )----");
      12:  dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
      13:  Console.WriteLine("---- Enumerate properties on cat (note isYoungAnimal doesn't appear )----");
      14:  catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
      15:   
      16:  Console.WriteLine("---- get attached property via GetValue ----");
      17:  Console.WriteLine("getting non browsable attached property on dog {0}", isYoungAnimal.GetValue(dogMi));
      18:  Console.WriteLine("getting non browsable attached property on cat {0}", isYoungAnimal.GetValue(catMi));

    Let’s break this down:

    • Lines 1-6 create a new attached property
      • IsBrowsable is false
      • No OwnerType being set
      • The Getter does some computation to return true or false
    • Lines 10-14 write out the properties (as above)
    • Lines 17-18 extract the value with AttachedPropertyInstance.GetValue(ModelItem)

    Let’s see the output there:

    ---- Enumerate properties on dog  (note isYoungAnimal doesn't appear )----
     Property : Name
     Property : Noise
     Property : Age
     Property : IsAnInterestingDog
    ---- Enumerate properties on cat (note isYoungAnimal doesn't appear )----
     Property : Name
     Property : Noise
     Property : Age
    ---- get attached property via GetValue ----
    getting non browsable attached property on dog False
    getting non browsable attached property on cat True

    As we can see, we’ve now injected this behavior, and we can extract the value. 

    Let’s get a little more advanced and do something with the setter.  Here, if isYoungAnimal is set to true, we will change the age (it’s a bit contrived, but shows the dataflow on simple objects, we’ll see in a minute a more interesting case).

       1:  // now, let's do something clever with the setter. 
       2:  Console.WriteLine("---- let's use the setter to have some side effect ----");
       3:  isYoungAnimal.Setter = ((mi, val) => { if (val) { mi.Properties["Age"].SetValue(10); } });
       4:  isYoungAnimal.SetValue(cat, true);
       5:  Console.WriteLine("cat's age now {0}", cat.Age);

    Pay attention to what the Setter does now.  We create the method through which subsequent SetValue’s will be pushed.  Here’s that output:

    ---- let's use the setter to have some side effect ----
    cat's age now 10

    Finally, let’s show an example of how this can really function as some nice sugar to eliminate the need for a lot of value converters in WPF by using this capability as a way to store the relationship somewhere (rather than just using at a nice proxy to change a value):

       1:  // now, let's have a browesable one with a setter.
       2:  // this plus dynamics are a mini "macro language" against the model items
       3:   
       4:  List<Object> FavoriteAnimals = new List<object>();
       5:   
       6:  // we maintain state in FavoriteAnimals, and use the getter/setter func
       7:  // in order to query or edit that collection.  Thus changes to an "instance"
       8:  // are tracked elsewhere.
       9:  AttachedProperty<bool> isFavoriteAnimal = new AttachedProperty<bool>
      10:  {
      11:      IsBrowsable = false,
      12:      Name = "IsFavoriteAnimal",
      13:      Getter = (mi => FavoriteAnimals.Contains(mi)),
      14:      Setter = ((mi, val) => 
      15:          {
      16:              if (val)
      17:                  FavoriteAnimals.Add(mi);
      18:              else
      19:              {
      20:                  FavoriteAnimals.Remove(mi);
      21:              }
      22:          })
      23:  };
      24:   
      25:   
      26:  aps.AddProperty(isFavoriteAnimal);
      27:   
      28:  dog.IsFavoriteAnimal = true;
      29:  // remove that cat that isn't there
      30:  cat.IsFavoriteAnimal = false;
      31:  cat.IsFavoriteAnimal = true;
      32:  cat.IsFavoriteAnimal = false;
      33:   
      34:  Console.WriteLine("Who are my favorite animal?");
      35:  FavoriteAnimals.ForEach(o => Console.WriteLine((o as ModelItem).Properties["Name"].ComputedValue.ToString()));

    Little bit of code, let’s break it down one last time:

    • Line 14 – Create a setter that acts upon the FavoriteAnimals collection to either add or remove the element
    • Line 28-32 – do a few different sets on this attached property
      • NOTE: you can’t do that in beta2 as the dynamic support hasn’t been turned on.  Rather you would have to do isFavoriteAnimal.SetValue(dogMi, true).
    • Line 35 then prints the output to the console, and as expected we only see the dog there:
    -- Who are my favorite animals?
    Sasha

    I will attach the whole code file at the bottom of this post, but this shows you how you can use the following:

    • Attached properties to create “computed values” on top of existing types
    • Attached properties to inject a new (and discoverable) property entry on top of the designer data model (in the form of a new property)
    • Using the Setter capability to both propagate real changes to the type, providing a nice way to give a cleaner interface, as well as use it as a mechanism to store data about the object outside of the object, but in a way that gives me access to it such that it seems like the object. 
      • This is some really nice syntactic sugar that we sprinkle on top of things

    What do I do now?

    Hopefully this post gave you some ideas about how the attached property mechanisms work within the WF4 designer.  These give you a nice way to complement the data model and create nice bindable targets that your WPF Views can layer right on top of.

    A few ideas for these things:

    • Use the Setters to clean up a “messy” activity API into a single property type that you then build a custom editor for in the property grid. 
    • Use the Getters (and the integration into the ModelProperty collection) in order to create computational properties that are used for displaying interesting information on the designer surface.
    • Figure out how to bridge the gap to take advantage of the XAML attached property storage mechanism, especially if you author runtime types that look for attached properties at runtime. 
    • Use these, with a combination of custom activity designers to extract and display interesting runtime data from a tracking store

     

    Full Code Posting

    using System;
    using System.Activities.Presentation;
    using System.Activities.Presentation.Model;
    using System.Collections.Generic;
    using System.Linq;
    
    namespace AttachedPropertiesBlogPosting
    {
        class Program
        {
            static void Main(string[] args)
            {
                EditingContext ec = new EditingContext();
                ModelTreeManager mtm = new ModelTreeManager(ec);
                mtm.Load(new object[] { new Dog { Name = "Sasha", Noise = "Snort", Age = 5 },
                                        new Cat { Name="higgs", Noise="boom", Age=1 } });
                dynamic root = mtm.Root;
                dynamic dog = root[0];
                dynamic cat = root[1];
                ModelItem dogMi = root[0] as ModelItem;
                ModelItem catMi = root[1] as ModelItem;
              
                // Add an attached Property
                AttachedProperty<bool> ap = new AttachedProperty<bool>
                {
                    IsBrowsable = true,
                    Name = "IsAnInterestingDog",
                    Getter = (mi => mi.Properties["Name"].ComputedValue.ToString() == "Sasha"),
                    OwnerType = typeof(Dog)
                };
                ec.Services.Publish<AttachedPropertiesService>(new AttachedPropertiesService());
                AttachedPropertiesService aps = ec.Services.GetService<AttachedPropertiesService>();
                aps.AddProperty(ap);
    
                Console.WriteLine("---- Enumerate properties on dog (note new property)----");
                dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
    
                Console.WriteLine("---- Enumerate properties on cat (note  no new property) ----");
                catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
    
    
                
                AttachedProperty<bool> isYoungAnimal = new AttachedProperty<bool>
                {
                    IsBrowsable = false,
                    Name = "IsYoungAnimal",
                    Getter = (mi => int.Parse(mi.Properties["Age"].ComputedValue.ToString()) < 2)
                };
    
                aps.AddProperty(isYoungAnimal);
    
                // expect to not see isYoungAnimal show up
                Console.WriteLine("---- Enumerate properties on dog  (note isYoungAnimal doesn't appear )----");
                dogMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
                Console.WriteLine("---- Enumerate properties on cat (note isYoungAnimal doesn't appear )----");
                catMi.Properties.ToList().ForEach(mp => Console.WriteLine(" Property : {0}", mp.Name));
    
                Console.WriteLine("---- get attached property via GetValue ----");
                Console.WriteLine("getting non browsable attached property on dog {0}", isYoungAnimal.GetValue(dogMi));
                Console.WriteLine("getting non browsable attached property on cat {0}", isYoungAnimal.GetValue(catMi));
                
                
                // now, let's do something clever with the setter. 
                Console.WriteLine("---- let's use the setter to have some side effect ----");
                isYoungAnimal.Setter = ((mi, val) => { if (val) { mi.Properties["Age"].SetValue(10); } });
                isYoungAnimal.SetValue(cat, true);
                Console.WriteLine("cat's age now {0}", cat.Age);
    
                // now, let's have a browesable one with a setter.
                // this plus dynamics are a mini "macro language" against the model items
    
                List<Object> FavoriteAnimals = new List<object>();
    
                // we maintain state in FavoriteAnimals, and use the getter/setter func
                // in order to query or edit that collection.  Thus changes to an "instance"
                // are tracked elsewhere.
                AttachedProperty<bool> isFavoriteAnimal = new AttachedProperty<bool>
                {
                    IsBrowsable = false,
                    Name = "IsFavoriteAnimal",
                    Getter = (mi => FavoriteAnimals.Contains(mi)),
                    Setter = ((mi, val) => 
                        {
                            if (val)
                                FavoriteAnimals.Add(mi);
                            else
                            {
                                FavoriteAnimals.Remove(mi);
                            }
                        })
                };
                aps.AddProperty(isFavoriteAnimal);
                dog.IsFavoriteAnimal = true;
                // remove that cat that isn't there
                cat.IsFavoriteAnimal = false;
                cat.IsFavoriteAnimal = true;
                cat.IsFavoriteAnimal = false;
                Console.WriteLine("Who are my favorite animals?");
                FavoriteAnimals.ForEach(o => Console.WriteLine((o as ModelItem).Properties["Name"].ComputedValue.ToString()));
                Console.ReadLine();
            }
        }
    
        public class Dog
        {
            public string Name { get; set; }
            public string Noise { get; set; }
            public int Age { get; set; }
        }
    
        public class Cat
        {
            public string Name { get; set; }
            public string Noise { get; set; }
            public int Age { get; set; }
        }
    }
  • mwinkle.blog

    WF4 ViewStateService

    • 2 Comments

    A comment posted by Notre asked for some more details about view state and attached property services, so I thought I would dive into those next.  I will follow-up in a subsequent post on the AttachedPropertyService, as there is a little bit more going on there.

    Motivation

    Why do I care about viewstate?  Well, usually it is because we want to write something down and store it for later that is not required for runtime.  A common example of viewstate is the position of nodes within a flowchart.  While not required to execute the flowchart, they are required to effectively view the flowchart. 

    Where to write them down?

    This was a question that caused a fair amount of debate on the team.  There are basically two places to write down things like view state in a file-based world. 

    1. In the source document itself
    2. In a document that stays close to the source (usually referred to as a sidecar file)

    We had customers asking for both.  The motivation for the first is that for things like flowchart, where I may always care about the visualization representation, I want to keep that metadata around and only deal with one element.  For the second, it is motivated by the reason that we want a clean source document that only describes the minimal artifact to run.  Now, there are certainly many stops along the spectrum (for instance, we might always want to keep annotations or source comments in the source document, and put positioning elsewhere).  For VS2010, we landed with a unified API to use, and we write in the source document.  This is something that is likely to change in future releases, as it does make things like textual diffs rather painful.

    So, that’s why we want to use it.

    How do we use it?

    We are going to create a simple activity designer that lets me write down a comment.

    A few simple steps:

    1. Create a new VS Project, let’s create an Activity Library
    2. Add a Designer to that activity library
    3. Add an attribute to the activity pointing to the designer
    4. Add a new WorkflowConsoleApp Project
    5. Build

    image

    Now, let’s go and make our activity designer a little interesting.

    Let’s add a text box and a button.  We’ll make the text of the button something obvious like “commit comment” The XAML for the activity designer looks like this:

    <sap:ActivityDesigner x:Class="simpleActivity.CommentingDesigner"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:sap="clr-namespace:System.Activities.Presentation;assembly=System.Activities.Presentation"
        xmlns:sapv="clr-namespace:System.Activities.Presentation.View;assembly=System.Activities.Presentation">
      <Grid>
            <StackPanel Name="stackPanel1" VerticalAlignment="Top" >
                <TextBox  Name="commentBlock"   />
                <Button Content="Load View State" Name="loadViewState" Click="loadViewState_Click" />
                <Button Content="Commit View State"  Name="button1" Click="button1_Click" />
            </StackPanel>
        </Grid>
    </sap:ActivityDesigner>

    Now, let’s add some code to the button (and to the initialization of the form)

    ViewStateService has a few useful methods on it.  I want to call out a subtle difference.  You will see StoreViewState and StoreViewStateWithUndo.  The primary distinction as the name implies is that one will simply write the view state down and will bypass the undo/redo stack.  This is for view state like an expanded/collapsed view.  You don’t really want ctl-z to simply flip expanded versus collapsed for you.  But for something like flowchart, where changing some of the viewstate, like position, might be such a thing that you want support for undoing the action.  That’s the primary difference.

    So, our code for the button looks like this:

    private void button1_Click(object sender, RoutedEventArgs e)
    {
        ViewStateService vss = this.Context.Services.GetService<ViewStateService>();
        vss.StoreViewStateWithUndo(this.ModelItem, "comment", commentBlock.Text);
    }

    Now, on load, we want to be able to populate the value, so we will use the RetrieveViewState method in order to extract this.

    private void loadViewState_Click(object sender, RoutedEventArgs e)
    {
        ViewStateService vss = this.Context.Services.GetService<ViewStateService>();
        commentBlock.Text = vss.RetrieveViewState(this.ModelItem, "comment") as string;
    }

    Now, let’s go back to our workflow project and put an instance of this activity on the surface:

    image

    Let’s add some viewstate information and commit it.  Now let’s look at the XAML:

    <s4:NotRealInterestingActivity Text="{x:Null}" sap:VirtualizedContainerService.HintSize="200,99">
      <sap:WorkflowViewStateService.ViewState>
        <scg3:Dictionary x:TypeArguments="x:String, x:Object">
          <x:String x:Key="comment">basic comment</x:String>
        </scg3:Dictionary>
      </sap:WorkflowViewStateService.ViewState>
    </s4:NotRealInterestingActivity>

    Ok, now, to show that we can pull this in, let’s change the text in the xaml and then reload our designer.

    image 

    You can muck around with ctl-z to see that this does get handled correctly via the undo. 

    The other important thing to note is that this takes an object, so your viewstate is not limited to strings, you can have more full featured objects if you’d like. Finally, the ViewStateService also has a ViewStateChanged you can subscribe to in order to handle, dispatch, and react to view state changes in the designer.

  • mwinkle.blog

    Where is System.Activities.Design in WF4 Beta2 and Beyond?

    • 0 Comments

    I got an email over the weekend asking about this, and I realized that it’s somewhat buried in this post here.  Anyway, in Beta1, you often saw System.Activities.Design.  For beta2 (and RTM), one important change

    System.Activities.Design  => System.Activities.Presentation

    The primary reason for this change is that the *.Design suffix is generally reserved for VS design extensibility.  As our designer ships in the framework, *.Design was not the correct suffix.  *.Presentation is where we landed. 

    Hoping that putting this in the title lands this high up in search queries so that this post might be useful for a few people.

  • mwinkle.blog

    WF4, WCF and AppFabric Sessions @ PDC09

    • 0 Comments

    I’m taking this week off to catch up on everything I haven’t done the last two months and to celebrate the Thanksgiving holiday here in the US.

    PDC was a blast!  It was incredibly awesome to meet so many folks interested in WF and talking with others about how they are using (or plan to use) WF4.  I’ll be following up with a more detailed post on my talk, including demos and code, but I wanted to give a summary of the talks that came from my team at this PDC.  Below is the diagram that breaks down some of the “capabilities” of AppFabric and I have color coded them for the various talks that we gave.  All of the PDC Sessions are available online here.

    image

    WF Talks

    WCF Talks

    AppFabric Talks

    There’s a ton of great content up at the PDC site, plenty to sit back and enjoy! 

  • Page 1 of 6 (148 items) 12345»