Yun Jin's WebLog

Workflow Foundation 4.0 Activity Data Model (III)

Yun Jin — Wed, 28 Apr 2010 21:43:00 GMT

I finally finished the long overdued last installment of the Activity Data Model series. In this post, I focused on workflow variables. Conceptually workflow variables are very similar to variables in procedure programming languages like C#. But one thing special about workflow variables is the concept of "public" versus "implementation" variables. "Public" variables means variables defined by activity users and could not be used by the activity's implementation. "Implementation" variables are very similiar to the concept of local varible in C#. Both variables and arguments are scoped in WF4, which is a big improvement compared to WF3.

The first wave of WF4 Activity Packs are released

Yun Jin — Sun, 25 Apr 2010 13:31:00 GMT

I'm very happy to annouce that my team just released 2 pack of Activities for WF4 on CodePlex!

· State Machine Activity Pack.
State Machine is one of the most common asks for WF4’s Activity Palette, especially for WF3 users. This Activity Pack includes a feature-rich State Machine supporting entry action, exit action, conditional transition, nested state, and etc. The designer supports free form editing experience similar to FlowChart. We also released all the source code so everyone could see how to implement such a complicated Activity. You are welcome to download the Activity, give it a try for your scenarios, and let us know how you think.

· ADO.Net Activity Pack.

WF4 does not have any out-of-box Activities for data access. Yet we believe data access is critical for workflow driven business applications. The set of Activities in this pack could execute SQL statements using ADO.Net. Designers for those Activities have some cool interaction with Visual Studio when it’s hosted in VS. For example, developers could access data source defined in VS’s solution explorer and connection string in config files. This is just a start, we are thinking hard about what data access experience should be for WF. So any of your feedback is appreciated.

Workflow Foundation 4.0 Activity Data Model (II)

Yun Jin — Wed, 07 Apr 2010 01:16:00 GMT

Here comes sequal of my WF4 Activity Data Model blog. This post focuses on WF arguments: the Argument class hierarchy which models arguments for WF; how to "declare" an argument for an activity; the automatic reflection; how to deal with dynamic arguments; what does expressions mean for WF; how to bind arguments to expressions; activity arguments evaluation order, and etc.

The next post will be about WF variables.

Workflow Foundation 4.0 Activity Data Model (I)

Yun Jin — Tue, 06 Apr 2010 01:37:00 GMT

This is my first post about WF4's Activity Data Model. The focus of this post is a high level principle which drives data model redesign between WF3 and WF4: separation of data and program. WF3 has Activity's runtime state stored in the Activity itself which caused a lot of problems. WF4 abstracts out concept like Argument and Variable and pushes the real storage to ActivityContext available only at runtime. This model is consistent with mainstream programming languages like C#.

I'll have more posts later to talk about WF Argument and Variable in details.

Workflow Foundation 4.0 Activity Model (II)

Yun Jin — Mon, 01 Feb 2010 14:27:00 GMT

I just posted the 2nd half of introduction to WF4's Activity Model. In this blog, I have given examples of building 2 activities with 4 different authoring styles: how to build a leaf activity (HttpGet) using CodeActivity and AsyncCodeActivity; and how to build a composite activity (World) using Activity and NativeActivity.

Workflow Foundation 4.0 Activity Model (I)

Yun Jin — Mon, 25 Jan 2010 13:57:00 GMT

I just posted one entry in our "Go with the Flow" team blog to explain Workflow foundation V4's Activity Model. In the blog, I discussed how Workflow is composed of Activities and introduced basic Activity programming model in WF4. I quickly went over the 4 Activity base classes (Activity, CodeActivity, AsyncCodeActivity, and NativeActivity) for 4 different programing style (from fully decarlative to 100% imperative programing). I'll post another blog later for the same topic to give few examples on how to use those 4 classes.

A developer's view of Workflow

Yun Jin — Fri, 15 Jan 2010 01:17:00 GMT

This is my first blog post about Windows Workflow Foundation (WF) on the "Go with the flow" team blog. In the post I briefly described some key benefits of Workflow as a programming language: Declarative programming, Continuation, Persistence, and instance management.

Hello world, again

Yun Jin — Fri, 15 Jan 2010 00:48:00 GMT

I've disappeared for more than 4 years in blog space. During this time period, I left CLR team and worked for Microsoft's High Performance Computing (HPC) team for 3 years to build Windows based compute cluster infrastructure. One thing I'm particularly proud of is that in Super Computing 2008, a system powered by our product (Windows HPC Server 2008) hits #10 in the world's Top 500 Super Computer list. Recently I joined Microsoft's Application Server Group (ASG) to work on Windows based application server products. Specially, my team focuses on Windows Workflow Foundation (WF) and Windows Communication Foundation (WCF). I decide to start the blogging hobby again to share more insights on WCF/WF. Please stay tuned. :)

Trivial debugging note - using WeakReference in finalizer

Yun Jin — Wed, 31 Aug 2005 19:42:00 GMT

Some time ago I saw a problem from a partner team in Microsoft that an InvalidOperationException is thrown from WeakReference.IsAlive. WeakReference wraps weak GC handle implemented in CLR's Execution Engine (GC handle is also exposed by System.Runtime.InteropServices.GCHandle which supports not only weak handles, but other types too). A weak GC handle will be allocated and assigned to the WeakReference object when the WeakReference object is created. As described by Jeffrey Richiter, the weak GC handle contains pointer to an object, if the object is collected by GC, the GC handle will be cleared to NULL. Most of time WeakReference.IsAlive returns true or false to indicate whether the tracked object is alive. The check is based on whether the underlying GC handle contains a non-NULL pointer or NULL. Similarly, WeakReference.get_Target will return a valid object reference or null. But after the WeakReference object itself becomes unrooted and finalized, the underlying GC handle will be destroyed and any call to IsAlive or get_Target on the WeakReference object will throw InvalidOperationException in V1.X.

How would any method being called on a finalized object? Well, if one object O has a field WR as WeakReference, when O become unrooted and there are no other roots for WR, both objects are considered to be dead and will be put into F-reachable queue for finalization(check also check Jeffrey's article). Since there are no guarantee about order of finalizers (things are a little bit different for critical finalizer), when O's finalizer is executed, WR may already be finalized. Thus inside O's finalizer or after O is resurrected, it could call methods on the finalized WR. In the example I mentioned at the beginning, the problem is some object's finalizer is calling IsAlive on its WeakReference field.

The guideline for finalization says not to use any finalizable field in finalizer, so it's fair for WeakReference's properties to throw exception if they are called in finalizer. But it might be hard for people to understand why IsAlive needs to throw. After all it's only used to check status and doesn't need to access the tracked object if it's already collected. I think the reasoning is that IsAlive is meant to check whether the underlying GC handle tracks a live object, but if the GC handle is already gone during WeakReference's finalization, we can't answer the question.

The most interesting part is to look at history of this design decision. In V1.X, both IsAlive and get_Target property throws exception after the WeakRerence object is finalized; in Beta2 of V2.0, IsAlive still throws, but get_Target won't throw, it will return null after finalization; after Beta2, we made a change so that IsAlive won't throw either, it will return false after finalization. Partly because there are too many people calling WeakReference.IsAlive in finalizers, and it is not a really dangerous thing to do. Note that WeakReference.set_Target always throws after finalization.

So on CLR V2.0 offical released build, you could safely use WeakReference in finalizer.But it is still good practice not to use finalizable objects in finalizer, including WeakReference.

Thread, System.Threading.Thread, and !Threads (III)

Yun Jin — Tue, 30 Aug 2005 19:55:00 GMT

I got email asking me to explain !Threads output in details. I think this is a good question and a good topic for another installment to the series.

Here is an example I'll use for this post:

0:055> !threads
ThreadCount: 202
UnstartedThread: 95
BackgroundThread: 1
PendingThread: 0
DeadThread: 47
                                  PreEmptive   GC Alloc                     Lock
        ID ThreadOBJ       State     GC       Context           Domain     Count APT Exception
0 0xed0 0x0014f260   0x2000020 Enabled 0x00000000:0x00000000 0x00149aa0     1 Ukn
1 0xa3c 0x00157d28   0x2001220 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn (Finalizer)
XXX      0 0x00166378      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
4 0x12cc 0x00166540   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 STA
5 0x12dc 0x00166708   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
3 0xe7c 0x00175b70   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00175d38      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00175f00      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x001760c8      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00176290      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn System.InvalidOperationException
XXX      0 0x00176458      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00176620      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x001767e8      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
12 0x7e0 0x001769b0   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
13 0x15e8 0x00178008   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
14 0x4d0 0x001781d0   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00178398      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00178560      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00178728      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x001788f0      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00178ab8      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00178c80      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00178e48      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
21 0x14f0 0x00179010   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
22 0x1708 0x001791d8   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
23 0x11f8 0x001793a0   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
24 0x224 0x00179568   0x2001020 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00179730      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x001798f8      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
XXX      0 0x00179ac0      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn System.InvalidOperationException
XXX      0 0x00179c88      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn System.InvalidOperationException
XXX      0 0x00179e50      0x1820 Enabled 0x00000000:0x00000000 0x00149aa0     0 Ukn
...

First !Threads gives some statistics about Thread Store.

ThreadCount: number of total C++ Thread objects in Thread Store.

UnstartedThread: number of C++ Thread objects marked as unstarted. Recall I mentioned in previous blog, if a user creates a C# Thread object, CLR will create an "unstarted" C++ Thread object. When Thread.Start is called on the C# object, CLR will create an OS thread and remove "unstarted" flag from the C++ Thread object.

BackgroundThread: number of C++ Threads (and the corresponding OS threads) considered as background. Being background simply means CLR won't wait the thread for shutting down. Threads created explicitly by using System.Threading.Thread.Start are by default foreground threads; whereas threads wandering into CLR from unmanaged world are by default background threads (Rotor: SetupThread in vm/threads.cpp calls SetBackground(TRUE)). However, whether a thread is background could be changed by using IsBackground property in C# Thread object.

PendingThreads: If an OS thread is created but its ThreadProc hasn't be executed to the place to decrement unstarted counter in Thread Store, the thread is considered to be pending. Number of this type of threads should be quite low.

DeadThreads: Number of C++ Thread objects whose OS threads are already dead but the C++ objects themselves are not deleted yet.

In Rotor, all the five numbers are actually stored in ThreadStore (vm/threads.h) object as its fields.

Then it comes a table of all C++ Thread objects in Thread Store. Let me explain each field.

The first column doesn't have a header. It is the OS thread ID given by debugger just for debugging readability. Because the numbers only exist in debugger process, not the debuggee process, you may see the number being different when you look at a live session than when you debug a dump taken from the same live session. For a "dead" or "unstarted" thread, this column is "XXX".

ID: this is the thread ID assigned by OS, it remains consistent during debugger sessions, but OS could recycle it.

ThreadOBJ: address of C++ Thread object. You could see contents of the object by "dt mscorwks!Thread <address>" if you have symbols for mscorwks.dll.

State: one of the most important fields of the table. For Rotor, it is the C++ Thread's m_State field. It is combination of bit masks to indicate what the status the Thread currently is. All possible states (bit masks) are defined as enum ThreadState in vm/Threads. We already covered several states like TS_Background, TS_Unstarted, and TS_Dead. More states include TS_AbortRequested (this thread is requested to be aborted), TS_AbortInitiated (abort process is already started for this thread), TS_GCSuspendPending (GC is trying to suspend this thread), and etc.

Preemptive GC: also very important. In Rotor, this is m_fPreemptiveGCDisabled field of C++ Thread class. It indicates what GC mode the thread is in: "enabled" in the table means the thread is in preemptive mode where GC could preempt this thread at any time; "disabled" means the thread is in cooperative mode where GC has to wait the thread to give up its current work (the work is related to GC objects so it can't allow GC to move the objects around). When the thread is executing managed code (the current IP is in managed code), it is always in cooperative mode; when the thread is in Execution Engine (unmanaged code), EE code could choose to stay in either mode and could switch mode at any time; when a thread are outside of CLR (e.g, calling into native code using interop), it is always in preemptive mode.

GC Alloc context: allocate context GC might use when it tries to allocate object for this thread. In Rotor, it is m_alloc_context in C++ Thread object.

Domain: which AppDomain the thread is currently in (Rotor: m_pDomain field of C++ Thread class). You could use !DumpDomain or "dt mscorwks!AppDomain" to dump details of the domain. A thread can only be in one domain at a time, but it could switch into different domains. Speical marks will be put on thread's stack to when it transit to another domain.

Lock count: how many locks this thread has taken (Rotor: m_dwLockCount field of C++ Thread class). The locks it tracks include the managed monitors (taken by lock(obj) in C#), BCL's ReaderWriterLock, and certain locks inside CLR's unmanaged code.

APT: COM apartment for the thread, whether the thread is in a single-threaded apartment (STA), multithreaded apartment(MTA) or unknown.

Exception: the last managed exception thrown from this thread. It is saved in a GC handle in the C++ Thread object (Rotor: m_LastThrownObjectHandle).

The last column also indicates which special thread this thread is. However, !Threads only recognize a limited type of special threads for this field, including Finalizer thread, GC thread, Threadpool Worker thread, and Threadpool Completion Port thread. And for special threads which doesn't have a C++ Thread object (a special thread doesn't need to run managed code like debugger helper thread and server GC thread), they can not be displayed here. In Whidbey, a "-special" option is added to !Threads command which will show all special threads in the process as a separate list. Here is a sample output:

0:007> !threads -special
ThreadCount: 4
UnstartedThread: 0
BackgroundThread: 3
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
                                      PreEmptive   GC Alloc           Lock
       ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception
   0    1 828 0029a030      a020 Disabled 06907c38:069081d4 0f59e038     2 MTA
   4    2 16fc 0029e980      b220 Enabled 0690424c:069061d4 0021f4a8     0 MTA (Finalizer)
   5    3 1c1c 002e71e8      1220 Enabled 028f20f8:028f3f94 0021f4a8     0 Ukn
   6    4 1244 0f6fa778    80a220 Enabled 00000000:00000000 0021f4a8     0 MTA (Threadpool Completion Port)

       OSID     Special thread type
    1    e20    DbgHelper
    2   1e1c    GC
    3   1ed4    GC
    4   16fc    Finalizer
    5   1c1c    ADUnloadHelper
    6   1244    Timer

This posting is provided "AS IS" with no warranties, and confers no rights.