Microsoft SharePoint Team Blog

I actually ran into a delay problem with the out of the box version of MOSS related to the Designer Workflow that Pauses for a certain duration.  I tested it with a few minute interval and the workflow would not start up.  So I googled the issue and someone had posted an stsadm command prompt that fixed it.  I have a sinus infection right now, however if you would like I can send a link of the post when I am feeling a little better.

Pues bien, nuevo año, nuevos desafíos, como diría mi jefe... Algo que ya había comentado en alguna oportunidad

Delay Activity is not working in State Machine Workflows. I have a complex workflow completely based on them and done other tests to find out what was wrong. The Delay has as Property TimeoutDuration - only this is taken into account and none of the code before it or initializing this timer. I tried to execute a code on the InitializeTimoutDuration but whatever is there it won't be considered, as well as code in InitializeState activity. If TimoutDuration is set to 0:0:0 the timer fires regardless of the code before it, i.e. referencing a DateTime in a list on an item. This TimeoutDuration has fixed values in the designer ranging from 0:0:0 to 1 day. I've been wondering from the very start why this value cannot be overwritten. The project will be sent to your email address and we hope you have a solution soon.

Hi everyone, If you're having trouble with Delay activities not returning in SharePoint workflows

Please hurry, don't follow the last sentence in this post "over the next couple of months". We need solution fast.

We have issues with any workflow that uses a Delay action.  Even the most basic workflow that does a delay for 5 minutes then tries to send a notification email.  The issue is quite simple - it never restarts after the delay.

At about the time you would expect it to be restarting after the delay action, there is an error in the ULS log, the details of which are:

Timestamp Process TID Area Category EventID Level Message Correlation  

11/14/2007 10:45:56.05 OWSTIMER.EXE (0x07EC) 0x0E58 Windows SharePoint Services Workflow Infrastructure 98d8 Unexpected System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.SharePoint.Workflow.SPWorkflowManager.RunWorkflowElev(SPWorkflow originalWorkflow, SPWorkflow workflow, Collection`1 events, SPRunWorkflowOptions runOptions)

I too (like so many others) found that Delay events in a State Machine workflow simply did not fire in Sharepoint. Then I found this:

Following this post:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1668527&SiteID=1&pageid=1

[Scroll down to the 21 June 2007 post by Fred Morrison]

I have found that if I restart the SPTimerV3 service in my PostBuildActions.bat after doing the deploy, the delay activities in the state machine appear to be firing correctly. This test was in my local VM, I am about to deploy this workflow to my test server and will let you know if it works there.

Further update:

• After deploying to my test server, the workflow/events were not firing.

• I stopped and restarted the “Windows SharePoint Services Timer” service and the events started firing.

Commands used:

Net stop SPTimerV3

Net start SPTimerV3

I also added these commands to my “postbuildactions.bat” file in my development project – right after the “iisreset”, and found that

Hope this helps

Tim Tuxworth

I'm not sure if this is the same issue, but since SP1 has been installed in our environments, all workflows that are set to start automatically (either when an item is created, or updated) will not kick off if the item is added by the "System Account".

This is especially an issue for environments where we have workflows set to start when items are created in document libraries that are e-mail enabled (since the timer service account is the only account that will enter those items).

These workflows will work as designed if an item is added by any other user, however items added by the System Account will not kick off the same workflow.

Also, these workflows will work as designed if started manually after the item is added by either a normal user or by the System Account.

I have not seen any further information online regarding this issue, so if anyone has a solution or temporary workaround until a final resolution is found, please pass it on.

["Brian"]

See example on Robert Shelton's blog on "How to Video: Programmatically Escalating an Overdue Task as a part of a Document Workflow in SharePoint (MOSS 2007)". Example has “Delay” activity http://www.sheltonblog.com/archive/2007/11/01/how-to-video-programmatically-escalating-an-overdue-task-document-workflow.aspx

@Brian

I encounterd the same issue without SP1 installed. I have a 2 machine installation with 1 web frontend and 1 app backend (the one where the incomming email smtp runs). Every once in a while, the on new event doesn't fire. I done a workaround and added code to start my wf programatically in my on new mail eventhandler.

We also had problems without SP1.

I use DelayActivty) as one of several "legs" of a ConditionedActivityGroup that includes a Replicator of WssTaskActivty (from ECM Starter Kit 1.2).  The DelayActivty is used to determine when, if ever, to send scalding reminder emails to those lazy good for nothing folks who ignore their assigned tasks :-).  

Essentially, the common Due Date/Time of all tasks minus a # of days (minutes during testing) to send out Reminders is used to determine how long to set the DelayActivity's TimeOutDuration.  Example:

           // Figure out how long to wait - make sure you have the HotFix in place or this won't work

           TimeSpan ts = new TimeSpan(0);

           ts = ReminderDate - DateTime.Now;

           DelayActivity da = sender as DelayActivity;

           da.TimeoutDuration = ts;

When the DelayActivity "pops", it checks the list of tasks that the Replicator created to see which ones are not complete and sends Emails to them.

Another DelayActivty, which is part of the same CAG does a similar calculation on DueDate:

           // Figure out how long to wait - make sure you have the HotFix in place or this won't work

           TimeSpan ts = new TimeSpan(0);

           ts = DueDate - DateTime.Now;

           DelayActivity da = sender as DelayActivity;

           da.TimeoutDuration = ts;

When the DueDate monitoring DelayActivty "pops", it sets the When condition of yet another leg of the same CAG to True.  That other leg of the CAG spins through the list of tasks and for any that are not completed, issues an AlterTask command to force the OnTaskChanged event to fire. Inside OnTaskChanged, I check for a special "AutoCompleting unfinished task" indicator and set the Description/Outcome of that task accordingly.

DelayActivty is crucial to being able to have tasks WITH ENFORCEABLE DEADLINES WITH REAL CONSEQUENCES FOR NOT DOING YOUR JOB (COMPLETE YOUR ASSIGNED TASKS ON TIME).  None of the out-of-the-box MOSS workflows provide this ENFORCEMENT OF TASK DUE DATES.

Once I put all the necessary HotFixes in, DelayActivity works fine in Sequential-type workflows.  Alas, as others have noted, it doesn't work in State Machine workflows.

Hi everyone,

First of all, thank you for all your feedback!  Keep it coming!  It looks like there may be multiple types of problems people are encountering that we haven't seen, so your data is super valuable.  

I also wanted to make sure you try the timer reset workaround that people have suggested; some of you have noticed that doing a reset of the timer service seems to fix things, so I wanted to clarify why this happens.  

Workflows start off in w3wp.exe, which loads the workflow assembly from the GAC.  When it hits a delay, it puts an event in the queue for OWSTimer.exe to pick up at the scheduled time. When the time comes, OWSTimer loads the workflow assembly from the GAC.

However, if you modify and redeploy the workflow assembly into the GAC and do an iisreset, starting a new workflow will make w3wp pick up the new one, while OWSTimer still has the old one cached.  Because of this mismatch, the workflow may crash and appear to hang indefinitely.  Thus, when you reset the timer, it flushes the cache, and on next load, it picks up the same assembly as w3wp.  Hence your workflows work again.

To help us find bugs other than this, please send me mail or post if you still have issues AFTER trying the following prerequisite workarounds:

1) Install the WF delay hotfix (http://www.microsoft.com/downloads/details.aspx?FamilyID=6096ce0f-d21e-47ac-afe2-d4e1c2fce670&displaylang=en)

2) Run the following commands on your front ends:

Net stop SPTimerV3

Net start SPTimerV3

When you send in info, it would be great to have the following additional information if available (feel free to follow up with this info if you haven’t sent it to me already):

1) Are you using WSS or MOSS?

2) What are the details of your symptoms?  Delay does not return ever, or delay returns at a much later time than expected (and how much later)?

3) What is your workflow timer interval set to?  You can use the following command to find this:

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>stsa

dm -o getproperty -propertyname "job-workflow" -url http://localhost

4) Check the WSS_Content SQL table to see if an event is actually registered at the specified time with the following query:

SELECT * FROM ScheduledWorkItems Where id = '2761E667-FECB-4A0B-8F4E-5D38C0644DA0' <- insert your broken workflow ID guid

Let me know if the event is missing, or if it was scheduled but never got picked up.

5) Are you running on a production environment with other types of load on your system, or is it an isolated test environment?

Thanks,

Eilene

In my development environment, my "job-workflow" interval is set to 2 minutes (default is 5), an extra level of overhead I can handle.  It allows me to "compress time" (days become minutes) in my Delay Activities to wait small amounts of time that won't get 'rounded up' to the nearest 5 minute interval like what happens in production (where the 5 minute minimum is still set by default).

Can someone tell me how can we track the active workflows on a site and more precisely the active delay activities and in how long they are due to "fire". Are there any administration tools or even API calls that can be used for such chores?

I don't seem to be able to track those anywhere to make sure that all those are still being tracked. As well, is there any sure way to "re-hydrate" all the workflows, as I am now seeing that often when we edit and recompile our workflow that the delay tasks especially don't seem to start back.

We have MOSS (no SP1) on 1 server and SQL 2005 on another server.

The workflow is build using SharePoint Designer and it uses the activity “Pause until Date”. It worked well for a couple months. Then we lost connection between the web server and database and we did a Shutdown + Restart for the web server followed by stop+restart SPTimerV3.

On ScheduleWorkItems table the pending workflows (created before the crash) have “InternalStatus”=9. Those created after and pending have “InternalStatus”=8.

Those created before the crash are still pending even the scheduled date has passed.

What does InternalState = 8 mean when I run the following query on my dev machine after starting a workflow that has a DelayActivity in it? Also, why do the times (DeliveryDate, Created) appear to be Universal Coordinated Time?

Note how I was lazy and used a 'everything in the past 2 weeks' WHERE clause in my query instead of plugging in a specif GUID.  In my case, I only get back one row, so my laziness is 'close enough' :-)

SELECT [SiteId]

     ,[Id]

     ,[DeliveryDate]

     ,[Type]

     ,[ParentId]

     ,[ItemId]

     ,[ItemGuid]

     ,[BatchId]

     ,[WebId]

     ,[UserId]

     ,[Created]

     ,[BinaryPayload]

     ,[TextPayload]

     ,[ProcessingId]

     ,[ProcessMachineId]

     ,[ProcessMachinePID]

     ,[InternalState]

 FROM [WSS_Content].[dbo].[ScheduledWorkItems] swi

WHERE swi.[Created] > DateAdd(Week, -2, GetDate())

ORDER BY

swi.[Created] DESC

I have an entry in Wss_Content database, table ScheduledWorkItems, that shows Created = 2008-03-28 20:33:10, DeliverDate = 2008-03-31 20:50:54, InternalState = 9.

Is it safe to delete this row?  It seems to be associated with a DelayActivity timer for one of my Workflows that was supposed to be delivered two days ago.

Or will SharePoint eventually clean this up by itself after some period of time (now isn't that ironic!) passes?

Microsoft: please publish the meanings of the values for InternalState so we can help you out.

When you can fix this bug????

Three months passed,what did you do???

Hi,guys

Delay activity troubled me for a long time.Today ,I found something which cause it can't wake.

In Sharepoint Database ,there's a table named "ScheduledWorkItems",which have a column named "DeliveryDate",

it decide that the delay acivity of workflow instance whether  wake,

sometimes the wss insert a wrong time (Max Date),so that  the instance can't wake forever,when I try to changed this  value to correct time,the instance will work normally .

as discussed above ,dealy activity  whether be  waked due to whether the time be inserted  corrected.

btw,I found that when a workflow  have parellel   delay activity , wss just  insert a record to  table of "ScheduledWorkItems", which make me confused.

do you have any suggestion?   share  it with us please.

Recent results from testing of turning off SPTimerV3 service for extended period of time (e.g., long maintenance cycle on the server) after workflow with DelayActivities has already started running:

1. Started workflow and waited for first of five DelayActivities to "pop" (timer event delivered to the workflow).

2. Stopped (not Restart) SPTimerV3 service.

3. Deliberately kept SPTimerV3 off the air for a period of time that guaranteed that two other DelayActivities in previously started workflow that should have "popped" did not pop because the timer service was off the air and therefore not available to deliver the event at the "DeliveryDate" specified in SQL Server table ScheduledWorkItems (adjusted for UTC).

4. Started SPTimerV3 before last two DelayActivities in my particular workflow were scheduled to "pop".

Questions to be answered by the test:

1. If SPTimerV3 service is off the air, are "missed" DelayActivity timers discarded, queued, or something else?

2. If missed timers are queued waiting for SPTimerV3 to come back on the air, what order are they delivered in?

3. If missed timers are queued, are they all delivered at once or are they "spaced apart" at some interval (5 minutes perhaps)?

Results:

1. After starting the SPTimerV3 service, the two seemingly "missed" timers associated with a couple of DelayActivities in my workflow "popped" at the next opportunity (next 5 minute interval boundary since my settings are the default).

2.   The order of the missed timers was preserved, but they were delivered "in the same bundle".

3. Both missed DelayActivity timers were delivered (late by design of my test) to the workflow by the SPTimerV3 timer service and the logic behind those DelayActivities kicked in.

4. All remaining timers associated with DelayActivities "popped" when they should.

Conclusion:

Keeping SPTimerV3 service off the air for and extended period of time, such as a long maintenance window (e.g., installing SQL Server 2005 post-SP2 HotFix Rollup #7) only delays delivery of timer events associated with DelayActivities in running workflows.  Yes, it's like a multi-vehicle "pile up" when they finally do get delivered and it's difficult to pin down exactly WHEN they'll be delivered, so depending on the business process your workflow models, you'll have to decide if this "late delivery pile up" is a good thing or a bad thing.

Has this been fixed yet?  It has been half a year since this blog was started but I still have workflows that will not dehydrate.

Fred,

What hotfixes are you referring to?  Where are they located?

Body: I just got a copy of &quot;Microsoft Office SharePoint Server 2007 Best Practices&quot; by &quot;Ben

I think we figured it out. At least for this specific problem (the "stuck" workflows taking a long time to resume). It turns out that in some environments, .NET 3.0 SP1 was not installed. Once I installed the service pack, things seemed to work properly. Still need to verify this in a couple other environments, but (fingers crossed) hopefully this has resolved our issue.