Workflow documents suddenly showing as 'Not Submitted' rather than 'Pending Approval' in Microsoft Dynamics GP 2010

When a user submits a document into the workflow system and approval is required, the document will have a status of 'Pending Approval'.  A user can view the workflow status from the window in Microsoft Dynamics GP 2010.  For example, in the Purchase Order Entry window below, you can see at the top of the window, the purchase order is 'Pending Approval by…'.

Have you run into the situation where Workflow for Microsoft Dynamics GP 2010 is working perfectly and then all of a sudden the workflow documents show as 'Not Submitted' rather than 'Pending Approval' when a user submits a document into the workflow system? In addition, no error is presented to the user upon submission.  For example, in the Purchase Order Entry window below, you can see at the top of the window, the purchase order is 'Not Submitted' even though the document has been submitted for approval.

You will also begin to see several errors logged about every 20 minutes in the Dynamics Event Viewer on the machine where Web Services is installed.  The Source of the error is Microsoft.Dynamics.Security.AdminService and the error details are below:

Event Error 1:

Exception type:
System.IO.FileNotFoundException

Exception message:
The system cannot find the file specified. (Exception from HRESULT: 0x80070002)

SecurityContext:null
Action:OnCacheUpdateTimerEvent

Input parameters:

Stack Trace:

at Microsoft.Interop.Security.AzRoles.AzAuthorizationStoreClass.UpdateCache(Object varReserved)
at Microsoft.Dynamics.Security.AzManHelper.UpdateCache()
at Microsoft.Dynamics.Security.AzManHelper.OnCacheUpdateTimerEvent(Object source, ElapsedEventArgs e)

Event Error 2:

Exception type:
System.Runtime.InteropServices.COMException 

Exception message:
The handle is invalid. (Exception
from HRESULT: 0x80070006 (E_HANDLE))  

SecurityContext:
ApplicationKey:null
ScopeKey:null

User:XXXXXXXXXXXXX

Action:Get

Input parameters:

<RequestObjects>

<ApplicationKey
xmlns:xsi="www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="www.w3.org/2001/XMLSchema">
<Id xmlns="schemas.microsoft.com/dynamics/security/2006/01">25cc1a21-2cc4-4b13-a1c8-eea186fb688a</Id>
</ApplicationKey></RequestObjects>

Stack Trace:   
at
Microsoft.Interop.Security.AzRoles.AzAuthorizationStoreClass.OpenApplication(String
bstrApplicationName, Object varReserved)   
at Microsoft.Dynamics.Security.AzManApplicationServiceImplementation.RetrieveForGetByKey(SecurityContext
context, Key key) at
Microsoft.Dynamics.Security.SecurityService.Get(SecurityContext context, Key
key)

Another symptom appears in Business Portal as well.  If you click to view any external list in Business Portal 5.1, you may also see the following error:

If you launch the Dynamics Security Console on the machine where Web Services is installed, you will see the following error and will not be able to see the security store.

 

Based on the error above, you will more than likely go to Services to see if the Microsoft Dynamics GP Service Host service is running.  This is the service that runs Web Services for Microsoft Dynamics GP 2010.  In Services, you will see the Microsoft Dynamics GP Service Host service is still running, therefore, this may cause you to question the above error in the Dynamics Security Console.

As we were troubleshooting the above issues, we asked what had changed in the environment since everything was working fine.  The answer we received in most cases was that the Microsoft SQL Server was rebooted or the Microsoft SQL Server instance had been stopped and started.  This started us down the path of investigating why a restart of the Microsoft SQL Server would potentially cause an issue.  In all cases the Web Services security store was held in a database in Microsoft SQL Server.  There is a timer that runs every 20 minutes on the server where Web Services is installed.  The purpose of the timer is to update the security store with any changes.  Because the connection to the Microsoft SQL Server was disconnected, this timer cannot make the connection to the Microsoft SQL Server even though the Microsoft SQL Server instance is back up and running.  This connection is running through the Windows Authorization Manager (AZMAN) pieces on the Windows operating system as well. 

This issue then causes all the issues we talked above:

1. Workflow documents now indicate 'Not Submitted' rather than 'Pending Approval'.
2. The external lists in Business Portal 5.1 will not work.
3. The Web Services Dynamics Security Console will produce an error.
4. An event will be logged every 20 minutes in the Event Viewer with the exception message "The handle is invalid. (Exception from HRESULT: 0x80070006 (E_HANDLE))"

The resolution for all issues listed above is to stop and start the Microsoft Dynamics GP Service Host service on the server where Web Services is installed.

More to come on this issue!

Enjoy!