In previous blogs I have discussed the use of the SubInACL tool for changing the Access Control Lists (ACL's) of services. The tool can be downloaded from http://www.microsoft.com/downloads/details.aspx?FamilyID=E8BA3E56-D8FE-4A91-93CF-ED6985E3927B&displaylang=en). Please be aware that to use the tool successfully you must install in on the target machine using the MSI file. If you simply copy the SubInACL.exe file directly from another machine to the target machine you will encounter errors when changing or viewing control lists.
Chris Forster
In most cases when flowing a distributed transaction between more than a single machine, a number of Transaction Managers (TM's) will be involved. In Microsoft’s case the TM implementation is the Microsoft Distributed Transaction Coordinator (MSDTC).
The MSDTC representing the machine where the transaction is begun is normally referenced to as the superior MSDTC. The MSDTC representing further machines involved in the transactions are normally referred to as subordinate MSDTC’s.
In Microsoft Windows there is a limit to the amount of subordinate MSDTC’s that can participate in a single transaction and this is set to 64.
During a transaction, usually we will enlist one or more Resource Managers (RM's). An example of an RM would be Microsoft SQL Server. In many scenarios it is the responsibility of the subordinate TM to enlist the RM it represents. In Microsoft Windows we have a limit in which each MSDTC involved in the transaction (each subordinate) can each enlist a maximum of 32 RM’s.
Chris Forster
Take the scenario whereby MSDTC is configured on Server A and allows remote administration. This means on server B you will be able to add Server A as a remote machine within the component services management snap-in. Once added you will be able to make administrative changes to the MSDTC on Server A from Server B, however it also means that you will be able to view the transaction lists and statistics for the MSDTC on Server A from Server B.
This statistical transaction information should be a duplication of what you would normally see locally, and instead of the information being sent from the MSDTC Transaction Manager on Server A to the Component Services UI on Server A it is sent from the MSDTC Transaction Manager on Server A to the Component Services UI on Server B.
In most scenarios this functionality works very well however there is a scenario where you will find a inconsistency. If Server A is for example a 64-bit version of the Microsoft Windows operating system and Server B is a 32-bit version of the Microsoft Windows operating system then you will see an inconsistency in the results displayed for the transaction statistic which could cause alarm. Likely from Server B, you will see a very large number of “Unknown” and “Total” transactions outcomes under the aggregate section; whereas from Server A you will not see the large number of unknown transaction outcomes and therefore the “Total” number of transaction will be less.
The cause of this issue is a problem in structures used when this information is passed back from the 64-Bit OS to the 32-Bit OS. It is simply a user interface issue and the underlying functionality of MSDTC is not affected. In conclusion, if you face this scenario you should trust the statistics shown from the user interface on the local machine where MSDTC is running over the results shown from the remote administration user interface.
Chris Forster
Take the scenario whereby you have a Windows 2008 Cluster used as a SQL Database Cluster. A common requirement of SQL Server is a Transaction Manager (MSDTC).
If your configuration is to have one clustered instance of MSDTC in its own resource group and for any of your SQL server instances to be in their own separate resource groups, then there is a potential for MSDTC and SQL Server to be active on different nodes at a given time.
Primarily the SQL Server process loads the MSDTC Proxy in order to communicate with the MSDTC Service. When this is on the same node there will likely be no problems, however if they are active on different nodes and you have the Windows 2008 Firewall enabled you may encounter problems. The communication between the MSDTC Proxy (loaded in to SQL) and the MSDTC service is done through RPC. The obviously symptom of such a problem will be that you cannot flow a transaction from your application towards the SQL Server instance in question, however more specifically if you connect to the SQL Server instance using the management studio and run a query “Begin distributed Transaction” it will fail.
If you examine the exception rules (for the MSDTC exceptions) within the “Windows firewall and Advanced Security” you will see the following rules:
Inbound Rules:
-Distributed Transaction Coordinator (RPC) -> Limited to SVCHOST Process communication
-Distributed Transaction Coordinator (RPC-EPMAP) -> Limited to SVCHOST Process communication
-Distributed Transaction Coordinator (TCP-IN) -> Limited to MSDTC Process communication
Outbound Rules:
-Distributed Transaction Coordinator (TCP-Out) -> Limited to MSDTC Process communication
As can seen there is no “TCP-IN” or “TCP-Out” for the SQL Server process. This is important as it is the SQL Server process initiating the contact with MSDTC. In order to keep security as tight as possible, instead of disabling the firewall, you can create these rules and ensure endpoint addresses must be within the cluster boundary. When creating these new rules, select the SQL Server process executable in the options (remember each is instance specific) and add all virtual and physical IP ranges within the cluster to the rules scope properties.
There are already configured DTC rules (named above) to MSDTC traffic from any application server to the Database Cluster so there is no concern about this for now.
Once these rules are enabled, repeat the “Begin Distributed Transaction” query when MSDTC is on a different node to SQL Server and it should now be successful.
Chris Forster
On a Windows 2003 cluster you can only ever have one instance of MSDTC running on the cluster at any one time. For this reason and to ensure the running instance of MSDTC is available to all nodes in the cluster, it must be configured as a clustered resource as per http://support.microsoft.com/kb/301600.
In order to configure MSDTC on the cluster a shared disk is required. This disk is where the MSDTC log file is held. The log file contains details and states of open transactions. This means if the MSDTC service was to stop for any reason, it can be restarted and the transaction could continue without being lost.
As stated in the KB article 301600 you cannot use a MNS disk as the disk for the MSDTC log file.
From the article http://technet.microsoft.com/en-us/library/bb123996.aspx we also state: “In Windows Server 2003, the MSDTC cluster resource requires the use of shared storage in the cluster.”
I have come across a number of scenarios whereby we did not meet these requirements yet still require MSDTC. An example of such a scenario is a Microsoft Exchange Cluster containing only an MNS disk with no shared disk installed. On these cluster you may need MSDTC to install certain updates or even to use some custom written COM+ add-on's.
First we must specify that MNS (Majority Node Set) is not a shared disk it is more a replication method and as stated above it cannot be used for the MSDTC log file.
In order to get this working in a Microsoft supported manner there are only two solutions:
- Install a shared disk; or
- Consider moving to Windows 2008 (Windows 2008 supports an active-active MSDTC configuration).
If these solutions are really not possible for your particular case there is one other possible option. I must explicitly state that this is a non-supported and not a fully tested workaround.
The method means that MSDTC would not be highly available on the cluster. It uses a local quorum disk and means MSDTC will be allowed to come online on a single (statically defined) node.
Even though the same node is the only place MSDTC will ever come online this is still a clustered resource, meaning a call from a passive node or active node to use MSDTC should work.
In order to try such a workaround, you should create a new cluster group. Within this cluster group create a regular IP Address and Network Name Resource.
The next step is to create a Local Quorum resource, the wizard to undertake this will present you the option of which nodes this resource can be used on. Ensure you select only the one node (e.g. Node A). Once completed then a “Distributed Transaction Coordinator” resource should be created and again only be allow to come online on the same single node (e.g. Node A). The dependency of this final resource should be the three resource named above. If this has been done correctly, you will arrive at drop down box where you will be given the drives available for the MSDTC log file; the local quorum should be listed.
(I must re-iterate that in the above description MSDTC will only come online on Node A, however can be utilised by other cluster nodes when Node A is online. If Node A goes offline or fails then MSDTC would not move to another node and MSDTC would not be available until Node A is back up and running.)
Chris Forster
On a Windows 2003 Cluster environment, if you have an application (e.g. COM+, winforms etc) which runs under a non-administrative account and trys to start Distributed Transactions (using MSDTC), you may find it will fail. Below are a few of the errors you could see:
-Event ID 4426 MSDTC Client
-Event ID 4376 MSDTC Client
-Event ID 4691 COM+
-DtcGetTransactionManagerEx(): hr = 0x80070005
-DtcGetTransactionManagerEx(): hr = 0x8004d027
Explaination
1) DtcGetTransactionManagerEx API initializes the MSDTC proxy, before making the connection to the MSDTC resource on the cluster. In order to figure out the running MSDTC instance, the proxy makes several cluster API calls that require the user calling this API to have "Full Control" access to the Cluster.
2) In addition, the proxy queries the Cluster API for Virtual Server that owns the MSDTC resource. When Kerberos is enabled, access to some of the registry parameters of this resource are removed from the Authenticated users and this results in the failure to get the Virtual Server Name that owns the MSDTC resource.
Possible Workaround:
1) In cluster administrator it is possible to “right click” on the cluster name and select “properties”. Under the "Security" tab it is possible to add the specific non-administrative account to the permissions list and give them permissions on the cluster.
Alternatively this step can be undertaken using the following command line:
Cluster ClusterName /prop "security descriptor"= DOMAIN\USER ,grant,f:security
Further details of this step can be found at: http://support.microsoft.com/default.aspx?id=316881
2) Install SubInACL (http://www.microsoft.com/downloads/details.aspx?FamilyID=E8BA3E56-D8FE-4A91-93CF-ED6985E3927B&displaylang=en0).
-From a command line please use the following two commands:
a) subinacl.exe /service MSDTC /grant="DOMAIN\USER"="Q"
b) subinacl.exe /service ClusSvc /grant="DOMAIN\USER"="QSEIL"
-Run the following from a command window:
sc sdset SCMANAGER D:(A;;CCGR;;;AU)(A;;CCLCRPWPRC;;;SY)(A;;KA;;;BA)S:(AU;FA;KA;;;WD)(AU;OIIOFA;GA;;;WD)
3) This step is only required if your "network name" resource for the group containing the MSDTC has the parameter property set for "Enable Kerberos Authentication".
- If possible this setting should be disabled, however if this is not the case then you need to open the registry and make a change to the permissions on a specific key.
Under the "My Computer\HKEY_LOCAL_MACHINE\Cluster\Resources\" there are a number of registry keys named as a GUID. Each of these corresponds to a resource configured within the cluster and contains a "name" property which will allow it to be identified. You need to locate the name of the resource which corresponds to the "network name" resource used by MSDTC. This key should be further expanded and you will find a sub-key names "Parameters". Right click on this key to select the "permissions". You need to add “Read” access for the Authenticated User group (or more specifically the user identity calling the DtcGetTransactionManagerEx API).
Chris Forster
The following may help you if your “COM+ Event System” service does not start successfully. The service stops responding (hangs) and the Services Management Console shows the service as "Starting". You may also find some other services such as the "Windows installer Service" will not start.
The first obvious check is to ensure that the RPC Service is correctly functioning, since this is one of the Event Systems dependencies. Normally another good suggestion has been to try to either re-register Event System or even in more extreme cases re-install COM+.
Re-registering EventSystem:
-Backup and Remove the following registry Key: "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventSystem" and reboot;
-Run "Regsvr32 es.dll" and reboot again.
Reinstalling COM+:
-Steps as per article: http://support.microsoft.com/kb/315296.
However in a recent issue, I have seen that this has not resolved the problem and trying to re-install COM+ also fails.
A worthwhile suggestion is to check the “DCOM Server Process Launcher” Service. This service should be configured to "automatic" start-up and should be "started". If you have found this is not the case, then make the change and reboot the machine. Following this the Event System Service may now starts successfully.
If (and only if) you have already tried to re-install COM+ before this change, you should again follow the steps in KB 315296 to ensure the COM+ installation now completes successfully.
Chris Forster
This post applies to Windows 2003 Clusters
1) First you should make a careful note of all the MSDTC security Settings within "Component Services" on the currently active cluster node. This can be found by right clicking the "My Computer" node of the DCOMCNFG tree and selecting "properties". On the tab list select "MSDTC" and then select the "Security Configuration" button.
2) Let us say for example, the MSDTC resource is currently within the Main cluster resource group and the MSDTC resource has a dependency on the main Quorum Drive Q:\ This means that the MSDTC log file is stored within a directory on this Q: drive.
3) All resources that use MSDTC should be brought offline whilst the cluster is being reconfigured e.g. The SQL group. This is not essential but is best practice.
4) Please ensure the MSDTC resource itself (which should be now set "not to effect the group") is brought offline, and for now please just delete the MSDTC resource in cluster administrator. On the Q:\ drive the directory Q:\MSDTC can also be deleted (this deletes the old msdtc log file, which will no longer be used -> Ensure first all outstanding transactions have been completed).
5) Following this you will find that no MSDTC.exe process is started on any of the cluster nodes. However the actual MSDTC service, which is installed on all of the nodes, is still physically there.
6) It is now time to correctly reinstall the MSDTC in its own resource group. The best way to do this is using method 1 from the article: http://support.microsoft.com/kb/301600. i.e. Create a new Cluster group with the 4 required resources:
-MSDTC Network Name
-MSDTC IP Address
-MSDTC Disk
-MSDTC Resource
7) Once completed, the MSDTC resource group should be brought online. A new MSDTC log file should have automatically been created on the drive being utilised by this new resource group.
8) On the active MSDTC node, check the MSDTC security setting are configured or changed to match those noted in step 1). If changes are required, MSDTC will prompt to be automatically restarted.
9) Finally, ensure all resources that were taken offline (e.g. SQL) are now brought back online.
Chris Forster
If you have a Resource Manager (for example SQL Server) that is failing to communicate with MSDTC (that you can see is started and working well), you maybe wondering the best way to troubleshoot such a problem?
One possible issue could be that the account under which the SQL Server (or other resource Manager) is running does not have permission on the MSDTC Service.
This can be checked in the ACL (Access Control List) using the tool named "subinacl" and downloaded from:
http://www.microsoft.com/downloads/details.aspx?familyid=e8ba3e56-d8fe-4a91-93cf-ed6985e3927b&displaylang=en
The command for checking the permissions on the msdtc service is:
"subinacl.exe /Service msdtc"
This command should output a number of different accounts or account groups. You must identify the permission for the account that the problematic resource manager is running.
From the Subinacl Readme file you can see the possible Service Permission options:
PACE Description
-F -> Full Control
-R -> Generic Read
-W -> Generic Write
-X -> Generic eXecute
-L -> Read controL
-Q -> Query Service Configuration
-S -> Query Service Status
-E -> Enumerate Dependent Services
-C -> Service Change Configuration
-T -> Start Service
-O -> Stop Service
-P -> Pause/Continue Service
-I -> Interrogate Service
-U -> Service User-Defined Control Commands
If you are missing a permission, normally the command to add the missing ACL is something like this:
subinacl /service msdtc /grant=<"DOMAIN\ACCOUNT" IN QUOTES>="QSETIL"
e.g.
subinacl /service msdtc /grant="Domain\SQLServiceAccount"="QSETIL"
Chris Forster
When you wish to flow transactions between a client and a WCF Service there are three key tasks to be considered:
1) The binding that you select (which is set on be both the client and service side), should be transaction aware. Within the configuration file you should also set the transaction flow attribute to true.
2) For each of the services operational contracts, the transaction flow option should also be specified, either 'NotAllowed', 'Allowed' or 'Mandatory'.
3) You must specifically mark the service methods that you wish to be transactional. This is set as an 'Operation Behaviour' attribute called 'Transaction Scope Required', and this should be set to true.
There is a great article with full details at: http://msdn.microsoft.com/msdnmag/issues/07/05/Foundations/default.aspx
Chris Forster
In both Windows Vista and Server 2008 there is now no longer a checkbox to enable TIP transaction. More information regarding a background of TIP can be found at: http://msdn2.microsoft.com/en-us/library/ms679484(VS.85).aspx.
TIP is something that Microsoft will hope to discourage due to efficiency and security problems which have led to its deprecation. Therefore not only is the user interface option gone, it is also disabled by default.
If there is a specific reason you need to allow TIP, this can be done through the registry:
Warning Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require that you reinstall the operating system. Modify the registry at your own risk.
To enable this it appears that the change must be applied directly to the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\Security
By setting: NetworkDtcAccessTip to 1 rather than 0.
It is important to note that there will be no guarantee that this feature will be supported in later operating system.
Chris Forster
In both Windows Vista and Windows Server 2008 the administration of MSDTC is a little different than Windows 2003 SP2 and Windows XP SP2. On a standalone machine, the difference is only in the layout of the options available, and with the exception of TIP, all functionality is still configurable through the user interface.
The new security features still remain as per <http://support.microsoft.com/kb/899191>. However the interface for these settings is a little different.
There is now a more explicit distinction between the specified local and remote Coordinators and the settings of these TM's. So as with Windows 2000, the “my computer” MSDTC properties only specify the local or remote coordinator:
Now the settings for the Coordinators are more explicitly (in location) associated with the specific DTC in question.
e.g. Component Services -> Computers -> My Computer -> Distributed Transaction Coordinator ->Local DTC
When you select the properties of the Local DTC, you will find the windows which gives the options you have come to expect. This includes tabs for Tracing, logging and security.
The main reason behind this is due to the new clustering features which allow both clustered and local instances of MSDTC to be configured independently. For more about MSDTC on Windows 2008 Clusters see http://technet.microsoft.com/en-us/library/cc730992.aspx.
As stated the missing setting that you maybe familiar with is to enable TIP, more information can be found: http://blogs.msdn.com/chrisforster/archive/2008/02/19/msdtc-in-windows-vista-and-windows-server-2008-removal-of-the-tip-checkbox.aspx.
Chris Forster
Since writing my last post, I thought it would be good to have some more information about how to actually embed a manifest as part of the build process in Visual Studio.
Unmanaged Projects
Embedding a manifest file for an unmanaged project: simply add the manifest in the project and Visual Studio will recognize it automatically.
Managed Projects
For a managed project it is not so simple:
The following extra steps are required for embedding it:
1) Add a new Item as an XMLFile called for instance "mymanifest.manifest"
2) Check the Build action property of the file, it should be “None” because VS doesn’t accept automatically the manifest file for managed code.
3) Add a resource file in order to embed the manifest file ManagedManifest.rc, check the build action property it should be “Content”
4) Open the RC file within VS editor and add the following :
#define RT_MANIFEST 24 // provided by WIN32 SDK
#define APP_MANIFEST 1 // provided by WIN32 SDK
APP_MANIFEST RT_MANIFEST mymanifest.manifest.
5) Add a custom Pre-Build event:
"C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin\rc.exe" "$(ProjectDir)ManagedManifest.rc"
6) Unload the project by right clicking on the project name in the Solution Explorer->Unload
7) Edit the project file (.csj)Right click on the project name in the Solution Explorer->Modify <project name>.csj
8) Add a new property group :
<PropertyGroup>
<Win32Resource>ManagedManifest.res</Win32Resource>
</PropertyGroup>
9) Close the file and reload it , right click on the project name in the Solution Explorer-> Reload
10) Build the solution
Chris Forster
Problem
Since upgrading to Windows 2003 SP1 or Windows Vista, applications that depend on the components in external manifest files, stop working. For example SxS Registration Free COM.
Reason
Windows 2003 Server and later Operating Systems including Windows Vista, have a change in the manifest reading behaviour. Previously, in Windows XP the external manifest took priority over any embedded manifest. Therefore if there was an embedded manifest, the external manifest was used in preference. In Windows Server 2003 and later, this behaviour is in fact the opposite, the embedded manifest is now used in preference to an external manifest.
To see the workaround and solution please review: http://blogs.msdn.com/chrisforster/pages/ChangeToManifest.aspx
Chris Forster
In this paper, the methods of environmental mapping with multi-agent mobile robots are discussed. Using three small mobile robots, proprietary software has been developed in C# in order to complete the tasks. The main methods of localisation, object detection, map representation and strategy will be described within the paper. The localisation includes different methods including one adapted from a system developed within the Cybernetics Department at Reading University. Problems associated with the technologies used, and solutions aimed to overcome these issues are also described. Some experimental results from the project are shown from the preset scenarios within the MADLAB laboratory.
Link to this Paper : http://blogs.msdn.com/chrisforster/pages/SCARP2006.aspx
Chris Forster
