Florin Lazar - Consistency Checkpoint : MSDTC

Phase0 in .Net System.Transactions

florinlazar — Sun, 09 Apr 2006 23:20:00 GMT

In a previous post (http://blogs.msdn.com/florinlazar/archive/2006/01/29/518956.aspx) I talked about Phase0 and some useful things it allows you to do. Although not referred as Phase0, phase 0 features are available in .Net System.Transactions in two forms. Actually in three, but the third is a derivate. I will make many references here to the previous post on Phase0 and thus I recommend reading it first.

The first form where Phase 0 shows up is through EnlistmentOptions.EnlistDuringPrepareRequired that you can pass to EnlistVolatile or EnlistDurable methods. This allows you to create new enlistments during Prepare phase (and after Commit was called) and thus it allows you to create the caching resource manager (RM) discussed in the earlier post. To create a cache RM, you will first call EnlistVolatile with EnlistmentOptions.EnlistDuringPrepareRequired, and then, during Prepare you will connect to the database to write the updates. The connection to the database will usually call EnlistDurable on the transaction. Similar to Phase0 in the native DTC API, during the Prepare phase, one can enlist again passing EnlistmentOptions.EnlistDuringPrepareRequired which will issue another round of Prepare on that enlistment and so on.

The second form comes with DependentTransaction. You get a DependentTransaction by calling the DependentClone method on a transaction object. You can get the other two scenarios enabled by Phase0, “early commit protection” and “async programming” by tweaking the DependentCloneOption to RollbackIfNotComplete and BlockCommitUntilComplete respectively. Then, you signal that the work is done by calling the Complete method of this dependent transaction.

Thus you will write the DoTransfer method in .Net as follows:

int DoTransfer(int amount, Transaction tx)
{
    // create the dependent clone
    DependentTransaction dtx = tx.DependentClone(DependentCloneOption. RollbackIfNotComplete);
    DB1.OpenConnection(tx);
    DB1.DoDebit(amount);
    DB2.OpenConnection(tx);
    DB2.DoCredit(amount);

// signal work complete on the “phase0” enlistment
dtx.Complete();
}

If the DependentTransaction is not Complete-ed before a Commit is issued by the client, the transaction will be aborted.

The “third” derivate form is TransactionScope itself. TransactionScope protects you against early commits by automatically handling under the covers a DependentTransaction with RollbackIfNotComplete. That is why TransactionScope is recommended for most of the transactions scenarios. Given this you can write with the same confidence the DoTransfer method as follows:

int DoTransfer(int amount, Transaction tx)
{
    // creating a TransactionScope by passing in the received transaction
    // this will set tx to be the ambient transaction and create a "rollback" dependent transaction
   using(TransactionScope ts = new TransactionScope(tx))
   {
       DB1.OpenConnection(); // note that now OpenConnection is picking the transaction from the ambient context instead of passing it as a parameter
       DB1.DoDebit(amount);
       DB2.OpenConnection();
       DB2.DoCredit(amount);

       ts.Complete();
   }
}

Other Scenarios enabled and/or made easier by DependentTransaction

The “Worker Threads Scenario”
This is explained both at http://pluralsight.com/blogs/jimjohn/archive/2005/05/01/7923.aspx and http://msdn2.microsoft.com/en-US/library/ms229976(VS.80).aspx

Passing transaction ownership in a sequence of threads
Described at http://pluralsight.com/blogs/jimjohn/archive/2006/01/01/17769.aspx.

Async Programming
This is the scenario described in my previous post where the client asks asynchronously a service to do some work as part of the transaction, but doesn’t want/need to wait until the service completes the work. The code will look like the following in .Net:

Client Code
void MainFunction()
{
    using(TransactionScope ts = new TransactionScope())
    {
        // dtx is a member of the current class of type DependentTransaction
        // its lifetime needs to be at least until the ack from the service is received
        this.dtx = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
        AsyncDoTransfer(tx, AckFromDoTransfer);
        ts.Complete();
     }
}
void AckFromDoTransfer()
{
     this.dtx.Complete();
}

Server Code
void AsyncDoTransfer(tx, AckFromDoTransfer)
{
DependentTransaction dtx = tx.DependentClone(DependentCloneOption. BlockCommitUntilComplete);
AckFromDoTransfer(); // the client can Commit safely from now on since I have dtx created on the service side

    // the dependent clone auto-created by TransactionScope doesn’t help in this scenario because it rollbacks on early commit while we need a blocking clone
    using(TransactionScope ts = new TransactionScope(tx))
    {
        // doing the work that takes a lot of time
        DB1.OpenConnection();
        DB1.DoDebit(amount);
        DB2.OpenConnection();
        DB2.DoCredit(amount);
        ts.Complete();
    }

// work if finished on the service so we can safely let the transaction continue
dtx.Complete();
}

Our Own Forum is Now Live! Please Join Us at the "Transactions Programming Forum"

florinlazar — Sat, 11 Mar 2006 02:34:00 GMT

I'm really happy to announce that our own forum, dedicated to transactions in Windows and .Net is now live at http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=388&SiteID=1

The RSS feed for the forum is http://forums.microsoft.com/MSDN/rss.aspx?ForumID=388&Mode=0&SiteID=1

I hope that by having a forum that is dedicated to Transactions technologies we can build a better channel for our customers to get quick and accurate answers to their questions. We encourage everyone in the community to participate in this forum.

I look forward to us working together to make this forum a valuable resource for Transactions discussions.

Dedicated Site for Transactions on Microsoft.com

florinlazar — Wed, 01 Mar 2006 13:01:00 GMT

In an effort to have a common place for all the transactions technologies we provide (and we started to have a good number of technologies that continues to grow), we have put together a webpage on Microsoft.com dedicated to transactions. The address is: http://www.microsoft.com/windowsserver2003/appserver/transmgmt.mspx

MSDTC: The Magic of Phase Zero (Phase0) – Or – When Using 2PC Transactions Is Not Enough

florinlazar — Mon, 30 Jan 2006 01:24:00 GMT

The most known technique of implementing distributed transaction is the "two-phase commit" (2PC). Here is a quick summary of how this technique works:
   - Active phase: After the transaction is created and before the "commit" message is issued by the creator, the transaction is in the "active" state. During this phase, resource managers can "enlist" and become part of the transaction. When "commit" is issued, the transaction moves to Phase1.
   - Phase1 (preparing phase): in which the superior transaction manager (TM), after receiving the Commit message from the creator of the transaction, "asks" the enlisted resource managers or the subordinate transaction managers to "prepare" for commit; if all respond with a successful "prepared" answer then the transaction moves to Phase 2, otherwise the transaction is aborted
   - Phase 2 (committing phase) the superior TM writes a durable log entry about the status of the transaction and then starts sending "commit" messages to all the resource managers and subordinate TMs part of the transaction. In the case of any type of error (communication, computer crash etc), the superior TM will continue to send the "commit" messages until a successful "committed" message is received from all the participants. When all the participants responded "committed", the log entry corresponding to the current transaction is removed from the log and the transaction ends.

In addition to the "two-phase commit" technique, MSDTC supports another phase called Phase Zero (Phase0) which occurs after the creator of the transaction calls Commit and before Phase1 starts. To participate in Phase0 you need to enlist as a Phase0 participant: http://msdn.microsoft.com/library/?url=/library/en-us/cossdk/htm/transactionphase0enlistment_6j04.asp?frame=true and http://msdn.microsoft.com/library/?url=/library/en-us/cossdk/htm/pgdtc_dev_8ldf.asp?frame=true. MSDTC will always start with Phase0 and remain there until all the Phase0 participants responded.

The great benefit of Phase0 is that during this phase, any type of enlistment (including Phase0) in the transaction it is still allowed. Once the transaction moves to Phase1, the new enlistments will be denied. Since you can create a new Phase0 enlistment during Phase0, MSDTC executes this phase in waves. After "commit" is issued, the TM will first send "Phase0Request" messages (http://msdn.microsoft.com/library/en-us/cossdk/htm/itransactionphase0notifyasync_8ywk.asp) to all known Phase0 enlistments. If all of them successfully replied Phase0Done (http://msdn.microsoft.com/library/en-us/cossdk/htm/itransactionphase0enlistmentasync_5pt1.asp), the TM will look for new Phase0 enlistments that occurred meanwhile and it will start another Phase0 wave if any is found and so on. Phase0 will end when no other new Phase0 enlistment is found after a wave. After Phase0 ends, the transaction moves to Phase1.

Phase0 allows the following three scenarios: caching resource managers, protection against "early" commits and "safe" propagation of transactions in async programming.

Caching Resource Managers
Using a Phase0 enlistment, a middle-tier component can delay the connection to the database(s) as long as possible, acting as a cache to the requests from the client and thus reducing network traffic and database locks. This works great for those scenarios in which the client does many changes to the same data before it finally decides to end and save the work. When the client "ends" the work, the transaction is committed, and the TM issues a Phase0Request to the caching component. Before replying Phase0Done, the component will open all the necessary connections to the databases that need to be involved and it will persist the final data. These databases will now be "durably" enlisted in the transaction. After Phase0Done is received, the TM will continue with the 2PC and commit the transaction. See also http://msdn.microsoft.com/library/?url=/library/en-us/cossdk/htm/pgdtc_dev_0y2b.asp?frame=true

Protection Against "Early" Commits
In general, 2PC transactions help to ensure integrity of data during different types of system errors. Using Phase0 you can also protect from badly written software.

Let me give an example: let's say a transaction is created on the client side and then propagated to a middle tier component that is asked to do work as part of that transaction. The work that this component needs to do involves two databases, DB1 and DB2, as follows:

int DoTransfer(int amount, transaction tx)
{
DB1.OpenConnection(tx);
DB1.DoDebit(amount);
DB2.OpenConnection(tx);
DB2.DoCredit(amount);
}

The component is using the received transaction to communicate with the two databases to ensure the integrity of its data. Both Debit and Credit operations need to be successful or none should occur. This looks great until now. But what if the client doesn't wait for DoTransfer to finish and calls Commit right after the DB1.DoDebit returned and before DB2.OpenConnection started? A badly written client can do that, maybe the programmer wanted to use multi-threading but didn't quite get it right. What will happen is that the component is left with inconsistent data in the databases, a debit operation that occurred without a credit operation. There is nothing wrong with the transaction itself; from the point of view of the 2PC transaction, only one enlistment occurred and that one responded successfully to both "prepare" and "commit" phases. It is the application logic that is wrong here.

Can you protect against this type of situation? Yes, you can. One way is to write bug-free code, but mistakes happen. If you don't own the client code, the problem is even harder. The best solution is to use a Phase0 enlistment and modify the middle tier code as follows:

int DoTransfer(int amount, transaction tx)
{
EnlistPhase0(tx); //Creates a Phase0 enlistment with the transactions

DB1.OpenConnection(tx);
DB1.DoDebit(amount);
DB2.OpenConnection(tx);
DB2.DoCredit(amount);

SignalWorkCompletedToPhase0Enlistment(tx); // signals the Phase0 enlistment that the work was completed
}

Your Phase0 enlistment should start with a "WorkCompleted" flag set to false. The method SignalWorkCompletedToPhase0Enlistment mentioned above will set this flag to true. If a Phase0Request is received and the flag is still false, it means that the Commit was issued before the DoTransfer finished its work. At this point, you have two options:
- one option (the recommended one when "early commits are not expected) is to abort the transaction and log some error; if you own the client code you might want to catch and fix these "early" commits
- the other option is to hold the Phase0Request, received while DoTransfer is still doing work, until the flag becomes true and only then let the transaction continue by replying with Phase0Done; use this option when you expect to receive "early" commits as part of your system logic and flow.

Asynchronous Programming
This is very similar to the early commits scenarios mentioned above, but this time, the early commits are "by design". One can encounter scenarios where a piece of the application starts a transaction and delegates the work asynchronously to other parts of the application (a different thread of execution, a remote location) and without waiting for a response continues with committing the transaction. In these scenarios, we will choose the second option from above, the one that blocks the commit until work is finished. The code is similar:
Client code:
  Transaction tx = StartTransaction();
  AsyncDoTransfer(tx); // like create a separate thread and let it do the transfer operation etc
  SomeOtherAsyncWork(tx);
  CommitTransaction(tx);

Server code:
  // inside AsyncDoTransfer that receives the transaction from the client
  EnlistPhase0AndBlock(tx); // if we get a Phase0Request we will block and wait until SignalWorkCompletedToPhase0Enlistment is called and only then reply with Phase0Done

DB1.OpenConnection(tx);
DB1.DoDebit(amount);
DB2.OpenConnection(tx);
DB2.DoCredit(amount);

SignalWorkCompletedToPhase0Enlistment(tx); // signals the Phase0 enlistment that the work was completed
return;

Looks nice and safe... Well, there is a problem even with this code. What if, Commit is called even before we are able to enlist in Phase0 in the AsyncDoTransfer method? We can end up again with some inconsistent data. The solution is to create some sort of acknowledgment from the server to the client saying that at least it got to the phase0 enlistment: "hey client, I know I'm slow and I will do your requested work later but here is my ack that you can Commit your transaction safely any time from now on". To accomplish this we will have to use phase0 enlistments on both the client and server side as follows:

Client Code:
  Void MainFunction()
  {
    Transaction tx = StartTransaction();

    EnlistPhase0AndBlock(tx); // if we get a Phase0Request we will block and wait until SignalWorkCompletedToPhase0Enlistment is called and only then reply with Phase0Done
    AsyncDoTransfer(tx, AckFromDoTransfer); // like sending an async message to a server and let it do the transfer operation etc

    CommitTransaction(tx);
}
void AckFromDoTransfer()
{
    SignalWorkCompletedToPhase0Enlistment(tx); // signals the Phase0 enlistment that the work was completed
}

Server Code:
  void AsyncDoTransfer(tx, AckFromDoTransfer)
  {
   EnlistPhase0AndBlock(tx); // if we get a Phase0Request we will block and wait until SignalWorkCompletedToPhase0Enlistment is called and only then reply with Phase0Done

    AckFromDoTransfer(); // the client can Commit safely the transaction from now, since I'm protected by phase 0

    // doing the work that takes a lot of time
    DB1.OpenConnection(tx);
   DB1.DoDebit(amount);
    DB2.OpenConnection(tx);
    DB2.DoCredit(amount);

SignalWorkCompletedToPhase0Enlistment(tx); // signals the Phase0 enlistment that the work was completed
}

The phase0 enlistment on the client side will make sure that we keep the transaction in phase0 until we get all the acks from our servers. Look at this as a method to pass phase0 "ownership" from client to servers.

All these work due to the "magic" of phase0 that allows "infinite" number of enlistments, until no new phase0 enlistments are being created and the existing ones replied with a successful Phase0Done. I compare the Phase0 enlistment to the AddRef method from the good old COM lifetime management. A Phase0 enlistment will call "AddRef" on the "active phase" of the transaction, while a Phase0Done will call "Release" on the counter. While the counter is higher than zero, the transaction will not be allowed to enter Phase1.

SQL Server 2005 with COM+/DTC is first in a new TPC-C result

florinlazar — Tue, 04 Oct 2005 08:29:00 GMT

The new results posted by Dell running SQL Server 2005 with COM+/DTC tops TPC-C by Price/Performance with .99 US$ price/tpmC: http://www.tpc.org/tpcc/results/tpcc_price_perf_results.asp

I can't wait to see some results with SQL Server 2005 combined with System.Transactions and our new "promotable transactions" (http://blogs.msdn.com/florinlazar/archive/2005/05/12/416805.aspx).

Fast transactions with System.Transactions and Microsoft SQL Server 2000

florinlazar — Fri, 30 Sep 2005 04:44:00 GMT

The simplest way to use transactions today with Microsoft SQL Server 2000, using .Net Framework 2.0, is as follows:

       static void Main(string[] args)
        {
            using (TransactionScope ts = new TransactionScope())
            {
                SqlConnection sqlConnection = new SqlConnection("connectionString");
                sqlConnection.Open();
                SqlCommand sqlCommand = new SqlCommand("INSERT INTO ...", sqlConnection);
                sqlCommand.ExecuteNonQuery();
                sqlConnection.Close();

                ts.Complete();
            }
        }

If you really care about speed, you will notice that the performance decreases compared to when transactions are not used. If you do a little bit of investigation, you will notice that in fact a MSDTC distributed transaction is created and used when the code is executed (you can see this using Component Services snap-in). But why?

If you run the same code against Microsoft SQL Server 2005, the performance doesn't decrease when compared to a similar code that doesn't use transactions. Good, but you might not have SQL Server 2005 (yet). So, what can you do?

Let's first explain the "why". In order to take advantage of lightweight transaction manager (or LTM) that comes with System.Transactions, your durable resource manager, or database in this case, needs to support a mechanism called "promotable transactions". I talked about how this can be accomplished at http://blogs.msdn.com/florinlazar/archive/2005/05/17/418595.aspx Since currently only SQL Server 2005 supports promotable transactions, when you use System.Transactions with SQL Server 2000, the lightweight transaction needs to be transformed into a MSDTC transaction, because this is the distributed transaction type that SQL Server 2000 understands. Having an MSDTC transaction involved, means there is some additional cost, and that is why you are seeing the perf hit. Just to make sure we are on sync on this, the perf hit is only when you compare it to the scenario that is not using transactions. If you compare System.Transactions with EnterpriseServices/COM+ scenarios using transactions, the perf is improved with System.Transactions.

And now, let's go over on what can you do, if you want to use System.Transactions with no perf hit, in a scenario where you only talk to a database server (Microsoft SQL Server 2000) and you might also involve volatile transacted resources, like a transacted hashtable. The solution is to use an "adapter" that enlists with the System.Transactions transaction using PSPE (http://blogs.msdn.com/florinlazar/archive/2005/05/17/418595.aspx) and coordinates the connection to the SQL Server using a SQL "local transaction". The code will have to look like this:

        static void Main(string[] args)
        {
            using (TransactionScope ts = new TransactionScope())
            {
                SqlConnection sqlConnection = new SqlConnection("connectionString;Enlist=false");
                DatabaseTransactionAdapter dbAdapter = new DatabaseTransactionAdapter(sqlConnection);
                sqlConnection.Open();
                dbAdapter.Begin();
                SqlCommand sqlCommand = new SqlCommand("INSERT INTO ...", sqlConnection);
                sqlCommand.Transaction = (SqlTransaction)dbAdapter.Transaction;
                sqlCommand.ExecuteNonQuery();

                ts.Complete();
            }
        }

In addition to the changes/additions in bold, you should also observe that “sqlConnection.Close();” was removed.

Now let’s dive into the details of the adapter. First, since the adapter will handle the connection to the SQL Server, you will have to specify in the connection string “Enlist=false”, thus telling to the SQL client to not enlist in the transaction, because this will determine the creation of a MSDTC transaction for reasons mentioned above. And you also must not close the connection, because the connection should stay open until the transaction is completed, which happens after the “using” statement ends. The adapter will take ownership of the connection lifetime and close it when it is done with it.

When Begin is called on the adapter, the adapter will enlist with the System.Transactions transaction, in other words, Transaction.Current, using EnlistPromotableSinglePhase. Later, LTM will call Initialize on the enlistment interface, and that is the time when the bridge to the SQL Server is established; the adapter will start an internal SQL transaction on the connection provided in the constructor using SqlConnection.BeginTransaction().

An additional step that might look unnecessary, at least for me, is that you need to set the internal SQL transaction from the connection to the sqlCommand.Transaction property. Manually. Why SqlCommand can’t get that automatically from the SqlConnection object, I don’t know. Maybe an expert in SQL objects can jump in and explain. And that is why the adapter needs to publish the SQL transaction in a property.

When the transaction completes, after exiting the using statement, LTM will notify the adapter through the IPromotableSinglePhaseNotification interface to commit or abort the transaction. Consequenlty the adapter will commit or abort the internal SQL transaction.

John Doty from my team, created a set of classes that does exactly this (the adapter works for both resource managers using IDbConnection and the MSMQ resource manager). They are available for download at http://download.microsoft.com/download/B/D/0/BD0D4D33-89DC-497E-B3F2-95871A03A5F7/PrivateTransactionAdapter.msi The installer will expand a TransactionAdapter.cs in C:\Documents and Settings\<currentUser>\My Documents\MSDN\Private Transaction Adapter. All you have to do is link the file to your project and use the adapter as described above.

DTCPing: Troubleshooting MSDTC Connectivity Issues

florinlazar — Fri, 16 Sep 2005 20:31:00 GMT

In today's world, security hardening is causing many headaches to software developers and admins. Especially when you have an application that needs to go beyond the boundaries of the local machine. MSDTC is a distributed app by definition and thus, getting transacted app that uses MSDTC to work in your network is not as easy as one would expect. You either get "New transaction cannot enlist in specified transaction coordinator" or 8004d00a.

There is a tool that can help you troubleshoot MSDTC connectivity issues, and that tool is called DTCPing: http://www.microsoft.com/downloads/details.aspx?FamilyID=5e325025-4dcd-4658-a549-1d549ac17644&DisplayLang=en

It was initially created to troubleshoot firewall issues: http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q306843 This KB article also gives you instructions on how to use DTCPing.

Most common errors that block MSDTC connectivity in a network are:
1. MSDTC Security settings are not configured properly: http://blogs.msdn.com/florinlazar/archive/2004/06/18/159127.aspx
2. MSDTC is not in the exception list in the firewall
3. The two machines involved in the transaction cannot see each other by the NetBIOS name. Try ping-ing the machines by name. If that fails, MSDTC will fail. You can solve this by updating your system32\drivers\etc\hosts file.
4. If the machines were imaged improperly, the CID for MSDTC under HKEY_CLASSES_ROOT\CID may be the same on the two machines, while they need to be unique. The KB 306843 mentioned above gives the instructions to correct this issue.

Transactions Sessions at PDC 2005

florinlazar — Wed, 14 Sep 2005 20:31:00 GMT

If you are at PDC this year, don't forget to attend the sessions related to transactions. An easy way to find these sessions is to go to http://commnet1.microsoftpdc.com/content/sessions.aspx and search by Keyword="transaction".

I especially recommend the session called "FUN320 - Windows Vista & "Longhorn" Server: Improving Reliability Using System.Transactions and the Transactional NTFS and Registry". Jim Johnson, our architect for transactions is presenting this session together with Dana Groff, the program manager in Core File Services focusing on transaction technologies.

Here is the full list:

FUN320 Windows Vista & "Longhorn" Server: Improving Reliability Using System.Transactions and the Transactional NTFS and Registry

Day/Time: Thursday, September 15 5:15 PM- 6:30 PM Room: 406 AB

Speaker(s): Dana Groff, Jim Johnson

Session Type(s): Breakout

Session Level(s): 300

Track(s): Fundamentals

You can build significantly more reliable applications by using the expanded and enhanced classes in the System.Transactions namespace. This session covers how to use new transaction-oriented programming techniques for application stability and robustness. Learn about the new systems in Windows Vista and "Longhorn" Server for supporting Transacted Files (TxF) and Transacted Registry (TxR) operations in both native and managed code.

FUN323 Microsoft Research: Future Possibilities in Concurrency

Day/Time: Friday, September 16 8:30 AM- 9:45 AM Room: 406 AB

Speaker(s): Tim Harris

Session Type(s): Breakout

Session Level(s): 300

Track(s): Fundamentals

Learn about the investment Microsoft Research is making in future languages and tools to help write software that will gain the maximum benefit from tomorrow's multi-processor/multi-core hardware. In this session, we start by introducing some of the tools and techniques we've developed for tracking down and preventing bugs in multi-threaded software. We then turn to the language features that we're prototyping in research: our work on transactional memory is developing a programming model where data can be shared directly between threads without needing to worry about locking or low-level deadlocks; our work on synchronization is leading to a unified abstraction for co-ordination between threads and between processes. We'll demo some of our prototype systems, showing how these techniques can lead to software which is not only easier to develop, but which can scale from uniprocessors up to highly parallel systems.

Distributed Atomic Transactions - Wishful Thinking?

Day/Time: Tuesday, September 13 9:00 PM- 10:00 PM Room: 511 C

Session Type(s): Birds of a Feather

Session Level(s): 300

Web Services, via SOAP, is the preferred choice of doing wide-reaching distributed communications today. However, one of the few factors that has hindered its adoption in the enterprise today is its lack of ability to handle transactions between remote un-trusted (independent) boundaries. There are two schools of thought today with regards to transactions --- The Atomic Way or The Compensating Way which pretty much takes out the atomicity of ACID. Indigo supports WS-AT via Microsoft DTC. Will this be widely adopted in the enterprise collaboration space? Or are we living in Fool's Paradise?

COM307 Windows Communications Foundation ("Indigo"): Writing Reliable and Transacted Distributed Applications

Day/Time: Wednesday, September 14 3:15 PM- 4:30 PM Room: 403 AB

Speaker(s): Shy Cohen

Session Type(s): Breakout

Session Level(s): 300

Track(s): Communications

What do you do when your network connection breaks half-way through a request? How do you synchronize state changes across different Web services? How do you overcome a system crash without losing important messages? Windows Communications Foundation (formerly codename "Indigo") provides simple and powerful reliability mechanisms that allow you to easily address these types of network and application issues. Take an in-depth look at reliable sessions, queues, and distributed transactions, and how these technologies are used to achieve reliable, transacted communication.

COM202 Windows Communications Foundation ("Indigo"): A Lap around the Windows Communications Foundation

Day/Time: Tuesday, September 13 1:00 PM- 2:15 PM Room: Halls C & D (Petree Hall)

Speaker(s): Omri Gazitt

Session Type(s): Breakout

Session Level(s): 200

Track(s): Communications

Windows Communications Foundation (formerly codename "Indigo") is Microsoft's new platform technology for building distributed applications, bringing the best of ASP.NET Web services, .NET Remoting, Enterprise Services, WSE, and System.Messaging into one unified framework. This session introduces the core concepts in the Windows Communications Foundation programming model-addresses, bindings, contracts, and behaviors-and presents the Windows Communications Foundation feature set through the lens of these concepts. This session covers security, reliable messaging, transactions, queues, hosting, transports, interop/integration, and more. If you have never seen an introduction to Windows Communication Framework, this session is for you.

The slides from these sessions are currently available at http://commnet.microsoftpdc.com/content/downloads.aspx

WS Specs for Transactions Updated

florinlazar — Tue, 23 Aug 2005 03:43:00 GMT

The WS specifications related to Transactions (WS-Coordination, WS-AtomicTransaction and WS-BusinessActivity) have just been updated. They are available at http://msdn.microsoft.com/webservices/webservices/understanding/specs/default.aspx?pull=/library/en-us/dnglobspec/html/wsatspecindex.asp.

Writing a resource manager that supports promotable transactions (or Promotable Single Phase Enlistment aka PSPE) in System.Transactions

florinlazar — Tue, 17 May 2005 17:14:00 GMT

A key feature that targets performance in System.Transactions is the Promotable Single Phase Enlistment. It allows a durable resource manager (RM) to host and "own" a transaction that can later be promoted to a distributed transaction (or MSDTC transaction) if necessary. This specific resource manager usually has its own internal non distributed transactions and it needs to support changing those transactions to distributed transactions at runtime.

The overview of steps involved in PSPE is:
1. The RM proxy enlists for PSPE using Transaction.EnlistPromotableSinglePhase. If the enlistment succeeds, the RM usually creates its internal transaction and associates it with the System.Transactions transaction. The notification are later sent to the RM proxy using the IPromotableSinglePhaseNotification interface
2. If the System.Transactions transaction never requires a promotion (see previous post http://blogs.msdn.com/florinlazar/archive/2005/05/12/416805.aspx) then, when the transaction is committed, the RM proxy will receive a SinglePhaseCommit, at which point it can commit the internal transaction that was initially created
3. If the System.Transactions transaction needs to be promoted (to support multiple RMs for instance), then System.Transactions will ask the RM to promote the transaction to a distributed transaction. The RM will have to promote the internal transaction to an MSDTC transaction, and associate it with the work already done. Later, when System.Transactions will commit its transaction, it will send a SinglePhaseCommit notification to the RM proxy and the RM will have to commit the distributed transaction that it created during promotion.

Please note that at step 1, I had an "if the PSPE enlistment succeeds" condition. This means that PSPE is not always allowed by System.Transactions. The cases when this can happen are: 1) the transaction is already a distributed transaction or 2) another RM has already done a PSPE enlistment. If the PSPE enlistment fails, the RM will have to follow the regular rules for enlistment (marshal the transaction to the RM and enlist using DurableEnlist).

Let's look at code now. I will have a client and a server communicating using remoting:

<client.cs>

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;
using System.Transactions;

namespace PSPEClient
{
    class Client
    {
        static void Main(string[] args)
        {
            using (TransactionScope ts = new TransactionScope())
            {
                /*section1begin
                DurableRM durableRM = new DurableRM();
                durableRM.OpenConnection();
                durableRM.DoWork();
                /*section1end*/
                
                DatabaseProxy dbProxy = new DatabaseProxy();
                dbProxy.OpenConnection();
                dbProxy.DoWork();
                
                /*section2begin
                DurableRM durableRM = new DurableRM();
                durableRM.OpenConnection();
                durableRM.DoWork();
                /*section2end*/
                
                ts.Complete();
            }
            System.Console.ReadLine();
        }
        
        class DatabaseProxy
        {
            private PSPEServer.PSPEDatabaseServer db;
            private InternalRM internalRM;
            
            public void OpenConnection()
            {
                System.Console.WriteLine("DatabaseProxy.OpenConnection");
                // connecting to the remote database
                TcpChannel tcpChannel = new TcpChannel();
                ChannelServices.RegisterChannel(tcpChannel);
                this.db = (PSPEServer.PSPEDatabaseServer)Activator.GetObject(
                    typeof(PSPEServer.PSPEDatabaseServer), "tcp://localhost:8085/MyDatabase");
                if (null == db)
                {
                    System.Console.WriteLine("Cannot connect to the server");
                }
                else
                {
                    System.Console.WriteLine("Internal tx id:" + db.Connect());
                }
                
                // enlisting in the transaction
                if (null != this.internalRM)
                {
                    throw new System.Exception("we don't support multiple connections, this is just a sample");
                }
                this.internalRM = new InternalRM(db);
                this.internalRM.Enlist();
            }
            
            public void DoWork()
            {
                System.Console.WriteLine("DatabaseProxy.DoWork");
                db.DoWork();
            }
            
            class InternalRM : IPromotableSinglePhaseNotification
            {
                #region IPromotableSinglePhaseNotification Members
                
                // This member will be called during the call to EnlistPromotableSinglePhase
                // The RM will usually allocate its internal transaction state here
                public void Initialize()
                {
                    System.Console.WriteLine("InternalRM.Initialize");
                }
                
                // This method will be called if the RM should Rollback the
                // transaction.  Note that this method will be called even if
                // the transaction has been promoted to a distributed transaction.
                public void Rollback(SinglePhaseEnlistment singlePhaseEnlistment)
                {
                    System.Console.WriteLine("InternalRM.Rollback");
                    db.RollbackWork();
                    singlePhaseEnlistment.Aborted();
                }
                
                // This method will be called when the RM should Commit the
                // transaction.  Note that this method will be called even if
                // the transaction has actually been promoted to a distributed
                // transaction.
                public void SinglePhaseCommit(SinglePhaseEnlistment singlePhaseEnlistment)
                {
                    System.Console.WriteLine("InternalRM.SinglePhaseCommit");
                    db.CommitWork();
                    singlePhaseEnlistment.Committed();
                }
                
                #endregion
            
                #region ITransactionPromoter Members
                
                // This method will be called if System.Transactions
                // determines that the transaction actually needs the support of
                // a fully distributed transaction manager.  The return value of
                // this method is a promoted representation of the transaction
                // usually in the form of transmitter/receiver propagation token
                public byte[] Promote()
                {
                    System.Console.WriteLine("InternalRm.Promote");
                    return db.Promote();
                }
                
                #endregion
            
                private PSPEServer.PSPEDatabaseServer db;
                
                public InternalRM(PSPEServer.PSPEDatabaseServer db)
                {
                    this.db = db;
                }
                
                public void Enlist()
                {
                    System.Console.WriteLine("InternalRM.Enlist");
                    if (null != Transaction.Current)
                    {
                        if (!Transaction.Current.EnlistPromotableSinglePhase(this))
                        {
                            System.Console.WriteLine("PSPE failed, doing regular Enlist");
                            // PSPE failed; we need to use the regular enlistment
                            db.Enlist(TransactionInterop.GetTransmitterPropagationToken(Transaction.Current));
                        }
                    }
                }
            }
        }
     
        class DurableRM : IEnlistmentNotification
        {
            
            #region IEnlistmentNotification Members
            
            public void Commit(Enlistment enlistment)
            {
                System.Console.WriteLine("DurableRM.Commit");
                enlistment.Done();
            }
            
            public void InDoubt(Enlistment enlistment)
            {
                System.Console.WriteLine("DurableRM.InDoubt");
                throw new Exception("The method or operation is not implemented.");
            }
            
            public void Prepare(PreparingEnlistment preparingEnlistment)
            {
                System.Console.WriteLine("DurableRM.Prepare");
                // first a durable RM will log preparingEnlistment.RecoveryInformation(), but this is just a sample
                preparingEnlistment.Prepared();
            }
            
            public void Rollback(Enlistment enlistment)
            {
                System.Console.WriteLine("DurableRM.Rollback");
                enlistment.Done();
            }
            
            #endregion
            
            public void OpenConnection()
            {
                System.Console.WriteLine("DurableRM.OpenConnection and enlist durable");
                if (null != Transaction.Current)
                {
                    Transaction.Current.EnlistDurable(Guid.NewGuid(), this, EnlistmentOptions.None);
                }
            }
            public void DoWork()
            {
                System.Console.WriteLine("DurableRM - DoWork");
            }
        }
    }
}

</client.cs>
<server.cs>

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;
using System.Transactions;
using System.Diagnostics;
 
namespace PSPEServer
{
    class Server
    {
        static void Main(string[] args)
        {
            TcpChannel tcpChannel = new TcpChannel(8085);
            ChannelServices.RegisterChannel(tcpChannel);
            RemotingConfiguration.RegisterWellKnownServiceType(Type.GetType("PSPEServer.PSPEDatabaseServer"),
                "MyDatabase", WellKnownObjectMode.Singleton);
            System.Console.WriteLine("Server running...");
            System.Console.ReadLine();
        }
    }
 
    public class PSPEDatabaseServer : MarshalByRefObject
    {
        private int internalTxID = 0;
        private CommittableTransaction tx;
        private InternalServerRM internalServerRM;

        public int Connect()
        {
            System.Console.WriteLine("client connected");
            return ++internalTxID;
        }

        public void DoWork()
        {
            System.Console.WriteLine("PSPEDBServer.DoWork");
        }

        public byte[] Promote()
        {
            System.Console.WriteLine("PSPEDBServer.Promote");
            this.tx = new CommittableTransaction();
            Debug.Assert(this.internalServerRM == null);
            // the following statement will cause the transaction to be promoted to MSDTC
            byte[] txToken = TransactionInterop.GetTransmitterPropagationToken(this.tx);
            Enlist(txToken);
            return txToken;
        }

        public void CommitWork()
        {
            System.Console.WriteLine("PSPEDBServer.CommitWork");
            if (tx != null)
            {
                // we have a distributed transaction, and so we have to commit it
                tx.Commit();
            }
            else
            {
                // we only have an internal tx
                System.Console.WriteLine("committing internal tx:" + internalTxID);
            }
        }

        public void RollbackWork()
        {
            System.Console.WriteLine("PSPEDBServer.RollbackWork");
            if (tx != null)
            {
                // we have a distributed transaction, and so we have to rollback it
                tx.Rollback();
            }
            else
            {
                // we only have an internal tx
                System.Console.WriteLine("aborting internal tx:" + internalTxID);
            }        
        }

        public void Enlist(byte[] txToken)
        {
            System.Console.WriteLine("PSPEDBServer.Enlist");
            this.internalServerRM = new InternalServerRM();
            this.internalServerRM.Enlist(txToken);
        }

        private class InternalServerRM : ISinglePhaseNotification
        {

            #region ISinglePhaseNotification Members

            public void SinglePhaseCommit(SinglePhaseEnlistment singlePhaseEnlistment)
            {
                System.Console.WriteLine("InternalServerRM.SPC");
                singlePhaseEnlistment.Committed();
            }

            #endregion

            #region IEnlistmentNotification Members

            public void Commit(Enlistment enlistment)
            {
                System.Console.WriteLine("InternalServerRM.Commit");
                enlistment.Done();
            }

            public void InDoubt(Enlistment enlistment)
            {
                System.Console.WriteLine("InternalServerRM.InDoubt");
                throw new Exception("The method or operation is not implemented.");
            }

            public void Prepare(PreparingEnlistment preparingEnlistment)
            {
                System.Console.WriteLine("InternalServerRM.Prepare");
                // first a durable RM will log preparingEnlistment.RecoveryInformation(), but this is just a sample
                preparingEnlistment.Prepared();
            }

            public void Rollback(Enlistment enlistment)
            {
                System.Console.WriteLine("InternalServerRM.Rollback");
                enlistment.Done();
            }

            #endregion

            private Guid rmGuid = new Guid("{B14FF9BB-8419-4dbc-A78C-3C1453D60AC4}");

            public void Enlist(byte[] txToken)
            {
                System.Console.WriteLine("InternalServerRM.Enlist");
                TransactionInterop.GetTransactionFromTransmitterPropagationToken(txToken).EnlistDurable(
                    this.rmGuid, this, EnlistmentOptions.None);
            }
        }
    }
}

</server.cs>

If we run the code, with only the PSPE enlistment, System.Transactions is managing the transaction and MSDTC is never involved; we get the following outputs:

<client>
DatabaseProxy.OpenConnection
Internal tx id:1
InternalRM.Enlist
InternalRM.Initialize
DatabaseProxy.DoWork
InternalRM.SinglePhaseCommit
</client>

<server>
Server running...
client connected
PSPEDBServer.DoWork
PSPEDBServer.CommitWork
committing internal tx:1
</server>

If we enlist a durable RM after the PSPE enlistment (uncomment section 2), the transaction is promoted to MSDTC when the durable enlistment is done and the outputs are:

<client>
DatabaseProxy.OpenConnection
Internal tx id:1
InternalRM.Enlist
InternalRM.Initialize
DatabaseProxy.DoWork
DurableRM.OpenConnection and enlist durable
InternalRm.Promote
DurableRM - DoWork
InternalRM.SinglePhaseCommit
DurableRM.Prepare
DurableRM.Commit
</client>

<server>
Server running...
client connected
PSPEDBServer.DoWork
PSPEDBServer.Promote
PSPEDBServer.Enlist
InternalServerRM.Enlist
PSPEDBServer.CommitWork
InternalServerRM.Prepare
InternalServerRM.Commit
</server>

And finally, if we enlist a durable RM before the PSPE enlistment (uncomment section 1), the PSPE enlistment will fail and thus the RM will have to follow the regular enlistment procedures. The outputs in this case are:

<client>
DurableRM.OpenConnection and enlist durable
DurableRM - DoWork
DatabaseProxy.OpenConnection
Internal tx id:1
InternalRM.Enlist
PSPE failed, doing regular Enlist
DatabaseProxy.DoWork
DurableRM.Prepare
DurableRM.Commit
</client>

<server>
Server running...
client connected
PSPEDBServer.Enlist
InternalServerRM.Enlist
PSPEDBServer.DoWork
InternalServerRM.Prepare
InternalServerRM.Commit
</server>

NOTE: the goal of my sample was to show the use of PSPE. For simplicity, I ignored dealing with the recovery (I will follow up with another post where I will target Durable Enlistments and recovery).

What is the Lighweight Transaction Manager and why do you need to know about it

florinlazar — Thu, 12 May 2005 13:49:00 GMT

A "hidden" feature available in System.Transactions is the Lightweight Transaction Manager (LTM). It provides support for light weight transactions that live in the current appdomain. The LTM acts as a façade layer for all the transactions started in System.Transactions and based on a set of rules, it will "fallback" to an MSDTC distributed transaction when it is necessary. I call it "fallback" because there is some performance penalty that you need to pay when this happens: the transaction will span beyond the appdomain and there will be cross-process messages that will be sent, since MSDTC resides in a different process. (Note that from the TM functionality point of view, you can also call this a "promotion", since the MSDTC is a more general purpose and more complete TM). The fallback/promotion happens behind the scenes without any action required from the application developer.

As long as the LTM deals with volatile resources (that enlist using Transaction.EnlistVolatile) and with at most one durable resource that supports single phase notifications (this resource enlists with Transaction.EnlistDurable and implements ISinglePhaseNotification) then the transaction remains in the ownership of LTM. The LTM restricts itself only to those resources that live in the same appdomain and for which "logging" (writing the transaction outcome to the disk) is not required.

The fallback will take place and the LTM will transfer the ownership of the transaction to MSDTC if any of the following is true:
- at least one durable resource that doesn't support single phase notifications is enlisted in the transaction
- at least two durable resources that support single phase notifications are enlisted in the transaction
- a request to "marshal" the transaction to a different appdomain or different process is done.

In between the LTM transaction and MSDTC transaction there is one intermediary type of transaction that we sometimes call it "database transaction" that is made available through the Promotable Single Phase Enlistment. PSPE is another important mechanism in System.Transactions for performance optimization. It allows a remote durable resource, located in a different appdomain/process/machine, to participate in the LTM transaction without causing it to "fallback" to an MSDTC transaction.

Since the LTM is a "hidden" feature, the only reason for which you need to know about its existence is when you care a lot about performance. You will want to delay or avoid the fallback to MSDTC as much as possible to gain performance and for that you need to know how and when the fallback is triggered.

What do you want to know about transactions?

florinlazar — Sun, 08 May 2005 00:02:00 GMT

Please don't hesitate to send me your suggestions or questions or topics you would like to be discussed in this blog. You can post a comment to this post or use http://blogs.msdn.com/florinlazar/contact.aspx to let me know. I'll prioritize the list and cover as much as I can.

Blog on Transactional NTFS

florinlazar — Wed, 27 Apr 2005 13:40:00 GMT

I found out about this blog a few days ago. Malcolm Smith, a developer from the Transactional NTFS team is blogging at http://blogs.msdn.com/because_we_can/

Using Windows XP SP2, Windows Server 2003 SP1 and MSDTC on Clusters

florinlazar — Wed, 27 Apr 2005 13:27:00 GMT

In order to allow Windows XP SP2 or Windows Server 2003 SP1 to talk to a remote MSDTC located in a cluster, the security level for MSDTC needs to be set to "Incoming Caller Authentication Required". (see http://blogs.msdn.com/florinlazar/archive/2004/06/18/159127.aspx for more details on MSDTC security levels).

MSDN Product Feedback Center

florinlazar — Thu, 21 Apr 2005 22:04:00 GMT

You've found a bug in a Microsoft product, and you don't know where to report it? Use MSDN Product Feedback Center http://lab.msdn.microsoft.com/productfeedback/default.aspx to submit it. You can collaborate with Microsoft developers, MVPs, and beta testers to identify problems and workarounds, and then track feedback online. You also get notifications of changes and fixes.