Florin Lazar - Consistency Checkpoint : promotable transactions

Supporting Promotable Transactions and Phase 0

florinlazar — Sat, 10 Feb 2007 16:48:00 GMT

If you are providing support for promotable transactions for your resource manager using PSPE then you need to remember to "support" Phase 0.

In general, a resource manager doesn't need to do anything special about Phase 0; this phase will be handled solely by the transaction manager. But, because in the case of promotable transactions using the PSPE mechanism, the resource manager acts as a special "transaction manager", your resource manager needs to be aware of Phase 0. Why? Because in this case, the resource manager "owns" the distributed transaction and System.Transaction will call Commit on the PSPE interface asking the resource manager to call Commit on the distributed transaction. The resource manager needs to be aware that even after it will call Commit on the distributed transaction it owns, there is a possibility that requests to enlist and do work as part of the same transaction are still possible, based on the Phase 0 rules. And these requests are supposed to be accepted. Don't worry, there will be no data corruption. MSDTC guarantees that Phase 0 happens before Phase 1 starts delivering its "prepare" messages.

What exactly you need to do to ensure Phase 0 support? Let's create the list:

when you receive the "commit" request through the PSPE interface, call Commit on the distributed transaction but do not start the commit process on your data yet; wait for the Prepare message on the distributed transaction from MSDTC before doing any work (this assumes you are enlisted with that transaction - you should be!); in other words, when you receive the IPromotableSinglePhaseNotification::SinglePhaseCommit simply "pass" along the Commit to the distributed trasaction you own and don't do anything else
don't get confused by the name of IPromotableSinglePhaseNotification::SinglePhaseCommit; it is not the same as the classic single phase commit from MSDTC; there might be phase 0 enlistments out there that might enlist other durable resource managers in the transaction, and thus a 2-Phase-Commit might be necessary
even after calling Commit on the distributed transaction, you should continue to accept new connections and work on the same transaction; don't worry, phase 1 didn't start yet, these are just phase 0 enlistments "flushing" their data to durable stores
once you received the "prepare" message from MSDTC on the distributed transaction, only then you should start your normal 2PC operation
if any new request is received on the same transaction after "prepare" was received from MSDTC, you should deny it as you do in any 2PC scenario after phase 1 started

Sounds like a lot of steps, but in reality, this can be said in a short summary: "when you receive IPromotableSinglePhaseNotification::SinglePhaseCommit, pass along the Commit to the distributed transaction you own and behave like it didn't happen :), i.e. accept new connection requests in the same transaction until MSDTC delivers you the "prepare" message.

Of course, if Promote was never called, then there isn't any distributed transaction involved and thus you can proceed with commiting the work when IPromotableSinglePhaseNotification::SinglePhaseCommit is called.

Thanks for supporting promotable transactions and Phase 0.

When transaction promotion goes "bad"

florinlazar — Thu, 08 Feb 2007 14:41:00 GMT

We already know the benefits of promotable transactions. They are all about performance, being very fast and lightweight.

But then you get into situations where you know for sure that you are going to talk to a second resource manager, so you know you will get to a distributed transaction anyway. Or, the resource managers out there that support promotable transactions might have limitations when you put them into edge cases. But you really want to use that edge case and it doesn't work.

Well, one solution is to "turn off" promotion and force a distributed transaction from the beginning such that all the resource managers involved will follow the classic route of enlisting in the transaction. We don't actually provide a flag today in System.Transactions to "turn off" promotable transactions, but you can achieve it by forcing the transaction to be a distributed transaction before you open any resource manager connection.

The way you do it in code is by calling:

TransactionInterop.GetTransmitterPropagationToken(tx);

Where tx is the transaction you want to promote.

If you are using TrasactionScope similar to:

using(TransactionScope ts = new TransactionScope())
{
    connection.Open();
    // do the work on the first connection
    connection2.Open();
    // do the rest of the work
    ts.Complete();
}

Then the code with the workaround should be:

using(TransactionScope ts = new TransactionScope();)
{
    TransactionInterop.GetTransmitterPropagationToken(Transaction.Current);
    connection.Open();
    // do the work on the first connection
    connection2.Open();
    // do the rest of the work
    ts.Complete();
}

With time, resource managers will get better. But today, you might find this <trick> useful.

Writing a resource manager that supports promotable transactions (or Promotable Single Phase Enlistment aka PSPE) in System.Transactions

florinlazar — Tue, 17 May 2005 17:14:00 GMT

A key feature that targets performance in System.Transactions is the Promotable Single Phase Enlistment. It allows a durable resource manager (RM) to host and "own" a transaction that can later be promoted to a distributed transaction (or MSDTC transaction) if necessary. This specific resource manager usually has its own internal non distributed transactions and it needs to support changing those transactions to distributed transactions at runtime.

The overview of steps involved in PSPE is:
1. The RM proxy enlists for PSPE using Transaction.EnlistPromotableSinglePhase. If the enlistment succeeds, the RM usually creates its internal transaction and associates it with the System.Transactions transaction. The notification are later sent to the RM proxy using the IPromotableSinglePhaseNotification interface
2. If the System.Transactions transaction never requires a promotion (see previous post http://blogs.msdn.com/florinlazar/archive/2005/05/12/416805.aspx) then, when the transaction is committed, the RM proxy will receive a SinglePhaseCommit, at which point it can commit the internal transaction that was initially created
3. If the System.Transactions transaction needs to be promoted (to support multiple RMs for instance), then System.Transactions will ask the RM to promote the transaction to a distributed transaction. The RM will have to promote the internal transaction to an MSDTC transaction, and associate it with the work already done. Later, when System.Transactions will commit its transaction, it will send a SinglePhaseCommit notification to the RM proxy and the RM will have to commit the distributed transaction that it created during promotion.

Please note that at step 1, I had an "if the PSPE enlistment succeeds" condition. This means that PSPE is not always allowed by System.Transactions. The cases when this can happen are: 1) the transaction is already a distributed transaction or 2) another RM has already done a PSPE enlistment. If the PSPE enlistment fails, the RM will have to follow the regular rules for enlistment (marshal the transaction to the RM and enlist using DurableEnlist).

Let's look at code now. I will have a client and a server communicating using remoting:

<client.cs>

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;
using System.Transactions;

namespace PSPEClient
{
    class Client
    {
        static void Main(string[] args)
        {
            using (TransactionScope ts = new TransactionScope())
            {
                /*section1begin
                DurableRM durableRM = new DurableRM();
                durableRM.OpenConnection();
                durableRM.DoWork();
                /*section1end*/
                
                DatabaseProxy dbProxy = new DatabaseProxy();
                dbProxy.OpenConnection();
                dbProxy.DoWork();
                
                /*section2begin
                DurableRM durableRM = new DurableRM();
                durableRM.OpenConnection();
                durableRM.DoWork();
                /*section2end*/
                
                ts.Complete();
            }
            System.Console.ReadLine();
        }
        
        class DatabaseProxy
        {
            private PSPEServer.PSPEDatabaseServer db;
            private InternalRM internalRM;
            
            public void OpenConnection()
            {
                System.Console.WriteLine("DatabaseProxy.OpenConnection");
                // connecting to the remote database
                TcpChannel tcpChannel = new TcpChannel();
                ChannelServices.RegisterChannel(tcpChannel);
                this.db = (PSPEServer.PSPEDatabaseServer)Activator.GetObject(
                    typeof(PSPEServer.PSPEDatabaseServer), "tcp://localhost:8085/MyDatabase");
                if (null == db)
                {
                    System.Console.WriteLine("Cannot connect to the server");
                }
                else
                {
                    System.Console.WriteLine("Internal tx id:" + db.Connect());
                }
                
                // enlisting in the transaction
                if (null != this.internalRM)
                {
                    throw new System.Exception("we don't support multiple connections, this is just a sample");
                }
                this.internalRM = new InternalRM(db);
                this.internalRM.Enlist();
            }
            
            public void DoWork()
            {
                System.Console.WriteLine("DatabaseProxy.DoWork");
                db.DoWork();
            }
            
            class InternalRM : IPromotableSinglePhaseNotification
            {
                #region IPromotableSinglePhaseNotification Members
                
                // This member will be called during the call to EnlistPromotableSinglePhase
                // The RM will usually allocate its internal transaction state here
                public void Initialize()
                {
                    System.Console.WriteLine("InternalRM.Initialize");
                }
                
                // This method will be called if the RM should Rollback the
                // transaction.  Note that this method will be called even if
                // the transaction has been promoted to a distributed transaction.
                public void Rollback(SinglePhaseEnlistment singlePhaseEnlistment)
                {
                    System.Console.WriteLine("InternalRM.Rollback");
                    db.RollbackWork();
                    singlePhaseEnlistment.Aborted();
                }
                
                // This method will be called when the RM should Commit the
                // transaction.  Note that this method will be called even if
                // the transaction has actually been promoted to a distributed
                // transaction.
                public void SinglePhaseCommit(SinglePhaseEnlistment singlePhaseEnlistment)
                {
                    System.Console.WriteLine("InternalRM.SinglePhaseCommit");
                    db.CommitWork();
                    singlePhaseEnlistment.Committed();
                }
                
                #endregion
            
                #region ITransactionPromoter Members
                
                // This method will be called if System.Transactions
                // determines that the transaction actually needs the support of
                // a fully distributed transaction manager.  The return value of
                // this method is a promoted representation of the transaction
                // usually in the form of transmitter/receiver propagation token
                public byte[] Promote()
                {
                    System.Console.WriteLine("InternalRm.Promote");
                    return db.Promote();
                }
                
                #endregion
            
                private PSPEServer.PSPEDatabaseServer db;
                
                public InternalRM(PSPEServer.PSPEDatabaseServer db)
                {
                    this.db = db;
                }
                
                public void Enlist()
                {
                    System.Console.WriteLine("InternalRM.Enlist");
                    if (null != Transaction.Current)
                    {
                        if (!Transaction.Current.EnlistPromotableSinglePhase(this))
                        {
                            System.Console.WriteLine("PSPE failed, doing regular Enlist");
                            // PSPE failed; we need to use the regular enlistment
                            db.Enlist(TransactionInterop.GetTransmitterPropagationToken(Transaction.Current));
                        }
                    }
                }
            }
        }
     
        class DurableRM : IEnlistmentNotification
        {
            
            #region IEnlistmentNotification Members
            
            public void Commit(Enlistment enlistment)
            {
                System.Console.WriteLine("DurableRM.Commit");
                enlistment.Done();
            }
            
            public void InDoubt(Enlistment enlistment)
            {
                System.Console.WriteLine("DurableRM.InDoubt");
                throw new Exception("The method or operation is not implemented.");
            }
            
            public void Prepare(PreparingEnlistment preparingEnlistment)
            {
                System.Console.WriteLine("DurableRM.Prepare");
                // first a durable RM will log preparingEnlistment.RecoveryInformation(), but this is just a sample
                preparingEnlistment.Prepared();
            }
            
            public void Rollback(Enlistment enlistment)
            {
                System.Console.WriteLine("DurableRM.Rollback");
                enlistment.Done();
            }
            
            #endregion
            
            public void OpenConnection()
            {
                System.Console.WriteLine("DurableRM.OpenConnection and enlist durable");
                if (null != Transaction.Current)
                {
                    Transaction.Current.EnlistDurable(Guid.NewGuid(), this, EnlistmentOptions.None);
                }
            }
            public void DoWork()
            {
                System.Console.WriteLine("DurableRM - DoWork");
            }
        }
    }
}

</client.cs>
<server.cs>

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;
using System.Transactions;
using System.Diagnostics;
 
namespace PSPEServer
{
    class Server
    {
        static void Main(string[] args)
        {
            TcpChannel tcpChannel = new TcpChannel(8085);
            ChannelServices.RegisterChannel(tcpChannel);
            RemotingConfiguration.RegisterWellKnownServiceType(Type.GetType("PSPEServer.PSPEDatabaseServer"),
                "MyDatabase", WellKnownObjectMode.Singleton);
            System.Console.WriteLine("Server running...");
            System.Console.ReadLine();
        }
    }
 
    public class PSPEDatabaseServer : MarshalByRefObject
    {
        private int internalTxID = 0;
        private CommittableTransaction tx;
        private InternalServerRM internalServerRM;

        public int Connect()
        {
            System.Console.WriteLine("client connected");
            return ++internalTxID;
        }

        public void DoWork()
        {
            System.Console.WriteLine("PSPEDBServer.DoWork");
        }

        public byte[] Promote()
        {
            System.Console.WriteLine("PSPEDBServer.Promote");
            this.tx = new CommittableTransaction();
            Debug.Assert(this.internalServerRM == null);
            // the following statement will cause the transaction to be promoted to MSDTC
            byte[] txToken = TransactionInterop.GetTransmitterPropagationToken(this.tx);
            Enlist(txToken);
            return txToken;
        }

        public void CommitWork()
        {
            System.Console.WriteLine("PSPEDBServer.CommitWork");
            if (tx != null)
            {
                // we have a distributed transaction, and so we have to commit it
                tx.Commit();
            }
            else
            {
                // we only have an internal tx
                System.Console.WriteLine("committing internal tx:" + internalTxID);
            }
        }

        public void RollbackWork()
        {
            System.Console.WriteLine("PSPEDBServer.RollbackWork");
            if (tx != null)
            {
                // we have a distributed transaction, and so we have to rollback it
                tx.Rollback();
            }
            else
            {
                // we only have an internal tx
                System.Console.WriteLine("aborting internal tx:" + internalTxID);
            }        
        }

        public void Enlist(byte[] txToken)
        {
            System.Console.WriteLine("PSPEDBServer.Enlist");
            this.internalServerRM = new InternalServerRM();
            this.internalServerRM.Enlist(txToken);
        }

        private class InternalServerRM : ISinglePhaseNotification
        {

            #region ISinglePhaseNotification Members

            public void SinglePhaseCommit(SinglePhaseEnlistment singlePhaseEnlistment)
            {
                System.Console.WriteLine("InternalServerRM.SPC");
                singlePhaseEnlistment.Committed();
            }

            #endregion

            #region IEnlistmentNotification Members

            public void Commit(Enlistment enlistment)
            {
                System.Console.WriteLine("InternalServerRM.Commit");
                enlistment.Done();
            }

            public void InDoubt(Enlistment enlistment)
            {
                System.Console.WriteLine("InternalServerRM.InDoubt");
                throw new Exception("The method or operation is not implemented.");
            }

            public void Prepare(PreparingEnlistment preparingEnlistment)
            {
                System.Console.WriteLine("InternalServerRM.Prepare");
                // first a durable RM will log preparingEnlistment.RecoveryInformation(), but this is just a sample
                preparingEnlistment.Prepared();
            }

            public void Rollback(Enlistment enlistment)
            {
                System.Console.WriteLine("InternalServerRM.Rollback");
                enlistment.Done();
            }

            #endregion

            private Guid rmGuid = new Guid("{B14FF9BB-8419-4dbc-A78C-3C1453D60AC4}");

            public void Enlist(byte[] txToken)
            {
                System.Console.WriteLine("InternalServerRM.Enlist");
                TransactionInterop.GetTransactionFromTransmitterPropagationToken(txToken).EnlistDurable(
                    this.rmGuid, this, EnlistmentOptions.None);
            }
        }
    }
}

</server.cs>

If we run the code, with only the PSPE enlistment, System.Transactions is managing the transaction and MSDTC is never involved; we get the following outputs:

<client>
DatabaseProxy.OpenConnection
Internal tx id:1
InternalRM.Enlist
InternalRM.Initialize
DatabaseProxy.DoWork
InternalRM.SinglePhaseCommit
</client>

<server>
Server running...
client connected
PSPEDBServer.DoWork
PSPEDBServer.CommitWork
committing internal tx:1
</server>

If we enlist a durable RM after the PSPE enlistment (uncomment section 2), the transaction is promoted to MSDTC when the durable enlistment is done and the outputs are:

<client>
DatabaseProxy.OpenConnection
Internal tx id:1
InternalRM.Enlist
InternalRM.Initialize
DatabaseProxy.DoWork
DurableRM.OpenConnection and enlist durable
InternalRm.Promote
DurableRM - DoWork
InternalRM.SinglePhaseCommit
DurableRM.Prepare
DurableRM.Commit
</client>

<server>
Server running...
client connected
PSPEDBServer.DoWork
PSPEDBServer.Promote
PSPEDBServer.Enlist
InternalServerRM.Enlist
PSPEDBServer.CommitWork
InternalServerRM.Prepare
InternalServerRM.Commit
</server>

And finally, if we enlist a durable RM before the PSPE enlistment (uncomment section 1), the PSPE enlistment will fail and thus the RM will have to follow the regular enlistment procedures. The outputs in this case are:

<client>
DurableRM.OpenConnection and enlist durable
DurableRM - DoWork
DatabaseProxy.OpenConnection
Internal tx id:1
InternalRM.Enlist
PSPE failed, doing regular Enlist
DatabaseProxy.DoWork
DurableRM.Prepare
DurableRM.Commit
</client>

<server>
Server running...
client connected
PSPEDBServer.Enlist
InternalServerRM.Enlist
PSPEDBServer.DoWork
InternalServerRM.Prepare
InternalServerRM.Commit
</server>

NOTE: the goal of my sample was to show the use of PSPE. For simplicity, I ignored dealing with the recovery (I will follow up with another post where I will target Durable Enlistments and recovery).