Florin Lazar - Consistency Checkpoint : Enterprise Services

Our Own Forum is Now Live! Please Join Us at the "Transactions Programming Forum"

florinlazar — Sat, 11 Mar 2006 02:34:00 GMT

I'm really happy to announce that our own forum, dedicated to transactions in Windows and .Net is now live at http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=388&SiteID=1

The RSS feed for the forum is http://forums.microsoft.com/MSDN/rss.aspx?ForumID=388&Mode=0&SiteID=1

I hope that by having a forum that is dedicated to Transactions technologies we can build a better channel for our customers to get quick and accurate answers to their questions. We encourage everyone in the community to participate in this forum.

I look forward to us working together to make this forum a valuable resource for Transactions discussions.

Why and When to use transactions?

florinlazar — Tue, 04 Oct 2005 11:07:00 GMT

Many people ask - do I need transactions? Why do I need them? When to use them?

The answer is simple: use them all the time, unless you have a very good reason not to (for instance, don't use atomic transactions for "long running activities" between businesses). The default should always be yes. You are in doubt? - use transactions.

Why are transactions beneficial? They help you deal with crashes, failures, data consistency, error handling, they help you write simpler code etc. And the list of benefits will continue to grow with time.

Don't think in terms of transactions only when you talk to two or more databases. Starting with technologies like LTM, transactions are made available to be used for any multi-action operation. "X = A+B; Y = C*D;" can be transactional or atomic, as some prefer to name it, and can be seen as a single unit of work.

That is also why support for transactions is available almost everywhere. In database systems, in COM+, in ASP.NET, in .Net Framework with System.Transactions and EntepriseServices, in Indigo, in Biztalk etc

You might be saying now: "but my friend X doesn't use transactions - why it is important that I use them?". It is mostly a problem on the education side. The industry didn't talk a lot about transactions and their usage until recently. The original scope of transactions was limited to database scenarios. But technologies around transactions evolved and continue to evolve, in terms on capabilities, ease of use, flexibility, performance etc.

We used to talk about composable objects (as in OOP). We currently talk a lot about composable web services (as in SOA). The technologies are already here to start talking about composable atomic units of work.

How about the future? One avenue is STM or "software transactional memory". Maybe another one is transactional support at hardware level.

Transactions Sessions at PDC 2005

florinlazar — Wed, 14 Sep 2005 20:31:00 GMT

If you are at PDC this year, don't forget to attend the sessions related to transactions. An easy way to find these sessions is to go to http://commnet1.microsoftpdc.com/content/sessions.aspx and search by Keyword="transaction".

I especially recommend the session called "FUN320 - Windows Vista & "Longhorn" Server: Improving Reliability Using System.Transactions and the Transactional NTFS and Registry". Jim Johnson, our architect for transactions is presenting this session together with Dana Groff, the program manager in Core File Services focusing on transaction technologies.

Here is the full list:

FUN320 Windows Vista & "Longhorn" Server: Improving Reliability Using System.Transactions and the Transactional NTFS and Registry

Day/Time: Thursday, September 15 5:15 PM- 6:30 PM Room: 406 AB

Speaker(s): Dana Groff, Jim Johnson

Session Type(s): Breakout

Session Level(s): 300

Track(s): Fundamentals

You can build significantly more reliable applications by using the expanded and enhanced classes in the System.Transactions namespace. This session covers how to use new transaction-oriented programming techniques for application stability and robustness. Learn about the new systems in Windows Vista and "Longhorn" Server for supporting Transacted Files (TxF) and Transacted Registry (TxR) operations in both native and managed code.

FUN323 Microsoft Research: Future Possibilities in Concurrency

Day/Time: Friday, September 16 8:30 AM- 9:45 AM Room: 406 AB

Speaker(s): Tim Harris

Session Type(s): Breakout

Session Level(s): 300

Track(s): Fundamentals

Learn about the investment Microsoft Research is making in future languages and tools to help write software that will gain the maximum benefit from tomorrow's multi-processor/multi-core hardware. In this session, we start by introducing some of the tools and techniques we've developed for tracking down and preventing bugs in multi-threaded software. We then turn to the language features that we're prototyping in research: our work on transactional memory is developing a programming model where data can be shared directly between threads without needing to worry about locking or low-level deadlocks; our work on synchronization is leading to a unified abstraction for co-ordination between threads and between processes. We'll demo some of our prototype systems, showing how these techniques can lead to software which is not only easier to develop, but which can scale from uniprocessors up to highly parallel systems.

Distributed Atomic Transactions - Wishful Thinking?

Day/Time: Tuesday, September 13 9:00 PM- 10:00 PM Room: 511 C

Session Type(s): Birds of a Feather

Session Level(s): 300

Web Services, via SOAP, is the preferred choice of doing wide-reaching distributed communications today. However, one of the few factors that has hindered its adoption in the enterprise today is its lack of ability to handle transactions between remote un-trusted (independent) boundaries. There are two schools of thought today with regards to transactions --- The Atomic Way or The Compensating Way which pretty much takes out the atomicity of ACID. Indigo supports WS-AT via Microsoft DTC. Will this be widely adopted in the enterprise collaboration space? Or are we living in Fool's Paradise?

COM307 Windows Communications Foundation ("Indigo"): Writing Reliable and Transacted Distributed Applications

Day/Time: Wednesday, September 14 3:15 PM- 4:30 PM Room: 403 AB

Speaker(s): Shy Cohen

Session Type(s): Breakout

Session Level(s): 300

Track(s): Communications

What do you do when your network connection breaks half-way through a request? How do you synchronize state changes across different Web services? How do you overcome a system crash without losing important messages? Windows Communications Foundation (formerly codename "Indigo") provides simple and powerful reliability mechanisms that allow you to easily address these types of network and application issues. Take an in-depth look at reliable sessions, queues, and distributed transactions, and how these technologies are used to achieve reliable, transacted communication.

COM202 Windows Communications Foundation ("Indigo"): A Lap around the Windows Communications Foundation

Day/Time: Tuesday, September 13 1:00 PM- 2:15 PM Room: Halls C & D (Petree Hall)

Speaker(s): Omri Gazitt

Session Type(s): Breakout

Session Level(s): 200

Track(s): Communications

Windows Communications Foundation (formerly codename "Indigo") is Microsoft's new platform technology for building distributed applications, bringing the best of ASP.NET Web services, .NET Remoting, Enterprise Services, WSE, and System.Messaging into one unified framework. This session introduces the core concepts in the Windows Communications Foundation programming model-addresses, bindings, contracts, and behaviors-and presents the Windows Communications Foundation feature set through the lens of these concepts. This session covers security, reliable messaging, transactions, queues, hosting, transports, interop/integration, and more. If you have never seen an introduction to Windows Communication Framework, this session is for you.

The slides from these sessions are currently available at http://commnet.microsoftpdc.com/content/downloads.aspx

Indigo and Avalon Beta1 RC Available for Download

florinlazar — Tue, 24 May 2005 01:03:00 GMT

"Indigo" and "Avalon" Beta1 RC are available for download at http://www.microsoft.com/downloads/details.aspx?FamilyId=B789BC8D-4F25-4823-B6AA-C5EDF432D0C1&displaylang=en. More info can be found at http://msdn.microsoft.com/Longhorn and http://msdn.microsoft.com/webservices.

What do you want to know about transactions?

florinlazar — Sun, 08 May 2005 00:02:00 GMT

Please don't hesitate to send me your suggestions or questions or topics you would like to be discussed in this blog. You can post a comment to this post or use http://blogs.msdn.com/florinlazar/contact.aspx to let me know. I'll prioritize the list and cover as much as I can.

MSDN Product Feedback Center

florinlazar — Thu, 21 Apr 2005 22:04:00 GMT

You've found a bug in a Microsoft product, and you don't know where to report it? Use MSDN Product Feedback Center http://lab.msdn.microsoft.com/productfeedback/default.aspx to submit it. You can collaborate with Microsoft developers, MVPs, and beta testers to identify problems and workarounds, and then track feedback online. You also get notifications of changes and fixes.

Transaction.Current and Ambient Transactions

florinlazar — Tue, 19 Apr 2005 12:07:00 GMT

Ambient transactions are defined as transactions that live in the current thread or object context that anybody interested can query for their existence and use them to do work as part of them. Their existence means that the user wants to do a series of actions in a transacted "all or nothing" way. They are not an enforcement, but merely a suggestion for the APIs that are going to be called. It is the API's responsibility to use the transaction when one exists. We call these APIs, transaction-aware APIs. The main customers of these ambient transactions were and continue to be the database proxies (ODBC, OLEDB, System.Data). But with .Net Framework 2.0 and System.Transactions we expect this to change.

At first it was MTS (later called COM+) with its "object context" that could have a transaction present in it. In an MTS/COM+ component, you would have to call CoGetObjectContext with IID_IObjectContextInfo to obtain an IObjectContextInfo pointer and then to call IObjectContextInfo::GetTransaction to get the "current" transaction.

Later, System.EnterpriseServices continued this in .Net with its ambient transaction that could be accessed using System.EnterpriseServices.ContextUtil.Transaction. This transaction was available inside a ServicedComponent or a COM+ context created with SWC (Services Without Components - see also http://blogs.msdn.com/florinlazar/archive/2004/07/24/194199.aspx).

System.Transactions and TransactionScope has its own ambient transaction in Transaction.Current which is a static property that lives in the current thread local storage. System.Data was updated to be aware of this new ambient transaction. In fact, we did a lot of work to ensure seamless interoperability with System.EnterpriseServices, and now System.Data is only looking at Transaction.Current. Transaction.Current can detect that the transaction was created inside a COM+ context and provide it to anyone asking for the ambient transaction. If you are writing transaction-aware APIs for .Net Framework 2.0, the recommended ambient transaction to query and use is System.Transactions.Transaction.Current.

XA Transactions in Windows XP SP2

florinlazar — Thu, 28 Oct 2004 10:11:00 GMT

Service Pack 2 adds to Windows XP the same level of security as in Windows Server 2003 for XA transactions (transactions that are used with XA-compliant resource managers like Oracle, IBM DB/2 etc). See my previous post at http://blogs.msdn.com/florinlazar/archive/2003/12/04/41370.aspx for the more details on how to enable XA transactions after installing XP SP2.

If you are using the default XA dll that COM+ provides, which is named mtxoci.dll, then in Windows XP SP2 you need to follow http://support.microsoft.com/default.aspx?scid=kb;EN-US;817066 and create a registry key entry for the "mtxoci.dll" as you would do for any other XA DLL. It would be something similar to:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\XADLL]

Key name: mtxoci.dll

Data: C:\Windows\system32\mtxoci.dll

Note: Replace the part in bold with your actual path to the %windir% folder.

Using distributed transactions in .Net 1.x without deriving from ServicedComponent

florinlazar — Sat, 24 Jul 2004 11:49:00 GMT

The most used feature of System.EnterpriseServices or COM+ is the distributed transaction support. And the automatic transaction programming model in ES using attributes ([Transaction] and [AutoComplete]) is great and nice but (it is always a but!)... you need to inherit from ServicedComponent and the Transaction attribute is only available at class level, and you need to register your component in the COM+ repository and the list can continue.

If doing this seems overkill to you, because all you need is a distributed transaction to protect your code/actions and you don't care of any of the others ES features (which are great ones nevertheless) then there is a solution for you: System.EnterpriseServices.ServiceDomain. Here is some sample code:

using System;
using System.EnterpriseServices;

namespace SDSample
{
   class Class1
   {
      [MTAThread]
      static void Main(string[] args)
      {
         ServiceConfig config = new ServiceConfig();
         config.Transaction = TransactionOption.Required;
         ServiceDomain.Enter(config);
         try
         {
            MyTxCode();
         }
         catch(Exception e)
         {
            // we got an exception
            Console.WriteLine(e.Message);
            // so, we should abort the transaction
            ContextUtil.SetAbort();
         }
         finally
         {
            ServiceDomain.Leave();
         }
      }

      // The code that I want to be transactional
      static void MyTxCode()
      {
         Console.WriteLine(ContextUtil.TransactionId);

         // Open connection to database 1
         // Execute update in database 1

         // Open connection to database 2
         // Execute update in database 2
      }
   }
}

Of course, you can go further and create a helper class, let’s call it ESTransactionScope (similar to System.Transactions.TransactionScope that will arrive in Whidbey) that will be very easy to use:

using System;
using System.EnterpriseServices;

namespace SDSample2
{
   class Class1
   {
      [MTAThread]
      static void Main(string[] args)
      {
         using( ESTransactionScope ts = new ESTransactionScope())
         {
           MyTxCode();

           // Everything went well, no exception thrown
           // so let’s vote for Commit
           ts.Complete();
         }
      }

      static void MyTxCode()
      {
         Console.WriteLine(ContextUtil.TransactionId);

         // Open connection to database 1
         // Execute update in database 1

         // Open connection to database 2
         // Execute update in database 2
      }
   }

   // Used to create transactional code blocks
   class ESTransactionScope : IDisposable
   {
      // Dispose must be called to exit the transactional block
      public void Dispose()
      {
         if(this.EnterSucceeded)
         {
            if(!this.Consistent)
            {
               ContextUtil.SetAbort();
            }
            ServiceDomain.Leave();
         }
      }

      // by calling this method, you mark the scope as being consistent
      // and ready to for commit
      // if the method is never called, upon dispose, the scope will abort the transaction
      public void Complete()
      {
         this.Consistent = true;
      }

      public ESTransactionScope()
      {
         EnterTxContext(TransactionOption.Required);
      }

      public ESTransactionScope(TransactionOption txOption)
      {
         EnterTxContext(txOption);
      }

      private void EnterTxContext(TransactionOption txOption)
      {
         ServiceConfig config = new ServiceConfig();
         config.Transaction = txOption;
         ServiceDomain.Enter(config);
         // Since Enter can throw, the next statement will track the success
         // In the case of success will we need to call Leave in Dispose
         this.EnterSucceeded = true;
      }

      // By default, the scope is inconsistent;
      // To Commit the transaction on exit, the Consistent flag
      // must be set to true before Dispose is called
      private bool Consistent = false;

      // Enter can throw, so we need to know if we need to call Leave in Dispose
      private bool EnterSucceeded = false;
   }
}

System.EnterpriseServices.ServiceDomain is available only on XP SP2 (or higher) and Windows Server 2003 and only in .Net 1.1.

If you need your app to work with .Net 1.0 or on Windows 2000 or XP pre-SP2, you can use the trick that Don Box posted at http://www.gotdotnet.com/team/dbox/default.aspx?key=2004-07-12T08:40:44Z It uses exactly one transactional ServicedComponent based class and a DoCallback method to which you pass the delegate to your MyTxCode function that needs to execute in a transaction.

.NET Enterprise Services Performance

florinlazar — Tue, 20 Jul 2004 01:04:00 GMT

Compare the performance of native COM+ and .Net Enterprise Services components and read some guidelines on how to get your EnterpriseServices to run faster at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncomser/html/entsvcperf.asp?frame=true

XP SP2 and Transactions

florinlazar — Fri, 18 Jun 2004 13:00:00 GMT

The preview (RC2) of Windows XP Service Pack 2 is available for download at http://www.microsoft.com/technet/prodtechnol/winxppro/sp2preview.mspx

The general info about it can be found at http://msdn.microsoft.com/security/productinfo/XPSP2/default.aspx and http://www.microsoft.com/technet/prodtechnol/winxppro/maintain/sp2chngs.mspx More info about the changes in MSDTC is also available at http://www.microsoft.com/technet/prodtechnol/winxppro/maintain/sp2otech.mspx#EBAA and http://support.microsoft.com/?id=899191

In this post I will cover what changes are introduced in this SP for distributed transactions, what is the impact on your applications and how can you re-enable your scenarios.

When you install XP SP2, all network MSDTC transactions will be disabled, even if network transactions had been previously enabled. This means that if you are using COM+ or Enterprise Services (or simple OleTx clients and resource managers) to flow transactions from or into the box, you will need to follow the steps defined below to re-enable your scenarios.

The first step is to enable network transactions in the Security Configuration dialog for MSDTC. To do this, open Control Panel\Administrative Tools\Component Services. Then select Component Services\Computer\My Computer, right-click and choose Properties. On the MSDTC tab, press "Security Configuration..." button and then select what you need. You will notice that the old checkbox “Network Transactions” has been replaced with a new group of settings named “Transaction Manager Communication”. This group contains two new checkboxes and 3 radio buttons, defined below.

“Allow Inbound” when enabled will allow a remote computer to flow transactions to the local computer; this is typically needed on the box hosting the MSDTC for a resource manager like Microsoft SQL Server. When enabled, “Allow Outbound” will allow the local computer to flow transactions to a remote computer; this is typically needed on the “client” box, where the transaction is initiated.

When “Mutual Authentication Required” is selected, the local MSDTC (proxy or service) will communicate with a remote MSDTC service using only encrypted messages and mutual authentication (Windows Domain authentication). If a secure communication cannot be established with the remote system, the communication will be denied. “Incoming Caller Authentication Required” means that if mutual authentication cannot be established, but the incoming caller can be authenticated, then the communication will be allowed. Currently only Windows 2003 Server and XP SP2 support the first two options. “No Authentication Required” means that the MSDTC communication on the network can fallback to a non authenticated and non encrypted communication if the attempts to start a secure communication will fail. The “no authentication required” option is for compat communications with previous OSes (W2K, XP RTM and XP SP1); this setting needs also to be used when the computers involved are located in two untrusted Windows domains or in a Windows workgroup. If your XP SP2 box is talking to a Windows 2003 system that has disabled it’s RPC security for MSDTC (using TurnOffRpcSecurity registry key - see http://blogs.msdn.com/florinlazar/archive/2004/03/02/82916.aspx for more info), then you will need to use this third option on the XP SP2 box to enable network transactions between the two systems.

The second step in enabling network transactions is related to the firewall. By default, after installing XP SP2, the Windows Firewall will be on. To enable network transactions through the firewall, you will need to add the msdtc.exe to the exception list of the firewall on all the machines involved in the transactions. You can do this using the UI in Control Panel\Windows Firewall or you can use this command: “netsh firewall set allowedprogram %windir%\system32\msdtc.exe MSDTC enable”.

Another configuration setting that you need to be aware (although I consider it to be an uncommon scenario) is RestrictRemoteClients registry key. If the value of this key is set to 2 (RPC_RESTRICT_REMOTE_CLIENT_HIGH) then MSDTC network transactions will not be able to work properly. MSDTC supports only RPC_RESTRICT_REMOTE_CLIENT_NONE (0) and RPC_RESTRICT_REMOTE_CLIENT_DEFAULT (1) values. See http://www.microsoft.com/technet/prodtechnol/winxppro/maintain/sp2netwk.mspx#XSLTsection128121120120 for more info on RestrictRemoteClients.

I recommend and I encourage you to give a try to this release candidate for XP SP2 on your test systems and send your feedback to the XP SP2 preview newsgroups: http://communities.microsoft.com/newsgroups/default.asp?icp=xpsp2&slcid=us . Thanks!

[Updated Sep 20, 2004]

What is MSDTC and why do I need to care about it?

florinlazar — Fri, 05 Mar 2004 04:52:00 GMT

I've been talking for a while about MSDTC and transactions without explaining them. I know many of you reading this blog already know the terms, but some of the developers just entering the enterprise space don't know if they should care or not about these subjects.

MSDTC is an acronym for Microsoft Distributed Transaction Coordinator. As the name says, MSDTC is a Windows service providing transaction infrastructure for distributed systems. In this case, a transaction means a general way of structuring the interactions between autonomous agents in a distributed system. Each transaction is a state transformation with four key properties - the ACID properties: Atomic (all or nothing), Consistent (legal), Isolated (independent of concurrent transactions) and Durable (once it happens, it cannot be abrogated). There are different techniques that implement the ACID properties but the most known one is two-phase commit.

In other words, transaction processing is a software technology that makes distributed computing reliable. You can see a transaction as a unit of work in which a series of operations occur. The transaction (with its ACID properties) is providing protection in the case when one or more of these operations fail at any point in time. By using transactions, you can vastly simplify the error recovery for your system.

The main actors in a transaction are: the transaction manager (MSDTC), the initiator (the application which started the transaction) and the resource managers (the entities that manage data and work). The flow of actions in a simplified form is:

1. The client application (the initiator) begins a transaction by requesting one from the transaction manager;

2. The client app asks the resource managers to do work as part of the same transaction; during this step, the resource managers register with the transaction manager for that transaction ("they enlist");

3. The client app commits the transaction;

4. The transaction manager coordinates with the resource managers to ensure that all succeed to do the requested work or none of the work if done, thus maintaining the ACID properties.

The main transactions standard currently supported by MSDTC is the OLE Transactions (or OleTx). MSDTC is also supporting other standards like XA (or X/Open Distributed Transaction Processing Standard) and TIP (Transaction Internet Protocol). In the future MSDTC will support WS-Coordination, WS-AtomicTransaction, and WS-BusinessActivity.

If you are developing COM+ or System.EnterpriseServices components and you are using the Transaction service then you are indirectly using MSDTC. The COM+ infrastructure is hiding all the details from you so you can focus on your business needs instead of implementation details. If you need to bypass COM+ and talk directly to MSDTC you can do it using the MSDTC proxy (msdtcprx.dll). For now, to directly access MSDTC from .Net apps you need to use COM Interop. For future apps, please refer to my previous post about .Net and Transactions.

Where to find answers to your questions

florinlazar — Tue, 10 Feb 2004 13:20:00 GMT

These are the newsgroups you should use when looking for answers related to COM+, Enterprise Services or MSDTC: microsoft.public.platformsdk.complus_mts and microsoft.public.dotnet.framework.component_services. For Indigo answers, use microsoft.public.windows.developer.winfx.indigo.

[Update March 30, 2005] We now have a dedicated forum for transactions technologies at http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=388&SiteID=1

Isolation Levels

florinlazar — Thu, 11 Dec 2003 03:22:00 GMT

MSDTC, COM+ or EnterpriseServices are allowing you to specify an isolation level for transactions so that you can increase concurrency to improve performance and scalability.

But, the value you set for the isolation level is used by MSDTC only to pass it to the resource manager(s), more like a hint rather than a restriction. It is the responsibility of the resource manager to use this isolation level accordingly.

The MSDTC transaction will always carry the same isolation level which is the one that was initially specified when the transaction was created. It cannot be changed later. Some resource managers allow you to change the isolation level. For instance, the isolation level that SQL Server is using on a connection can be changed using "SET TRANSACTION ISOLATION LEVEL". Changing the isolation level at statement level inside SQL Server or any other resource manager is invisible to MSDTC. This is why mixing MSDTC transaction isolation levels with resource manager statement level isolation levels is not recommended.

Enable Network DTC Access in Windows Server 2003

florinlazar — Fri, 05 Dec 2003 04:47:00 GMT

When you install Windows Server 2003, most of the services will be off by default, including MSDTC network access. If you are using COM+/Enterprise Services and [Transactions], you are most likely to hit errors when you want to go distributed: flow transactions to another computer, talk to a resource manager as SQL server on another machine etc. To enable your distributed scenarios you need to enable MSDTC network transactions.

One way to do it is to follow the steps in this KB article: http://support.microsoft.com/default.aspx?scid=kb;en-us;817064. But if you like being in control, you can use Control Panel\Administrative Tools\Component Services. Then select Component Services\Computer\My Computer, right-click and choose Properties. On the MSDTC tab, press "Security Configuration..." button and then carefully select only what you need. For instance to flow transactions to another machine you only need to enable "Network DTC Access\Network Transactions". You can do the same thing programmatically using IDtcNetworkAccessConfig implemented by the transaction object of the DTC proxy.

Windows Server 2003 is opening a new trend that we all need to get used to. It is not anymore "if I don't need it, I should probably disable it"; now it is "if I need it, I have to enable it".

[Originally posted Oct 21, 2003]