The Windows Server Core support team has produced a useful blog post that explains how to understand and implement multiple instances of MSDTC, a feature new to Windows Server 2008.

How to Configure Multiple Instances of Distributed Transaction Coordinator (DTC) on a Windows Server Failover Cluster 2008

The example used to illustrate the process uses of MSMQ which makes the blog post doubly helpful in my eyes.

And there is an impressive reference list at the bottom for further reading.