Clemens Vasters, Bldg 42 : BizTalk Services

Announcing the Microsoft Code-Name "BizTalk Services" R12 Release

clemensv — Tue, 15 Jul 2008 18:30:37 GMT

We’re thrilled to announce that the BizTalk Services “R12” Community Technology Preview (CTP) is now available for general use.

“BizTalk Services” is the code-name for a platform-in-the-cloud offering from Microsoft. Currently in active development, BizTalk Services provides Messaging, Workflow, and Identity functionality to enable disparate applications to connect quickly and easily.
Combined together in an integrated offering, these capabilities deliver a Service Bus architectural pattern that is immediately usable by applications that need to connect across the Internet.

Many enterprises employ the ‘Enterprise Service Bus’ pattern to interconnect disparate systems within an organizational domain. Built on Microsoft platform technology, an ESB might include building blocks such as Windows Server, Active Directory, BizTalk Server, as well as the Windows Communication Foundation and Windows Workflow Foundation technologies included in the .NET Framework. “BizTalk Services” extends the concept of an ESB to truly exploit the Internet, for instance by exposing individual service endpoints in a secure fashion or by selectively federating elements of distinct identity systems to facilitate cross-company collaboration.

For ISVs and Solution Providers creating specialized business solutions that enable collaboration and information exchange across increasingly mobile and distributed work-forces, “BizTalk Services” provides the cloud-based platform building blocks to create sophisticated (Internet-) Service Bus solutions with broad reach that could otherwise only be realized by operating dedicated Data Centers of significant complexity – which is often out of reach for both, ISVs and their customers.

Major Changes

With the release of BizTalk Services “R12”, developers must update all clients and SDK installations to the new release.

New in R12 - Workflow

The most exciting new capability we’ve added in the “R12” CTP is Workflow. These new cloud-based Workflow capabilities enable ‘service orchestration’ from the cloud. This specialized cloud-based, or hosted, Windows Workflow Foundation runtime can orchestrate services that connect to systems in your enterprise, or to systems running anywhere on the Internet via Web services messages. This new power and capability will enable an entirely new set of application scenarios, and we’re very excited to see what people will do with it.

In the SDK you will find samples showing how to create and control Workflow instances hosted on the BizTalk Services cloud, including a sample Workflow implementation that monitors the availability of a website and fires multicast events into the service bus indicating the state.

New in R12 - Identity

For R12, the BizTalk Services Identity Service has been expanded and enhanced to enable more flexibility for scenarios demanded by our customers. R12 introduces a new approach for creating, viewing, and managing access control rules. This approach relies on a few key principles outlined below:

• Every Identity Service account owns a Security Token Service (STS).
• An STS is composed of one or more scopes.
• A scope contains zero or more access control rules.
• An STS owner can grant another Identity Service account permission to edit the access control rules in a scope

A practical illustration to clarify:. The Messaging Service owns an STS whose root scope is http://connect.biztalk.net/services/. When you create a new account (newaccount) in the Identity Service, the messaging service creates a new scope http://connect.biztalk.net/services/newaccount. The Messaging Service then grants (newaccount) the permission to create access control rules in that scope. Any communication endpoints hosted there can thus be secured by the owner of the scope. Rules from R11 accounts have been migrated to the “root” scope of the new account.

On the protocols front, we’ve added several new capabilities for ‘REST’ services. We now support integration with Windows Live ID and have added RFC2617 Basic and HTTPS/Client Certificate support for acquiring security tokens using simple HTTP GET requests.

New in R12 - Messaging

Connectivity Modes

The most fundamental new feature area in the Messaging service are the new ‘connectivity mode’ settings on the RelayBinding. Before this release, BizTalk Services clients and listeners always required outbound TCP ports 808 and 818 to be available for connecting to the BizTalk Services cloud for all connection modes except the clients of a listener running with ConnectionMode.RelayedHttp.

In this release we are introducing three different connectivity modes: Tcp, Http, and AutoDetect. The connectivity mode can be set on a static property of the RelayBinding. The Communication\ExploringFeatures\ConnectionModes\Multicast sample shows how. For clarity: ‘Connection Mode’ defines the type of end-to-end connection that is to be established through the Relay. ‘Connectivity Mode’ defines how a particular endpoint connects up to the Relay.

The ‘Tcp’ connectivity mode is the most efficient one and works as in previous releases. The ‘Http’ mode is new. It creates a volatile FIFO buffer for messages in the BizTalk Services cloud and polls for messages using HTTP ‘parked requests’. The Http model exhibits delivery latency characteristics similar to Tcp mode, albeit with slightly higher bandwidth consumption on idle connections. The ‘AutoDetect’ mode will check whether TCP connectivity is available and will choose ‘Tcp’ if that’s the case and ‘Http’ otherwise.

The new HTTP-based connectivity option is only effective for the RelayedOneway, RelayedMulticast and RelayedDuplex connection modes. RelayedDuplexSession, HybridDuplexSession, and RelayedHttp (listener only) still require TCP connectivity at this time.

Transport Credentials and Unauthenticated Access

Also, in the “R12” release, the model for specifying the client credentials for the Relay has now been closely aligned with the standard WCF client credentials model. Instead of picking and instantiating token providers, there is now a TransportClientEndpointBehavior that holds all credential information and credential types. The samples in the Communication\ExploringFeatures\RelayAuthentication of the SDK download clarify the use of this new behavior.

We have added a pair of ‘WebNoAuth’ samples which introduce a new capability that we had a lot of requests for: Unauthenticated client access. When registering a service listener you can now explicitly waive the authentication requirement for clients connecting to your service. This is very useful in Web scenarios where you want to enable any HTTP client to connect to your service and don’t want them to authenticate in any way. For the time being we suggest that you always use this new unauthenticated access mode for RelayedHttp services until we release the update for the ‘Web’ client authentication capability.

For R12, we have omitted the ‘Web’ (REST) samples for Relay authentication since that area is undergoing some substantial protocol changes. The update for this will be released soon. In the interim, existing applications that were built on a prior release of the BizTalk Services SDK to use the authentication technique shown in the R11 ‘Web’ sample must be modified to use unauthenticated access as shown in the new ‘WebNoAuth’ sample.

Give it a try

The new BizTalk Services “R12” CTP is online and available now for your use. The SDK is available at http://labs.biztalk.net. If you already have an account for BizTalk Services, your accounts and settings have been migrated to the new environment. If you don’t have an account yet, just sign up, download the SDK, and get started creating the new generation of connected applications.

BizTalk Services R11/R12 upgrade in progress

clemensv — Tue, 15 Jul 2008 14:32:18 GMT

The BizTalk Services CTP will be switched from the "R11" to the "R12" release starting in about 30 minutes and we expect to have a 2 hour time window (1400h-1600h PT/2300h-0100h UTC) where existing service accounts are being rolled over to the new release. We're expecting to be done with the migration by 1600h. Once the migration is done we'll give you an update on what's new in R12.

BizTalk Services: Update to the services (and SDK) scheduled

clemensv — Sat, 12 Jul 2008 14:02:35 GMT

Heads up: If things go as planned, the BizTalk Services cloud will be unavailable for a few hours during the day on Tuesday 7/15 (U.S. Pacific Time) since we're doing an update to the services and to the SDK. I will post an update with the exact time window some time on Monday. Once we're back up and have verified that everything is working as intended we'll let you know about it and tell you what's new.

Applications built on the R11 release (the release currently running in the data center) will have to be recompiled (and in some instances slightly changed) against the new R12 release's assemblies to run with R12. We've done some protocol adjustments in R12 that make this necessary - mind that we're still in "experimentation-only preview" mode here. Theory suggests that some compiled R11 applications will work against the R12 cloud, but it's not a combination we're explicitly testing as of yet. We obviously have that sort of backwards compatibility on the radar (it's SOA, should be easy, right?) but it'll likely take us a couple more revisions before we're happy enough with the baseline protocols.

[Update: The switch to R12 will happen between 1400-1600 PT/2300-0100 UTC. More later]

Federated Identity with BizTalk Services

clemensv — Mon, 31 Mar 2008 13:08:04 GMT

I highly recommend reading Vittorio's most excellent and illuminating blog entry for how to use the new features we've added to BizTalk Identity Services for allowing you to use 3rd Party Managed Cards.

BizTalk Services "R11" CTP Comes with a Surprise

clemensv — Mon, 31 Mar 2008 11:56:40 GMT

A flock of pigs has been doing aerobatics high up over Microsoft Campus in Redmond in the past three weeks. Neither City of Redmond nor Microsoft spokespeople returned calls requesting comments in time for this article. An Microsoft worker who requested anonymity and has seen the pigs flying overhead commented that "they are as good as the Blue Angels at Seafair, just funnier" and "they seem to circle over building 42 a lot, but I wouldn't know why".

In related news ...

We wrapped up the BizTalk Services "R11" CTP this last Thursday and put the latest SDK release up on http://labs.biztalk.net/. As you may or may not know, "BizTalk Services" is the codename for Microsoft's cloud-based Identity and Connectivity services - with a significant set of further services in the pipeline. The R11 release is a major milestone for the data center side of BizTalk Services, but we've also added several new client-facing features, especially on the Identity services. You can now authenticate using a certificate in addition to username and CardSpace authentication, we have enabled support for 3rd party managed CardSpace cards, and there is extended support for claims based authorization.

Now the surprising bit:

Only about an hour before we locked down the SDK on Thursday, we checked a sample into the samples tree that has a rather unusual set of prerequisites for something coming out of Microsoft:

Runtime: Java EE 5 on Sun Glassfish v2 + Sun WSIT/Metro (JAX-WS extensions), Tool: Netbeans 6.0 IDE.

The sample shows how to use the BizTalk Services Identity Security Token Service (STS) to secure the communication between a Java client and a Java service providing federated authentication and claims-based authorization.

The sample, which you can find in ./Samples/OtherPlatforms/StandaloneAccessControl/JavaEE5 once you installed the SDK, is a pure Java sample not requiring any of our bits on either the service or client side. The interaction with our services is purely happening on the wire.

If you are a "Javahead", it might seem odd that we're shipping this sample inside a Windows-only MSI installer and I will agree that that's odd. It's simply a function of timing and the point in time when we knew that we could get it done (some more on that below). For the next BizTalk Services SDK release I expect there to be an additional .jar file for the Java samples.

It's important to note that this isn't just a thing we did as a one-time thing and because we could. We have done a significant amount of work on the backend protocol implementations to start opening up a very broad set of scenarios on the BizTalk Services Connectivity services for platforms other than .NET. We already have a set of additional Java EE samples lined up for when we enable that functionality on the backend. However, since getting security and identity working is a prerequisite for making all other services work, that's where we started. There'll be more and there'll be more platform and language choice than Java down the road.

Just to be perfectly clear: Around here we strongly believe that .NET and the Windows Communication Foundation in particular is the most advanced platform to build services, irrespective of whether they are of the WS-* or REST variety. If you care about my personal opinion, I'll say that several months of research into the capabilities of other platforms has only reaffirmed that belief for me and I don't even need to put a Microsoft hat on to say that.

But we recognize and respect that there are a great variety of individual reasons why people might not be using .NET and WCF. The obvious one is "platform". If you run on Linux or Unix and/or if your deployment target is a Java Application Server, then your platform is very likely not .NET. It's something else. If that's your world, we still think that our services are something that's useful for your applications and we want to show you why. And it is absolutely not enough for us to say "here is the wire protocol documentation; go party!". Only Code is Truth.

I'm also writing "Only Code is Truth" also because we've found - perhaps not too surprisingly - that there is a significant difference between reading and implementing the WS-* specs and having things actually work. And here I get to the point where a round of public "Thank You" is due:

The Metro team over at Sun Microsystems has made a very significant contribution to making this all work. Before we started making changes to accommodate Java, there would have been very little hope for anyone to get this seemingly simple scenario to work. We had to make quite a few changes even though our service did follow the specs.

While we were adjusting our backend STS accordingly, the Sun Metro team worked on a set of issues that we identified on their end (with fantastic turnaround times) and worked those into their public nightly builds. The Sun team also 'promoted' a nightly build of Metro 1.2 to a semi-permanent download location (the first 1.2 build that got that treatment), because it is the build tested to successfully interop with our SDK release, even though that build is known to have some regressions for some of their other test scenarios. As they work towards wrapping up their 1.2 release and fix those other bugs, we’ll continue to test and talk to help that the interop scenarios keep working.

As a result of this collaboration, Metro 1.2 is going to be a better and more interoperable release for the Sun's customers and the greater Java community and BizTalk Services as well as our future identity products will be better and more interoperable, too. Win-Win. Thank you, Sun.

As a goodie, I put some code into the Java sample that might be useful even if you don't even care about our services. Since configuring the Java certificate stores for standalone applications can be really painful, I added some simple code that's using a week-old feature of the latest Metro 1.2 bits that allows configuring the Truststores/Keystores dynamically and pull the stores from the client's .jar at runtime. The code also has an authorization utility class that shows how to get and evaluate claims on the service side by pulling the SAML token out of the context and pulling the correct attributes from the token.

Have fun.

[By the way, this is not an April Fool's joke, in case you were wondering]

[WebGet] BizTalk

clemensv — Tue, 08 May 2007 02:00:16 GMT

Steve Maine explains what's in the newest revision of the BizTalk Services SDK, including quite a few (standalone-) surprises for WCF and WF developers. In case you haven't noticed, we've dropped a new and substantially expanded build of the SDK just a week after we published the first SDK.

Stop. Don't leave yet. Before you say "What do I care about BizTalk?", you should know that while BizTalk has been more or less associated with the BizTalk Server 200x product line in the past few years, (Codename-) BizTalk Services is a complementary set of functionality that's not only interesting to BizTalk Server customers, but really to all .NET developers.

Weird? Flip flopping? Confusing? No. The fact that BizTalk is not only BizTalk Server isn't really new. When BizTalk came out back in 2000 and I was very closely looking at what's going on (get it used for $2), the definition read like this in the press release:

The BizTalk Initiative represents the collective set of investments that Microsoft is making to facilitate business process integration within and between organizations using Internet-standard protocols and formats. It includes the BizTalk Framework, the BizTalk.org community and business document library, as well as BizTalk Server 2000, a business process orchestration server and tools for developing, executing and managing distributed business processes. These investments are being made in conjunction with industry standards groups, technology and service providers, as well as key global organizations.

While the envisioned schema exchange BizTalk.org fell flat since industry-wide message-level-schema standardization for "everything" more or less didn't happen in the way people initially expected, what came out of this initiative as a significant element was that the set of specifications then known as the BizTalk Framework 2.0 that acted as a foundation for quite a few of the WS-* specifications and the BizTalk Server product which evolved into a very successful and leading SOA/BPM suite that's soon seeing its next release, BizTalk Server 2006 R2. Fast forward, read Steven Martin's blog entry where he writes:

[...] We see BizTalk Services as a complement to "traditional" BizTalk Server uses on premise. As you need to coordinate SOA on a broader scale beyond the organization, we see the introduction of hosted services as one way to help support federation of business process, messaging, and identity across boundaries. Over time, we want to ensure that BizTalk Server customers will be able to easily use the cloud services in conjunction with their premise technology. [...]

So all in all, a very sane way to think about BizTalk is that the software and services we publish under that name are providing functionality for messaging, process management and connectivity that go beyond the capability of the core .NET Framework.

Some fun with the BizTalk Services ISB

clemensv — Tue, 08 May 2007 00:00:21 GMT

I wrote a slightly Twitter-inspired, fun app over the weekend that's using the BizTalk Services Connectivity service and relay. In the spirit of Software+Services I'm going to give you half of it [for now] ;-) You must have the BizTalk Services SDK installed to run the sample.

The server app, which I'm keeping to myself for the next few days as part of the experiment, is an extension (add-in) to Windows Live Messenger. The Messenger add-in monitors all chats with tweetiebot@hotmail.com and keeps circular buffer with the last 40 incoming messages. Using the client (which is in the attached archive), you can get a list of "Tweets" and add a new one (same as chatting)

[ServiceContract(Name = "TweetieBot", Namespace = http://samples.vasters.com/2007/05/tweetiebot)]
public interface ITweetieBot
{
[OperationContract]
IList<Tweet> GetTweets(DateTime? since);
[OperationContract]
void Tweet(string nickname, string text);
}

or you can subscribe to new tweets and get them as they arrive

[ServiceContract(Name = "TweetieEvents", Namespace = http://samples.vasters.com/2007/05/tweetiebot)]
public interface ITweetieEvents
{
[OperationContract(IsOneWay=true)]
void OnTweet(Tweet tweet);
}

The client application hooks up to the client (that lives right on my desktop machine) through the BizTalk Services ISB and the server fires events back through the ISB relay into the client as new tweets arrive. So when you run the attached client app, you'll find that it starts with a dump of the current log of the bot and then keeps spitting out events as they arrive.

The client is actually pretty simple. The EventsClient is the subscriber for the pub/sub service (ConnectionMode.RelayMulticast) that writes out the received events to the console. The rest all happens in Main (parsing an validating the command line argument) and in Run.

    class Program
    {
        class EventsClient : ITweetieEvents
        {
            public void OnTweet(Tweet tweet)
            {
                Console.WriteLine("[{0}] {1}:{2}", tweet.Time, tweet.User, tweet.Text);
            }
        }

        static void Main(string[] args)
        {
            string usageMessage = "Usage: IMBotClient <messenger-email-address>";
            if (args.Length == 0)
            {
                Console.WriteLine(usageMessage);
            }
            else
            {
                if (!Regex.IsMatch(args[0], @"^([\w\-\.]+)@((\[([0-9]{1,3}\.){3}[0-9]{1,3}\])|(([\w\-]+\.)+)([a-zA-Z]{2,4}))$"))
                {
                    Console.WriteLine(usageMessage);
                    Console.WriteLine("'{0}' is not a valid email address");
                }
                else
                {
                    Run(args[0]);
                }
            }
        }

        private static void Run(string emailAddress)
        {
            EndpointAddress serviceAddress =
                new EndpointAddress(String.Format(String.Format("sb://{0}/services/tweetiebot/{1}/service",
                                    RelayBinding.DefaultRelayHostName, Uri.EscapeDataString(emailAddress))));
            EndpointAddress eventsAddress =
                new EndpointAddress(String.Format(String.Format("sb://{0}/services/tweetiebot/{1}/events",
                                    RelayBinding.DefaultRelayHostName, Uri.EscapeDataString(emailAddress))));

The URI scheme for services that hook into the ISB is "sb:" and the default address of the relay is encoded in the SDK assemblies. We set up two endpoints here. One for the client channel to fetch the initial list and one for the event subscriber.

            RelayBinding relayBinding = new RelayBinding();

            ServiceHost eventsHost = new ServiceHost(typeof(EventsClient));
            RelayBinding eventBinding = new RelayBinding(RelayConnectionMode.RelayedMulticast);
            eventsHost.AddServiceEndpoint(typeof(ITweetieEvents), eventBinding, eventsAddress.ToString());
            eventsHost.Open();

            ChannelFactory<TweetieBotChannel> channelFactory = new ChannelFactory<TweetieBotChannel>(relayBinding, serviceAddress);
            TweetieBotChannel channel = channelFactory.CreateChannel();
            channel.Open();

The two *.Open() calls will each prompt for a CardSpace authentication, so you will have to be registered to run the sample. Once you have opened the channels (and my service is running), you'll be able to pull the list of current tweets. Meanwhile, whenever a new event pops up, the EventsClient above will write out a new line.

            IList<Tweet> tweets = channel.GetTweets(lastTime);
            foreach (Tweet tweet in tweets)
            {
                Console.WriteLine("[{0}] {1}:{2}", tweet.Time, tweet.User, tweet.Text);
            }

            Console.WriteLine("Press ENTER to quit at any time");
            Console.ReadLine();

            eventsHost.Close();
            channel.Close();
            channelFactory.Close();
        }

So when you run the app, you can chat (anyone can, you don't need to be a buddy) tweetiebot@hotmail.com through Live Messenger and you'll see your chat lines (and potentially others') popping out as events from the service bus.

To run the sample with my bot, you need to call the client with "IMBotClient tweetiebot@hotmail.com" and select your BizTalk Services Information Card twice as you are prompted.

Privacy notice: I'm anonymizing the name of the contact only insofar as I'm clipping anything including and following the "at" sign of the user that chats the bot. So whatever you say is published as "emailname: text line"

IMBotClient.zip (3.61 KB)