Justin Smith's Blog : BizTalk.NET Services

First week at the new job

justinjsmith — Sat, 22 Mar 2008 20:01:50 GMT

This week I started a new job at Microsoft - I've moved from Developer and Platform Evangelism to the Connected Systems Division. I'm now a program manager on the BizTalk Services team (http://biztalk.net). It was hard to leave DPE, but this seems like the right opportunity. For those that haven't seen BizTalk.net, it's an incubation project for messaging, security, and workflow services that run at scale in Microsoft's hosted environment.

The week was a whirlwind, and I am having a blast. The team is very agile, and it has the atmosphere of a startup. Truly cool.

Robots and BizTalk Services

justinjsmith — Thu, 28 Feb 2008 21:58:23 GMT

I just posted a new screencast on Channel 9: http://channel9.msdn.com/Showpost.aspx?postid=386824.

It's purpose is to show off the power of claims and claims transformation, especially in a hosted service like BizTalk Services.

Claims and the Calculator sample in the BizTalk Services SDK

justinjsmith — Thu, 28 Feb 2008 02:01:00 GMT

The identity provider in BizTalk Services is a way for applications to delegate identity and access control to a hosted service. In other words, an application can use BizTalk Services to answer the all-important question "Who are you and what are you allowed to do?". If you haven't checked it out, I strongly recommend it (http://biztalk.net).

The identity provider can be used without the connectivity service. The calculator sample in the SDK is an example (as of R10, it installs by default in the C:\Program Files\Microsoft BizTalk Services SDK\Samples\AccessControl directory). This entry examines the calculator sample that uses a certificate, and this blog post is not a substitute for reading the readme...

As indicated in the readme, running this sample requires you to login to http://biztalk.net, go to "Manage Access Control", "Rules", and setup a few claims mappings. After you login, click on the following (right hand side):

The Rules link takes you to a page that lets you map input claims to output claims. In the case of the Calculator sample, you are going to map BizTalk.NET usernames to Resource+Operation claims. These Resource+Operation claims are demanded by the calculator service (running on your machine). WCF shields you from quite a bit of the protocol level goo here. The following is an example of what it looks like in the web UI:

There's lots of interesting things going on here. For now, let me just focus on the Output Claim Value field. It's the concatenation of (in this case) the service URL, "#", the service contract name, ".", and the action of the operation:

<Value> ::= <Service Url> "#" <Service Contract Name> "." <Operation Action>

This may sound very scientific, but the reality is that it's simply a matter of a choice made by the ServiceAuthorizationManager derived type that's included in the Calculator sample (available at C:\Program Files\Microsoft BizTalk Services SDK\Samples\AccessControl\CalculatorServiceWithCertificate\FederatedAccessManager). You can change it to be whatever you want (of <Read> or <Write> could have been substituted for contract name + operation action).

After you setup the input and output claim mapping for the calculator service, you have a system like the following:

Step 1: Present identity claims to BizTalk.NET

Step 2: Receive the claims that are mapped to that identity

Step 3: Send those claims to the Calculator Service

Step 4: If the claim (URL+ContractName+Action) is present, the ServiceAuthorizationManager allows the WCF infrastructure to invoke the calculator implementation.

In effect, the calculator service delegates identity and access control to BizTalk Services.

I think that's way cool.

PictureServices and BizTalk Services

justinjsmith — Sat, 19 Jan 2008 03:47:00 GMT

Previous posts have talked a bit about PictureServices. Now I'd like to run through what it took to bring PictureServices to the BizTalk Services. I've talked a bit in other posts about BizTalk Services, but it has some interesting and very useful messaging features. For starters, BizTalk Services has an endpoint that can do HTTP transforms on messages. It goes like this:

MyApp connects to BizTalk Services
BizTalk Services and MyApp use WS-* security, coupled with the WCF binary message encoding.
Other clients (Java, PHP, Ruby, whatever) can hit an HTTP endpoint hosted in the BizTalk Services "mesh". We will call this endpoint the HTTP Endpoint.
Upon receipt of a message, the HTTP Endpoint tries to dispatch that message (or request) to MyApp.

If we use this set of steps with PictureServices, then you have a way to serve local pictures over the internet. If you just look at this task, there are lots of other ways to do this. I'm not suggesting that BizTalk Services is the next cool way to share photos. Instead, I think there are lots of cool apps that can take advantage of this kind of feature set.

Down to nuts and bolts - the service contract. PictureServices defines a service contract with operations for RSS, ATOM, and Simple list extensions. For the sake of time, I opted to have a single feature: get an RSS feed. Here's what the service contract looks like:

[ServiceContract]
interface IBTSPictureSyndication
{
  [OperationContract(Action="GET", ReplyAction="GETRESPONSE")]
  Message GetFeed();
}

The return type is System.ServiceModel.Channels.Message because the BizTalk Services SDK does not have full parity with the .NET Framework 3.5's capabilities. This will happen over time. To be honest, most of the head scratching I went through was due to the disparity between these two APIs.

Another interesting bit is the implementation of the GetFeed method. The syndication, the images, and all the thumbnail HTTP GETs (remember they are transformed) are going to flow through this one method. As a result, we have to test the requested address to determine which resource is being requested.

For that we turn to our new best friend: the UriTemplate. This is a new type introduced in .NET 3.5 - it makes URI parsing much easier (among other things).

There are 3 conditions I test for:

1) a request for the whole feed

2) a request for an image

3) a request for a thumbnail.

Here's the implementation:

[ServiceBehavior(AddressFilterMode=AddressFilterMode.Prefix)]
sealed class BTSPictureSyndication : IBTSPictureSyndication
{
    String feedUri = "/Feed/";
    String pictureUri = "/Feed/Picture/{pictureId}";
    String thumbnailUri = "/Feed/Picture/t/{pictureId}";

    PictureService service = new PictureService();

    public Message GetFeed()
    {
        // get the To address and the base address
        Uri to = OperationContext.Current.RequestContext.RequestMessage.Headers.To;
        Uri baseAddress = new Uri(ConfigurationManager.AppSettings["baseAddress"]);

        // check to see if it's a request for the main feed
        UriTemplate feedTemplate = new UriTemplate(feedUri);
        UriTemplateMatch feedMatch = feedTemplate.Match(baseAddress, to);

        if (feedMatch != null)
        {
            return GetMainFeed();
        }

        // check to see if it's a request for a picture
        UriTemplate pictureTemplate = new UriTemplate(pictureUri);
        UriTemplateMatch pictureMatch = pictureTemplate.Match(baseAddress, to);

        if (pictureMatch != null)
        {
            String pictureId = pictureMatch.BoundVariables["pictureId"];
            return GetPicture(pictureId);
        }

        // check to see if it's a request for a thumbnail
        UriTemplate thumbnailTemplate = new UriTemplate(thumbnailUri);
        UriTemplateMatch thumbnailMatch = thumbnailTemplate.Match(baseAddress, to);

        if (thumbnailMatch != null)
        {
            String pictureId = thumbnailMatch.BoundVariables["pictureId"];
            return GetPictureThumbnail(pictureId);
        }

        // we don't know what it is, so throw
        throw new InvalidOperationException(String.Format("the address {0} was not matched", to.ToString()));

    }

    // use the service object to get the picture
    private Message GetPicture(String pictureId)
    {
        return StreamMessageHelper.CreateMessage(OperationContext.Current.IncomingMessageVersion,

        "GETRESPONSE", service.GetPicture(pictureId));
    }

    // use the service object to get a thumbnail
    private Message GetPictureThumbnail(String pictureId)
    {
        return StreamMessageHelper.CreateMessage(OperationContext.Current.IncomingMessageVersion,

        "GETRESPONSE", service.GetPictureThumbnail(pictureId));
    }

    // get the main feed (RSS)
    private Message GetMainFeed()
    {
        Rss20FeedFormatter formatter = service.GetPicturesAsRss();

        MemoryStream stream = new MemoryStream();
        XmlWriter writer = XmlWriter.Create(stream);

        formatter.WriteTo(writer);

        writer.Close();
        stream.Position = 0;

        return StreamMessageHelper.CreateMessage(OperationContext.Current.IncomingMessageVersion, "GETRESPONSE", stream);
    }
}

Another thing that's worth pointing out is the presence of the AddressFilterMode property on the ServiceBehavior attribute annotation. This setting tells the WCF dispatching infrastructure to allow prefix matches to filter through to the method. In practical terms, this means that a request to http://foo/bar/baz would get dispatched to the same method as http://foo/bar. That's how we are returning the full feed, the image, or a thumbnail.

Here's a zipped version of the project (note that you must also have PictureServices).

Introduction to BizTalk.NET Services

justinjsmith — Sun, 10 Jun 2007 20:54:11 GMT

Background

BizTalk.NET Services (http://labs.biztalk.net) is a nascent and experiemental project that is all about generic, secure connectivity. Many WCF customers ask us how to notify client applications that something has happened in a service. If the client is addressable and listening, this is a fairly easy question to address. In fact, there is an SDK sample that addresses (pun intended) that exact scenario. However, when one of the clients is not addressable (behind a firewall or NAT), the problem becomes much harder. After all, how can a client receive incoming messages if it's listening address is virtually invisible outside the firewall or NAT?

This problem is not new. Connected applications like Groove, FolderShare, Skype, and all manner of Instant Messenger have had to solve this problem. Each of these applications has solved this problem in their own way, and with their own resources. Even though each of these business had to solve the problem, the physical characteristics of networks and the commonality of the requirement fostered (perhaps) similar design charachteristics. In the cases I am aware of, instances of these connected applications (i.e. clients) call out from behind a firewall or NAT to one or more servers in the internet. For the duration of that established socket, the server is able to send data back to the client. If, during the lifetime of that socket, another instance of the same connected application calls into the server, then the server is able to pass data from one instance of the connected application to another.

Though I don't have much emprical evidence to prove it across the board, my guess is that the connectivity quest undertaken by these development teams was not cheap or easy. Nonetheless, connected application vendors have all incurred this cost, and, in some form or fashion, solved the connected application problem in a proprietary manner. In some cases (like Live Messenger and Yahoo Messenger), some level or interoperability is possible, but they are generally point solutions.

Where does this leave organizations that don't have the budget, expertise, or time to implement a custom connectivity solution? This is the problem space where BizTalk.NET Services thrives. An organization can use BizTalk.NET Services to handle the connectivity, and frees up development resources to focus on the functionality of the application. The hope is that alleviating the connectivity burden allows organizations to deliver better applications faster and cheaper.

State of the Project

As of the date of this post, BizTalk.NET Services is in CTP form, and the team has adopted an Agile mindset. They drop code frequently, and alter the trajectory of the project based on feedback (and other things). As I understand it, the team had not committed to a release date, and there are many discussions internally regarding how these services will be made available to the community and customers.

In my discussions with the team, there is no doubt that they are eager to receive feedback about existing and non-existing features. As an example, the latest drop includes support for username / password security. Before this drop, the product had support for CardSpace security only. The inclusion of username / password security was largely driven by community feedback.

I encourage you to download the BizTalk.NET Services SDK and write some code with it. You can find it here (http://labs.biztalk.net). That site also includes links for how to provide feedback to the team.

How BizTalk.NET Services Surfaces to the Developer

In it's current form, BizTalk.NET Services is visible to developers via a new WCF binding (named the RelayBinding). The RelayBinding type utilizes WCF extensibility points and creates a channel stack that knows how to communicate with the BizTalk.NET Services servers in the cloud. More specifically, the RelayBinding creates a channel stack that includes a new transport channel that can connect with the BizTalk.NET Services servers in the cloud. The URI passed to this channel has a new scheme that helps distinguish it from other transports (sb://). The RelayBinding can be used with the existing ServiceHost implementations for listening, as well as the existing ChannelFactory<T> implementations to send messages to other messaging participants.

The transport channels do the heavy lifting of connecting to the servers in the cloud as well as participating in sending and receiving messages. The servers in the cloud do the heavy lifting of connecting applications. The compostion of the URI and the configuration of the RelayBinding object (and the resultant channel stack) dictate which applications are connected and how they are connected to one another.

Several types of connections are possible, and these will be the subject of another post (or several other posts).

One of the takeaways with this model is that user or applicaiton code is not hosted in the cloud. The services in the cloud broker the connections between instances of a client application.

Futures

As I understand it, the team's work items include furthering the scalability of this system, the inclusion of workflow into the services, and enhancing the security features of the services. Keep in mind that this project is nascent, and the team is intentionally agile. If you have opinions or feedback about any of these features, I encourage you to contact the team via the address shown earlier.

I fully expect to write much more about this cool new project, so stay tuned.