Yasser Shohoud

What I've Been Up To

yassers929 — Tue, 17 Feb 2009 18:21:00 GMT

I've been away from blogging for a loong time so have some catching up to do. I spent most of last year building out the new Dallas Microsoft Technology Center while delivering customer engagements and working on a few side projects. The MTC build out is mostly done (a few items remain but I'm spending little time on it) and has turned out pretty nice. You can see a synth of the reception area here but you really have to come visit and see it for yourself. Recently, the Dallas Morning News wrote a nice article about the Dallas MTC and what we do for customers. On Friday 2/13, we hosted a minority students day here in the Microsoft Las Colinas campus (see here) and we had the kids visit the MTC where we exposed them to different careers within the tech industry. It was a lot of fun and I was surprised a few times to find out that kids have used our technologies e.g. XBox 360, Zune, Windows movie maker (hey maybe we're doing good marketing with this age group after all!). The local NBC channel was here and some of us were on the news that night.
Now that the facility is built, it's time to shape and define my team and make sure we stay sharp and on top of our game. As for what I'm doing now, I'm still doing customer engagements mostly around .NET architecture and development and SharePoint. I'm also getting a lot of exposure to infrastructure including server/storage hardware and networking as I manage the partners that provide us with equipment for our MTC. Just recently I was recruited to help build the next generation MTC tools on Dynamics CRM so I'm about to start learning CRM.
Outside of work, I've started school again, this time for an MBA. My first course is on Operations Management which I'm finding quite interesting thus far. I'm planning to be done in 3 years assuming I can take 5 courses per year which hopefully I can do.

Big week for Dallas MTC

yassers929 — Sat, 15 Nov 2008 21:27:00 GMT

This week is shaping up to be a big week for the Dallas MTC. Monday we have our official launch party where we expect 285 attendees from Micrsosoft and partners (we've been operational since 10/6 but we're now making it official). Then Tuesday thru Thursday we have the World Wide MTC Summit where managers from all 16 world-wide MTCs get together to share best practices and plan for the year ahead. Thursday and Friday we have another major event with the MTC partners. Not to mention a few customer engagements going on next week as well. It'll be crazy busy and lots of fun :-)

Getting back into blogging

yassers929 — Fri, 07 Nov 2008 17:36:00 GMT

It's been over a year since I last blogged over on yassers.com. I'm trying to get back into it now and will move my blog back here to MSDN blogs. I know what you're thinking: "he moves around too much" and I agree with you :-) Not only am I moving my blog for the second time in as many years, I've also physically moved twice in the last two years. The first move was when I moved from Seattle (where I worked on the WCF team) to Austin (where I worked on the Austin Microsoft Technology Center team). About a year after that, Microsoft decided to move the Austin MTC to Dallas for very good reasons. So I picked up and moved again to the Dallas area (I live in Grapevine, TX which is about 12 miles from the office in Irving, TX).

I spent most of the last year building out the new MTC in Dallas and we just recently moved in. It's an awesome place to work and I'll be blogging about that more in the weeks to come. I also plan to blog about the various technologies we have at the MTC as well as some of the new technologies Microsoft is announcing. Thanks for reading!

Blog moved

yassers929 — Thu, 14 Sep 2006 04:34:00 GMT

I've moved my web site and blog to yassers.com. The blog is at http://blog.yassers.com. Some of the more recent entries on this blog (the ones included in this blogs current RSS feed) are duplicated there as well.

What Exactly Is BizTalk?

yassers929 — Tue, 25 Jul 2006 21:21:00 GMT

I now work as a Technology Architect at the Microsoft Technology Center in Austin which is part of the Microsoft Sales org. In this role I help customers understand how to architect solutions to their business problems/needs using Microsoft technologies. As a result I'm learning a lot of technologies that, as a member of the WCF product team, I just wasn't focused on in the past few years. When I learn a new technology I like to start by coming up with (or learning) a short and accurate description of that technology's focus, i.e. what is it for?

One of the technologies I'm (re)learning is BizTalk Server. So I looked around for a short and accurate description of what BizTalk is. I knew what it was, but I wanted a 2 - 3 sentence description. I found some interesting documents like the BizTalk Overview and David Chappell's excellent Understanding BizTalk Server 2006 (a must read). While those docs offered valuable information, I couldn't find the sentences I was looking for. This is probably due in part to my coming from a development background and wanting to relate things back to the developer world. So I came up with this short and, in my opinion, accurate description:

BizTalk is to business process automation what Visual Studio and the .NET Framework are to application development. It is a runtime and tools that enable you to automate and manage business processes more productively.

That's basically it. If I only get 10 seconds to explain what BizTalk is to a customer, the above description is what I'm most comfortable with. If I have more time, I would elaborate on that description as follows.

The term "application development" is worth further exploration. Looking at the kinds of applications that people build with Visual Studio and the .NET Framework today, the vast majority are focused on specific tasks rather than end to end processes. Not to say that such applications are isolated, they're actually often connected to other apps or data sources via Web services or other means. But the focus of the app is to automate a particular task or set of related tasks and the connectivity serves that purpose. However the business reality is that businesses run on processes which span multiple tasks and of course people. So to automate a business process end-to-end you wind up wanting to connect some of the applications that were written to automate the tasks that make up that business process. As an example, consider the task of updating contact information in the company's address book for a newly hired employee. That employee may launch a .NET desktop app or go to an intranet Web site and enter they home address, cell phone number, emergency contact etc. Now zoom out and look at the big picture and what you see is an end-to-end on-boarding process that includes things like provisioning credentials, certificates, smart card, automatic deposit setup, hardware procurement, office furniture procurement, 401K account setup, setting up a mentor, and the variety of other things a new employee needs to get/set up. The address book update app automates one task in the end to end process but there's no app that automates the entire process. BizTalk Server allows you to connect existing apps that automate the various tasks of this process and automate the tasks or parts that don't yet have automation to form a complete end-to-end process automation solution. It also gives you an operational or runtime view of that automation allowing you to see all the moving parts from one vantage point and start/stop/change/maintain them.

While it's certainly possible to build VS and .NET apps that automate end-to-end business processes, it's not the most productive use of the customer's/developer's time because they'd have to reinvent (or re-implement) several wheels in the process. It's a bit like building an document management Web site using nothing but ASP.NET: You can do it, but you can do it a lot better and faster if you used Windows SharePoint Services (WSS). The operations folks would also love you for it because of the management features they get out of the box with WSS.

Living In Austin

yassers929 — Sun, 25 Jun 2006 16:47:00 GMT

On Saturday 6/17 I boarded a flight from Seattle to Austin and I didn't have a return ticket. You see, for family reasons I moved to Austin and joined the Microsoft Technology Center. The goal of the MTC is to help customers map their business problems to solutions based on Microsoft products and services. MTC Architects work with customers in three primary ways: Strategy Briefings (1day), Architecture Design Sessions(2days), and Proof-of-Concept Workshop(2-3weeks). You can read more about these activities at the above link, but basically they are about giving the customer the big picture of Microsoft technologies that can help with their business problems, drilling deeper into customer needs and architecting a solution to meet those needs, and building a proof of concept of that solution with the customer's own developers.
Needless to say, this role is quite different from my previous role on the WCF product team and I'm excited about that change and looking forward to spending a lot more time with customers and getting first hand experience with the customer-facing side of Microsoft. It's only been a week but I'm already having a ton of fun. For my first two weeks I'm shadowing Christian Thilmany in a POC based on Windows Sharepoing Services and Microsoft Office Sharepoint Server. Both technologies and in beta and I have no prior hands-on experience with them so I'm learning and having fun doing it. In addition to the exciting work, people on the MTC team have been warmly welcoming me and offering assistance whenever I need it.
Besides the exciting new role and the great team, Austin has some bonus features. There are some really great schools here (we're looking to live in NW Austin around Canyon Vista Middle and Westwood High Schools), UT Austin is a high ranking university, and the Austin live music scene provides some unique entertainment opportunities. Yesterday I discovered another bonus feature: Twisty roads. Moving here from Seattle's East Side (foot hills of the Cascades), I was worried I won't find good places to ride my bike. So yesterday I went out exploring and I found several well paved, twisty roads that were nearly empty at 5PM on a Saturday. In fact I got tired and turned back before the road showed any signs of ending. Turns out because Austin is in the Texas Hill Country, there are plenty of hills, canyons, and lakes and and abundance of twisty roads. Couple that with the drier and warmer weather, and you're pretty close to sport bike rider's heaven! I'm already loving it here.

More on the WCF RSS Sample

yassers929 — Wed, 21 Jun 2006 21:31:00 GMT

Last week we put up a sample named WCF RSS Toolkit on wcf.netfx3.com and I posted a sample service using it here. Since I wrote most of the toolkit’s code I thought I’d talk a bit about its goals, design and the future. There were two primary goals for this sample:

To explore/prototype RSS/Atom support in WCF and get customer feedback
Provide a sample to show WCF developers how to expose a service as RSS and Atom over HTTP GET using WCF v1

Note the “and” not “or” in #2, meaning the same service implementation can be exposed as RSS, Atom, or any other format simultaneously. If you know WCF, you’ll know this means generating multiple wire formats (RSS, Atom, whatever) from the same contract which can be done with some extensibility. The key is to not rely on the contract projecting the desired info set, rather to transform whatever info set projected by the contract to the desired info set. That transformation would be different depending on the desired wire format.

To make it easier for developers to emit RSS and Atom, I defined a DataContract with some of the commonly used RSS/Atom data and I defined a transform from that DataContract’s info set to RSS and another transform to Atom. All of this is pluggable/replaceable: You can use your own DataContract if you like, the transforms will transform what they recognize and copy the rest out as extension elements. You can also plug in your own transform to support new formats. Since it’s source, it’s also customizable so you can modify the DataContract and/or the transform if you don’t like how I make DataContract members to RSS/Atom elements.

There was also the HTTP GET part of the equation. WCF’s HTTP transport supports GET out of the box but there are other places in the stack that need to be extended for the scenarios I had in mind. For example, dispatching to the right operation based on the path segment following the endpoint’s address in the request URL. Also, parsing the request query string and decoding parameters from it. And of course, supporting ETag and Last-Modified HTTP header. All of these are commonly needed in RSS/Atom scenarios and when we do build RSS/Atom support into WCF, we will have these features. For the sample however, the first two are included in the sample but the last one isn’t. The good news is that the WCF HTTP transport gives you full read/write access to HTTP headers via the Message properties HttpRequestMessageProperty and HttpResponseMessageProperty so you could build ETag support on top of the current WCF bits. In fact, one of the benefits of having the RSS toolkit sample source code is it gives you a starting point that you can take and modify with ETag support and other features you want. You can even publish you modified sample for the community to learn from and use.

WCF RSS Sample

yassers929 — Thu, 15 Jun 2006 19:22:00 GMT

Mark is looking for some sample code for the recently released WCF RSS Toolkit. I've put the readme up here. I won't repeat the whole readme here but here's a peek at the programming model. There are several shortcuts for feeds from databases and feeds using existing data types so you should really check out the readme.

An example service:

using System;
using System.ServiceModel;
using System.Collections.Generic;
using Microsoft.ServiceModel.Samples.Syndication;
namespace Microsoft.ServiceModel.Samples.Syndication
{

    [ServiceContract]
    public class ItemListService
    {
       [OperationContract]
       ContentFeed<FeedItem> GetItemList()
       {

//instantiate the feed
ContentFeed<FeedItem> feed = new ContentFeed<FeedItem>();

//populate the list of feed items
feed.FeedItems = GenerateItemList();

            //set some feed-level properties
            feed.FeedAuthor = wcfuser@contoso.com;
            feed.FeedTitle = "WCF Item List!";

...

//return the feed
return feed;

}

ItemList<FeedItem> GenerateItemList()

{

ItemList<FeedItem> list = new ItemList<FeedItem>();

FeedItem item1 = new FeedItem();
item1.ItemTitle = "This is item 1";

...

list.Add(item1);

...

return list;

}

To expose RSS, Atom and SOAP endpoints in code (see readme and samples in toolkit for config-based and hosting in IIS examples):

using System.ServiceModel;
using Microsoft.ServiceModel.Samples.Syndication;
namespace Microsoft.ServiceModel.Samples.Syndication
{

class Program
{

static void Main(string[] args)
{

            ContentFeedHost host = new ContentFeedHost(
                typeof(ItemListService),
                new Uri("http://localhost:8077/ItemListService"));

//endpoints defined in code

            // RSS endpoint will be listening at
            //http://localhost:8077/ItemListService/rss
            host.AddRssEndpoint(typeof(ItemListService), "rss");

            // Atom endpoint will be listening at
            //http://localhost:8077/ItemListService/atom
            host.AddAtomEndpoint(typeof(ItemListService), "atom");

            // WS-* SOAP endpoint will be listening at
            //http://localhost:8077/ItemListService/soap
            host.AddServiceEndpoint(typeof(ItemListService),
                                    new WsHttpBinding(), "soap");

host.Open();

            Console.WriteLine("Service started ...");
            Console.WriteLine("Press [Enter] to exit");
            Console.ReadLine();

host.Close();

}

Enjoy!

WCF Custom Channel Ecosystem

yassers929 — Mon, 10 Apr 2006 15:05:00 GMT

Back in January, a few of us on the WCF team kicked off a side project dubbed "the channel ecosystem project". The goal is to create a community and an ecosystem around WCF custom channels. The first step was to make it easier for developers to build custom channels. To do this we decided to provide detailed documentation, samples, and a couple of tools. We've been creating this content over the last 3 - 4 months as a part-time effort next to our full-time jobs. Thanks to the hard work of many people, the content is now in a state where it can be shared with the world as a pre-release to solicit feedback. It's been published on windowscommunication.net. The samples are also linked from the samples gallery.
We want to hear your feedback on this content as well as what else you'd like to see to help you build custom channels. Send feedback to ces at microsoft.com.

It's here

yassers929 — Sun, 12 Mar 2006 08:35:00 GMT

I came home today to find a large box waiting for me; it's my new iMac G5! This is the 20" intel dual core version and it's absolutely stunning. Now I must admit that this is my first Apple computer. My first computer was a Texas Instruments TI 99/4A (complete with the Extended Basic cartridge). My second computer was a Commodore 128. After that it's been PCs, PS/2s. and more PCs. But now I own a Mac and I'm glad I do. This computer is a pleasing to look at and fun to use. But the thing that's most amazing thus far is how easy it is to use evening for scenarios that I thought would be considered "advanced". It took less than 15 minutes from the minute I started opening the box till I was browsing the Web on my wireless LAN. I then decided to go all out and attempt to connect to my home file server to get to my music and photos. Now this is a task I don't take lightly. Every single time I try to do this with a new or newly reinstalled Windows computer it never works the first time. I typically have to do hacks like use the server's IP address or put an entry in the hosts file. Of course I expected it to be doubly hard on a Mac because I've never used one of these before and, well, it's a Mac so how could it possibly be easier to talk to a Windows server than from a Windows PC? Well, I fumbled along until I found the help, I searched for connecting to Windows PCs and I found a sequence of 3 steps that supposedly end in seeing the Windows PC and its shares. Oddly enough, I followed those steps and there was my home server and its shares right there on my Mac! That's just amazing. Next, I decided to try and connect my cell phone and my wife's cell phone to the Mac using Bluetooth. Painlessly, everything worked the first time. This machine brings new meaning to the words "it just works". I put in The Matrix DVD and fiddled around with the iPod-shaped remote control. The 10' UI is smooth and clean and video playback on the 20" screen makes for a great viewing experience. But the built-in speakers are weak and have poor bass. Guess I need to buy a nicer pair of speakers. Then my kids came and we played with the photo booth app. This is a little app that lets you take pictures with the built-in camera and apply one of about 20 silly effects. The app has a clever trick to make up for poor lighting: It uses the massive 20" LCD as a flash turning it bright white for a second while the camera takes a picture. My kids loved playing around and making silly faces. We then took the pictures into iPhoto and made a slideshow with transitions and music. Took two minutes and turned out great. Wow. Best of all, there are absolutely no wires except for the power cord (my wife is really impressed by that). I want to keep it clean so I'll look for a pair of wireless speakers. I'm looking forward to learning more about Mac OS/X and using it for all sorts of fun projects.

Moving lots of data

yassers929 — Sun, 22 Jan 2006 09:19:00 GMT

Recently I've been getting lots of questions about moving large files (or lots of data) between a WCF service and client. The question comes in multiple forms, e.g.

How do I send a file that's many GB in size from the service to the client?
Should I use MTOM to send large files?
I have a huge object graph and I want to send it over the wire, how should I do it?

So let's explore the problems associated with moving lots of data and the solutions offered by WCF.
Problem: Bandwidth Utilization
Sending gigabytes of data means lots of bandwidth usage. While that may not be a problem if it happens infrequently on a 100 Mbit or 1 Gigabit LAN, it is definitely an issue when bandwidth is scarce and/or is being paid for. In interop scenarios, where messages are encoded as text XML, the encoding of binary data using Base64 exacerbates the problem because it inflates the size by 1/3. There are three solutions to this problem depending on your scenario. First, compression could really help especially if the data is text or gets encoded as text (using Base64). Compression/decompression can be implemented using a custom WCF encoder/decoder (and IIS 6 offers built-in response compression). Second, you can avoid using text encoding when interop is not required. WCF provides a binary encoding which is far more bandwidth-efficient than the text encoding especially when sending binary data. Third, if you want interop and you need to send large binary content, you can use MTOM which allows you to send the binary content outside of the SOAP envelope (as a separate part of a multi-part MIME message) without Base64 encoding it.

Problem: Memory Utilization
By default, WCF buffers messages to support protocols like WS-ReliableMessaging and WS-Security that require buffered messages. For extremely large messages this can lead to out-of-memory conditions especially on servers that try to send or receive multiple of those messages simultaneously. Fortunately, WCF supports streaming on HTTP, TCP, and Named Pipes allowing you to send infinitely large messages without hitting out-of-memory exceptions (actually, message size is constrained to Int64.MaxValue or 9,223,372,036,854,775,807 but hopefully that's infinite as far as your app is concerned).

Problem: Recovering from Failures
So what happens if half way through sending your 4GB stream the TCP connection fails? Well, your app must catch the exception and recover. If the other side has been processing the stream as it receives it (e.g. saving it to disk) you may be able to have the sending app coordinate with the receiving app to figure out what was the last received byte and restart from there.
Another alternative is to use Reliable Messaging to recover from connection failures. RM will detect the failure and automatically re-establish a connection and resend the failed message. The problem here is that when you're streaming, the entire stream is one message so resending means basically starting over. Another problem, is that RM requires buffering (so it can resend on failure) so it doesn't work with streaming! The solution is to use chunking. This is where the sending application divides up the 4GB file into say 1 million messages each 4KB in size and sends them in buffered mode. The receiving application reconsitutes the file by appending all 1 million messages to form the original 4GB file. Here, RM buffers only a few of the 4KB messages at a time. In case of a failure, RM will automatically resend the 1 or few failed messages. The chunking/dechunking functionality can be encapsulated in a general purpose chunking channel that lets the applications program against a Stream and handles the chunking under the covers. This is what my sample chunking channel from PDC does. Note that because of the chunking and the protocol overhead added by RM, the resulting throughput is expected to be lower than direct streaming. However, you are getting reliability which comes in handy when the connection fails especially when you're almost done sending the 4GB and don't want to start all over!

To summarize, my recommendations are:
1. Use binary encoding if you have WCF on both sides or MTOM if you need interop.
2. Consider compression if you are forced to use text encoding or if the data itself is highly compressible (e.g. large text data).
3. Use streaming, or if you want reliability, use chunking. One restriction to keep in mind is that to stream, there can be only one parameter (or return value, depending on which direction you're streaming) and it must be of a type that derives from System.IO.Stream.

Meet the Channel Model: ICommunicationObject

yassers929 — Mon, 09 Jan 2006 03:14:00 GMT

A Common a State Machine

Objects that deal with communication, e.g. sockets, usually present a state machine whose state transitions relate to allocating network resources, making or accepting connections, closing connections and aborting communication. The channel state machine provides a uniform model of the states of a communication object that abstracts the underlying implementation of that object. The interface provides a set of states, state transition methods, and state transition events. All channels, channel factories and channel listeners implement the channel state machine.

ICommunicationObject

ICommunicationObject is a CLR interface that describes the channel state machine contract:

public interface ICommunicationObject : IDisposable
{
    CommunicationState State { get; }

    event EventHandler Closed;
    event EventHandler Closing;
    event EventHandler Faulted;
    event EventHandler Opened;
    event EventHandler Opening;

    void Abort();

    void Close();
    void Close(TimeSpan timeout);
    IAsyncResult BeginClose(AsyncCallback callback, object state);
    IAsyncResult BeginClose(TimeSpan timeout, 
                            AsyncCallback callback, object state);
    void EndClose(IAsyncResult result);

    void Open();
    void Open(TimeSpan timeout);
    IAsyncResult BeginOpen(AsyncCallback callback, object state);
    IAsyncResult BeginOpen(TimeSpan timeout, 
                           AsyncCallback callback, object state);
    void EndOpen(IAsyncResult result);
}

The events Closed, Closing, Faulted, Opened and Opening signal an external observer after a state transition occurs.
The methods Abort, Close, and Open (and their async equivalents) cause state transitions.
The state property returns the current state as defined by CommunicationState:

public enum CommunicationState
{
    Created,
    Opening,
    Opened,
    Closing,
    Closed,
    Faulted
}

An ICommunicationObject starts out in the Created state where it’s various properties can be configured. Once in the Opened state, the object is usable for sending and/or receiving messages but its properties are considered immutable. If an unrecoverable error occurs, the object transitions to the Faulted state where it can be inspected for information about the error and ultimately closed. When in the Closed state the object has essentially reached the end of the state machine. In general, once an object transitions from one state to the next, it does not go back to a previous state.

States and Transition
The figure below shows the ICommunicationObject states and state transitions. State transitions can be caused by calling one of the three methods: Abort, Open, or Close. They could also be caused by calling other implementation-specific methods. Transitioning to the Faulted state could happen as a result of errors while opening or after having opened the communication object. Every ICommunicationObject starts out in the Created state. In this state, an application can configure the object by setting its properties (e.g. setting the address of a channel listener by calling its SetUri method). Once an object is in a state other than Created, it is considered immutable. For example, calling SetUri on a channel listener in the Opened state should throw an InvalidOperationException.

WCF provides an abstract base class named CommunicationObject which implements ICommunicationObject and acts as the base of all channel objects. That will be the subject of my next posting on the channel model.

WCF Architecture Overview

yassers929 — Fri, 30 Dec 2005 07:13:00 GMT

This is a document I wrote a while ago and has been sitting around waiting to be published. Well, it finally is! Enjoy.

Looking for people who want to change the world

yassers929 — Tue, 29 Nov 2005 22:31:00 GMT

We have a few open positions on the Indigo (WCF) team for people who want to change the world. In this role, you would get the opportunity to really change how millions of developers write distributed applications (if you're a distributed app developer yourself, you know that's a big part of your world!). I'm not exaggerating here, we're not just talking tweaks or incremental improvements, I really mean significant change.
We're looking for program managers to help innovate, design, build and ship Indigo v1 and future versions/technologies in the distributed applications space. We have positions for both experienced folks and recent college grads. If you or someone you know is interested, just email me at yassers@microsoft.com.

Meet the WCF Channel Model – Part 2

yassers929 — Sat, 22 Oct 2005 21:31:00 GMT

Most WCF developers won’t interact directly with channels or have to think about them much. The Service Model, WCF’s programming model, provides abstractions that layer on top of channels to give a “method and typed parameters” programming model. However, many developers will want to write custom channels for the reasons I mentioned in my previous post. For those developers, it’s interesting to understand how the channels are used (indirectly) by a developer building a WCF service and/or client.

WCF services and clients are based on the concept of endpoints. An endpoint consists of an address, a binding and a contract. The address is the endpoint’s network location (the where). The binding specifies how the endpoint communicates with the world (the how) and the contract is the collection of operations the endpoint exposes (the what). For the purpose of this discussion, the binding is the most interesting of the three.

A service developer defines an endpoint by specifying its three components. For example:

public static void Main()
{
ServiceHost host = new ServiceHost(typeof(StreamServer));

host.AddEndpoint(
             typeof(IStreamService),
             new ChunkingTcpBinding(),
             new Uri("net.tcp://localhost/StreamService"));

host.Open();

Console.WriteLine("stream server started....");
Console.ReadLine();
}

The call to host.AddEndpoint defines the endpoint. The call to host.Open creates the channel stack that will listen for and process messages targeted at the endpoint. Each binding is really a collection of binding elements each contributing to building the channel stack. For example, this is the ChunkingTcpBinding:

public class ChunkingTcpBinding : Binding
{
public ChunkingTcpBinding()
{
       ...
}
public override BindingElementCollection CreateBindingElements()
{
      BindingElementCollection col =
                new BindingElementCollection();
      ChunkingBindingElement cbe = new ChunkingBindingElement();
      TcpTransportBindingElement tcp =
                new TcpTransportBindingElement();
      tcp.TransferMode = TransferMode.Buffered;
      tcp.MaxMessageSize = Int32.MaxValue;
      col.Add(cbe);
      col.Add(tcp);
      return col.Clone();
   }

}

Each binding element inherits from BindingElement which has a couple of interesting methods:

public abstract class BindingElement
{

   public virtual IChannelListener<TChannel>
           BuildChannelListener<TChannel>(ChannelBuildContext context)
                          where TChannel : class, IChannel
   {
   }

   public virtual IChannelFactory<TChannel>
           BuildChannelFactory<TChannel>(ChannelBuildContext context)
   {
   }

... other methods not shown ...

}

The Service Model calls BuildChannelListener to get back a ChannelListener that is added to the channel stack to listen for incoming channels. Think of an incoming channel as an incoming connection request. Once established, it can be used for receiving messages. Here it is visually:

On the send side (e.g. on the client side), the Service Model calls BuildChannelFactory to get back a ChannelFactory. A ChannelFactory is used to build a channel that can be used for sending messages. Here is this part visually:

You may be wondering why there are two factory abstractions: binding element and channelfactory/channellistener. Turns out there are different two categories of things one may want to do in a factory class that creates channels and those two categories are best separated in two factories. The categories are: programming model-specific work and channel-specific work. Programming model-specific work that belongs in binding elements includes things like varying behavior based on settings on the endpoint’s contract or implementation and reading from config. Channel-specific work that belongs in channel factories/listeners includes things like pooling resources across channels (e.g. network connections).

The next interesting thing to talk about is how channel factories create channels and the concept of channel shapes. That will be the subject of my next posting.

Meet the WCF Channel Model - Part 1

yassers929 — Wed, 12 Oct 2005 20:37:00 GMT

For the past 5 months or so, I’ve been working on the WCF Core Communications team (with Kenny, Matt, and Steve) where we focus on WCF transports and hosting features. In this period, I’ve learned a lot about the WCF channel model and I’m still learning a lot. I want to share some of what I’ve learned with you on this blog. So here’s a first of what I hope to be many posts on this topic. Kenny also has several posts about this subject on his blog.

The WCF runtime consists of two layers: The channel stack and the dispatcher/proxy. The channel stack is responsible for sending/receiving messages between the sender and receiver while the dispatcher/proxy layer is responsible for translating those messages to/from service method calls.

The channel stack is where WCF gets a lot of its power and versatility. It’s similar to the traditional OSI model in that there are cleanly separated layers with each layer using the services of, and only calling on, the next layer down. It’s also very different from the OSI model in that there aren’t n well defined layers each with their specific role. Instead, there are two types of channels with implementations of these types allowed to compose in different ways depending on the application’s needs.

At the bottom of the stack is a transport adapter channel (aka a transport) whose role is to adapt an underlying transport to WCF’s channel model. For example, WCF ships with HTTP, TCP, Named Pipes, Peer to Peer, and MSMQ transport adapters allowing you to use any of these technologies as a means of transporting messages between sender and receiver. Each transport adapter can use an encoder that transforms WCF messages to the appropriate format for the transport. WCF ships with three encoders: text XML, binary XML, and MTOM. You can mix and match transport adapters and encoders unless the transport adapter has some restriction on the type of encoder it needs.

Above the transport adapter is zero or more protocol channels. These are channels that perform some kind of message transformation as a means of communicating with the equivalent protocol channel on the other side. For example, WCF ships with security, reliable messaging, and transaction flow channels. Each of these channels add/read headers to/from the message and some of them modify message content and even send their own messages (reliable messaging sends ACK messages).

One of the key differences between WCF and existing distributed app technologies like ASMX or ES, is this ability to use any transport (through transport adapters) and compose any number of protocol implementations within the communications stack. .NET Remoting has similar features (extensibility allowing you to plug in custom transport channels and chains of message sinks that could act as protocols) but WCF takes this further by providing a richer channel model that captures the fundamental communication patterns (input, output, duplex, request/reply).

So why is all this flexibility and extensibility good? Those of you who know me will know I’m a pragmatic guy so I would not say something is good just because it’s cool technology. WCF is the next generation distributed applications framework. It will be used for a long time and for many diverse scenarios many of which we don’t even know yet. Product extensibility is essential in this case to satisfy such diverse scenarios. For example WCF/WSE 3.0 interop over HTTP is supported by WCF out of the box but there are some scenarios where customers may want to interoperate over TCP. In our PDC talk, Kenny showed a custom TCP transport adapter that allows Indigo to talk to WSE 3.0 over TCP. The fact that it was possible, and relatively easy (OK, Kenny owns our TCP transport adapter so he knows this stuff inside out) to enable such a scenario speaks volumes for WCF’s channel extensibility. Another powerful scenario is when a customer wants to use WCF to communicate with a existing system that exposes its functions only through a proprietary network API. The customer can write a custom transport adapter that adapts the proprietary network API to WCF’s channel model allowing WCF developers to access the existing system without having to learn a new API.

These rich scenarios are what excite me the most about WCF’s channel extensibility. As more customers start playing with WCF, I think we’ll see a proliferation of custom channels both from ISV’s as well as within the enterprise.

Emacs for 2007

yassers929 — Tue, 13 Sep 2005 22:42:00 GMT

Don Box, Chris Anderson, Anders Hejlsberg and Scott Guthrie did a 40 minute demo in Jim Allchin's keynote this morning. The demo started showing off the new Language Integrated Query or LINQ technology. Anders and Don wrote an app to query the list of running process with more than 4MB of working set. They then joined that result with a list of process descriptions from a SQL database. The description for devenv.exe was "Emacs for 2007".
Next, Don and Chris wrote an Indigo service to expose this list of processes to other apps. Then Scott got up and talked about Atlas and showed how easy it is to write an interactive Web page that gets data from an Indigo service. I think there were two key points in this part of the demo that may have been lost:

1. The Web page had a text box and a "Get data" button. Using traditional ASP.NET, clicking on the button would submit a form to the server where the request would be processed (possibly invoking an Indigo service) then the response would be returned as an entire new page to the browser. In this demo using Atlas, client-side javascript sends a request to the Indigo service directly and displays the returned data without reloading the entire page.
2. The format used for communicating between browser and Indigo service is a simple text format known as JavaScript Object Notation or JSON. This means no XML parser is required on the client side. It also means that Indigo is versatile enough to speak HTTP GET and extensible enough to understand non-XML formats.

Just to reiterate, the Web client Scott showed used just JavaScript and a simple text format to talk to Indigo. No SOAP and no XML. This client also works on non IE browsers and was shown running on the Mac in the keynote.
We've done some work on Indigo to support HTTPisms like query string and various HTTP verbs (e.g. GET). This feature (and the Indigo JASON encoder that Steve Maine wrote) is the corner stone of enabling Atlas to communicate with Indigo services. The Indigo PDC bits have this support built in so once you get your hands on them, you can try that feature out and send us feedback.

Bizarre afternoon in L.A.

yassers929 — Tue, 13 Sep 2005 05:08:00 GMT

This morning, I flew out to LA for PDC. The plane landed around 12:15PM, nice and early. The ride to the hotel was mostly uneventful until we got to downtown and I noticed all the traffic lights were flashing red and drivers were confused at every major intersection. After dodging collisions a couple of times, the driver dropped me off at the Westin. I go to check in and they tell me I can't because all of downtown LA has no power. I'm thinking "you've got to be kidding! Does this really happen?". So I go leave my bags with the bell captain and head over to the restaurant for a nice cold salad (no power means no hot food).
About an hour later, power returns and I go to check in. I run into Steve Swartz standing in the check in line and we chat about CommunicationObject. When I finally get to the counter, the lady informs me I was supposed to have arrived yesterday. Apparently, the reservation says I arrive on Sunday and leave the following Sunday but I actually arrived today (Monday) and leave on Friday. I must have entered the wrong dates when I made the reservation. Anyway, there are only smoking rooms now so I settle for one and go up planning to work on my demo.
I set the laptop down and pull the adapter out of the bag only to find it's missing the tip that connects into the laptop is missing (it's a universal adapter with interchangeable tips). I call the concierge and ask for a nearby electronics store she tells me there's one across the street. Great, things are starting to look up. So I walk down to California Computer Center. The helpful clerk there shows me three different Targa adapters only one of which has my laptop listed on the back (I have a Compaq Evo N800w). The clerk says "be sure you get the right one because we don't return them." So I buy it (relying on the list on its back) and walk back to the hotel room. I plug it in, it lights up but my laptop gets no power. I go online and lookup the number for Targa, I call them and they say it looks like a faulty unit so I should exchange it. So I walk back downstairs to CCC and ask for an exchange. They're out. So I argue with them for 15 minutes until I convince them to refund my money. I ask for the nearest Radio Shack which turns out to be 3 blocks away. I walk down there and find they have only one adapter left which happens to work with my laptop. I'm back in business!
Next, came my adventure in getting a Starbucks coffee. Never mind.
Well, despite the rocky start, I'm psyched about PDC and the talk that Kenny and I are giving about writing custom channels for WCF (a.k.a Indigo). I plan to hang out in the Communications Track lounge so stop by and say hi.

Indigo Versioning

yassers929 — Tue, 21 Jun 2005 07:28:00 GMT

I've been thinking about Indigo versioning a lot recently. Basically there are three axes:

1. Versioning of protocol implementation: Say Indigo v1 implements WS-SomeProtocol v1, how do we go about that implementation so that when WS-SomeProtocol v2 comes out and we want to implement it in Indigo v2 we can do so without breaking anything. I believe we have a good answer for this.
2. Versioning of applications: How do app developers version their Indigo apps? This is essentially a set of guidance and some features to support that guidance. I believe we have a good answer for this as well.
3. Indigo side by side support: For various reasons, Indigo needs to support side by side execution. There are two main opinions on what the versioning isolation boundary ought to be:
- One opinion is that the isolation boundary ought to be appdomain. This means you can have two appdomains in the same process and each appdomain running its own version of Indigo. The reasoning is that an Indigo application may want to use a 3rd party component that also uses Indigo. Because the app and the component are versioned independently, they may need different versions of Indigo. The app can load the component in a separate appdomain and use some form of x-appdomain communication mechanism to talk to the component.
- A second opinion is that the isolation boundary ought to be process. This means you can have two processes on the same machine and each process running its own version of Indigo. All appdomains in a process use the same Indigo version. The reasoning is that today the CLR and the .NET Framework are both process-wide (one version per process) so if an application and a component are using different Indigo versions, they may (maybe even likely) be using different versions of the .NET Fx so they can't be loaded in the same process anyway. Also, from an app developer point of view, since I already have to think in terms of process-wide for the .NET Fx, why should I have to think in terms of appdomain-wide for Indigo? It's easier for me as an app developer to just think of one versioning isolation boundary which, today, is process.

What are your thoughts on #3? Have you versioned applications from .NET Fx 1.0 to 1.1? to 2.0? How did you do it/plan to do it?

How do folks deal with moving from one ASP.NET version to the next? Do you upgrade the entire machine or do you run multiple application pools each with its own ASP.NET version?

I'd love to get your thoughts on this. Please email yassers _at_ microsoft _dot_ you know what.

It's fun being a test engineer on Indigo

yassers929 — Tue, 22 Mar 2005 20:33:00 GMT

Recently, I spent a few days writing test cases for the MetadataExchange endpoint feature in Indigo. This started as a way to help out our overly thin-stretched test team so that we can get this feature in Indigo Beta1. It has since turned into an educational experience for me.

It’s increased my, already tremendous, appreciation of the work our testers do. Not only do they think about and design test cases to achieve optimum coverage with minimal number of test cases, they also write and maintain sophisticated distributed test frameworks (complete with distributed shared state and synchronization) for automating Indigo tests. This framework is so versatile and powerful that I think it would make a useful product for anyone testing distributed applications.

Going forward, this is experience will be extremely useful for me when making test case suggestions to our test team and when reviewing test specifications.

Want to join the Indigo test team? See Indigo Needs Testers.

ASMX documentation

yassers929 — Thu, 27 Jan 2005 01:35:00 GMT

I'm currently reviewing the docs for Whidbey version of ASMX (ASP.NET Web services). What are the top 1 - 3 ASMX docs topics that you'd like to see improved?
I'm enabling comments for a while so you can reply to this. I'm sure I'll have to turn them off again though because I get spammed.

Disabling Comments

yassers929 — Fri, 31 Dec 2004 19:06:00 GMT

My blog has recently been subjected to "comment spam" so I'm disabling comments again. It's a shame because I love to hear your opinions/questions but I have over 300 spam comments just in the last week.

Let's all become cavemen

yassers929 — Thu, 11 Nov 2004 05:11:00 GMT

I just came across a statement made by a prominent industry figure about how WS-* is becoming so complex that it is not practical to implement a WS-* architecture with no tool/stack support. Well, what's wrong with relying on tools/stacks? Isn't that how humans have made progress? That is, a specialized few focus on a complex area and produce a useful package that can be used by others in tackling completely different complexity etc. Does a construction engineer have to understand the inner workings of all construction tools? Seems to me that if we start eschewing tools as a principal, we will all end up more or less cavemen. I'd rather use tools.

Web services features in .NET 2.0 (VS Connections talk)

yassers929 — Thu, 11 Nov 2004 01:24:00 GMT

My VS Connections talk's slides and code are available here.

Web services and large content in .NET 2.0

yassers929 — Wed, 10 Nov 2004 19:05:00 GMT

There are a couple of features in .NET 2.0 (Whidbey) that help solve the problems of dealing with large data. The two main issues with sending large data in Web service messages are: 1. working set (memory) due to buffering by the serialization engine and 2. bandwidth consumption due to 33% inflation after Base64 encoding.

Chunking to constrain working set
First, let's talk about working set. In .NET 2.0, you can implement IXmlSerializable on both client and server side and use that to chunk data to/from the network. This allows you send/receive arbitrarily large content without consuming arbitrarily large working set. Here's an example:

On the server side, here's what you do to chunk large content back to the client:

First: The Web method needs to turn off ASP.NET buffering and return a type that implements IXmlSerializable:

[WebMethod]

[SoapDocumentMethod(ParameterStyle= SoapParameterStyle.Bare)]

public SongStream DownloadSong(DownloadAuthorization Authorization, string filePath)

{

//turn off response buffering

HttpContext.Current.Response.Buffer = false;

//return song

SongStream song = new SongStream(filePath);

return song;

}

Second: The type that implements IXmlSerializable needs to send chunks in its WriteXml method:

[XmlSchemaProvider("MySchema")]

public class SongStream : IXmlSerializable

{

private const string ns = "http://demos.teched2004.com/webservices";

private string filePath;

public SongStream()

{

//default constructor for serializer

}

public SongStream(string filePath)

{

this.filePath = filePath;

}

public static XmlQualifiedName MySchema(XmlSchemaSet xs)

{

//this method is called by the framework to get the schema for this type

// here we return an existing schema from disk

XmlSerializer schemaSerializer = new XmlSerializer(typeof(XmlSchema));

string xsdPath = null;

xsdPath = HttpContext.Current.Server.MapPath("SongStream.xsd");

XmlSchema s = (XmlSchema)schemaSerializer.Deserialize(

new XmlTextReader(xsdPath), null);

xs.XmlResolver = new XmlUrlResolver();

xs.Add(s);

return new XmlQualifiedName("songStream", ns);

}

void IXmlSerializable.WriteXml(System.Xml.XmlWriter writer)

{

//this is the chunking part

//note that ASP.NET buffering must be turned off for this to really chunk

int bufferSize = 4096;

char[] songBytes = new char[bufferSize];

FileStream inFile = File.Open(this.filePath, FileMode.Open, FileAccess.Read);

long length = inFile.Length;

//write filename

writer.WriteElementString("fileName", ns, Path.GetFileNameWithoutExtension(this.filePath));

//write size

writer.WriteElementString("size", ns, length.ToString());

//write song bytes

writer.WriteStartElement("song", ns);

StreamReader sr = new StreamReader(inFile, true);

int readLen = sr.Read(songBytes, 0, bufferSize);

while (readLen > 0)

{

writer.WriteStartElement("chunk", ns);

writer.WriteChars(songBytes, 0, readLen);

writer.WriteEndElement();

writer.Flush();

readLen = sr.Read(songBytes, 0, bufferSize);

}

writer.WriteEndElement();

inFile.Close();

}

System.Xml.Schema.XmlSchema IXmlSerializable.GetSchema()

{

throw new System.NotImplementedException();

}

void IXmlSerializable.ReadXml(System.Xml.XmlReader reader)

{

throw new System.NotImplementedException();

}

On the client side, here's what you do to chunk large content from the server:

First: The Web service method named DownloadSong in the proxy must be changed to return a type that implements IXmlSerializable. You can use a SchemaImporterExtension to do this automatically when the proxy is generated, for now, let's just assume you somehow edited the proxy code and changed it to return a type named SongFile which resides on the client and implements IXmlSerializable.

Second: The type used on the client that implements IXmlSerializable (in this case called SongFile) needs to read chunks from the network stream and write them to disk to avoid arbitrarily large buffers. This particular implementation also fires progress events which can be used to update a UI e.g. a progress bar.

public class SongFile : IXmlSerializable

{

public static event ProgressMade OnProgress;

public SongFile()

{

}

private const string ns = "http://demos.teched2004.com/webservices";

public static string MusicPath;

private string filePath;

private double size;

void IXmlSerializable.ReadXml(System.Xml.XmlReader reader)

{

reader.ReadStartElement("DownloadSongResult", ns);

ReadFileName(reader);

ReadSongSize(reader);

ReadAndSaveSong(reader);

reader.ReadEndElement();

}

void ReadFileName(XmlReader reader)

{

string fileName = reader.ReadElementString("fileName", ns);

this.filePath =

Path.Combine(MusicPath, Path.ChangeExtension(fileName, ".mp3"));

}

void ReadSongSize(XmlReader reader)

{

this.size = Convert.ToDouble(reader.ReadElementString("size", ns));

}

void ReadAndSaveSong(XmlReader reader)

{

FileStream outFile = File.Open(

this.filePath, FileMode.Create, FileAccess.Write);

string songBase64;

byte[] songBytes;

reader.ReadStartElement("song", ns);

double totalRead=0;

while(true)

{

if (reader.IsStartElement("chunk", ns))

{

songBase64 = reader.ReadElementString();

totalRead += songBase64.Length;

songBytes = Convert.FromBase64String(songBase64);

outFile.Write(songBytes, 0, songBytes.Length);

outFile.Flush();

if (OnProgress != null)

{

OnProgress(100 * (totalRead / size));

}

else

{

break;

}

outFile.Close();

reader.ReadEndElement();

}

public void Play()

{

System.Diagnostics.Process.Start(this.filePath);

}

By the way, you can implement IXmlSerializable today in .NET 1.0 and 1.1 and use it to chunk content. In .NET 2.0 we improve the experience by allowing you to emit a meaningful schema and introducing SchemaImporterExtensions which allow you to generate the right code on the client side.

Reducing Bandwidth Utilization
IIS 6.0 makes it easy to compress replies including Web services replies. In .NET 2.0, the client will automatically tell the server that it accepts gzip compression and it will automatically decompress replies. This lets you send Base64 encoded data (text) and compress it which can reduce its size by up to 10 : 1. Note that we don't have a client side compression feature so if you are sending Base64 data from the client and you want to compress it, you would need to use 3rd party compression library or roll your own.