mjm

Auto-Open (part 1)

2006-11-13T19:38:00Z

Several objects in WCF support ICommunicationObject. ICommunicationObjects have a well-defined life-cycle:

Created -> Opening -> Opened -> Closing -> Closed

There is also a Faulted state and a few more transitions than these, but they are not relevant to auto-open.

Most ICommunicationObjects can only be used in the Opened state. For example, most channels will throw if you try to Send a Message when the channel is not Opened.

However, for usability, a few objects support auto-open. This feature allows you to call an I/O method while the object is in the Created (or Opening) state. The object's I/O methods ensure that the object transitions to the Opened state before calling the inner channel. Clients generated by svcutil.exe, client channels created by ChannelFactory<T>, and ChannelFactory<T> itself all support auto-open.

Normally this is great. The channel stack is very explicit about state transitions, which improves debugging and simplifies the contract that channel authors must implement. At the same time, the user experience is very simple--create a thing, configure it, and go.

There is one interesting implementation detail though, that complicates this picture. It does not usually cause problems in real applications but does cause confusion among people experimenting with WCF sessions and concurrency.

The short story is that if you are using a session and you need to send multiple messages concurrently and you care that the first couple messages are actually sent concurrently, you should not use auto-open. Instead, you should explicitly Open your client/channel before using it from multiple threads.

The long story is... longer.

If you use ChannelFactory<T> to create a channel, or new a svcutil-generated client, the top-most layer of the channel stack is ServiceChannel. ServiceChannel is an internal class that is added automatically by ChannelFactory<T>. Its main job is translating typed calls to untyped ones. For example, when you call this:

channel.SubmitPurchaseOrder(purchaseOrder);

ServiceChannel does some work and then calls this:

channel.Send(message);

The interesting implementation detail enters because ServiceChannel both supports auto-open and does not know whether its user cares about Message ordering. "What have these to do with each other?" you ask. Excellent question--I'm glad you asked.

Let's look at how auto-open could work. The simplest implementation would just check this.State and call Open if it is Created. This does not work for multiple threads though, because it may result in multiple Open calls.

One improvement would be to allow the first request to proceed, but then block all subsequent requests until Open returns. The problem with this approach is that when the first call releases the lock, a second call that came in later may race ahead and call Send on the underlying channel first. This could happen even if the application uses WS-RM with OrderedSession enabled. That would be bad, because it would mean ServiceChannel broke RM's guarantee. One could argue that this is okay, since the user has multiple threads and therefore the order never was deterministic. However, deterministically ordered async calls can cause the same problem.

To fix this, as long auto-open might be blocking a previous request, ServiceChannel must block each subsequent request until it is sure that all those previous requests have called the inner channel. The only way to be sure that previous requests have called the inner channel is to wait until the [Begin]{Send|Request} call has returned. However, ServiceChannel does not have to do this indefinitely. Once there are no pending calls, there can be no previous blocked calls, so subsequent calls can run freely.

This means that the first couple calls on a channel may get synchronized, but after that things should open up. If you want to avoid that initial congestion, you can explicitly open the channel.

ServiceChannel could have supported ordering only when the binding says it also supports ordering. However, it turns out most sessionful bindings support ordering so we would not gain much. Ideally ServiceChannel could get an explicit statement from the developer (like an attribute property) that says, "Yes, I care about ordering." There are several problems with that though--one is that this information is most useful on the client but that would require a WCF-specific extension to metadata to get it there.

Passing objects by reference in Indigo

2005-05-12T18:41:00Z

I'll be honest. The feature I really want to talk about in this part of Indigo is instancing and Duplex channels, but I'll do that soon. I have heard concerns about Indigo's support for distributed objects. I hope to address those concerns as well as provide some background for discussing Duplex.

I should begin by saying that passing object references remotely is bad. Don't ever do this. Shame on you for reading this blog entry. Passing objects by reference kills scalability. Passing objects by reference introduces coupling and therefore makes your application more fragile. "Ryan" passed objects by reference once--that very day his Smartphone met an unfortunate end, he got a speeding ticket, and dandelions took over his front yard. You get the idea.

Of course I'm mostly kidding, except for the part about losing scalability and having fragile apps. Now that the Best-Practice-Police are satisfied, let's see what kind of trouble we can get into.

Reference Passing in .NET Remoting

Using .NET Remoting to pass references is simple, automatic, and completely transparent to the caller. All I need to do is create an object whose type derives from MarshalByRefObject. I can then call a method on a proxy and pass the object by reference:

proxy.Method(myMarshalByRefObject);
myMarshalByRefObject.Callback("local?");

The implementer of Method() can take the object and start using it:

void Method(MyMarshalByRefObject myMarshalByRefObject)
{
    myMarshalByRefObject.Callback("remote?");
}

Because we passed a reference to Method(), both Callback()s execute on the same CLR object. .NET Remoting does a lot of work to make this seamless, and it is pretty amazingly cool. Of course, there is a downside--it is so easy you might even do it accidentally. (Did you know System.Uri is MarshalByRefObject? Oops.)

Reference Passing in Indigo

This could make an interesting out-of-context sound-bite: "If you want to do bad things, Indigo makes your job harder." Indigo does require that you be explicit when you want to pass an object by reference. The server side is not too bad--just add this attribute to your service class:

[ServiceBehavior(InstanceMode=InstanceMode.SharedSession)]

This turns on all the server-side infrastructure goo that lets you connect new channels to objects that already exist.

On the client, a serializable object reference in Indigo looks like an EndpointAddress. This is true both in code and on the wire. Typically the EndpointAddress will be the service's EndpointAddress plus one or more InstanceHeaders. Indigo provides an explicit method to convert from a proxy to a serializable EndpointAddress:

EndpointAddress address = ((IProxyChannel)proxy).ResolveInstance();

This is a lossy transformation--type/contract information is not preserved. Indigo also provides an explicit method to convert from a deserialized EndpointAddress to a proxy:

IHello proxy = ChannelFactory.CreateChannel<IHello>(address);

Here is a sample that demonstrates this conversion. I traded realism for simplicity, so you have to use your imagination a bit. The SharedSessionCounter has a name and keeps a running total. The sample creates two of these counters, A and B, and always increments A by 1 and B by 10 to illustrate what is happening. Initially it creates the two objects, then pretends to pass both by reference as EndpointAddresses, then connects a new channel to each object.

Object Reference.

Casting

Suppose I have a proxy of type IHello, and the service also implements IHiThere. How do I cast the IHello to IHiThere? DON'T! I would not recommend this, since the client would have an implicit dependency on the internal implementation of the service. Instead, be explicit and have your service implement a third contract that combines the other two:

[ServiceContract]
interface IGreeting : IHello, IHiThere
{
}

Then you can create an IGreeting channel and do casting (if needed) locally.

If for some reason you really must connect to the same instance using a different contract, you can call CreateChannel again with a different channel type. Note, though, that if something goes wrong here you will only find out by sending a message, and even then it will be difficult to find out programmatically.

Lifetime

In books about distributed object technologies, lifetime management generally takes at least a chapter to explain. I'm sure Indigo will be no exception. However, here is an attempt to summarize how instances shut down when using SharedSession with a small collection of haiku:

Once all the channels
Go away, the timer starts.
Connect soon or lose.

When timer expires,
Ask extensibility.
Can this instance live?

Now that all agree,
Farewell, my little object.
Mizu no Oto.

Caveat 1: If new work comes in before the object goes away, the process starts over.
Caveat 2: If someone explicitly calls Close(), BeginClose() or Abort(), the ServiceSite starts closing immediately.
Caveat 3: There is a reason I was not an English major.

By the way, in case the InstanceMode value names don't make sense, they have to do with the lifetime of the instance. So PerCall means the instance lasts as long as the call. PrivateSession means the instance is not shareable and lasts as long as the sessionful channel. SharedSession means the instance is shareable and lasts as long as there are sessionful channels connected. And of course, Singletons last "forever", for sufficiently small values of forever.

Alternate EndpointAddress

Suppose I have a service that listens on multiple addresses. If you were watching closely, perhaps it bothered you that ResolveInstance returns just one address. Which one is it? What about the others?

Well, it isn't pretty. IProxyChannel has a collection of headers for exactly this purpose. Here's a method you can use to make it easier:

EndpointAddress GetInstanceAddressForEndpoint(IProxyChannel proxy, EndpointAddress endpoint)
{
    // Populate the headers
    proxy.ResolveInstance();

    // Build the address:
    EndpointAddressBuilder builder = new EndpointAddressBuilder(endpoint);
    builder.InstanceHeaders.AddRange(proxy.InstanceHeaders);

    // This is the new address:
    EndpointAddress instanceAtEndpoint = builder.ToEndpointAddress();
    return instanceAtEndpoint;
}

Here is the previous sample, modified to create the initial channels over TCP/IP and then connect the second channels using Named Pipes. It uses an overload of the method above to manage this binding change. I tried to minimize changes, so looking at a diff between this and the previous sample may be helpful.

Object Reference with Multiple Bindings.

Interoperability

Note that ResolveInstance uses a protocol that is not meant to be particularly interoperable. I doubt it will ever be in a WS-* spec. That said, it should not be too difficult to write an interface or WSDL that expresses this protocol. Of course, I haven't tried that yet.

More than you ever wanted to know about reference passing in Indigo...

This posting is provided "AS IS" with no warranties, and confers no rights.

Asynchronous Programming in Indigo

2005-05-04T17:22:00Z

In this post I am going to talk about the async programming model in Indigo. Indigo supports the Asynchronous Programming pattern common in the .NET Framework on the client and the server. An overview of how to use the Asynchronous Programming pattern in the .NET Framework can be found here.

As a concrete example, suppose we have the following contract:

[ServiceContract]
interface ISleep
{
    [OperationContract]
    void Sleep(int milliseconds);
}

Indigo supports expressing this contract on the client or server as:

[ServiceContract]
interface ISleep
{
    [OperationContract(AsyncPattern=true)]
    IAsyncResult BeginSleep(int milliseconds, AsyncCallback callback, object state);

    void EndSleep(IAsyncResult result);
}

On the client, this is great for introducing parallelism without getting too wacky with threads, though there are some interesting issues we'll discuss later.

On the server, this is non-trivial to implement, but once you do you can do long-running asynchronous I/O operations without eating a thread the whole time. This turns out to be useful when building scalable services.

Here is a sample that shows a client making async calls on a sync server. The output shows that the calls run concurrently on the server, even though the client uses just one thread. At the end, the client just calls End to wait for the results.

Client Async

Here is another sample that shows an asynchronous middle-tier service. Both the client and the backend server are implemented synchronously. The output demonstrates that a thread is only active on the middle tier just before and just after the backend server call.

Middle-tier Async

Sync vs. Async is a Local Thing

As you might expect, Indigo allows the same operation to be defined both synchronously and asynchronously on the same interface. However, it may surprise you that calling either from the client results in the same message sent to the server. This has interesting implications.

One implication is that an async client can talk to a sync server and vice versa, as shown in the samples above.

Another is that if the server defines sync and async variants of the same operation, we have a demux problem. Since both versions of the method have the same wire format, nothing about the message says which one to call. In this case, Indigo chooses the synchronous version because the synchronous version generally results in higher throughput.

Sometimes people expect the sync method on the client to invoke the sync method on the server, and the async method on the client to invoke the async method on the server. This is NOT how it works.

If you care about whether Indigo invokes you synchronously or asynchronously (and you probably should), you need to ensure that the interface you implement only has one variant of each method. Since async is local in Indigo, you can still have both variants on the client.

How does this work?

Let's dissect at a simple async call. To illustrate and keep the discussion simple, I am assuming an async request-reply scenario occuring on the client using TCP, where the caller passes a callback.

The pseudocode below tries to show a thread's callstack over time. Time progresses vertically, and entering/exiting a scope indicates calling/returning from a method.

Client

Calls to an async proxy method ultimately resolve down to an async I/O. If the I/O does not complete synchronously, call returns control to the caller, who goes back to doing whatever it wants.

...
proxy.BeginCall()
{
    channel.BeginSend()
    {
        Start async I/O         ---Request-->
    }
    channel.BeginReceive();
}
...

At some point (if all goes well), a reply comes back and the BeginReceive completes. This calls the callback that the proxy passed to BeginReceive, which can then call the user's callback:

I/O-CompletionPortCallback()    <---Reply---
{
    ReceiveCallback()
    {
        channel.EndReceive();

        UserCallback()
        {
            Message reply = proxy.EndCall()
            ...
        }
    }
}

This leaves out a lot of details, but hopefully gives you some idea of what is going on here.

Stack Dive

Before we look at the server, I want to digress into an issue that can plague naive uses of the async pattern.

Thinking about the pseudocode above, imagine if one of the things in the "..." inside UserCallback() was calling BeginCall again. This is probably a bug. I/O may complete synchronously, which results in calling I/O-CompletionPortCallback inside of the channel.BeginSend call.

There are scenarios where this could end up recursing indefinitely. Recursing indefinitely is generally a Bad Idea.

This is what the CompletedSynchronously property on IAsyncResult is for. In your callback, you can continue working if IAsyncResult.CompletedSynchronously is false on your parameter. In your caller, you can continue working if IAsyncResult.CompletedSynchronously is true on the return value. This prevents unwanted recursion.

Of course, if you don't do anything interesting in your callback, you don't have to worry about this.

Here is an example (minus error handling) of how to do an infinite loop calling an async method repeatedly while avoiding stack dive:

void AsyncLoop()
{
    for (;;)
    {
        IAsyncResult result = proxy.BeginCall(this.Callback, proxy);

        if (!result.CompletedSynchronously)
            break;

        this.InvokeEnd(proxy, result);
    }   
}

void Callback(IAsyncResult result)
{
    if (result.CompletedSynchronously)
       return;

    this.InvokeEnd((IMyProxyType)result.AsyncState, result);
    this.AsyncLoop();
}

void InvokeEnd(IMyProxyType proxy, IAsyncResult result)
{
    proxy.EndCall(result);
}

Server

On the server, when a message arrives for your service operation, Indigo calls your corresponding Begin method and passes a callback. At this point, you own the request--assuming you don't complete synchronously, you can return the thread to the system. You are responsible for making sure that something wakes up eventually that calls the callback.

If you are implementing an async server method, it is important to set CompletedSynchronously correctly before letting Indigo see the IAsyncResult. Setting CompletedSynchronously to true means you are calling the callback method on the same thread that called your Begin method. It is also important that the IAsyncResult returned from Begin is the same IAsyncResult passed to the callback. Errors in these cases typically result in a hang or an Exception.

Exceptions

How do Exceptions work with all of this? It turns out that Exceptions can sometimes come back from the Begin call. This means whatever Exception handling you are doing has to be done in two places instead of one. In case you were wondering why I broke out InvokeEnd() above as a separate method, this allows you to put your Exception handling for the EndCall in a single method. So Begin and End can both throw on client and server.

What about the callback method? If I invoke a callback method when an async method completes and it throws an Exception, what does that Exception mean? I'm supposed to be doing the Exception throwing, since I implemented the method. Normally I would throw an Exception back to my caller from End. However, the callback is my only mechanism for telling my caller to call End, and it just threw. At this point, neither of us is able to recover the system to a known Correct state.

Should I let it go? Nobody is below me on the stack to handle that Exception. (Or worse, someone is on the stack eating all Exceptions and the Exception silently disappears).

Should I try calling again? I am likely to end up with more (possibly unbounded) Exceptions.

Should I swallow all Exceptions? This is a possibility, but extremely dangerous in cases where the Exception is AccessViolation or something else that could result in data corruption.

Indigo Beta 1 rethrows the Exception on a clean thread, with the goal of triggering the default CLR behavior for unhandled Exceptions. By default this unloads the AppDomain or process. However, I believe there is a way to configure a standalone App to behave differently. In WebHost (or as a Windows Service), this works well, since ASP.NET just spins up a new AppDomain. In self-host this is not quite as nice (your process exits), but at least you didn't go on to corrupt your database or open a security hole in your application.

Indigo will not normally throw Exceptions at you from its callback on the server. We provide a custom error handling mechanism for the application to handle Exceptions, and we use that to avoid throwing Exceptions here. Of course, an error handler may throw, or you may get catastrophic Exceptions like OutOfMemory or ThreadAbort, but these should not be handled. Therefore, I believe it is reasonable for async server implementations to either let any Exceptions fall through, or throw the Exception on a clean thread.

Threading considerations

One of the features of Indigo I hope to talk more about later is its integration with the CLR SynchronizationContext. I want to just briefly mention that Indigo does capture and use the SynchronizationContext to ensure that the callback for an async operation does occur on the correct thread.

What this means for you is that if you are using Indigo from your Avalon or WinForms application, and you do an async call to Indigo, you don't have to do any crazy threading stuff like control.Invoke if you call UI components from the callback thread.

Hopefully this gives you some idea of how the .NET Framework Asynchronous Programming pattern fits into Indigo.

This posting is provided "AS IS" with no warranties, and confers no rights.

The Obligatory Self-Introduction

2005-05-04T17:06:00Z

Hi!

My name is Michael Marucheck. I am a developer at Microsoft working on Indigo. My focus is the ServiceModel runtime. This is where instancing, threading, throttling, context, serialization, proxy/stub, several extensibility points, and things like ServiceHost and ServiceSite live.

It feels a bit presumptuous to write a blog. Several blogs I read are written by far more interesting and intimidating people than myself. Although I have (sort of) maintained a personal web page for about a decade now, blogging somehow feels different.

At the same time, I am very excited to finally get to talk to people about this project I have worked on for the past few years. Indigo lets you do some pretty cool stuff, and it's great to get to explain some of what it does, how it works, and why it works that way.

This posting is provided "AS IS" with no warranties, and confers no rights.