Andrew Arnott : WCF

GZip encoder for Compact WCF

andarno — Wed, 01 Oct 2008 05:13:07 GMT

The GZip encoder is not included in NetCF as it is on the desktop WCF, but it isn't hard to build yourself and in fact we have released a sample of exactly how to add your own GZip encoder to NetCF:

http://go.microsoft.com/fwlink/?LinkId=108652

So many people have asked me for it lately I thought I'd better just post the link. Have fun!

Using the XmlSerializer as an XmlObjectSerializer with WCF

andarno — Fri, 18 Jan 2008 23:06:54 GMT

The Windows Communication Foundation introduced the new DataContractSerializer to replace the XmlSerializer in many scenarios. The DataContractSerializer is faster than the XmlSerializer, but has certain limitations. For example it does not support serializing an object's members as XML attributes. When you write a WCF client that calls a service, WCF will generate a proxy class that defaults to using the DataContractSerializer and falls back to the XmlSerializer if the service uses features that the DataContractSerializer does not support. This fallback mechanism is automatic if you're using WCF's service model.

But if you're working at WCF's messaging layer, you must explicitly choose which serializer to use. You can use any serializer that derives from System.Runtime.Serialization.XmlObjectSerializer. System.Runtime.Serialization.DataContractSerializer already derives from this class -- but System.Xml.Serialization.XmlSerializer does not. So you have some work to do before you can use the XmlSerializer as your serializer of choice at the messaging layer of WCF.

In order to use the XmlSerializer to send messages with WCF, we have to define a class that derives from XmlObjectSerializer and uses the XmlSerializer internally. In essence we will create a wrapper class for the XmlSerializer that makes it look and work like an XmlObjectSerializer. XmlObjectSerializer is an abstract class with only a few methods we need to implement.

Below is a sample of a class that will do just that.

namespace Microsoft.ServiceModel.Samples
{
    using System;
    using System.Diagnostics;
    using System.IO;
    using System.Text;
    using System.Xml;
    using System.Xml.Serialization;
    using System.Runtime.Serialization;

    public class CFMessagingSerializer : XmlObjectSerializer
    {
        readonly Type objectType;
        XmlSerializer serializer;

        public CFMessagingSerializer(Type objectType)
            : this(objectType, null, null)
        {
        }
        
        public CFMessagingSerializer(Type objectType, string wrapperName, string wrapperNamespace)
        {
            if (objectType == null)
                throw new ArgumentNullException("objectType");
            if ((wrapperName == null) != (wrapperNamespace == null))
                throw new ArgumentException("wrapperName and wrapperNamespace must be either both null or both non-null.");
            if (wrapperName == string.Empty)
                throw new ArgumentException("Cannot be the empty string.", "wrapperName");
            
            this.objectType = objectType;
            if (wrapperName != null)
            {
                XmlRootAttribute root = new XmlRootAttribute(wrapperName);
                root.Namespace = wrapperNamespace;
                this.serializer = new XmlSerializer(objectType, root);
            }
            else
                this.serializer = new XmlSerializer(objectType);
        }

        public override bool IsStartObject(XmlDictionaryReader reader)
        {
            throw new NotImplementedException();
        }

        public override object ReadObject(XmlDictionaryReader reader, bool verifyObjectName)
        {
            Debug.Assert(serializer != null);
            if (reader == null) throw new ArgumentNullException("reader");
            if (!verifyObjectName)
                throw new NotSupportedException();

            return serializer.Deserialize(reader);
        }
        
        public override void WriteStartObject(XmlDictionaryWriter writer, object graph)
        {
            throw new NotImplementedException();
        }

        public override void WriteObjectContent(XmlDictionaryWriter writer, object graph)
        {
            if (writer == null) throw new ArgumentNullException("writer");
            if (writer.WriteState != WriteState.Element)
                throw new SerializationException(string.Format("WriteState '{0}' not valid. Caller must write start element before serializing in contentOnly mode.",
                    writer.WriteState));
            using (MemoryStream memoryStream = new MemoryStream())
            {
                using (XmlDictionaryWriter bufferWriter = XmlDictionaryWriter.CreateTextWriter(memoryStream, Encoding.UTF8))
                {
                    serializer.Serialize(bufferWriter, graph);
                    bufferWriter.Flush();
                    memoryStream.Position = 0;
                    using (XmlReader reader = new XmlTextReader(memoryStream))
                    {
                        reader.MoveToContent();
                        writer.WriteAttributes(reader, false);
                        if (reader.Read()) // move off start node (we want to skip it)
                        {
                            while (reader.NodeType != XmlNodeType.EndElement) // also skip end node.
                                writer.WriteNode(reader, false); // this will take us to the start of the next child node, or the end node.
                            reader.ReadEndElement(); // not necessary, but clean
                        }
                    }
                }
            }
        }

        public override void WriteEndObject(XmlDictionaryWriter writer)
        {
            throw new NotImplementedException();
        }
        
        public override void WriteObject(XmlDictionaryWriter writer, object graph)
        {
            Debug.Assert(serializer != null);
            if (writer == null) throw new ArgumentNullException("writer");
            serializer.Serialize(writer, graph);
        }
    }
}

.NET Compact Framework 3.5 Power Toys RTW

andarno — Tue, 11 Dec 2007 01:49:10 GMT

The .NET Compact Framework 3.5 Power Toys have been released. You can download them from here.

Tools included in this package:
Remote Performance Monitor and GC Heap Viewer – Provides real time counter data (ranging from Garbage Collector activity to type loading info) on a running NETCF application. The GC Heap Viewer feature allows you to capture the managed heap at any moment your app is running to view live references, and allows you to compare multiple snapshots to find memory leak issues.
NETCF CLR Profiler – CLR Profiler is an instrumenting allocation profiler for NETCF applications. It provides detailed allocation visualizations, allocation callstacks visualizations and useful for diagnosing memory management issues.
App Configuration Tool (NetCFcfg.exe) - On-device tool for specifying what version of the NETCF runtime an application will run against, displaying installed versions of NETCF and displaying info about DLLs in the GAC.
NETCF ServiceModel Metadata Tool – The .NET Compact Framework ServiceModel Metadata Tool (netcfsvcutil.exe) allows you to generate a Windows Communication Foundation (WCF) client proxy to help developers consume WCF services on device. Like svcutil.exe, which is the desktop version of the utility, netcfsvcutil.exe is a command-line tool that generates service model code from metadata documents and generates metadata documents from service model code.
Remote Logging Configuration Tool – The Logging Configuration Tool enables users to easily configure logging options on a NETCF device including: loader, interop, network, error and finalizer logs.
NETCF Network Log Viewer – A utility for viewing NETCF network log data.

Why .NET Compact Framework fails to call some HTTPS web servers

andarno — Tue, 20 Nov 2007 01:18:04 GMT

A bug was discovered recently in the .NET Compact Framework version 2.0 (SP2 and earlier) and 3.5 that causes HttpWebRequest's directed at some HTTPS servers to fail with this error: (for web services)

System.Net.WebException: Unable to read data from the transport connection. ---> System.Net.Sockets.SocketException: Unknown error (0x0).
   at System.Net.HttpWebRequest.fillBuffer(HttpWebRequest request, Connection connection, CoreResponseData data)
   at System.Net.HttpWebRequest.getLine(HttpWebRequest request, Connection connection, CoreResponseData data)
   at System.Net.HttpWebRequest.parseResponse(HttpWebRequest request, Connection connection, Boolean defaultKeepAlive)
   at System.Net.HttpWebRequest.startReceiving(Connection connection)
   at System.Net.Connection.startReceiving(Object ignored)
   at System.Threading.ThreadPool.WorkItem.doWork(Object o)
   at System.Threading.Timer.ring()

   at System.Net.HttpWebRequest.finishGetResponse()
   at System.Net.HttpWebRequest.GetResponse()
   // ...
   at Program.Main(String[] args)

For standard HttpWebRequest calls from your app the stack trace would be different, although the exception message and cause would be the same.

While there are multiple reasons this exception can be produced, one thing that can cause it is if the server sends an empty encryption packet to the NetCF client. Here is an illustration of the encryption process that occurs on the server for HTTPS:

First, a memory buffer is initialized with unencrypted data that the server wants to send to the client over the network.

Step 1:

unencrypted data

Next, the server makes a call to EncryptMessage and the unencrypted data is encrypted in place. A header and footer is added to the buffer:

Step 2:

header

encrypted data

footer

This header if 5 bytes long, and the footer is several more bytes. This entire packet, which is necessarily longer than the original unencrypted data, is then sent down the wire where the process is reversed using a call to DecryptMessage.

The trouble with NetCF's SSL stack is when the server encrypts a buffer of zero length and sends it to the client:

Step 1:		0 bytes of unencrypted data
Step 2:	header	encrypted representation of a zero-length buffer	footer

Although the original data had a length of zero, the encrypted version of it actually has a non-zero length. When this packet is sent to a NetCF client, current versions of NetCF decrypt the packet and return a zero length buffer to the caller. The semantics of a network Read method is traditionally that it blocks until some data is received, and if a zero length buffer is returned it is the sign that the socket was closed. Because NetCF will return an empty buffer after decrypting an "empty" encryption packet, the caller may misinterpret this as a sign of a disconnected socket and terminate the connection.

In fact this is exactly what happens in NetCF's web services code when calling services over SSL that respond with empty encryption packets. As a result the connection fails before the response is fully received and an exception is thrown.

What causes the server to send these empty encryption packets? Technically it's legally allowed for these empty packets to be sent (although they are pointless), so depending on your server and its configuration this may never happen or it may happen regularly.

Unfortunately there is no way to get NetCF to ignore these packets, so the only workarounds are these, in order of simplicity:

Don't use SSL to make your request (security implications here of course by sending your request and its response in clear text)
Reconfigure your existing server in some way to get it to avoid generating these empty encryption packets.
Build a new web server that will forward your device's requests on a separate connection to the ultimate target server and forward the responses back to the device. This would work for web sites and web services. But this new front-end server (which serves something of the role of a proxy server) would have to be configured to not generate these empty encryption packets.
[Added 1/2/08] Build a device-side web proxy that calls into the SSPI functions itself (native would probably be easier than managed for this). Then have your managed app call into the proxy. This proxy would be responsible for consuming the empty packets and re-encrypting everything (if necessary to secure IPC) for the app on the same device without any empty SSL packets.
Wait until a future version of NetCF that fixes this bug.
Write your own HTTPS client using native code or P/Invoke'ing from NetCF (difficult to get right).

So how do you know for sure if the HTTPS server is sending empty encryption packets that are causing your NetCF clients to fail? I wrote a native C++ app that calls a web server on an HTTPS connection and detects these packets. (No, I'm afraid it can't filter them out for NetCF clients) By running this app against your web server the app will tell you whether your server generates these empty packets. At least then you know what's going on.

[Updated Feb 1, 2008] You can download the source code from its resource page on the MSDN Code Gallery.

What do you think of the new WCF 'Store and forward' Mail Transport?

andarno — Mon, 29 Oct 2007 22:53:53 GMT

With the .NET Compact Framework 3.5 release which comes with Visual Studio 2008, a new Windows Communication Foundation transport is introduced that uses email as the communication mechanism.

Others have already blogged about this new transport including Roman Batoukov and David Kline.

I'd like to take a poll of the audience to see how you like the transport, including the documentation and samples that we ship with it. Can you think of uses for an email transport? Do you have any trouble figuring out how to use it? Will you use it exclusively on desktop machines, mobile devices or both?

Please comment below and let me know what you're doing with it! We'd like to know about any pain points you run into so we can fix it in the next version.

Calling WCF services from NetCF 3.5 using Compact WCF and NetCFSvcUtil.exe

andarno — Thu, 13 Sep 2007 02:29:00 GMT

The .NET Compact Framework 3.5 adds a subset of the Windows Communication Foundation (WCF or "Indigo") to smart devices, allowing them to communicate with desktop WCF components with all the flexibility of multiple, interchangeable service bindings and endpoints. Although it ships out of the box only with support for message-level communication (no service calls, per se), we have a code generation tool that will allow you to call WCF or other WSDL-publishing services.

Desktop has an svcutil.exe tool that generates client proxy code, but the source code it generates cannot compile against NetCF libraries because NetCF does not include all the classes referenced from that source code. While you can modify the code by hand to eventually get it to work with NetCF, we offer an easier way.

The NetCFSvcUtil.exe tool that ships as part of the Power Toys for .NET Compact Framework 3.5 supports many of the same command-line switches that svcutil.exe does, but performs a few additional/modified steps:

Verifies that the service being consumed offers endpoints compatible with the feature subset included in NetCF 3.5.
Generates the client proxy class in C# or VB that compiles against NetCF or desktop WCF.
Generates code for a CFClientBase class that fills in where NetCF doesn't support calling WCF services.
Does not generate a .config file to store the endpoint information. This information is stored in the client proxy code. If an X509 certificate required by the endpoint, that must be passed into the proxy class' constructor by your code.

Assuming that you have a service running at http://localhost/service/service.svc on your desktop computer and a device application in your C:\deviceapp directory, you could generate the code to call the service from NetCF using the following command line. Be sure NetCFSvcUtil.exe is in your path. Also be sure to use a machine name that can be reached from your device. If you use "http://localhost/..." as your argument to the tool, your device will look for localhost (which is the device itself!) and not find the service that is running on your Windows box.

C:\deviceapp>netcfsvcutil.exe http://<machinename>/service/service.svc

The tool will generate and list two source files. The source file named after your service will be the one that contains the proxy class you'll use from your app. You can use it just like you would the svcutil.exe-generated code, except for the provision of secure endpoints made in my numbered list above.

To use these source files in your application, add them to your Visual Studio project and be sure you have System.ServiceModel.dll and System.Runtime.Serialization.dll in your project's Referenced Assemblies list in Solution Explorer. Calling your service from your device application can be as simple as a couple of lines of code:

CalculatorServiceClient client = new CalculatorServiceClient();
MessageBox.Show(client.Add(3, 5));

Some features that WSDLs and the desktop .NET WCF supports are not supported by the .NET Compact Framework 3.5. Although NetCFSvcUtil.exe generates code to enable some of these features, here is a list of features that remain unavailable for our first version of Compact WCF when calling a WCF service on desktop:

Fault contracts (they are not automatically deserialized for you, but you get the raw xml that you can parse manually)
Custom message headers (including peer hop counts)
Callback contracts
Sessions
Mandatory transactions

And the only desktop binding NetCF 3.5 supports is basicHttpBinding, with optional WS-Security (X.509 only) and no support for WS-ReliableMessaging. NetCF 3.5 adds the new mail transport as well, but NetCFSvcUtil does not support generating proxies for services exposed only on the mail transport.

It's worthwhile to note that any service or client that supports WSDL contracts should be able to intercommunicate, regardless of underlying implementation. This means that if you have a desktop WCF service that you want to call from NetCF, you may also use the already-existing Add Web Reference in Visual Studio to generate code for your client proxy class (the command line equivalent is wsdl.exe). The resulting proxy class may not have as friendly of a public API for you to use, and it won't use the WCF stack, but it is a valid alternative that will work in many cases.

The reverse is also true: you can use WCF to call into old-style .asmx web services. It is not necessary to even know which web service implementation is used on a server. So long as it publishes a WSDL, you're in luck. If it doesn't publish a WSDL, you may still be able to generate a client proxy for it as long as it uses WCF service model and you have access to the managed service assemblies. But that's another topic.

Power Toys for .NET Compact Framework 3.5 CTP released

andarno — Thu, 13 Sep 2007 02:17:00 GMT

The .NET Compact Framework team just released the Consumer Technology Preview (CTP) of the Power Toys for .NET Compact Framework 3.5. These Power Toys are tools to help you build, debug and optimizie your NetCF 3.5 applications.

My personal favorite is the NetCF ServiceModel Metadata Tool (NetCFSvcUtil.exe) which allows NetCF clients to call web services using the Compact WCF stack.

Download it and check it out!

The WCF subset supported by NetCF

andarno — Tue, 21 Aug 2007 02:23:00 GMT

[Updated: 21Nov07 to clarify that custom headers are supported, but not in NetCFSvcUtil proxy generation]
[Updated: 27Aug07 to correct Gzip sample, and clarify on transports & extensibility]
[Updated: 23Aug07 to add SecurityAlgorithmSuite enumerable]

Many people have been asking about what subset of .NET 3.0's Windows Communication Foundation (WCF) will be supported by the .NET Compact Framework 3.5. Well, here is a table I put together with the answer to that question:

Feature	Desktop WCF	Compact WCF
Bindings:
· BasicHttpBinding	Yes	Yes
· CustomBinding	Yes	Yes
· WindowsMobileMailBinding	N/A	Yes
· ExchangeWebServiceMailBinding	Yes, via NetCF install	Yes
Formatters:
· SoapFormatter	Yes	Yes
· BinaryFormatter	Yes	No
Encoders:
· TextMessageEncoder	Yes	Yes
· BinaryMessageEncodingBindingElement	Yes	No
· MTOMEncoder	Yes	No
· GzipEncoder	Sample available	Sample available
Transports:
· HttpTransportBindingElement	Yes	Yes
· HttpsTransportBindingElement	Yes	Yes
· MailTransportBindingElement	Yes, via NetCF install	Yes
· MsmqTransportBindingElement	Yes	No
· TcpTransportBindingElement	Yes	No
· (other transports)	Yes	No
XmlDictionaryReader/Writer	Yes	Yes; stub around XmlTextReader/Writer
DataContractSerializer	Yes	No; but can be wire-compatible with DCS via XmlSerializer
Service proxy generation	Yes; via SvcUtil.exe	Yes; via NetCFSvcUtil.exe, not integrated into VS2008
· Non-HTTP transports in generated proxies	Yes	Not built-in
· Custom headers in generated proxies	Yes	Not built-in
WS-Addressing	Yes	Yes
WS-Security message level security
· X.509	Yes	Yes
· Username/password	Yes	No
· SecurityAlgorithmSuite.Basic256Rsa15	Yes	Yes
· SecurityAlgorithmSuite.Basic256	Yes	No
WS-ReliableMessaging	Yes	No
Patterns
· Service model	Yes	No
· Message layer programming	Yes	Yes
· Buffered messages	Yes	Yes
· Streaming messages	Yes	No
· Endpoint descriptions in .config files	Yes	No
Channel extensibility	Yes	Yes
Security channel extensibility	Yes	No