Sowmy Srinivasan's Blog

New additions in 3.5 Sp1

2008-11-21T06:30:00Z

In .NET 3.5 SP1 the following features are added to the DataContractSerializer.

1. Support for POCO types. Aaron has a nice writeup here. I have updated my post on Serialization Rules to include this.

2. Support for get-only collection properties.

3. Support for schema-verifiable multiple object reference (i.e using the same instance of an object in two places)

Another new feature worth mentioning is the support of XmlSerializer for serialization of faults. I will write more about each of these features in the near future.

Serializing internal types using XmlSerializer

2008-10-05T01:38:00Z

Being able to serialize internal types is one of the common requests seen by the XmlSerializer team. It is a reasonable request from people shipping libraries. They do not want to make the XmlSerializer types public just for the sake of the serializer. I recently moved from the team that wrote the XmlSerializer to a team that consumes XmlSerializer. When I came across a similar request I said, "No way. Use DataContractSerializer".

The reason is simple. XmlSerializer works by generating code. The generated code lives in a dynamically generated assembly and needs to access the types being serialized. Since XmlSerializer was developed in a time before the advent of lightweight code generation, the generated code cannot access anything other than public types in another assembly. Hence the types being serialized has to be public.

I hear astute readers whisper "It does not have to be public if 'InternalsVisibleTo' attribute is used".

I say, "Right, but the name of the generated assembly is not known upfront. To which assembly do you make the internals visible to?"

Astute readers : "the assembly name is known if one uses 'sgen.exe'"

Me: "For sgen to generate serializer for your types, they have to be public"

Astute readers : "We could do a two pass compilation. One pass for sgen with types as public and another pass for shipping with types as internals."

They may be right! If I ask the astute readers to write me a sample they would probably write something like this. (Disclaimer: This is not the official solution. YMMV)

using System;

using System.IO;

using System.Xml.Serialization;

using System.Runtime.CompilerServices;

using System.Reflection;

[assembly: InternalsVisibleTo("Program.XmlSerializers")]

namespace InternalTypesInXmlSerializer

{

class Program

{

static void Main(string[] args)

{

Address address = new Address();

address.Street = "One Microsoft Way";

address.City = "Redmond";

address.Zip = 98053;

Order order = new Order();

order.BillTo = address;

order.ShipTo = address;

XmlSerializer xmlSerializer = GetSerializer(typeof(Order));

xmlSerializer.Serialize(Console.Out, order);

}

static XmlSerializer GetSerializer(Type type)

{

#if Pass1

return new XmlSerializer(type);

#else

Assembly serializersDll = Assembly.Load("Program.XmlSerializers");

Type xmlSerializerFactoryType = serializersDll.GetType("Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializerContract");

MethodInfo getSerializerMethod = xmlSerializerFactoryType.GetMethod("GetSerializer", BindingFlags.Public | BindingFlags.Instance);

return (XmlSerializer)getSerializerMethod.Invoke(Activator.CreateInstance(xmlSerializerFactoryType), new object[] { type });

#endif

}

#if Pass1

public class Address

#else

internal class Address

#endif

{

public string Street;

public string City;

public int Zip;

}

#if Pass1

public class Order

#else

internal class Order

#endif

{

public Address ShipTo;

public Address BillTo;

}

Some astute 'hacking' readers may go as far as giving me the build.cmd to compile it.

csc /d:Pass1 program.cs

sgen program.exe

csc program.cs

Remoting posts

2006-07-28T03:58:00Z

As Doug indicated I was going to post my deranged .NET Remoting ramblings in my blog. But then John had created a cool blog especially for our Remoting users. So I'll start putting my Remoting posts there.

All About KnownTypes

2006-06-06T10:47:00Z

One of the common errors in Deserialization is “Element ''http://mycompany.com/:shape'' contains data of the 'http://mycompany.com/:Circle' data contract. The deserializer has no knowledge of any type that maps to this contract. Add the type corresponding to 'Circle' to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding it to the list of known types passed to DataContractSerializer.” The following code will throw such an exception.

[DataContract(Namespace = “http://mycompany.com/”)]

public class Shape{…}

[DataContract(Namespace = “http://mycompany.com/”)]

public class Circle : Shape {…}

[ServiceContract]

public interface IMyServer

{

[OperationContract]

bool AddShape(Shape shape);

}

IMyServer client = new ChannelFactory<IMyServer>(binding, endPoint).CreateChannel();

client.AddShape(new Circle());

The contract specifies Shape but the client passes Circle a subtype of Shape. Since Circle is not a part of the contract the DataContractSerializer fails to deserialize.

Shared Contract vs Shared Type Model:

Remoting users would have never faced this problem. This is due to the fact that, in remoting, the CLR type name is serialized to the wire. The serialized type name is of the following format.

MyCompany.Library.Circle, MyAssembly, Version=2.0.0.0, Culture=neutral, PublicKeyToken=XXXXXX, processorArchitecture=MSIL

The reader uses the Assembly.Load API to load the assembly referred in the wire. In this approach the writer controls the type loaded by the reader. While this removes the hassle of the reader having to know the type being deserialized it has a couple of drawbacks.

1. It disallows the writer and reader using different types which is common in an interoperable webservice scenario.

2. Letting the writer load an arbitrary type into the reader’s appdomain is not a sound idea from security standpoint.

NetDataContractSerializer, shipped in WCF, supports sharing types as described above. DataContractSerializer (which is used by default in WCF services) follows a different approach. Like the XmlSerializer (which is used by the ASMX webservices), the DataContractSerializer shares contracts instead of types. This allows the reader to control the deserializing type. From the webservice contract the reader specifies the type to deserialize. In the derived type scenario the contract name (i.e the XSD schema type name) is written on the wire. The contract name is a XML qualified name like 'http://mycompany.com/:Circle'.

While contract name is inferable from the CLR type the reverse is not possible since there can be multiple CLR types with the same contract name. To enable this reverse lookup the user needs to pass a set of types called the KnownTypes. The deserializer resolves the type qname in the wire using these types. It must be noted that unlike the remoting scenario the deserializer never loads the type based on the information from the wire.

Declaring Known types:

ASMX uses XmlInclude and SoapInclude attribute to denote known types. WCF allows more than one way to specify the known types.

1. The base type can specify the derived as a known type using the KnownTypeAttribute as shown below.

[DataContract(Namespace = “http://mycompany.com/”)]

[KnownType(typeof(Circle))]

public class Shape{…}

2. Known types can be specified in the config as shown below.

<system.runtime.serialization>

<add type="MyCompany.Library.Shape,

MyAssembly, Version=2.0.0.0, Culture=neutral,

PublicKeyToken=XXXXXX, processorArchitecture=MSIL">

<knownType type="MyCompany.Library.Circle,

MyAssembly, Version=2.0.0.0, Culture=neutral,

PublicKeyToken=XXXXXX, processorArchitecture=MSIL"/>

</add>

</declaredTypes>

</dataContractSerializer>

</system.runtime.serialization>

</configuration>

The config defines Circle as a known type for Shape is used. This is equivalent to defining KnownType in the base type as described in #1.

3. Known types can also be passed to the constructor of the DataContractSerializer

4. In service model known types can be added via the ServiceKnownType attribute along with either ServiceContract or OperationContract.

[ServiceContract]

public interface IMyServer

{

[OperationContract]

[ServiceKnownType(typeof(Circle))]

bool AddShape(Shape shape);

}

Declaring ServiceKnownType on an operation causes the known type to be applied for all the parameters for the operation. Declaring on the service contract causes known type to be applied for all the parameters of all the operations.

5. The known types specified via ServiceKnownType attribute end up in the OperationDescription. Known types can also be added to OperationDescription.KnownTypes imperatively. Known types in the description are passed to the constructor of DataContractSerializer. These known types are also passed as extra types to XmlSerializer in the XmlSerializerFormat mode.

Known Type Scope:

Deserialization is a process of constructing the object tree. Known types passed to the constructor of DataContractSerializer apply to all nodes of the tree. A KnownType declared on a class not only applies to the class itself but for the entire subtree.

[DataContract(Namespace = “http://mycompany.com/”)]

[KnownType(typeof(Square))]

[KnownType(typeof(Circle))]

public class Shapes

{

[DataMember]

Shape[] shapeList;

}

In the above example the types Square and Circle are ‘in scope’ KnownType not only when Shapes is deserialized but also when the Shape[] is deserialized.

It is an error to have more than one KnownType with the same contract name at a given scope (i.e in the list of KnownTypes on a class or the types passed to the constructor of the DataContractSerializer). However there could be such duplication across scopes. In such cases the type in the inner most scope is used. To understand the scoping rules consider the class Shape being defined as follows.

[DataContract(Namespace = “http://mycompany.com/”)]

[KnownType(typeof(AnotherSquare))]

[KnownType(typeof(AnotherCircle))]

public class Shape

{

}

If AnotherSquare has the same contract name as Square then AnotherSquare will be used when the contract name of Square is seen on the wire.

To summarize the order in which known types are applied is shown below.

Data contract primitive type
Known types provided via KnownTypeAttribute from the inner most scope to the outer most.
Known types passed to the constructor of the DataContractSerializer
The declared member type.
The root type passed to the constructor of DataContractSerializer

Known Types for Generic Types:

Consider Circle and Shape in the previous example to be generic type. Ideally one would want to define the known type as follows.

[DataContract(Namespace = “http://mycompany.com/”)]

[KnownType(typeof(Circle<>))] //NOT ALLOWED

public class Shape<T>{…}

Unfortunately the CLR does not allow generic types in attributes. Thus the above declaration will not work. Fortunately the DataContractSerializer supports another way to provide generic known type. Instead of specifying the known type directly in the KnownTypeAttrribute the user can also point a method from KnownTypeAttribute. The method can return an array of KnownTypes. This will allow generic known types as shown below.

[KnownType("GetKnownTypes")]

public class Shape<T>

{

static Type[] GetKnownTypes()

{

return new Type[] { typeof(Circle<T>) };

}

The method approach also allows the type authors to be able to control the known types dynamically at runtime. The method pointed by KnownType attribute must be static, take no parameters and return an IEnumerable<Type>. It is an error to mix a static known type with the method based dynamic known type for a given class.

ServiceKnownType supports a similar model to specify dynamic known types as shown below.

[ServiceContract]

[ServiceKnownType(Method = “GetKnownTypes”, Type = typeof(KnownTypeProvider))]

public interface IMyServer<T>

{

[OperationContract]

bool AddShape(Shape shape);

}

static class KnownTypesProvider

{

static Type[] GetKnownTypes(ICustomAttributeProvider knownTypeAttributeTarget)

{

Type contractType = (Type)knownTypeAttributeTarget;

return new Type[]{contractType.GetGenericArguments()[0]};

}

Note that the attribute refers to a method name and a type name. The type name is needed since the ServiceKnownType attribute can go on interface or interface methods and the Known type method cannot be implemented in the interface. The other difference between ServiceKnownType and KnownType is in the method definition. In ServiceKnownType the method takes an additional ICustomAttributeProvider as parameter. DataContractSerializer passes the ServiceContract class or MethodInfo of the OperationContract method when invoking this method.

Generic Types in Config

Generic known types can also be defined in config as shown below.

<system.runtime.serialization>

<add type="MyCompany.Library.Shape`1,

MyAssembly, Version=2.0.0.0, Culture=neutral,

PublicKeyToken=XXXXXX, processorArchitecture=MSIL">

<knownType type="MyCompany.Library.Circle`1,

MyAssembly, Version=2.0.0.0, Culture=neutral,

PublicKeyToken=XXXXXX, processorArchitecture=MSIL">

</knownType>

</add>

</declaredTypes>

</dataContractSerializer>

</system.runtime.serialization>

</configuration>

The above config specifies that the generic parameter for Circle is the same as the generic parameter for the declared type Shape. The config allows the definition of known type of arbitrary complexity. For example if it is needed to define Circle<Dictionary<string, T>> as the known type of Shape<T> (of course this is purely academic) it can be done as follows.

<system.runtime.serialization>

<add type="MyCompany.Library.Shape`1,

MyAssembly, Version=2.0.0.0, Culture=neutral,

PublicKeyToken=XXXXXX, processorArchitecture=MSIL">

<knownType type="MyCompany.Library.Circle`1,

MyAssembly, Version=2.0.0.0, Culture=neutral,

PublicKeyToken=XXXXXX, processorArchitecture=MSIL">

</parameter>

</knownType>

</add>

</declaredTypes>

</dataContractSerializer>

</system.runtime.serialization>

</configuration>

Note the use config element “parameter” with the attributes ‘type’ and ‘index’.

Common KnownType Scenarios:

In addition to the inherited scenario described earlier, there are some typical scenarios that require known types.

When passing a DataContract class instance as an item to non generic collection class such as ArrayList. This due to the fact that ArrayList is projected as an array of objects.
When passing a DataContract class instance as an item in ISerializable class such as System.Exception. This is due to the fact that ISerializable is an untyped (System.Object based) bag.

While these can also be solved by declaring known type, it is recommended to consider other approaches. Scenario #1 can be worked around by using a generic collection. Scenario #2 will never arise if the user follows the fault contract guidance and explicitly declare fault types as FaultContract and not use exception for faults. In general it is recommended to avoid ISerializable in a contract oriented application.

Why prevent DataContract + IXmlSerializable

2006-05-14T19:56:00Z

I left one thing unsaid in the serialization rules and Aaron's sharp eyes caught it promptly. As he mentioned in his blog, mixing interface programming model (such as IXmlSerializable or ISerializable) with DataContract programming model is disallowed in V1. Here is an example of such a class.

[DataContract]

public class XmlDataContractType : IXmlSerializable

{

[DataMember]

public int MyDataMember;

//IXmlSerializable Members

}

As Aaron mentioned we could have chosen IXmlSerializable in this case. I agree that it is natural for the interface to trump attribute since it impacts the derived types as well. However, this meant that the DataContract does not impact anyone.

On the other hand, picking DataContract enables scenarios where a type is serialized with multiple serializers. Both IXmlSerializable and ISerializable are programming models that are also supported by other serializers (XmlSerializer and BinaryFormatter). A user might want to make a type IXmlSerializable or ISerializable to be used by ‘legacy’ serializers and make it a DataContract type for WCF. But this approach fails as soon as we consider inherited types. Consider two derived types of XmlDataContractType as defined below.

public class DerivedXmlType : XmlDataContractType

{

}

[DataContract]

public class DerivedDataContractType : XmlDataContractType

{

}

Class DerivedXmlType does not define DataContract but is an IXmlSerializable by virtue of inheritance. In this case IXmlSerializable projection of XmlDataContractType should be used. Since DerivedDataContractType is a DataContract it will end up using the DataContract projection of XmlDataContractType. We definitely did not want to encourage this dual projection of XmlDataContractType.

Preserving Object Reference in WCF

2006-03-26T09:04:00Z

By default object references are not preserved by the DataContractSerializer; Values of an object referenced multiple times is serialized multiple times. If the object is part of mutual (cyclic) reference (e.g. circular linked list) an exception is thrown during serialization.

DataContractSerializer can be made to preserve object reference by passing true for parameter PreserveObjectReference when constructing DataContractSerializer as shown below.

new DataContractSerializer(type, name, ns, knownTypes,

0x7FFF /*maxItemsInObjectGraph*/,

false/*ignoreExtensionDataObject*/,

true/*preserveObjectReferences*/,

null/*dataContractSurrogate*/);

Enabling in Service Operation

To enable this option in service operation one must pass an instance of DataContractSerializer to the WCF runtime. This can be done by sub classing DataContractSerializerOperationBehavior and overriding CreateSerializer method as shown below.

class ReferencePreservingDataContractSerializerOperationBehavior

:DataContractSerializerOperationBehavior

{

public ReferencePreservingDataContractSerializerOperationBehavior(

OperationDescription operationDescription)

: base(operationDescription) { }

public override XmlObjectSerializer CreateSerializer(

Type type, string name, string ns, IList<Type> knownTypes)

{

return CreateDataContractSerializer(type, name, ns, knownTypes);

}

private static XmlObjectSerializer CreateDataContractSerializer(

Type type, string name, string ns, IList<Type> knownTypes)

{

return CreateDataContractSerializer(type, name, ns, knownTypes);

}

public override XmlObjectSerializer CreateSerializer(

Type type, XmlDictionaryString name, XmlDictionaryString ns,

IList<Type> knownTypes)

{

return new DataContractSerializer(type, name, ns, knownTypes,

0x7FFF /*maxItemsInObjectGraph*/,

false/*ignoreExtensionDataObject*/,

true/*preserveObjectReferences*/,

null/*dataContractSurrogate*/);

}

The behavior must be added to all the operations on the server side contract. For a self hosted server it can be done as shown below.

ServiceHost host = new ServiceHost(

typeof(FooContractImpl), new Uri(address));

host.AddServiceEndpoint(typeof(FooContract), binding, address);

foreach (ServiceEndpoint endpoint in host.Description.Endpoints)

SetDataContractSerializerBehavior(endpoint.Contract);

host.Open();

Similary the behavior must be added to all the operations on the client side as shown below.

ChannelFactory<FooContract> factory = new ChannelFactory<FooContract>(binding, new EndpointAddress(address));

SetDataContractSerializerBehavior(factory.Endpoint.Contract);

FooContract proxy = factory.CreateChannel();

The SetDataContractSerializerBehavior is defined as shown below.

private static void SetDataContractSerializerBehavior(ContractDescription contractDescription)

{

foreach (OperationDescription operation in contractDescription.Operations)

{

operation.Behaviors.Add(new ReferencePreservingDataContractSerializerOperationBehavior(operation));

}

Enabling via Attribute

This is all good. However wouldn’t it be great if it is possible avoid all the imperative code and instead define an attribute as shown below?

[ServiceContract]

public interface FooContract

{

[OperationContract]

[ReferencePreservingDataContractFormat]

Node EchoNode(Node node);

}

Yes this can be done by defining a custom attribute that implements IOperationBehavior as shown below.

public class ReferencePreservingDataContractFormatAttribute : Attribute, IOperationBehavior

{

#region IOperationBehavior Members

public void AddBindingParameters(OperationDescription description, BindingParameterCollection parameters)

{

}

public void ApplyClientBehavior(OperationDescription description,System.ServiceModel.Dispatcher.ClientOperation proxy)

{

IOperationBehavior innerBehavior = new ReferencePreservingDataContractSerializerOperationBehavior(description);

innerBehavior.ApplyClientBehavior(description, proxy);

}

public void ApplyDispatchBehavior(OperationDescription description,System.ServiceModel.Dispatcher.DispatchOperation dispatch)

{

IOperationBehavior innerBehavior = new ReferencePreservingDataContractSerializerOperationBehavior(description);

innerBehavior.ApplyDispatchBehavior(description, dispatch);

}

public void Validate(OperationDescription description)

{

}

#endregion

}

The complete sample is attached.

WCF Serialization Programming Model

2006-02-22T12:58:00Z

DataContract is the default serialization programming model for WCF. However WCF supports more than just the types marked wth DataContract attribute. It supports serialization of the following kinds of types in the default mode.

CLR built-in types
Byte array, DateTime, TimeSpan, GUID, Uri, XmlQualifiedName, XmlElement and XmlNode array [This includes XElement and XNode array from .NET 3.5]
Enums
Types marked with DataContract or CollectionDataContract attribute
Types that implement IXmlSerializable
Arrays and Collection classes including List<T>, Dictionary<K,V> and Hashtable.
Types marked with Serializable attribute including those that implement ISerializable.
Types with none of the above attributes (POCO) but with a default constructor (can be non-public). [This is supported only from .NET 3.5 SP1]

Some types may implement more than one of the above programming model. In such cases a programming model is chosen based on its priority as given by the above list. For example, Hashtable is a collection class but also implements ISerializable and it is serialized as a collection type. DataSet implements both IXmlSerializable and ISerializable and it is serialized as IXmlSerializable.

Here is a DataContract class that no one will ever see in real world. Figuring out which programming model is implemented by these types is left as an exercise to the readers.

[DataContract]

public class ComplexDataContract

{

[DataMember]

string name;

[DataMember]

System.Version version;

[DataMember]

System.IO.FileInfo fileInfo;

[DataMember]

System.Collections.Generic.List<System.Uri> uriList;

[DataMember]

System.Data.SqlTypes.SqlInt32 sqlInt;

[DataMember]

System.Drawing.KnownColor knownColor;

}

Hello, world!

2006-02-22T12:52:00Z

Here is yet another Indigo blog. Having worked on ‘Indigo’, now known as WCF, for four years I am excited to talk about it.

My name is Sowmy Srinivasan and I am a developer in the WCF team. My team works on serialization of data and management of related metadata. The intent of this blog is to clarify design nuances of the features I had worked on and provide adequate information for everyone to start on this great technology, specifically about the features such as DataContractSerializer, XmlSerializer, DataContract, MessageContract, Message, MTOM, XSD, WSDL, XmlDictionaryReader, XmlDictionaryWriter, XPath Filter engine etc.

Let the Rambling begin.