Lifeng's Blog : WCF

Edit .svcmap file to access "hidden" functions of service references

Lifeng Lu — Fri, 01 Aug 2008 20:40:00 GMT

Not all code generation options of service references are exposed through its UI. To prevent users to be confused by some advanced options, the configuration dialog is designed to only expose options which are most likely to be used. We can only access other options by editing the .svcmap file. Unfortunately, the file is hidden by default, so people won't discover it easily, and most people will prevent to edit hidden files as well to prevent corrupting the project by accident.

Well, this .svcmap file is editable, and Xml Intellisense actually works here, so changing it is not that difficult. Certainly, we are not interested to change those options which can be changed easily through the configuration dialog. Here are something you can do with editing this file, but otherwise not doable:

1, You can enable sharing service contract types:

by adding ServiceContractMapping in the ServiceContractMappings section, you will force the code generator to use a shared service contract type defined in a referenced assembly (you cannot share a type defined in the current project.) The service reference will still generate configuration to the app.config file.

2, you can define complex collection mappings:

the configuration dialog only support some limited options. Adding CollectionMapping in the file, you get similar function you can get by using svcutil command line.

3, namespace mappings:

it works like /namespace option in the svcutil command line

4, you can add additional MatadataSources!

actually, you can reference two services in one service reference, and share the data types. Adding additional MetadataSource to the file by hand is the only way to do that today. Once you added, it will be maintained, and it will not be lost, when you update the service. (Actually, you don't need add MetadataFiles by hand. Just add a MetadataSource into the file, save it, and Update the reference, all metadata files will be downloaded automatically.

5, add additional Metadata files

It is possible to add additional metadata files to a service reference, just copy them to the folder, and add additional MetadataFile node in the .svcmap file to reference them. They will be used in the code generator. All MetadataFile without a SourceId will not be automatically updated, so they will stay even when you update the service reference.

6, add private configuration file

You can add a special extension file with name = "Reference.config" (very similar to configuration.svcinfo). With the configuration file, you can add additional extensions to the code generator. (Unlike adding them to the machine level configuration, things in the Reference.config only affect the code generator in one reference. Special security settings to enable downloading metadata files from a secured server is also possible through a such config file.

7, additional extension files

The additional code generator extensions can locate those extensions file registed in the Extension section. It is possible to set additional options through those files.

Side by side comparsion between svcutil and the service reference in VS

Lifeng Lu — Tue, 14 Aug 2007 02:47:00 GMT

The Service Reference is new feature added into VS Orcas, so we don't have to use svcutil when using VS IDE. Here is the side by side comparsion table between svcutil and the service reference in Orcas:

Svcutil command line	VS service reference (.svcmap)	VS UI
/out	(always delivery from the name of .svcmap)
/config	(always the config file of the project)
/mergeConfig	Always true
/noConfig	Not supported
/dataContractOnly	Not supported
/language	Always based on the project
/namespace	<NamespaceMappings> But *-> ReferenceName is always added
/messageContract	<GenerateMessageContracts>	Yes
/enableDataBinding	<EnableDataBinding>
/serializable	<GenerateSerializableTypes>
/async	<GenerateAsynchronousMethods>	Yes
/internal	<GenerateInternalTypes>	Yes
/reference	<ReferenceAllAssemblies> <ReferencedAssemblies> <ReferencedDataContractTypes> <ServiceContractMappings> Can pick up existing reference assemblies of the project automatically	Limited
/collectionType	<CollectionMappings>	Limited
/excludeType	<ExcludedTypes>
/noStdLib	Through <ReferencedAssemblies>	Yes
/serializer	<Serializer>
/importXmlTypes	<ImportXmlTypes>
/targetClientVersion	Based on the project
/t:metadata	Not supported
/validation	Not supported
/t:xmlSerializer	Not supported
Multiple url/file	<MetadataSources>
/svcutilConfig	<ExtensionFile Name=”Reference.config” />

Essentially, the svcutil is built for several tasks like exporting metadata, validating the service. But the service reference is built for generating proxy/configuration only. However, in the proxy/configuration generating scenario, the function of the service reference and svcutil are overlapped. The service reference supports most options svcutil supports in its command line. However, to access those options, editing the .svcmap file is necessary (only limited options are exposed through UI.) By saving the options in a file, however, it is easier to repeat the process, when the service is updated.

The advantage to use the service reference is clear when using the IDE. We don't lose most ablility of the svcutil tool, but gain some convenience: options like the language/targetPlatform don't have to be chose again, but will be picked up from the project system automatically. The result config is also automatically merged, and previous injected configuration will be tracked, and removed when the reference is removed or updated, so it won't inject lots of duplicated items. (That function is limited in this version if binding in the configuration is editted.) The IDE also supports better experience when the service is secured.

Passing DataTable across web/wcf services

Lifeng Lu — Wed, 01 Aug 2007 23:46:00 GMT

Passing DataTable (without embeded into a dataset) across web services is supported in .Net 2.0 framework. However we found a bug in this area, which might affect using this feature with strong typed dataTable. For example, when a web service function wants to return a DataTable to the client, an independent dataTable (a dataTable which is not a part of a dataset) will work, but a dataTable inside a dataSet won't work. What happens is that the client side couldn't deserialize the dataTable from wire. The root reason is the DataTable inside a dataset inherits a schema namespace from the dataSet, but an independent datatable has an empty namespace (unless we change it). Data cannot be changed between two dataTables with different namespace.

The problem is that the instance of the dataTable on the client side is created by framework directly. It is impossible to change the namespace without changing generated strong type dataset code. To work around this issue, the server side need copy data to an independent dataTable before passing it to the client.

That affects both .Net 2.0 web services and 3.0 Windows communication fundation services.

.svcmap file

Lifeng Lu — Sat, 16 Jun 2007 02:08:00 GMT

After we add a WCF service reference into a Visual Studio project, a .svcmap file will be added to the project, and it contains most information of the service reference. Actually, it is the only essential file of the reference. In most case, we can remove all other files in the service reference, and still can pull down all the files from services by updating the service reference. By default the .svcmap file is hidden in the solution explorer unless we turn on "show all files..." option.

Do I need understand the .svcmap file? No, you don't have to, if you just want to play with the new WCF feature, just like you don't have to know much about command line options of the svcutil tool before you start to use that. Actually, the file contains options for the proxy code generator, makes it equlivent to those command line options in many cases. However, when we need deal with some slightly complex scenarios, like consuming an old web services, sometimes without turning those options, we don't get useful proxy code. That is when we need look into this file.

Some common code generator options are exposed through a "service reference settings" dialog (not in beta1). But many of them are not exposed through any UI. To change those options, we will have to open and edit the .svcmap file inside a service. Fortunatly, the format of the file is fairly simple, and the schema of that file is shipped with the Visual Studio, so the Xml Editor will provide intellisense when we edit the file.

Although the most options in the .svcmap file works exactly the same way as command line options of svcutil tool, there is some difference between them. For example, when you turn on type sharing in existing assemblies, the svcutil will share service contract types as well as data contract types. But the code generator in VS will only share data contract types. Service contract types will not be shared automatically. A white list must be added in the .svcmap file to make service contract type sharing to work.

Other than the code generator options, there are some interesting things we can do when we edit the file directly. One sample is that there is a MetadataSource section in the file, which contains where to download metadata file. By default, there is always one MetadataSource when the service reference is created through UI. It doesn't have to be this way. For example, if the metadata is not exposed by the service, but you get them through other way of communication, like through an email. We can copy all metadata files into the service reference folder, and reference them directly in the .svcmap file, and delete all MetadataSources. The result will be a service reference, which generates valid code, but will not try to download files from a server. Another possible is to add two sources, so the metadata will be pulled into one reference, and code will be generated together. It is similar like providing two URLs in the svcutil tool.

We can also apply things similar to '/toolconfig' in the svcutil command line. That will allow us to add/remove WSDL/Policy importer extensions. The easiest way is to add those tool configuration in the web.config/app.config of the project. However, that solution has two problems. First, we usually don't want to ship those design time options in the product. The web.config/app.config is a part of the product, so we do mix the design thing and runtime thing in the same place. Second, if we have two service references, and we want different importer extensions for them, there is no way to do so. Fortunately, it is possible to give every service reference a seperated tool config file. The way to do this is to add a Reference.config file in the service reference folder, and reference that file as an extension file in the .svcmap file, just like how .svcinfo file is referenced.

Files in a service (WCF) references

Lifeng Lu — Wed, 09 May 2007 02:32:00 GMT

The new service reference is persisted in a similar way how the old web reference is persisted in a visual studio project. All files including metadata file downloaded from the server will be persisted into a single folder, which defines the CLR namespace proxy code lives in.

Files in a service reference is not shown in the solution explorer by default, but it is easy to see them by turning on "Show All Files".

Those metadata file includes all WSDL and XSD files. DISCO files will be also kept, if the metadata is downloaded from a disco port, although those disco files are never used in configuration/proxy generation.

All those metadata files and code generator operations are persisted in a .svcmap file. The .svcmap file is the basic unit for the proxy/configuration generator. We may add multiple metadata sources into one .svcmap file, so the code generator will handle them at the same time. If the metadata is property prepared, the code generator will generate shared data contracts for multiple service contracts. This is just one function that you have to get by editing the .svcmap file directly, because it is not exposed through simplified UI.

Besides the .svcmap file, there is a '.svcinfo' file which tracks the configuration added to the app.config/web.config, so that portion can be removed when the service reference is removed from the project. When we update the reference after the service is updated, we will try to update the app.config file, but not just inject another section of configuration into the file, like what svcutil did. The format of this file is not designed to read/edit by human, it is better not to mess up this file.

However, currently, the fuction to maintain the app.config is faily weak. It is certainly better than svcutil, with which you have to maintain the file by yourself. But while the function works fine when you live with the configuration generated automatically, it doesn't work well after you change it. In most case, it will not take out the configuration changed by you, but will inject the new section side by side with it. That makes that function less useful in those advanced scenarios.

There are talks about what to do when the injected configuration has been updated, but the extensibility of the configuration in WCF makes it more complex than what can be done in a short product cycle.

.svcinfo file is persisted as one extension file of the .svcmap file. The .svcmap file allows other customized extension files, so actually a customized WSDL importer/policy importer extensions could pick up options from one extension files to control options in code generator. BTW, just like svcutil, those importer extensions are supported in VS. Without the toolconfig parameter, you need add those extensions in the machine.config or app.config of the project. The drawback to put those design time configuration into app.config is that you would leak them into the runtime. In beta 2, it is possible to put those design time configuration in a seperated config file.

So, an advanced user could write an extension to change generated configurations by changing certain values, or remove not needed ports. It could be done in a general way, and its real behavior can be controlled by an extension file, which could sepecify like which port need be removed, and what binding parameter need be changed, so you don't have to manage the app.config file by hand.

Windows communication framework client in web site projects

Lifeng Lu — Thu, 26 Apr 2007 21:09:00 GMT

The Visual Studio Orcas beta1 supports generating and using WCF client in the ASP.Net web site projects. From the platform point of view, ASP.Net wants to enable an user to build a web site without any Visual Studio tools. The feature to generate WCF client has been built with the same sprit. With Visual studio, you can easily add a WCF client, but without it, you can create your own by creating a 'svcmap' file. The 'svcmap' file will be processed by a build provider at the runtime to generate WCF client code, which will be compiled on the fly, so web site code could consume those classes.

However, that comes with a price. Because the build provider hasn't been built into the 3.0 framework (it is added in 3.5 framework), so it becomes not possible to consume WCF this way in a project targetting 3.0 framework. That is why we can create a WCF server with a 3.0 project, but couldn't consume it in the same project. (Of couse, if you use svcutil tool, it is still valid to generate WCF client, and consume it in a 3.0 web site project. But you have to do everything by hand.)

However, a build provider might work well to do some simple code generation. It actually is not a good platform to geneate complex proxy code. The problem is that it can either sucess without any message, or fail with one message string, so it becomes no way to output waring messages, if anything might be wrong. That is why it could be difficult to figure out a problem in the WCF client in a web site project. Using a web application project might be a better choice, or maybe I will add the same WCF client to a client project to see warning messages.

The svcmap file is also used in the client project, so it is actually easy to copy one from a client project to a web site project. Metadata files pulled down from services are saved in individular xsd/wsdl files, and actually referenced by the svcmap file. It is somewhat similar to the discomap file in a web reference in VS 8. The svcmap file also contains code generation options. Many options of the svcutil tool are supported. However, those options are not supported in the beta 1 product. Unlike the old web reference, default proxy works fine in most cases. Sometime, it is necessary to adjust those WCF client options to get right proxy code, that is especially true when you consume a service built with old web service platform.

Where is "Add Web Reference" after I install Orcas Beta 1

Lifeng Lu — Tue, 24 Apr 2007 20:30:00 GMT

When you create a project targetting .Net framework 3.x in Orcas Beta 1, the "Add Web Reference..." menu is replaced by "Add Service Reference..." menu. "Add Service Reference..." will create a WCF (Windows Communication Framework) client to a web service. The proxy generated by WCF is very different from old web reference proxy. You can consume old web reference through new WCF client.

The WCF client is not supported when your project is targetting .Net Framework 2.0. You are still able to add old web reference to those projects. When you upgrade a 2.0 project to 3.x, if your project contains any old web reference, you will still be able to add extra old web reference. In that case, both "add web reference..." and "add service reference..." will be shown.

So if you do need add old web reference to a project targetting 3.x framework, the workaround is that you can change the targetFramework to 2.0, add one web reference and change back the target framework.

BTW, in Orcas beta 2, you will be able to add Web Reference to 3.x projects.