John W Powell : WCF

Consume SharePoint Web Services with WCF using the Repository, Gateway, Mapper, Domain Model and Factory Design Patterns

johnwpowell — Sun, 04 Jan 2009 00:53:00 GMT

The SharePoint developer community has produced a wealth of knowledge and code samples that demonstrate how to consume SharePoint web services and leverage them in domain-specific applications. This information is often task-centric, such as "How to add an Item to a List using SharePoint Web Services." In this article, we will take a framework-centric approach and apply sound, fundamental object-oriented (OO) design principles to implement a reusable library for working with the SharePoint web services.

Motivation and Goals

My motivation for this article is to help you create a foundation for a framework that can be used in one, or reused across many SharePoint-integrated applications. I also want to demonstrate how good software design principles can (and should) be applied to SharePoint development, and confess my own sins for not always following them. For some reason, I often find myself thinking procedurally when developing a SharePoint customization. Perhaps it's because the product is a 70% solution and requires a different mindset and approach than a large greenfield software project does. Maybe customers expect results faster when you build it on SharePoint, and I've taken shortcuts to meet them. Or maybe I've just been lazy, but in any case, here are some of the issues I've seen or been responsible for:

Re-writing the same SharePoint code on each new project
Not refactoring within and across applications
Tightly coupling application code to the XML input and output from SharePoint services
Lack of a domain model; designing procedurally instead of applying domain-driven design (DDD)
Not having well-defined layers and strategies
Failure to design for testability and not writing unit tests

Here are some of the goals I would like to achieve with this framework:

Implement a domain model that is independent and decoupled from the SharePoint services
Define a set of patterns and strategies for communicating with the SharePoint services
Create a reusable library that can be used in many different domain-specific applications
Use DDD and Test-Driven-Design (TDD)
Apply good OO principals and patterns
Use Windows Communication Foundation (WCF)

Design

In the next section, I'll walk through the implementation and the process I went through to arrive at the design below. Generally speaking, I used DDD and TDD (where it made sense) and followed the Single Responsibility principle. If you are not familiar with this, it is often described as "a class should only have one reason to change." I call it the "and test." I describe the purpose of the class and if I have to use the word and, I consider refactoring some of the functionality to another class.

In the diagram below, the client application uses the repository and domain classes. The repository is responsible for retrieving domain objects based on criteria. The repository uses one or more service gateways to get the information it needs. The service gateway communicates with the SharePoint web services using a WCF client. The service calls typically return XML responses, and it is the job of the service response mapper to translate the XML to a domain object. Finally, the WCF client factory creates clients configured to call the SharePoint services.

Get the code

Implementation

In this example, the goal is to get all the lists in a given site. Following DDD, we will start the design in the domain layer and create a list class that has a title property:

Create the list repository class to define the method clients will call the get the lists in a given site:

Following TDD, we'll create a unit test that will fail until we implement the repository:

Return to the repository class and add the implementation to call the lists service gateway which we haven't created yet. How did I know to create a lists service gateway? I decided to have a gateway that corresponds to each SharePoint web service and did some research to find the exact service and method that would return the lists for a given site. The implementation below is very simple, but you can imagine a more complex repository that uses several service gateways and does some additional processing to construct a domain object graph.

Create the lists service gateway so the project will compile:

Create a failing unit test for the lists service gateway:

Next we'll add a reference to the SharePoint Lists service. Note: Visual Studio will add an app.config and a Properties > DataSources folder and you can delete both of them. We are going to configure WCF programmatically.

Return to the lists service gateway and fill in the implementation. Now I'm pulling a "Julia Childs" on you. I already went through several iterations of TDD and refactoring to arrive at the implementation below. Initially this method had all the code to call the web service, and there was no WCF client factory nor a mapper. After implementing this logic in several gateways, I refactored the commonality to classes. It made sense to have a factory to construct WCF clients in a consistent way and allow me to make a change in one place if I discovered something new. The Single Responsibility "and test" led me to refactor the mapping code to a separate class and the end result is a much cleaner implementation:

Next, we'll implement the WCF client factory. As you can see, it has been through a few iterations of TDD and refactoring starting with the service url class. This class contains constants and methods for working with SharePoint service urls. The other thing you'll notice is the the WCF configuration that is typically done in a configuration file is being done programmatically in the service binding factory class.

Here is the implementation of the service binding factory:

The gateway uses a mapper to map the XML response from the SharePoint web service to a domain object. The XML response is in the form of <element attribute1...n /> which, in this case, translates nicely to domain objects and properties. The code below uses Linq to find all List elements and for each one it calls MapInternal which creates a list object from the element:

Now if we return to the repository GetLists unit test, it passes and output each list title in the site:

Summary

In this article and walkthrough, I demonstrated how DDD, TDD, patterns and good OO principles can be applied to implement a framework that consumes SharePoint web services. Even though I focused on the web services, the SharePoint API can also be regarded as a service and the same fundamentals can be applied to developing applications that run on the server. I hope this inspires and helps you to implement your own framework and improve the quality and reuse of your SharePoint code.

Get the code

References and Additional Reading

Build PowerShell CmdLets to call SharePoint Web Services using WCF

johnwpowell — Mon, 03 Nov 2008 00:22:00 GMT

In this post I'll show you how to build a PowerShell cmdlet that consumes SharePoint web services using WCF. What I like about this approach is that it provides the ability to use some of the SharePoint API without having to be logged in to the SharePoint server. Although you can accomplish the same thing using PowerShell scripts and functions, in this post we'll build a SharePoint cmdlet using C#.

About the Approach

There are several advantages of combining PowerShell and SharePoint web services:

Consume the SharePoint API without logging on to a SharePoint server
Provide a layer of abstraction and reusable task-oriented building blocks
Reduce risk of performance issues caused by PowerShell scripts that don't dispose of unmanaged SharePoint resources

There are also some drawbacks to writing custom cmdlets that consume SharePoint web services:

SharePoint web services do not expose the full API; you may need to augment them with business-specific functionality
Cmdlets require compilation and additional overhead over script file

Getting Started

In this walkthrough, I'm using Visual Studio 2008 on a SharePoint virtual machine. Although you could use a Visual Studio PowerShell project template, we'll are going to build everything by hand so there are no mysteries. So let's get started by creating a new class library project:

Add a reference to PowerShell which is contained in System.Management.Automation assembly. If you've installed the Windows SDK, you can find it in the following location: C:\Program Files\Reference Assemblies\Microsoft\WindowsPowerShell\v1.0

Since we are creating a class that will both describe and install a custom snap in, we need to reference System.Configuration.Install

Create the Snap In and Cmdlet

Add a new class that inherits CustomPSSnapIn. Add the RunInstaller attribute to the class and describe the snap in by overriding the Name, Vendor and Description properties. Your code should end up looking like this:

Add a new class that inherits from Cmdlet. We'll build a cmdlet to get information about a SharePoint site, so name it "GetSPWebCmdlet." Add the Cmdlet attribute to the class. You'll notice the cmdlet name, "Get-SPWeb" is a combination of the noun string SPWeb and verb enumeration VerbsCommon.Get specified by the Cmdlet attribute:

Override the Cmdlets property in the snap in class and add the Get-SPWeb cmdlet to the Cmdlets collection:

Configure Visual Studio to Install and Add the Snap In When the Project is Debugged

Before we add any code to the cmdlet, let's configure Visual Studio to automatically deploy the snap in to make debugging easier. We'll use a post-build event to install the snap in, so in the spirit of learning PowerShell, we'll write the post-build script in PowerShell! Add a new text file to your project named PostBuildEvent.ps1. Add the following script to the file:

trap
{
   $errorTxt = $_.Exception.Message
   write-output "$errorTxt"
   break
}

$targetDir = $args[0]
$targetFileName = $args[1]
$snapInName = $args[2]

write-output "Deploying..."
join-path $targetDir $targetFileName
write-output "Snap In Name: $snapInName"

set-location $targetDir
set-alias installutil $env:windir\Microsoft.NET\Framework\v2.0.50727\installutil
installutil $targetFileName

Notice the script file takes some arguments (targetDir, targetFileName and snapInName). Using that information, it installs the SnapIn (and our SharePoint cmdlet). Configure Visual Studio to pass the arguments to the post-build event:

After a SnapIn is installed, you have to add it to your environment before it can be used. As you can see, even though our custom snap in has been installed, it is not listed when you call Get-PSSnapin:

To add it, call Add-PSSnapin passing in the name of the assembly:

We don't want to add the snap in after each code change, so we are going to automate it by adding a console file that will add the snap in into our environment when the project is debugged. Add a new file named SharePointSamples.Console.psc1 to your project. Put the following xml in the file:

<?xml version="1.0" encoding="utf-8"?>
<PSConsoleFile ConsoleSchemaVersion="1.0">
<PSVersion>1.0</PSVersion>
<PSSnapIns>
<PSSnapIn Name="SharePointSamples.PowerShell" />
</PSSnapIns>
</PSConsoleFile>

In the project configuration, Debug settings, configure Visual Studio to start PowerShell when you debug and in the command-line arguments section, use the -PSConsoleFile argument to specify the console file you created:

Implement the Cmdlet

Now that deployment and debugging are taken care of, add the following code to the cmdlet:

The cmdlet takes the url of the site as an argument. Arguments are implemented as class properties decorated with the Parameter attribute. The work is implemented in the ProcessRecord method. As you can see, I already did some re-factoring and created a service locator class that will give me an instance of the SharePoint Webs service (using WCF). The code then invokes the GetWeb method and writes out the resulting xml.

Here is the code for the service locator. As you can see it configures the SharePoint Webs proxy:

Most of the key WCF configuration settings for integrated authentication are implemented in the BindingFactory class:

Test the Cmdlet

To test the cmdlet, run the project. When the PowerShell command prompt opens, type Get-SPWeb. You will be prompted for the site url if you don't provide it:

Summary

In this walkthrough, you learned how to leverage PowerShell, WCF and the SharePoint web services to implement a custom cmdlet. Using this approach, you can expand the administrative capabilities of SharePoint and build reusable, task-oriented building blocks to increase your productivity and capability.

Get the code for this article.

Additional Reading

How to Create a Reusable WCF Service Host for Windows Services and Self-Hosting

johnwpowell — Sat, 20 Sep 2008 23:45:00 GMT

In this article, I'll show you how to create a reusable service host that can be run from either from the console (self-hosted) or installed as a Windows service. I'll also demonstrate how to use a configuration file instead of code to control which WCF services the host starts. Finally, I'll show how you can make the Windows service installer flexible to allow you to run multiple versions of the service on the same machine. Before getting into implementation details, I'd like to share my motivations behind this solution.

Why I Developed This Solution

If you have been doing WCF development, you have probably experienced some pain around service hosting. Some of the things I found frustrating on my first WCF project included:

Making a decision on the type of service host
Discovering it's more work to host as a Windows service than IIS!
Installing, starting, stopping and uninstalling a service during development
Re-writing the same Windows service plumbing for each WCF service
Having to change code if I wanted to add or remove the WCF services that are hosted
Trying to host multiple versions of a service on the same machine

Having learned a few things, but by no means an expert, I decided to implement a more flexible and reusable service host to meet these goals:

Create a single Windows service project that can be shared by all WCF services in the solution
Keep the service project separate from the service host project
Ability to run the service host from the console or a Windows service
Ability to test the service host during development without having to install it
Ability to host multiple versions of a service on the same machine in production
Ability to control which WCF services are hosted without having to make code changes

How it Works

Now that you understand the why, I'll show you how the service host works starting with the entry point: Main(). The output type of a Windows service project is an executable. When the service starts, it calls the executable. Here is the default Main method for a Windows service project:

As you can see this code creates an instance of the service and runs it. To execute this code, you have to install the service and start it. Although it seems like you could run the executable from the command line, you will receive the following error if you try:

So to get around this, we are going to send an argument to the executable that switches it between a Windows service and self-hosting. During development, we can configure Visual Studio to pass the command line argument to test the code. In production, this same mechanism can provide additional diagnostic capabilities. Here is the new implementation of Main():

Running the Service Host from the Command Line

When run from the command line using the self-hosting argument, the service host starts the WCF services and outputs information to the console:

Running the Service Host as a Windows Service

To run the service host as a Windows service, you first must install it. To do this, copy the output of the service host project to a directory on the server. If there are two versions of the service to run, you'd copy the output to different folders (you also need to configure the services to have different endpoints in the configuration file).

To install the service, run installutil on the service host executable, passing different parameters for the name and displayname, for example:

cd C:\Windows\Microsoft.NET\Framework64\v2.0.50727

installutil /name=CalcV1 /displayname="Calc V1" /description="Sample Calculator Service" "c:\CalcV1\WcfSample.Service.Host.exe"
installutil /name=CalcV2 /displayname="Calc V2" /description="Sample Calculator Service" "c:\CalcV2\WcfSample.Service.Host.exe"

After installing, you will see the services in Windows:

To update the WCF service, stop the Windows service, copy the new WCF service dll and restart the Windows service. There shouldn't be a need to update the service host executable. To uninstall the service, use the /u argument, for example:

installutil /u /name=CalcV1 "c:\CalcV1\WcfSample.Service.Host.exe"
installutil /u /name=CalcV2 "c:\CalcV2\WcfSample.Service.Host.exe"

Using the Same Service Host for Multiple WCF Services

In the previous section, you saw parameters passed into the service installer (note the forward slash parameter name format such as /name). In the service host project, there is an installer component. In that component, you can access the parameters through the Context object. So instead of hard-coding the service name, we set the service name from the parameters. That's all there is to it.

Controlling Which WCF Services are Hosting Using a Configuration File

Now, let's examine how to control which WCF services are hosted using a configuration file. I was initially planning to iterate the service elements in the system.serviceModel section and start all services. You can see an example in this post and this post. I found two problems with this approach. First, you can't control which services to start--like if I wanted to have the configuration for five WCF services, but only start one. Second, the service name element doesn't allow you to specify an assembly name, just the namespace and class. I need the assembly name to load the WCF service type at runtime. To overcome both of these problems, I created a configuration section that allows you to specify which services to start. Here is a sample configuration showing this:

The service container class gets the custom configuration section and iterates through the services list and starts each one:

Separating the WCF Service from the Service Host

One of the goals I had for this solution was to separate concerns by keeping the WCF service separate from the service host. In the example below, you can see how the Calculator service is in a separate project than the Host. The only coupling between these two projects is necessary. The Host project has a project reference to the WCF service project so we can load the service assembly at runtime. In this example, there is only one service project, but you can imagine a separate project for each type of service.

Conclusion and Link to Download the Code

This article presented an approach for creating a reusable WCF service host that enables you to specify which WCF services to run using configuration and support hosting multiple versions of a service on the same machine. Hopefully, the techniques presented here improve your WCF service development and deployment experience.

Get the Code