Smart Client Data : Blogicles

Splitting Typed DataSets from TableAdapters - Sharing Validation Code across the tiers

Steve.Lasker — Tue, 21 Feb 2006 21:06:00 GMT

You’re building a Smart Client application. You want this Smart Client to be smart enough to provide validation feedback to the user as they enter information. However, in order to make the application scale to thousands of users you can’t open connections directly to the database from the client because you’re server can’t handle that many connections. You probably don’t want the data access information to be available on the client anyway so you turn to Web Services to help abstract the intimate knowledge about your database. This way the client only has Web Service address and authentication information. Since web services can be consumed by many different types of clients you don’t want to trust that all the validation rules have been enforced. So, what do you do?

You could put all the validation rules in the database, but that means your database has that much more processing to do, and your Smart Client won’t be so smart. It will still have to make round trips to the database each time a user tabs off a control.
You could use the XSD schema validation extensions, but they aren’t all that extensible and they don’t deal with dynamic values.
You could write the validation code twice. Once for the server, then again for the client. But that wouldn’t be all that maintainable.

The likely solution is to simply write the validation logic and share it on the client and within the web service. But wait, that’s type sharing… that’s bad… Well, yes and no. In the old days of COM/DCOM and even with Remoting you really were sharing the same actual object. The client would instantiate a component and it would actually run on a remote server. In addition to scalability issues, this also had a maintenance issue. If you ever updated the component on the server, even if it didn’t involve changing the public interface, all the clients had to be updated as well. To make this more complicated, since clients are running statefull operations on shared servers you usually had to completely shut down the entire system to make a simple update. Web Services solves these problems as the object being used and what travels across the wire are really individual things. The serialization format of a particular object has no real direct correlation to the actual object. Yes, certain objects serialize certain types of formats, but that’s not a hard and fast rule. I can talk in English, French, German, Hungarian, but I’d still be saying the same thing. Likewise, 3 different people can all say the same thing in English, and it would have the same meaning. Once you separate the wire protocol from the functionality you can change the internal implementation all you want, and as long as you don’t change the public interface and your clients can continue to consume the service.

Now I know that many would argue that changing functionality within a service is breaking the contract. And to some extent you’re correct. If you’re exposing a public service, and don’t own all the clients then you do have an obligation to maintain consistency. However, if you do own the clients and the services are only consumed as part of your application then this becomes a very flexible model.

Take the following scenario:

You roll out your application over the weekend. Thousands of users fire up the app first thing Monday morning. They all get the updated version with ClickOnce deployment. Life is good. Later that day you notice that you forgot to put a validation rule in that all orders can’t be shipped for 7 days after the order date. You quickly add the validation rule to enforce DueDate must be Today + 7 Days and you add a default value that DueDate is Today + 7 days, but how do you deploy this? You could announce over the PA system that everyone must exit the sales system for 5 minutes which effectively shuts the company down and directly affects the profits of the day. That probably wouldn’t reflect well on your next review.

Or, you could update the validation logic on the server without touching the client. Since the public interface didn’t change, the clients continue to function. Since ASP.net Web Services use something called Shadow Copied assemblies, you aren’t blocked from updating an assembly in the web server. Once the file is finished being copied, the webserver detects that one of the assemblies that’s using has changed and reloads that assembly into memory. The next request will immediately get the new functionality. At this point the client doesn’t have the validation logic, however, when the sales rep attempts to save an order the additional validation logic will kick in and send back the error to the client. If the developer updated the client app with the new assembly then the clients will get updated the next time they restart their app.

You could use the ClickOnce background APIs to constantly check for updates, and that’s a good thing but you don’t want to be checking or updates ever 10 seconds. That sort of defeats the purpose here. In the above example, you were able to immediately affect a change without interrupting any of the sales reps which means no downtime, and you’ve got a great review again.

So, how do you implement all this? The basic concepts, including some of the proxy generation issues of Web Services equally apply to custom objects, but for this article I’ll focus on how to leverage Typed DataSets to enable validation on the client but enforce it on the server.

One of the major features of Visual Studio 2005 is the new typed data access components called TableAdapters. These can be used within your Data Access Layer to handle all the CRUD operations to the database. TableAdapters are effectively strongly typed DataAdaters. These bring parity with the Typed DataSet experience in Visual Studio 2002. One of the VS 2005 enhancements of Typed DataSets is the use of partial classes. In 2002/03 it was very difficult to add custom logic to the Typed DataSet. Since Visual Studio uses something called Single File Generators to generate the Typed Datasets, any code the developer added to the Typed DataSet would be lost the next time the generator ran. Developers would sometimes inherit from the Typed DataSet but that had lots of other complications as well. It further complicated things as you now had two types; the one generated by Visual Studio and the one that had your validation code.

Visual Studio 2005 Typed DataSets leverage a feature known as partial classes. Using partial classes developers can now directly enhance the functionality of the generated Typed DataSet without the possibility of being overwritten by the Single File Generator. This means we can now easily add our validation code directly to the Typed DataSet and enforce it on the server. However, because Visual studio generates the Typed DataSet in the same file and project as the TableAdapters we don’t have an easy way to leverage the Typed DataSet on the client. We could add a reference to the assembly that contains the Typed DataSet and TableAdapters on the client, but now the client has the intimate knowledge of the server that we’re trying to avoid?

Great, so now what? Well, it turns out it’s not all that difficult to separate these. It’s not very discoverable, but it’s not that difficult. What we’re going to do is separate the Typed DataSet from the DataLayer into its own DataEntities assembly. We’ll create a Web Service to return the Typed DataSet and use the DataLayer assembly behind the web service. The client application will share the reference to the DataEntities assembly on the client but it won’t have the DataLayer so we’ll keep the intimates of the database away from the client. Since Web Services serializes DataSets as XML we can leverage the same type on both sides of the wire, but we’re not actually sharing the same instance so we get a lot of flexibility in how we update the application.

The rest of this article focuses on a walkthrough for how to accomplish this. I’ll highlight some of the best practices for this scenario to minimize complexities for working around the default behavior of the tool.

To get things started we’ll create a solution with the projects to represent the logical and physical tiers.

Creating the solution

1. Using Visual Studio 2005 create a new solution containing 4 projects. It doesn’t matter whether you’re using VB or C#.

Project Name	Project Type & Description
Client	A Windows Forms project
DataEntities	A Class Library that will contain our Typed DataSet with validation logic
DataLayer	A Class Library that will load and save the Typed DataSet in the DataEntities dll
BizServices	A WebService that will simply wrap calls to the DataLayer

2. Delete Class1 in the Class Libraries

3. Delete the Service.asmx and Service.vb/cs file in the App_Code directory

4. You should now have a solution that looks something like the following:

Adding a Typed DataSet/TableAdapters to the DataLayer

1. With the DataLayer selectged, open the Data Sources Window. (You can do this using the Data Menu and select Show Data Sources).

2. Add a new Data Source and choose Database

3. Add a connection to your database. In this walkthrough we’ll use the Northwind database

4. Using the treeview, choose your tables. We’ll simply select the Orders table for this walkthrough. You could use sprocs or views, but to keep this walkthrough focused, we’ll just select the orders table and press finish.

5. You should now have a Typed DataSet and TableAdapter for the Northwind.Orders table

Moving the DataSet to the DataEntities project

We’ll now cut and paste the Typed DataSet definition from the DataLayer to the DataEntities project to isolate the intimate knowledge of the database from the data entities.

Show the hidden files by pressing “Show All Files” button in the top of the solution explorer
Open the NorthwindDataSet.Designer.vb/cs file
In this file you’ll see two major sections. The definition of the Typed DataSet and a namespace for all the TableAdapters associated with the Typed DataSet
It’s easiest to see the source if you collapse the regions for the Typed DataSet and the TableAdapter namespace

Copy the source code from the first line through the definition of the Typed DataSet
In the DataEntities project add a new Class File and name it the same as your DataSet. You’ll want to add the .Designer extension to maintain separation between the generated code and the validation code we’ll add later on. In this case we’ll create NorthwindDataSet.Designer.vb
In this file paste over the entire contents with the source you copied from the DataLayer.dll
Back in the DataLayer project, delete the DataSet definition. Be sure to leave the headers, Imports and if using VB, the Option Strict code

Fixing up the references

If you build the project you’ll now see a bunch of errors in the task list. The TableAdapters reference a type that is no longer “visible”.

In the DataLayer project, add a reference to the DataEntities project. This will solve the issue of finding the types.
You’ll now need to add a project level imports to resolve any namespace differences between the entities and DataLayer.
VB: Select the DataLayer project and double click the My Project node to open the Project Properties. Select the References tab and using the Imported Namespaces check the DataEntities namespace. The errors should now disappear from the task list.

C#: Open the NorthwindDataSet.Designer.cs file and add an additional Using statement for the DataEntities project:
using DataEntities;

Once you build the solution you should no longer see any build errors.

We now have data access and data entities isolated and the have the proper references. Notice that while the DataLayer does have a reference to the DataEntites the opposite is not true. This means the DataEntiteis can be used without any database intimates. The next problem is how to get the DataEntites from the DataLayer to the client. This is where Web Services come in. Using Web Services we can abstract the client form the server logic. Note that this same process will work for the upcoming Windows Communication Framework in WinFX.

Exposing the DataEntities via Web Services

In order to expose the DataEntities via Web Service we’ll create some thin wrappers to delegate any calls to the DataLayer

In the BizServices web service project select add new Web Service
Name the Web Service Orders. Visual Studio will create two files. One for IIS to expose as an address, Orders.asmx. And another for the code, Orders.vb/cs
In the BizServices project add references to the DataEntities and the DataLayer projects.
In the Orders.vb/cs file add the following code to expose the Load and Save methods:

Private _ordersTableAdapter As DataLayer.NorthwindDataSetTableAdapters.OrdersTableAdapter

Public Sub New()

_ordersTableAdapter = New DataLayer.NorthwindDataSetTableAdapters.OrdersTableAdapter()

End Sub

<WebMethod()> _

Public Function GetOrders() As DataEntities.NorthwindDataSet.OrdersDataTable

Return _ordersTableAdapter.GetData()

End Function

<WebMethod()> _

Public Function SaveOrders(ByVal orders As DataEntities.NorthwindDataSet.OrdersDataTable) As DataEntities.NorthwindDataSet.OrdersDataTable

If orders.HasErrors Then

Return orders

Else

_ordersTableAdapter.Update(orders)

Return orders

End If

End Function

We now have the ability to return and save Orders to the database. The sample above does do some validation testing within the web service which you’d likely want to place in a separate assembly. You could place it in the DataLayer as well The point is you want to minimize the actual code in the Web Services project so you’ll have more flexibility for where to use this code. For instance, when WCF ships you’ll likely want to leverage some of the new transports. By keeping the code in the web service to an absolution minimum you’ll ease your migration process..
Another thing worth notice is while this sample focuses on a simple 1:1 mapping of the database to the returned DataSet, that isn’t the limitation of DataSets. The TableAdapter wizards are optimized for relatively simple mappings, but using ADO.net commands, Stored Procedures and/or Views, you can do quite a bit of additional logic to differentiate the schemas of your DataSet from the underlying database. Using the merge capabilities of DataSet, you can have a single DataSet or even a single DataTable represent data from two or more database systems. But, that’s a discussion for another day.

Consuming the DataEntities in your WinForms client

We’ll now add the ability to load and save a form full of customers. In the client project we’ll add a Web reference to our BizServices to get the load and save functionality and we’ll add an additional reference to the DataEntities dll.to leverage common validation logic on the client and the server. Note that we won’t have any reference to the DataLayer dll so the client will be clean of data access knowledge.

Using Solution Explorer select Add Web Reference in the Client project. You could use the Data Sources Window, but we’ll want to rename the service and this feature isn’t available in the Data Sources Window wizard.
In the Web Reference wizard browse Web services in this solution. You should now see your Orders service.
Since LocalHost is fairly meaningless, select the Orders Service and rename it OrdersService. Press finish to continue
If you notice the Data Sources Window you’ll see the dataset returned by the web service is now available. What’s important to note is while this dataset is the same “shape” as the data set returned by the Web Service, it’s not the same Type as the DataSet in the DataEntities project. The Web Reference Dialog created a proxy type for the dataset. If you show the hidden files you can navigate through the OrdersService, Reference.map to find the refernce.vb file. This contains the code necessary to call the OrdersService, but it also contains a type definition for the NorthwindDataSet. While it is declared as Partial, we’re not going to leverage this feature.

Due to time constraints in the Visual Studio 2005 schedule we were unable to complete a feature we refer to as Proxy Type Sharing. If you consume multiple web services from the same project that return the same type, Visual Studio will create multiple proxy types on the client. This means you could actually wind up with several TypedDataSets on the client that all share the same shape. However, even if we fixed Proxy Type Sharing, we still wouldn’t have exactly what we need in this scenario. In this case, we already have the type on the client, so we don’t need a proxy. In a few steps, we’ll see how we consolidate these.

Merging the proxy and original types

Although we’ll use the Web Service to get and save orders, we’re not going to use this TypedDataSet to represent our data within our client app. Instead we’re going to use the DataEntities dataset

With the Client project active, use the Data Sources Window to add an Object Data Source. Add a reference to the DataEntities project and select the NorthwindDataSet as your Object DataSource.
Visual studio differentiates Object Data Sources from Database DataSources by providing a different design experience. When the developer chooses Database datasources, Visual Studio creates Typed DataSets and TableAdapters. When you drag & drop Database DataSources from the Data Sources Window Visual Studio will add additional load and save code to your form. For Object Data Sources, Visual Studio doesn’t make any assumptions about how you load and save your objects. Typed DataSets are simply very specialized data object containers that have all the necessary plumbing for serializing and databinding.
With Form1 open, select the DataEntities.Northwind.Orders object in the Data Sources Window and drag it to the form. Be sure you’re getting the right DataSet, it can be confusing as they’re both named similarly.
1. You can remove the Web Service DataSet proxy from the Data Sources Window with a little work around. If you show hidden files in the Client project and navigate through OrdersService -> Reference.map you’ll see a NorthwindDataSet.datasource file. This file is what Visual Studio uses to show object in the Data Sources Window. If you delete this file the entry will be removed from the Data Sources Window but the Web Service will still exist. If you ever update the web reference the .datasource file will be recreated.
Double click the form to get the form.load event and add the following code to merge the DataSet returned by the web service with the DataEntities DataSet.
1. We could do another work around and change the code generated by the web service proxy generator, however this is much more complicated to maintain. Since DataSets and DataTables support merge capabilities, we can avoid changing the generated code. For plain old objects, (POOs) that don’t support merge capabilities, you’ll likely want to edit the proxy generated code.

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

Me.NorthwindDataSet.Orders.Merge(My.WebServices.Orders.GetOrders())

End Sub

Notice that DataTables now support merging directly. In 1.x, you had to merge the entire DataSet. Also note that we were able to return an individual Typed DataTable rather then the entire DataSet. This only works for Typed DataTables, You can’t return a single UnTyped DataTable

Activate the form designer again and enable the Save button that was created in the BindingNavigatgor
Double click the Save button and add the following code to commit any changes made by the user and send the changes to the server.

Private Sub OrdersBindingNavigatorSaveItem_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles OrdersBindingNavigatorSaveItem.Click

Me.Validate()

Me.OrdersBindingSource.EndEdit()

Dim orderServiceOrderTable As New Client.OrdersService.NorthwindDataSet.OrdersDataTable()

orderServiceOrderTable.Merge(Me.NorthwindDataSet.Orders.GetChanges())

orderServiceOrderTable = My.WebServices.Orders.SaveOrders(orderServiceOrderTable)

If orderServiceOrderTable.HasErrors Then

MessageBox.Show(Me, "Errors on attempt to save")

End If

Me.NorthwindDataSet.Orders.Merge(orderServiceOrderTable)

End Sub

Hit F5 to run you’re app
If you look in the bin directory of the Client app you’ll notice we only have the DataEntities.dll and Client.exe files. The DataLayer which contains all the intimate knowledge of the Databas are left out of the picture.
Using ClickOnce, publish the Client project and install it on your machine. Be sure to use the default behavior so you get the Start menu item

Adding Validation Code

Now that we’ve got the client project setup and deployed, let’s add some validation logic. In order to demonstrate the lack of tight coupling, we’ll add the validation logic to the DataEntities project, but we won’t actually deploy this to the client. So, when testing this, be sure not to republish the client.

In the DataEntities project add a new class file. Name it NorthwindDataSet.vb/cs. Note that we’re using the same standard as forms and the DataSet designer to isolate designer generated code from your code.
In the file add the following code:

Private Sub OrdersDataTable_ColumnChanged(ByVal sender As Object, ByVal e As System.Data.DataColumnChangeEventArgs) Handles Me.ColumnChanged

' We use the Changed event as we try to avoid deleting information a user enters

' Rather we show them what they typed, but provide information indicating the specific error

' This helps users understand if they mistyped, or the information is just not permitted

' Use strongly typed names of the columns to benefit from compile time verification

' If the user changed the OrderDate or ShippedDate columns, verify the dates

If e.Column.ColumnName = Me.OrderDateColumn.ColumnName Or _

e.Column.ColumnName = Me.ShippedDateColumn.ColumnName Then

ValidateDates(e.Row)

End If

End Sub

Private Sub ValidateDates(ByVal row As OrdersRow)

' Check for specific column errors and set or clear the errors

' Using this model, DataSets will surface the errors using the IDataError interface

' DataBinding in the DataGridView and ErrorProvider will pick up these errors

' and surface them to the user

' You may want to use strongly typed Resources for the error strings.

If row.OrderDate > row.ShippedDate Then

row.SetColumnError(Me.OrderDateColumn, "Can't ship before it's been ordered")

row.SetColumnError(Me.ShippedDateColumn, "Can't ship before it's been ordered")

Else

row.SetColumnError(Me.OrderDateColumn, "")

row.SetColumnError(Me.ShippedDateColumn, "")

End If

End Sub

Rebuild the entire solution to make sure the updated DataEntites dll gets pushed up to the Web Service project.
With the app still running, change the OrderDate to be after the ShippedDate. Note that if you closed the app from above, simply start it again from the start menu so you get the version that doesn’t have the updated DataEnties on the client. Notice that you don’t see any errors in the DataGridView.
Press the save button. You may notice a slight delay as the Web Service re-computes the updated assembly, but when it returns you should now see ErrorProviders indicating that the ShipDate can’t be before the OrderDate.

Updating the client

To update the client, simply republish the Client project using the ClickOnce deployment wizard and restart your client app.

Wrapping Up

With the above scenarios you were able to make changes to the DataEntities without breaking the client applications. You’re users maintained their productivity, yet you were able to quickly add new validation rules. The clients could then be updated to complete the update, but you didn’t break you’re users workflow in the process. With the basic tooling support in Visual Studio and these tricks you can achieve a truly factored ‘N scale application that leverages code where you want, scales to thousands of users, can be updated mid day without kicking your users off the system and delivers on the promise of Smart Clients for productive users.

Post Whidbey, we are working to enable this scenario without all the work arounds noted above, but for now…

Steve

Working with local databases

SmartClientData — Fri, 26 Aug 2005 23:24:00 GMT

Overview

In Visual Studio 2005 we added a number of features to help developers build and deploy applications that need a local data store. Here's a quick peek at how it works.

In order to work with a local database file, you can simply add the file to your project (e.g. using the Project/Add Existing Item... menu). We currently support adding SQL Server data files (.mdf), Jet (Access) data files (.mdb) and SQL Mobile data files (.sdf). Note that in order to be able to use .mdf files, you need to have installed SQL Server Express. SQL Express is available on the VS CD or at http://go.microsoft.com/fwlink/?LinkId=49251. With SQL Server Express installed, you will also be able to create new databases through 'Project/Add New Item…/Database'.

Once the database file is in the project, VS will do a few things:

1. It will automatically add a connection in the Database Explorer so you can edit the database schema or the data.

2. It will make sure that the connection strings are serialized using a relative path (more on this below).

3. The first time the file is added, VS will also launch the Data Source wizard to create a new typed dataset.

Full path vs relative path

One of the reasons why it was hard to work with database files before is that the full path to the database was serialized in different places. This made it harder to share a project and also to deploy the application. In this version, the .NET runtime added support for what we call the DataDirectory macro. This allows Visual Studio to put a special variable in the connection string that will be expanded at run-time. So instead of having a connection string like this:

"Data Source=.\SQLExpress;AttachDbFileName=c:\program files\app\data.mdf"

You can have a connection string like this:

"Data Source=.\SQLExpress;AttachDbFileName=|DataDirectory|\data.mdf"

This connection string syntax is supported by the SqlClient and OleDb managed providers.

By default, the |DataDirectory| variable will be expanded as follow:

      - For applications placed in a directory on the user machine, this will be the app's (.exe) folder.
      - For apps running under ClickOnce, this will be a special data folder created by ClickOnce
      - For Web apps, this will be the App_Data folder

Under the hood, the value for |DataDirectory| simply comes from a property on the app domain. It is possible to change that value and override the default behavior by doing this:

AppDomain.CurrentDomain.SetData("DataDirectory", newpath)

For customizing the connection string at runtime, please see our team blog at: http://blogs.msdn.com/smartclientdata/archive/2005/07/25/443034.aspx

Where is my data? -- Understanding the file copy for desktop projects

One of the things to know when working with local database files is that they are treated as any other content files. For desktop projects, it means that by default, the database file will be copied to the output folder (aka bin) each time the project is built. After F5, here's what it would look like on disk

MyProject\Data.mdf

MyProject\MyApp.vb

MyProject\Bin\Debug\Data.mdf

MyProject\Bin\Debug\MyApp.exe

At design-time, MyProject\Data.mdf is used by the data tools. At run-time, the app will be using the database under the output folder. As a result of the copy, many people have the impression that the app did not save the data to the database file. In fact, this is simply because there are two copies of the data file involved. Same applies when looking at the schema/data through the database explorer. The tools are using the copy in the project, not the one in the bin folder.

If this copy behavior is not what you want, there are few ways to work around it:

1. If you select local database file in the Solution Explorer window, you will see a property called ‘Copy to Output’ in the Properties window. By default it is set to ‘Copy Always’ which means that on each build, the data files in the project folder will be copied to the output folder overwriting the existing data files if any. You can set this property to Copy Never and then manually put a copy of the data file in the output folder. This way, on subsequent builds, the project system will leave the datafile in the output folder and not try to overwrite it with the one from the project. The downside is that you still have two copies so after you modify the database file using the app, if you want to work on those changes in the project, you need to copy it to the project manually and vise-versa.

2. You can leave the data file outside the project and create a connection to it in Database Explorer. When the IDE asks you to bring the file into the project, just say no. This way, both the design-time and the run-time will be using the same data file but the downside is that the path in the connection string will be hard coded and therefore it’ll be harder to share the project and deploy the app. Before deploying the app, just make sure to replace the full path in the settings with a relative path.

Antoine
Software Design Engineer
Visual Studio Data Design-time

Drag Once w/Custom Controls

SmartClientData — Wed, 24 Aug 2005 17:26:00 GMT

Article written for CoDe magazine describing how to customize the controls listed in the Data Sources Window for Drag Once Databinding

http://www.code-magazine.com/Article.aspx?quickid=0411071

Steve

Drag Once Databinding

SmartClientData — Wed, 24 Aug 2005 17:24:00 GMT

Article written for CoDe magazine relating to the Drag Once Databinding features in Smart Client Applicaitons

www.Code-Magazine.com/Article.aspx?quickid=0409051

Steve

The little database that could. Deploying SQL Mobile as a local / single user database across Microsoft Mobile devices

SmartClientData — Fri, 15 Jul 2005 02:31:00 GMT

The little database that could. Deploying SQL Server Mobile 2005 as a local / single user database across Microsoft Mobile devices including the Tablet PC...(read more)