The Synchronizer : SQL2008

Sync Designer in VS2008 SP1

Rafik — Fri, 30 May 2008 23:43:00 GMT

I am sure you already know that SP1 Beta for Visual Studio 2008 is available to download. There are plenty of enhancements that I am sure the VS folks are covering all over the net with different blogs. As a sync guy, I am on the lookout for everything new in sync and SP1 has made a lot of improvement to the Sync Designer introduced for the first time with Visual Studio 2008.

The designer was revamped quite a bit and it much more developer friendly now. The biggest improvement however is the ability to tap into SQL Server 2008 Change Tracking feature. The designer gives you the ability to use Change Tracking instead of the traditional trigger based model (a.k.a. coupled tracking as tracking columns are coupled with data columns in the base table).

Under the covers, the Sync Designer uses a newer version of the SqlAdapterBuilder to generate the adapter that works against SQL Change Tracking. The new class is part of the newer beta bits of the Sync Services for ADO.NET. Developers can dynamically generate the commands be configuring the builder to run against an instance of SQL Server 2008 with Change Tracking enabled on the target table.

Guy Burstein has written a really good post on the new changes in the sync designer with nice snapshots including the one I posted here. Make sure to check it out.

Update: Just to let you know, I left Microsoft to start a new company, Raveable Hotel Reviews. See Examples: Romantic Hotels in Myrtle Beach, Top 10 Hotels in Seattle, Best Hotels in Miami, Best Hotels in San Francisco, Hotels with in-room jacuzzi and 5 Star Hotels in Los Angeles. Your feedback is welcome on twitter.com/raveable, raveable blog.

I am not actively blogging about Sync Technologies. Please see Sync Team Blog for more updated content.

Metadata is the Hart and Sole of Peer Sync

Rafik — Fri, 30 May 2008 01:09:00 GMT

Occasionally, I receive questions about using SQL Change Tracking with PeerSyncProvider. People who examine SQL Change Tracking in preview version of SQL Server 2008 have become fans rather quickly and see the value of using it. Using change tracking with ServerSyncProvider scenarios is a piece of cake and will get easier with the VS2008 SP1. Now how can you do the same with PeerSyncProvider?
The short answer is that it is not so easy. Not as easy as with ServerSyncProvider. The reasons might not be so obvious and I see many attempts of writing the commands for PeerSyncProvider adapters to work against SQL Change Tracking. While it might work for very limited scenarios, it is largely incorrect.

Again, the question is why?

Well, let’s think a little bit about the following SyncAdapter commands for the PeerSyncProvider:

InsertMetadataCommand
UpdateMetadataCommand
DeleteMetadataCommand

Notice that these commands are intended to make changes to the tracking metadata. That’s rather interesting and clearly different from the commands you supply to for ServerSyncProvider. Now it is becoming clear that we have a problem since SQL Change Tracking metadata (i.e. CHANGETABLE) are all read-only!

General Purpose Tracking Infrastructure

Remember that SQL Change Tracking is a general purpose tracking mechanism. You enable change tracking for a table with the sole purpose to be able to ask “what have changed since given version?” If applications are allowed to make changes to the metadata then this question is not reliable and thus useless. Making change tracking metadata read only makes all the sense in the world for its general purpose nature.

Direct Metadata Manipulation

An obvious question here is: Why make direct changes to the metadata? Changes to the base table will be reflected in the CHANGETABLE, wouldn’t that achieve the same effect?
Well, for peer to peer data synchronization… you need more control than that. Let’s take the following two cases:

Synchronizing Deletes

Let’s consider the following example, three nodes A, B, C. The nodes are connected as shown in the diagram. Follow the following scenario:

Make a change R on node A
Let A sync with B, B now has R
B deletes R
Let B sync with C. B will send the deleted row metadata to C

As B received the metadata for the deleted row, it needs to persist is. R never make it to B thus a normal delete command won’t generate the metadata we desire as it simply won’t do anything.

As a side note, the delete metadata must be persisted on B. Since B could be connected to another node (A, for example) which have R and thus needs to pass on the delete during subsequent sync. In P2P environment, one cannot make assumption about the other nodes and must maintain metadata for every deletes.

At a first glance one can come up with a clever work around for this scenario which is insert R then delete it right away. SQL Change tracking should record that, right? That’s right but the workaround cannot survive two or more tables with PK-FK relations which is the reality of most databases. Thus this workaround is short lived.

Resolving Conflicts

When conflicts are detected and a winner is chosen, one needs to update the metadata such that this winner will propagate across the topology of nodes. That said, the version of the row needs to be pumped. Now consider the case where the local row was selected as the winner. With SQL Change Tracking, there is no way to make changes to the version as the metadata is not editable.

My point in this post is to answer a common question and illustrate the underlying technical reasons behind the limitation. That being said, you can still leverage SQL Change Tracking for Peer Scenarios albeit you need to use a side table to store the changes to the metadata. I am sure some of the readers will find this as a worthwhile challenge to tackle. Happy Sync’ing!

Update: Just to let you know, I left Microsoft to start a new company, Raveable Hotel Reviews. See examples: Top 10 Hotels in Myrtle Beach, Best Hotels in New York City, Romantic Hotels in Seattle, Hotels with a kitchen or kitchenette and Top 10 Hotels in San Francisco. Your feedback is welcome on twitter.com/raveable, raveable blog.

I am not actively blogging about Sync Technologies. Please see Sync Team Blog for more updated content.

New Release: Windows Mobile Line of Business Solution Accelerator 2008

Rafik — Sun, 16 Mar 2008 22:42:00 GMT

Rob Tiffany, the mobile accelerator guy, just announced the new release of Windows Mobile Line of Business Solution Accelerator 2008. It a collection of windows mobile centric samples that showcase different new technologies and help developers greatly accelerate building their applications, thus the name “Accelerator”. Rob also has a great blog that I read regularly. Here is a quick overview of what this new release contains:

Adapt your App
Create a single binary that runs unchanged on Windows Mobile Standard or Pro, Portrait or Landscape, Rectangle or Square. No more wasting time building separate executables to accommodate different screen sizes or input methods.
Sync Services for ADO.NET
Synchronize your data between SQL Server 2008 and SQL Server Compact 3.5 using the new Sync Framework. Keep all your occasionally-connected mobile workers on the same page.
Windows Communication Foundation (WCF) Store and Forward
Reliably push messages to servers or other devices via Exchange Server 2007. Programmatically notify peer devices that they have new orders waiting for them and need to sync.
MapPoint
Guide delivery drivers to their customers via either the shortest or quickest route. Integrated mapping means you’ll never get lost again.
LINQ
Use the new Language Integrated Query to filter results from Generic Object Collections. Query both your objects and XML using a familiar, SQL-like syntax to boost developer productivity.
Custom Controls
Capture signatures and dazzle your end-users with 3D and Alpha-blended controls that alter their behavior depending on the platform they’re running on.
Managed Stored Procedures and Triggers
The pluggable data layer allows you to say goodbye to compiling Dynamic SQL inside your code and fires events to react to INSERT, UPDATE, and DELETE operations.
Notifications and Online Help
Formerly only supported on Pro, say hello to Popup Notifications and Online Help on Standard. Popup Notifications, also known as “toast,” display an HTML message and then disappear after a pre-determined amount of time. Using Online Help on every screen reduces your application training costs.
Language Switching and Localization
Change Language/Regional Settings inside your app and watch text and Online Help speak a different language. Don’t wait until your application is finished to realize that it needs to be world-ready.
Time to Market
Stop reinventing the wheel and use this Accelerator as the foundation for your next Windows Mobile development effort. If you don’t want to use the whole thing, pick and choose the components that are the best fit for your project.

I am playing with it right now.. pretty cool stuff. Download Now

Update: Just to let you know, I left Microsoft to start a new company, Raveable Hotel Reviews. See examples: Top 10 Hotels in Myrtle Beach, Best Hotels in NYC, Romantic Hotels in Seattle, Kid Friendly Hotels in Orlando and Top 10 Hotels in San Diego. Your feedback is welcome on twitter.com/raveable, raveable blog.

I am not actively blogging about Sync Technologies. Please see Sync Team Blog for more updated content.

SQL Server Change Tracking vs min_active_rowversion()

Rafik — Wed, 06 Feb 2008 23:02:00 GMT

In a bit old post I wrote about min_active_rowversion() as the best way to get the new anchor showing you that @@DBTS has its flaws and thus should be avoided. Min_active_rowversion() was introduced in SP2 for SQL Server 2005. If you have code that still uses @@DBTS, you are running the risk of missing changes from the server, do yourself a favor and change it now.

Despite the a fact that min_active_rowversion() addressed a major problem for sync developers, it did not solve it. Check out the diagram below:

T1 made a change to R1 row which received a timestamp of 50. The transaction did not commit and when you call min_active_rowversion() you will end up with the value of 50. That’s quite what we expect; sync should then synchronize up to 50-1 = 49 anchor. So what if we have few more transactions, like T2, that started after T1 and committed soon thereafter. If T1 did not commit, then min_active_rowversion() will remain at 50 until T1 commits. This could be a problem in many situations. You see, min_active_rowversion() did address the problem but did not solve it. Still much better that @@DBTS.

If we dig deeper into what the problem really is, we find that timestamp value is assigned at the statement execution time and not at transaction commit time. Had the row version be assigned at the transaction commit time, we would have a real solution for this problem. Fortunately, that’s exactly what SQL Server Change Tracking does for you in SQL Server 2008. Each transaction gets a version at the commit time and all the rows affected by the transaction will get that version. Check out the diagram below:

SQL Change Tracking will assign version at transaction commit time and all the rows affected by the transaction will have the same version. Now, when you try to sync before T1 or T2 starts, you should get 49, whereas after T2 commits, you will receive a value of 50. You see, now you are not stuck waiting for that long transaction to commits to sync changes that happened in subsequent transactions. Pretty cool, huh! …

SQL Server Change Tracking is a reliable way to track changes and designed from the ground up to address the needs of data synchronization applications!

I am not actively blogging about Sync Technologies. Please see Sync Team Blog for more updated content.

Katmai is Slowly Becoming the Best Platform for Building Offline Applications

Rafik — Wed, 13 Jun 2007 01:15:00 GMT

Katmai is the code name of the next release of SQL Server slated for next year. Last week, CTP3 was released to public, more info here. The new features included in CTP3 are eye popping especially if you are in the sync space. You might think that Sync Services was the driver for many of these features, however, that is not true. CTP3 has new components that I think are very useful and will help developers build offline application faster. That’s not to say that there won’t be sync specific features coming up later on, but I will refrain from mentioning anything now and wait till things unfold. For now, let’s talk about CTP3 features:

Change Data Capture (CDC)

Change Data Capture (CDC) is a generic component that will track database changes asynchronously and expose the changes through a relational interface which can be consumed easily.
Through this interface, users can very easily track changes based on their specific requirements and consume the change data using T-SQL or other data access methods.
CDC can also be used as for building download-only offline application. Why not upload too? Because the tracking is asynchronous, thus you cannot rely on the metadata for conflict detection and resolution.

MERGE SQL Statement

Common data warehouse scenarios require developers to either insert information or update information. SQL Server 2008 will provide new statements within the SQL language to enable developers to provide this functionality with a single statement.
This is very useful for writing sync commands or stored procedure; you don’t need to write a separate command or sproc for insert and update. One merge statement would work for both.

There are more features in the CTP3, but I am only interested in those which make sync developers live easier. It seems to me that build up of features that are useful for developing sync application will make Katmai the best platform for authoring offline applications. Keep an eye on it!

Update: Just to let you know, I left Microsoft to start a new company, Raveable Hotel Reviews. See Examples: Romantic Hotels in Myrtle Beach, Top 10 Hotels in Seattle,Best Hotels in Miami, Best Hotels in San Francisco, Hotels with in-room jacuzzi and Best Hotels in Los Angeles. Your feedback is welcome on twitter.com/raveable, raveable blog.