SQL Server Compact - Compact & Capable : MergeReplication

Merge replication vs. Sync Services for Compact

SQLCEBLOG — Mon, 09 Nov 2009 14:12:00 GMT

SQL Server Compact supports predominantly two different sync technologies. “Merge replication” and “Sync Services”. Users might be using one or the other, and interested in knowing the differences between the two technologies, from a use-case point of view. Still, there might be others interested in knowing, which technology better suites their need, by comparing and choosing the right one. One simple rule of thumb here: Merge replication is designed keeping enterprises in mind, where Sync Services is a framework for developer community/social networking/hobbyist programmers etc… (everything else). “Merge replication” is a solution, which is suitable for Enterprises mostly, in preference to Sync services. However, Sync services is an open-ended, extensible framework on top of which, one can do almost everything that Merge replication does. This article concentrates mostly on the merge replication angle in terms of where it makese more sense and where it doesn't.

Where does "Merge" make more sense?

In the enterprise scenario. In my opinion, enterprises like things faster, better (without any issues), and easily do-able/repeatable. If a solution (like “Merge replication”) allows them to do things like “setting up a publication, subscribers etc…”, although with significant coding, people are not going to like it. So, tools and wizards for enterprises are mandatory (Also give them a way to re-do the setups, typically outputting scripts from these tools/wizards). To avoid the setup/upgrade glitches, provide an out-of-the-box integration story, with other often used components (like Windows, IIS, SQL Server, HTTP etc…). Also, tighter integration helps performance. Usually, in enterprise contexts, setting up a “data server and clients” is a long-term commitment, so, tighter integration is fine there. All these are basic requirements for the “Enterprise” oriented “merge replication” technology.

Note: There are many ways in which a tighter integration with other components helps deployments.

1. Efficiency/performance

2. Automation possibility and tooling

3. Better end-2-end support/solution story for the customers

4. Ability to monitor and troubleshoot parts of the system

5. Essentially, you leave all the hardwork to us, and in the end, expect something that automatically works J (Means, reduced cost of development and testing for you)

Also, my take is that “Enterprises like rich (and relevant) features”. Give them something that can easily model/extend their business logic, and that could be of great value to them. Some of the features of “Merge replication”, in this category are,

1. Automatic partition management

2. Custom conflict resolution hooks

3. Business logic plugins to do custom processing, in the process of merge replication

a. Example is, if one wants to charge the client for synching

b. One wants to compute his share whenever a salesman syncs to the server

4. Retention cleanup, which can be used to (weakly) set a stringent policy about incremental syncs.

a. Like, every salesman must sync at least once in a week or every branch office, must sync at least once a day.

5. Incremental schema changes replication

6. Integration with SQL Server mirroring

7. Integration with SQL Server backup and restore.

Other valuable features are, “Automatic identity range management” etc. I hope now it is very clear as to, why “merge replication” technology is good for enterprise businesses. Enterprises do not mind, buying into a special architecture, if that provides value. Enterprises can setup a particular set (at the least) of windows machines, with IIS and SQL Server, and run merge replication. So, a tailored solution, like merge replication is best suited for them.

Where does Sync play better role compared to Merge?

Now, let’s examine another use-case. Incidentally, in this use-case, the features of merge replication, do not seem much relevant. You want to write a stock-alert application, and link it with a stock-tick source on a website. On closer examination, this is also a “data sync” scenario, so, merge replication could be used here. Trying to use merge replication here, elicits many incompatibilities/redundancies, that are not relevant to this use-case.

First of all, the nature of the data source is unknown, it can change while the app is running too. The web-site might use SQL backend now, and something else tomorrow. The transport is not known, it is over HTTP, but, can have any format (plain HTML/Json etc…). There is no need of rich data semantics here, as one is just comparing a single piece time-series data, and deciding to throw the alert or not. Also, one does not need a big database at client, all he does is read and discard (or read, alert and discard). Clearly, using merge replication here is an overkill, and unsuitable. So, you should use “Sync services”.

If you want to write a very quick sync app, and use it anywhere*, “Sync services” is the answer. Because Sync Services, is not tied to any particular, server architecture or transport mechanism, one can use it anywhere*. For quick development of sync apps, Sync Services is integrated with Visual Studio. But, this is only to develop the app quickly, not to deploy it. Visual Studio is not even needed for plain development of any sync app. It is just there to make it easy and fast.

Sync services is amazingly extensible, componentized, customizable. While merge replication answers enterprise use-case, which is a typical use-case for data synchronization, virtually all other use-cases are addressable by “Sync services”. If for some reason, people do not want to use a particular server/transport architecture, (or can’t use a single architecture), Sync services is the way to go. Besides, Sync services is a free platform to develop apps on. It is a great tool to enable non-(traditional)-enterprise related, businesses/users to realize the new models of interaction cropping up almost everywhere now. Sync services can be made to have feature-parity with “merge replication” solution through coding, although, I hope you would agree that it is not the intent of Sync services. I quoted that here, only to make you realize the potential of Sync services platform.

When an open-ended sync solution needs to be developed, Sync Services Framework is the one to use. Many times in usage, applications to sync data, need heterogeneous servers/transports etc. This is a very important capability, for example, for news generation and propagation web-site. News could be present in various web-sites, and also inside documents/databases of different formats. Also, the solution should be open-ended and extensible, so that, it can be easily tailored for a new data source. The importance of “Sync services” in enabling such scenarios, should not be underestimated. Sync Services, is coming out strong, and there is more to watch out in this field, going forward. With this, we conclude the explanations of the technologies, from the use-cases point of view.

The list of features of merge replication and Sync services are tabulated below:

Feature	Supported in merge replication	Supported in Sync Services
Enterprise-centric
Type of the technology	Solution	Framework
Target users	Enterprises with DBAs	Developers/Hobbyists/Social networking Communities
Integration with DBA tools (SQL Server)	Yes	No
More tooling support	Yes	No
Pluggable business logic hooks	Yes	No
Conflict resolution support	Built-in + support for custom***	Built-in + extensible**
Schema propagation support	Initial + any changes	Limited, available by extension**
Partitioning support	Built-in	Available by extension
Server type (State-full or State-less)	State-full	State-less
Integration with Server mirroring (SQL Server)	Yes	No
Integration with Server backup/restore (SQL Server)	Yes	No
Identity ranges management	Yes	No

Developer centric
Type of the technology	Solution	Framework
Target users	Enterprises with DBAs	Developers/Hobbyists/Social networking Communities
Developer platform support	No	Yes, Visual Studio integration provided
Pluggable transport	No (HTTP with IIS server only)	Yes (Web service model is possible, by extension)
Heterogeneous server	No (Only SQL Server)	Any server is good (by extension).
Network architectures	Fixed, 3-tier	Variable, 2-tier to N-tier
Supports web services model	No	Yes
Ability to work with other sync platforms/frameworks	No	Yes (by extension)
Exposed API surface for tracking****	No	Yes

Common functionality provided
Subscribed database deployment	Yes	Yes
Type of change tracking used	ROWGUID	PK or ROWGUID
Sync directions allowed	All	All
Programmability layer	Native & managed	Managed

Semantics provided
Auto-management of dependent tables	Yes	No
Sync granularity enforced	Yes (Publication level)	No (table level)
Schema propagation	Yes	Limited extent
Built-in conflict resolution strategies (also customizable)	Yes	Only some limited number of built-ins

* Well, almost everywhere.

** Whenever we use, “available by extension” or simply “by extension”, that means developer should write code to achieve the desired behavior.

*** (For this document,) the difference between customizable and extensible is the following: customization is a type of extension, where, the application architecture is not affected majorly. Like, register a COM dll, to do conflict resolution etc…

**** Tracking is a mechanism used to tag all changes. Changes could be Inserts/Updates/Deletes on tracked tables. This module is used to detect changes between successive syncs, so that the data can be forwarded to the other party. Exposing the tracking API (enable/disable tracking etc…), helps write applications like peer-peer sync, transaction notifications etc…

Wrap-up:

We have looked at the use-cases where merge replication technology is suitable and where it is not. There are scenarios where “Sync services” comes out as a candidate for solution, and there are scenarios for merge replication too. When considering the right technology for using, there is no silver bullet; there is no panacea that works for all needs. The requirements of your use determine the “right” technology for you. Merge replication provides a solution, but, there are many scenarios that it can’t help you with. Sync services can be made to work in “any” scenario (including the one that merge replication provides out-of-the-box J), but, are you fine with building such a (mammoth) architecture yourself, and incurring the various costs involved? In merge replication, we take the pains and give you a solution, although, for a special, tailored need, is also happens to be the most common and justified one. In sync services, you are on your own, but, it gives many benefits that you can’t do without in many situations.

Contributor: Udaya Bhanu Goteti

Overview of SqlCeReplication methods - LoadProperties and SaveProperties

SQLCEBLOG — Fri, 06 Feb 2009 18:22:00 GMT

Whenever sync is started, developers provide all the necessary properties to an object of SqlCeReplication class. These properties are provided through the constructors or by directly setting the public properties of the object. In this approach developers need to provide this data for every sync, which forces them to store this data somewhere in their code. Even if the properties seldom change, developers provide this data all the time. This data includes passwords such as DistributorPassword, InternetPassword, InternetProxyPassword and PublisherPassword which developers are better off, not saving or handling in their code.

This blog article throws light on SaveProperties/LoadProperties methods which provide a good solution to issues described above. SaveProperties saves all the properties a developer can provide to SqlCeReplication object (except for two which are described below) in a system table called __sysMergeSubscriptionProperties within the replicated database. The benefits of this approach are:

1) The replicated database can be encrypted securing this data inside the sdf file.

2) Passwords are encrypted with the unique device hardware key enabling an extra layer of security.

3) Data that seldom changes need not be explicitly provided by developer for every sync

Code example for calling SaveProperties:

// Example code for SaveProperties starts

repl = new SqlCeReplication();

repl.InternetUrl = "http://www.adventure-works.com/sqlmobile/sqlcesa35.dll";

repl.InternetLogin = "MyInternetLogin";

repl.InternetPassword = "<enterStrongPassword>";

repl.Publisher = "MyPublisher";

repl.PublisherDatabase = "MyPublisherDatabase";

repl.PublisherLogin = "MyPublisherLogin";

repl.PublisherPassword = "<enterStrongPassword>";

repl.Publication = "MyPublication";

repl.Subscriber = "MySubscriber";

repl.SubscriberConnectionString = "Data Source=MyDatabase.sdf";

repl.SaveProperties();

// Example code for SaveProperties ends

Note: SaveProperties will succeed only if the Subscription is already created on the machine.

This simplifies subsequent syncs in that developers need to provide only SubscriberConnectionString (to identify the local database) to SqlCeReplication object and call LoadProperties. If __sysMergeSubscriptionProperties table has only one record this data is automatically loaded into SqlCeReplication object. If there are more records, then in additional to SubscriberConnectionString, developers need to provide Publication, Publisher and PublisherDatabase to uniquely identify a saved profile for a subscription.

Code example for calling LoadProperties:

// Example code for LoadProperties starts

repl = new SqlCeReplication();

repl.SubscriberConnectionString = "Data Source='Test.sdf'; Pwd='<enterStrongPassword>'";

repl.LoadProperties(); // assuming only one subscription’s properties are saved

// Example code for LoadProperties ends

LoadProperties/SaveProperties thus provide an efficient way to store and retrieve data relieving developers from maintaining sensitive and unchanging data.

Properties not saved by SaveProperties:

SendTimeout and ReceiveTimeout properties are not saved via SaveProperties. These values must be explicitly provided to SqlCeReplication object for every sync.

Note:

There are two things to note while using this functionality.

1) Before calling SaveProperties, please make sure all necessary properties are set on the SqlCeReplication object. If we call SaveProperties with required properties missing, this data will be set to default even if there is a saved profile already present for this subscription in the Properties table.

2) Passwords stored are encrypted with the unique device hardware key. Therefore if the database file is moved to a different machine LoadProperties will fail to retrieve the correct password. Developers need to provide the password and other required properties, and call SaveProperties again to refresh the saved profile on the new machine.

For more information please refer MSDN documentation:

a) SqlCeReplication.SaveProperties - http://msdn.microsoft.com/en-us/library/system.data.sqlserverce.sqlcereplication.saveproperties.aspx

b) b) SqlCeReplication.LoadProperties - http://msdn.microsoft.com/en-us/library/system.data.sqlserverce.sqlcereplication.loadproperties.aspx

Thanks,
Balasubramaniam Bodedhala

Katmai new datatypes support

SQLCEBLOG — Wed, 25 Jun 2008 15:31:00 GMT

In Katmai, new datatypes are introduced. Some of these (including some older types in Yukon) are supported for merge replication synching with SQL Compact in SQL Compact 3.5 SP1. These types are mapped to downlevel compatible types for SSC. These types and their mappings are, as follows:

SQL type Compact type Format

Date nvarchar(10) ‘yyyy-mm-dd’

Time nvarchar(16) ‘hh:mi:ss.nnnnnnn”

Datetime2 nvarchar(27) ‘yyyy-mm-dd hh:mi:ss.nnnnnnn’

DatetimeOffset nvarchar(34) ‘yyyy-mm-dd hh:mi:ss.nnnnnnn [+|-]hh:mi’

Geometry image

Geography image

The format of strings used when inserting/updating the data on SQL Compact, should be same as that given above, or as the type demands, in case of Spatial types.

Spatial types are interesting, because, one can map them to image columns on the client without data loss. The data can even be manipulated on the client side, if the required spatial type libraries are present. Other new types in Katmai, like, HierarchyID etc... are not supported for merge with SQL Compact.

Thanks

Udaya

Connectivity Cross Version Compatibility (SQL Server Compact 3.5)

Laxmi Narsimha Rao ORUGANTI — Wed, 19 Dec 2007 03:47:00 GMT

Connectivity Cross Version Compatibility

This blog post explains the Merge Replication connectivity cross version compatibility scenarios for the SQL Server Compact 3.5 release. For more information about the connectivity cross version compatibility scenarios in SQL Server Compact 3.1, see SQL Server Compact 3.1 Connectivity Tools.

This post is not applicable/refer-to Sync Services for ADO.NET. Please refer to this post by Mr. Steve Lasker to know the differences between Sync Services for ADO.NET and Merge Replication.

Product Version Information:

The following table demonstrates the product names and code names used throughout this blog post.

Version	Product Official Name(s)	Product Codename	Nickname
Client 2.0	SQL Server 2000 for Windows CE SQL Server Compact 2.0	<None>	SQL CE 2.0
Client 3.0	SQL Server 2005 Mobile Edition SQL Server Compact 3.0	<None>	SQL CE 3.0
Client 3.1	SQL Server 2005 Compact Edition SQL Server Compact 3.1	<None>	SQL CE 3.1
Client 3.5	SQL Server Compact 3.5	<None>	SQL CE 3.5
Server 8.0	SQL Server 2000	Shiloh	SQL Server 8.0
Server 9.0	SQL Server 2005	Yukon	SQL Server 9.0
Server 10.0	SQL Server 2008	Katmai	SQL Server 10.0

Summary:

1) SQL Server Compact 3.5 can synchronize with only SQL Server 2005 not with SQL Server 2000.

2) SQL Server Compact 3.5 Server Tools package enables synchronization between SQL Server Compact client versions 3.0, 3.1, and 3.5 and SQL Server 2005 through SQL Server Compact3.5 IIS components.

3) SQL Server Compact 3.5 SP1 can synchronize with both SQL Server 2005 and SQL Server 2008.

4) SQL Server Compact 3.5 SP1 Server Tools package enables synchronization between SQL Server Compact client versions 3.0, 3.1, 3.5 and SQL Server 2008 through SQL Server Compact 3.5 SP1 IIS components.

Server Tools Release Plan:

Our release plan for server tools package is as follows: In Phase1, we are releasing SQL Server Compact 3.5 Server Tools package for SQL Server 2005 (Yukon) and SQL Server Compact 3.5 SP1 Beta Server Tools package for SQL Server 2008 (Katmai). In Phase2, we will release the SQL Server Compact 3.5 SP1 Server Tools package for both SQL Server 2005 (Yukon) and SQL Server 2008 (Katmai). This new package will replace the Phase 1 packages.

Important:

This blog post does not describe the following deployment scenarios:

1) Upgrading clients from SQL Server Compact version 2.0 to versions 3.0 and 3.1

2) Upgrading from SQL Server 2000 to SQL Server 2005

3) Mixed installations including SQL Server compact client versions 2.0, 3.0, 3.1 and SQL Server or publisher versions SQL Server 2000 (8.0), and SQL Server 2005 (9.0)

For more information about these deployment scenarios, please see SQL Server Compact 3.1 Connectivity Tools documentation.

General Information for Deployment:

1) You can always install SQL Server Compact client versions 3.0 and 3.5 side-by-side.

2) The version of SQL Server IIS components must be equal to or lesser than the version of SQL Server Publisher.

3) You can install the SQL Server IIS components side-by-side . When installing side-by-side, the SQL Server IIS components should be in two different physical and virtual directories.

4) SQL Server IIS components and SQL Server Compact IIS components reside in IIS server.

5) You can install the SQL Server IIS components from the SQL Server Installation CD. In feature selection page, choose Shared Features, and then Client Tools to install the SQL Server IIS components.

The new rules that apply only to SQL Server Compact v3.x:

1) The version of SQL Server Compact v3.x client components must be equal to or lesser than the version of SQL Server Compact v3.x IIS components. In other words, SQL Server Compact 3.0 client can connect to 3.0, 3.1, 3.5 IIS components. SQL Server Compact 3.1 client can connect to 3.1, 3.5 IIS components. SQL Server Compact 3.5 client can connect to 3.5 IIS components.

2) SQL Server Compact 3.x IIS Components are not side-by-side installable.

1 Cross Version Compatibility

1.1 Cross Version Compatibility Support Matrix

The following table provides the cross version compatibility support for each different type of deployment scenario. The table includes:

· “Dx”: An identifier for each different scenario.

· “Existing Support”: A deployment that is already supported. For more information, see SQL Server Compact 3.1 Connectivity Tools.

· “New Support”: A new supported deployment. This new deployment is explained in this blog post later.

· “Pending Support”: Specifies that SQL Server Compact 3.5 RTM does not support synchronization to SQL Server 2008 but this support will be enabled in SQL Server Compact 3.5 SP1.

Deployment Scenario No	SQL Server Compact Client Version	SQL Server Publisher Version	Supported (Yes/No)	Notes
D1	2.0	N/A	N/A	N/A
D2	3.0/3.1	SQL Server 2000 SP3a or later	Yes	Existing Support
D3	3.0/3.1	SQL Server 2005	Yes	Existing Support
D4	3.0/3.1	SQL Server 2008	Yes	New Support
D5	3.5 RTM or later	SQL Server 2000	No	Retired/Deprecated
D6	3.5 RTM or later	SQL Server 2005	Yes	New Support
D7	3.5 RTM	SQL Server 2008	No	Pending Support
D8	3.5 SP1 or later	SQL Server 2008	Yes	New Support

SQL Server version--à	SQL Server 2000	SQL Server 2005	SQL Server 2008
SQL Server Compact Version ↓	SQL Server 2000	SQL Server 2005	SQL Server 2008
2.0	Old	Old	Not Supported
3.0/3.1	Old	Old	New
3.5 RTM	Not Supported	New	Not Supported
3.5 SP1 or later	Not Supported	New	New

1.2 Compatibility with SQL Server and IIS components

The following table summarizes the compatibility with SQL Server Compact, SQL Server and server IIS components. For more information, see Section 2 Deployment Scenarios. This section requires thorough understanding of Merge Replication deployment.

SQL Server Compact Client Version	SQL Server Compact IIS Components Version	SQL Server IIS Components Version	SQL Server Publisher Version
2.0	N/A	N/A	N/A
3.0/3.1	3.1	8.0	8.0
3.0/3.1	3.1	9.0	9.0
3.0/3.1	3.5 SP1+	10.0	10.0
3.5 RTM+	N/A	N/A	8.0
3.5 RTM+	3.5 RTM+	9.0	9.0
3.5 RTM	N/A	N/A	10.0
3.5 SP1+	3.5 SP1+	10.0	10.0

2 Deployment Scenarios

This section describes how to upgrade the existing deployment and how to set up a new deployment. In addition, this section also describes how to setup a deployment of the mixed subscribers.

The mixed subscribers represent the SQL Server Compact clients that are of different versions. For example, a single deployment might have different versions of SQL Server Compact clients connecting to the same SQL Server Publisher through either same or different IIS servers. In this case, these SQL Server Compact clients are called as mixed subscribers. For more information on how to configure the IIS Server Environment, see http://msdn2.microsoft.com/en-us/library/ms172441(SQL.100).aspx.

Note: SQL Server Compact includes a connectivity wizard, "Configure Web Synchronization Wizard", which helps you configure a virtual directory in Internet Information Services (IIS). Whenever a virtual directory URL is changed, you need to update the Virtual Directory Information to refer to a new URL. For more information about how to do this, see http://msdn2.microsoft.com/en-us/library/ms172450(SQL.100).aspx and http://msdn2.microsoft.com/en-us/library/ms171862(SQL.100).aspx. You also need to update the clients to point to this new URL.

2.1 New Deployment

This section describes how to set up a new deployment for the new deployment scenarios introduced with SQL Server Compact 3.5 and/or SQL Server 2008.

2.1.1 SQL Server Compact 3.5 RTM+ and SQL Server 2005 (D6)

1) Install SQL Server 2005 on a Publisher computer

2) Install SQL Server Compact 3.5 RTM+ on a client computer

3) Install SQL Server 2005 IIS Components on IIS server

4) Install SQL Server Compact 3.5 RTM+ Server Tools Package on IIS server

5) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.1.2 SQL Server Compact versions 3.0 or 3.1 and SQL Server 2008 (D4)

1) Install SQL Server 2008 on a Publisher computer

2) Install SQL Server Compact versions 3.0 and 3.1 on a client computer

3) Install SQL Server 2008 IIS Components on IIS server

4) Install SQL Server Compact 3.5 SP1+ Server Tools Package on IIS server

5) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.1.3 SQL Server Compact 3.5 SP1+ and SQL Server 2008 (D8)

1) Install SQL Server 2008 on a Publisher computer

2) Install SQL Server Compact 3.5 SP1+ on a client computer

3) Install SQL Server 2008 IIS Components on IIS server

4) Install SQL Server Compact 3.5 SP1+ Server Tools Package on IIS server

5) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2 Upgrading an Existing Deployment (Full)

2.2.1 Server Upgrade Only: from SQL Server 2000 or 2005 to SQL Server 2008

2.2.1.1 SQL Server Compact version 2.0 clients

This scenario is not supported. In this case, clients must be upgraded along with the server.

2.2.1.2 SQL Server Compact versions 3.0 and 3.1 clients

1) Upgrade SQL Server

2) Retain SQL Server versions 3.0 and 3.1 on a client computer

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server

4) Optionally uninstall SQL Server 2000/2005 IIS Components on IIS server

5) Remove the SQL Server Compact 3.1 Server Agent ISAPI Virtual Directories

6) Install SQL Server 2008 IIS Components on IIS server

7) Install SQL Server Compact 3.5 SP1+ Server Tools Package on IIS server

8) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2.2 Client Upgrade Only

2.2.2.1 Upgrade from SQL Server Compact 2.0 to SQL Server Compact 3.5

2.2.2.1.1 SQL Server 2000

This scenario is not supported. In this case, the server must be upgraded.

2.2.2.1.2 SQL Server 2005

1) Upgrade SQL Server Compact clients

2) Retain SQL Server on a Server computer

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server (if any)

4) Uninstall SQL Server Compact 2.0 Server Tools Package on IIS server

5) Optionally uninstall SQL Server 2000 IIS Components on IIS server

6) Install/Retain SQL Server 2005 IIS Components on IIS server

7) Remove the SQL Server Compact 2.0 Server Agent ISAPI Virtual Directories

8) Install SQL Server Compact 3.5 RTM+ Server Tools Package on IIS server

9) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2.2.2 Upgrade from SQL Server Compact versions 3.0 and 3.1 to SQL Server Compact 3.5

2.2.2.2.1 SQL Server 2000

This scenario is not supported. In this case, the server must be upgraded along with the clients.

2.2.2.2.2 SQL Server 2005

1) Upgrade SQL Server Compact Clients

2) Retain SQL Server on a Server computer

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server

4) Retain SQL Server 2005 IIS Components on IIS server

5) Remove the SQL Server Compact 3.1 Server Agent ISAPI Virtual Directories

6) Install SQL Server Compact 3.5 RTM+ Server Tools Package on IIS server

7) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2.2.2.3 SQL Server 2008

This deployment scenario describes an incremental upgrade where servers are upgraded from SQL Server 2000 or 2005 to SQL Server 2008 as described in section 2.2.1.2 above and then clients are getting upgraded. This deployment scenario might be useful in some cases where you might choose to upgrade the servers first and then the clients.

1) Upgrade SQL Server Compact clients

2) Retain SQL Server on a Server computer

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server

4) Retain SQL Server 2008 IIS Components on IIS server

5) Remove the SQL Server Compact 3.1 Server Agent ISAPI Virtual Directories

6) Install SQL Server Compact 3.5 SP+ Server Tools Package on IIS server

7) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2.3 Client and Server Upgrade in Parallel

2.2.3.1 SQL Server Compact Clients: from 2.0 to 3.5, SQL Servers: from 2000 to 2005

1) Upgrade SQL Server Compact clients

2) Upgrade SQL Server

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server (if any)

4) Uninstall SQL Server Compact 2.0 Server Tools Package on IIS server

5) Uninstall SQL Server 2000 IIS Components on IIS server (optional)

6) Remove the SQL Server Compact 2.0 Server Agent ISAPI Virtual Directories

7) Install SQL Server 2005 IIS Components on IIS server

8) Install SQL Server Compact 3.5 RTM+ Server Tools Package on IIS server

9) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2.3.2 SQL Server Compact Clients: from 2.0 to 3.5, SQL Servers: from 2000 or 2005 to 2008

1) Upgrade SQL Server Compact clients

2) Upgrade SQL Server

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server (if any)

4) Uninstall SQL Server Compact 2.0 Server Tools Package on IIS server

5) Uninstall SQL Server 2000/2005 IIS Components on IIS server (optional)

6) Remove the SQL Server Compact 2.0 Server Agent ISAPI Virtual Directories

7) Install SQL Server 2008 IIS Components on IIS server

8) Install SQL Server Compact 3.5 SP1+ Server Tools Package on IIS server

9) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2.3.3 Server Compact Clients: from versions 3.0 or 3.1 to 3.5, SQL Server versions: from 2000 to 2005

1) Upgrade SQL Server Compact clients

2) Upgrade SQL Server

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server

4) Uninstall SQL Server 2000 IIS Components on IIS server (optional)

5) Remove the SQL Server Compact versions 3.0 or 3.1 Server Agent ISAPI Virtual Directories

6) Install SQL Server 2005 IIS Components on IIS server

7) Install SQL Server Compact 3.5 RTM+ Server Tools Package on IIS server

8) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.2.3.4 Server Compact Clients: from versions 3.0 or 3.1 to 3.5, SQL Server versions: from 2000 or 2005 to 2008

1) Upgrade SQL Server Compact clients

2) Upgrade SQL Server

3) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server

4) Uninstall SQL Server 2000/2005 IIS Components on IIS server (optional)

5) Remove the SQL Server Compact version 3.0/3.1 Server Agent ISAPI Virtual Directories

6) Install SQL Server 2008 IIS Components on IIS server

7) Install SQL Server Compact 3.5 SP1+ Server Tools Package on IIS server

8) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

2.3 Mixed Deployments

Mixed deployments might be due to choice or due to an incremental upgrade, etc. This section describes how to setup IIS server in mixed deployment scenarios and in what cases multiple IIS servers are required.

The scope of mixed subscribers in this section is limited to the SQL Server Compact subscribers of multiple versions. In other words, this section explains only mixed of versions of SQL Server Compact clients. This section does not explain the mixed deployment scenarios including different types of subscribers such as SQL Server Compact subscriber and SQL Express subscribers together.

2.3.1.1 SQL Server Compact Client versions: 2.0, 3.5; SQL Server versions: Any

This deployment scenario describes a deployment, which might include SQL Server Compact 2.0 or 3.5 subscribers and any version of SQL Server publishers.

SQL Server Compact 2.0 and 3.5 cannot be subscribers to the same publisher even with different publications. This combination is not supported. Either SQL Server Compact 2.0 subscribers must be upgraded to SQL Server Compact versions 3.0 or 3.1 or 3.5, or there must be different publishers for different SQL Server Compact clients in the deployments.

If there are different publishers; you should make sure to have different IIS servers configured.

For more information about how to setup two completely disjoint parallel deployments, please see New Deployment.

2.3.1.2 SQL Server Compact Client versions: 2.0, 3.0 or 3.1; SQL Server versions: Any

SQL Server Compact 2.0 cannot be a subscriber to SQL Server 2008; therefore this deployment cannot be a valid mixed deployment. For more information about this kind of mixed deployment, see SQL Server Compact 3.1 Connectivity Tools.

2.3.1.3 SQL Server Compact Client versions: Any; SQL Server versions: 2000, 2008

This deployment scenario describes a deployment, which might include SQL Server Compact 2.0 or 3.0 or 3.1 or 3.5 subscribers and SQL Server 2000 or 2008 publishers.

Important: SQL Server Compact 3.5 cannot be a subscriber to SQL Server 2000 publisher.

If there are different publishers and different sets of subscribers, you should make sure to have different IIS servers configured. In this case, you might need separate set of deployments, such as:

· Subscriber Set1 merge replicates through IIS Server1 with SQL Server1

· Subscriber Set2 merge replicates through IIS Server2 with SQL Server2

Note that this deployment scenario is not a mixed deployment but it is much more similar to the two completely disjoint parallel setups which can be configured as specified in section New Deployment above.

2.3.1.4 SQL Server Compact client versions: 3.0, 3.1, 3.5; SQL Server versions: 2005, 2008

In this deployment scenario, one IIS server can serve all the SQL Server Compact subscribers with or without multiple publishers in the deployment. Note that SQL Server Compact 3.5 IIS Components can connect to any SQL Server Compact version 3.x client component. Therefore, a single SQL Server Compact 3.5 Server Agent URL can be used by all SQL Server Compact version 3.x Clients.

Note that when using SQL Server 2008 as a Publisher, the SQL Server Compact 3.5 IIS components must be of version 3.5 SP1 or greater.

To setup this kind of mixed deployment, you should follow these steps:

1) Upgrade clients and servers (if necessary)

2) Uninstall SQL Server Compact 3.1 Server Tools Package on IIS server (if any)

3) Uninstall SQL Server Compact 2.0 Server Tools Package on IIS server (if any)

4) Uninstall SQL Server 2000 IIS Components on IIS server (optional)

5) Remove the SQL Server Compact 2.0 Server Agent ISAPI Virtual Directories

6) Remove the SQL Server Compact versions 3.0/3.1 Server Agent ISAPI Virtual Directories

7) Install/Retain SQL Server 2005 IIS Components on IIS server(if any publisher is SQL Server 2005)

8) Install/Retain SQL Server 2008 IIS Components on IIS server(if any publisher is SQL Server 2008)

9) Install SQL Server Compact 3.5 RTM+ Server Tools Package on IIS server(if all publishers are SQL Server 2005)

10) Install SQL Server Compact 3.5 SP1+ Server Tools Package on IIS server(if any publisher is SQL Server 2008)

11) Configure Web Synchronization with SQL Server Compact 3.5 Server Agent ISAPI

You might choose to configure multiple IIS servers for different set of publisher and subscriber combinations. If you do that, this case cannot be a mixed deployment anymore. It would be much similar to two completely disjoint parallel setups which can be configured as specified in section New Deployment above.

Thanks

Merge Subscribe Once, Copy and Deploy Multiple Times

SQLCEBLOG — Tue, 26 Jun 2007 21:38:00 GMT

"Merge Subscribe Once, Copy and Deploy Multiple Times" - This has been the real good feature I like and unfortunately not many people knew about it. This is applicable for v3.0 and above. Here is a short story of it:

The initial subscription is going to bring down lot of data from server to client in merge replication. It means that the requirement for bandwidth, memory, processing power is going to be high for the creation of subscription. However, for devices all these requirements are not easy and even if they are available, it takes very long time to get the subscription created on the client database. Imagine a deployment of 1000+ devices (usual number) and doing this creation of subscription for every device. This has been the major pain and customer feedback we received in v2.0 and wanted to address this in v3.0 and we didJ.

Before going into details on how it works and using it, I first want to call out the limitations/conditions with it:

C1) It is useful only in case of static filtering or no filtering. If your publication has dynamic filtering, then this does not work.

C2) Every subscribed DB copy must do an empty/dummy sync (to update the metadata both on client and server about this copy) from deployed device/SD Card/desktop before using it for local database operations/in application.

You would have loved to have:

1) Subscribe to the publication once (on the desktop! where there is required processing power, memory and optional bandwidth)

2) Copy the *same* subscribed database file to 1000+ SD Cards/Devices/Desktops

3) What else, start using it** J

It works exactly as you wantedJ. The mark ** on (3) meant to hint the condition (C2) above. Note this also means that you can deploy the database file along with your application :)

Namaste / Namaskaram / Thanks,

Laxmi Narsimha Rao ORUGANTI

HOWTO: Pending Changes for a Merge Subscription (Server Side)

SQLCEBLOG — Tue, 26 Jun 2007 19:55:00 GMT

Earlier in the other blog article we discussed on “HOWTO: Pending Changes for a Merge Subscription (Client Side)” and this article will focus on finding the pending changes on server side for download. This blog article requires a little understanding of Remote Data Access (RDA). Note that, RDA does NOT need any extra configuration and can use/piggyback on the current merge replication configuration both on IIS and in SQL Server.

1) Create a stored procedure on server side like the one mentioned below

====================================================================================

CREATE PROC sp_IsThereAnyChangesToReplicateServerSide

DECLARE @AnythingToReplicateServerSide int

DECLARE @pending_deletes int

DECLARE @pending_ins_and_upd int

SET @AnythingToReplicateServerSide = 0

CREATE TABLE #My_PendingChanges (destination_server sysname, pub_name sysname, destination_db_name sysname, is_dest_subscriber bit, article_name sysname, pending_deletes int, pending_ins_and_upd int)

INSERT INTO #My_PendingChanges EXEC sp_showpendingchanges @publication = 'Repl_Mobile_Publication'

SELECT @pending_deletes = pending_deletes, @pending_ins_and_upd = pending_ins_and_upd FROM #My_PendingChanges

IF (@pending_deletes <> 0 OR @pending_ins_and_upd <> 0)

SET @AnythingToReplicateServerSide = 1

DROP TABLE #My_PendingChanges

SELECT @AnythingToReplicateServerSide

====================================================================================

a. You can make it smarter by specifying a @destination_server, @publication to sp_ShowPendingChanges.

2) Call this stored procedure from client using RDA.Pull which will create local table

Thanks,

Laxmi Narsimha Rao ORUGANTI

HTTP/Internet Proxy Detection - Using WINCE API

SQLCEBLOG — Tue, 19 Jun 2007 23:28:00 GMT

SqlCeReplication and SqlCeRemoteDataAccess have InternetProxyServer, InternetProxyLogin, InternetProxyPassword properties which the application can set to make these sync protocols communicate via/thru the desired proxy.

However, then the question comes on how to programmatically detect the HTTP/Internet proxy settings from the app to set the properties on SQL CE classes. Here is the short story of how to detect the settings on Windows CE/Mobile devices.

DLL/Module Name and Reference

API

Data Structures

CellCore.dll

Connection Manager API

ConnMgrMapURL

ConnMgrEstablishConnectionSync

ConnMgrProviderMessage

ConnMgrReleaseConnection

CONNMGR_CONNECTIONINFO

PROXY_CONFIG

1) Call ConnMgrMapURL function to map the InternetUrl to a network Guid and an index

a. This function takes the URL you specify and determines the best network connection to use to access that URL.

2) Fill CONNMGR_CONNECTIONINFO with the retrieved network Guid

a. dwParam: CONNMGR_PARAM_GUIDDESTNET

b. dwPriority: CONNMGR_PRIORITY_USERINTERACTIVE

3) Call ConnMgrEstablishConnectionSync with the above filled structure to get a connection HANDLE

a. This function uses ConnMgrEstablishConnection and ConnMgrConnectionStatus internally to attempt to fully establish a connection before returning to the caller. The ConnMgrEstablishConnectionSync function does not provide an internal message pump; therefore, the message queues of any user interface threads that call into this function are blocked until the function returns.

4) Call ConnMgrProviderMessage with the received connection handle and Guid provider as IID_ConnPrv_IProxyExtension, to get PROXY_CONFIG

a. Make sure to set PROXY_CONFIG.dwType: CONNMGR_FLAG_PROXY_HTTP before calling

5) Call ConnMgrReleaseConnection to close the connection HANDLE

Now, PROXY_CONFIG has all the required details of the HTTP/Internet Proxy for ready use.

Thanks,

Laxmi Narsimha Rao ORUGANTI

HOWTO: Pending Changes for a Merge Subscription (Client Side)

SQLCEBLOG — Wed, 18 Apr 2007 23:31:00 GMT

Often there will be need to find out if there are any pending changes on client side to upload. Note that, It is always better to avoid empty sync both from the resource (network bandwidth, web server, SQL Server) usage and performance. This blog post discusses on how to find if there are any pending changes to be uploaded for a particular subscription.

In the earlier blog post “Merge Replication system tables”, we discussed on how to find the list of tables/articles for a particular publication/subscription.

For each table in the *interested* merge subscription,

SELECT MAX(__sysIG) AS MaxIG FROM UserTable;

SELECT MAX(__sysCG) AS MaxCG FROM UserTable;

__sysMergeSubscriptions AS sms;

SQL Server 9.0 and 10.0 (SQL Server Codename YUKON and KATMAI),

If (MaxIG > sms.LastUploadedGen OR MaxCG > sms.LastUploadedGen), then there are pending changes for this User Table

SQL Server 8.0 (SQL Server Codename SHILOH),

If (MaxIG > sms.SentGen80 OR MaxCG > sms.SentGen80), then there are pending changes for this User Table

If none of the tables have pending changes, then user can avoid the empty sync. (Note: Finding the pending changes on SQL Server side for download is not covered)

Thanks,

Laxmi Narsimha Rao ORUGANTI

Merge Replication System Tables

SQLCEBLOG — Wed, 18 Apr 2007 23:11:00 GMT

The earlier blog post “How to: Detect the database is mere replicated”, gave a list of the system tables that get created in a merge subscription database. This blog post explains brief description of each of the system tables. There is another blog post “RDA Subscriptions” that describes briefly the RDA system tables.

__sysMergeSubscriptions – Each row in this table represents a merge subscription. Note that, multiple merge subscriptions can co-exist in a single client database.

__sysMergeArticles – Each row in this table represents a merge subscription article.

__sysMergeSubscriptionProperties – Each row in this table stores merge subscription properties that are on set SqlCeReplication object. For example, Internet URL/Login/Password, Publisher Machine/UserId/Password/Database/Network, Distributor Network/Machine/Database/UserId/Password … etc.

List of tables that belong to a particular publication:

SELECT TableName FROM __sysMergeArticles AS sma INNER JOIN __sysMergeSubscriptions AS sms ON sma.SubscriptionId == sms.SubscriptionId AND sms. Publication == @PublicationName.

Every merge article/table is tracked by SQL Server Compact engine. Tracking infrastructure creates three system tables. Namely,

__sysTrackedObjects – Each row in this table represents a database object that is tracked. It can be a table, column … etc (Note: Merge replication supports column level tracking)

__sysDeletedRows/__sysRowTrack – Each row in this table represents a deleted row of a tracked table

Thanks,

Laxmi Narsimha Rao ORUGANTI

Replication and Database Mirroring (Failover) Support

SQLCEBLOG — Tue, 20 Feb 2007 21:07:00 GMT

Replication and Database Mirroring (Failover) support for Merge Replication between SQL Server 2005 SP1+ and SQL Server 2005 Compact Edition (Mobile Edition SP1+) is supported but it was not documented as the support was added after RTM and the documentation was not updated. Then comes for me an opportunity to BLOG about it J (apart from filing a DOC BUG). Here the story goes:

The database mirroring for replication is documented @ http://msdn2.microsoft.com/en-us/library/ms151799.aspx

One important point you should not is that for database mirroring to work, you should be running SQL Server services under a domain account (by default it is under LOCAL SYSTEM account).

Coming to what is supported (With an assumption you know what is meant by Principal (P), Mirror (M), Witness (W)):

Test case description	Initial Sync	Incremental Sync
Sync with Principal – Both are UP	PASS	PASS
Sync with Principal – Mirror is DOWN	PASS	PASS
Sync with Mirror – Both are UP	FAIL	PASS
Sync with Mirror – Principal is DOWN	FAIL	PASS

Legend: Old Principal Now Mirror (OPNM), New Principal Old Mirror (NPOM)

Key points to note:

1) Sync with Mirror means mirror became a principal due to fail over

2) No need to change the subscription properties when there is a fail over.

3) In the initial subscription time/phase, we connect directly to the Publisher supplied in the subscription properties to find the publisher version so that we invoke the right merge replication mode.

4) Direct connections are not allowed to the Mirror

5) Points (3) and (4) make the initial subscription fail if we the user specifies the Publisher as OPNM, no matter whether the OPNM is UP or DOWN. However, if the user specifies NPOM or the current principal as Publisher while creating subscription, subscription succeeds.

Thanks,

Laxmi Narsimha Rao ORUGANTI & Balachandar P

HOWTO: Detect the database is Merge Replicated?

SQLCEBLOG — Mon, 19 Feb 2007 19:59:00 GMT

Sometimes an application may want to know whether the database is in good shape in terms of Merge Replication especially when the merge replication is aborted by the user, or due to connection failure or many other reasons. Here are the things application should ideally check if the database is merge replicated correctly and is usable further.

1. Ensure that the following system tables exist

a. __sysMergeSubscriptions

b. __sysMergeArticles

c. __sysTrackedObjects

d. __sysRowTrack

e. __sysDeletedRows

[Note: Tables (c), (d), (e) would be created only when the client side tracking of changes is needed. If all the articles are download only then you may not these tables at all]

2. For each merge subscription in this database,

a. An entry/row should exist in __sysMergeSubscriptions

b. A valid sync partner should exist:

SELECT * FROM __sysTrackedObjects AS sto JOIN __sysMergeSubscriptions AS sms ON sto.N = sms.SyncPartnerId AND sto.T=’P’ AND sto.SV = sms.Publication

3. For each subscribed table,

a. An entry/row should exist in __sysMergeArticles

b. The table should have the following seven system columns:

i. __sysIG, __sysCG, __sysCD, __sysP1, __sysMC, __sysMCS, __sysSR

Thanks,

Dan Shaver

Annoying error - 28559 - Encountered problems when opening the database

SQLCEBLOG — Wed, 07 Feb 2007 08:14:00 GMT

We have been hearing lot of complaints on how to get more information for error 28559 when doing RDA or Merge Replication.

If you are using managed API (SqlCeReplication OR SqlCeRemoteDataAccess), then it should be straight forward for you as you would be receiving SqlCeErrorCollection in SqlCeException.Errors which would contain list of errors. This list would tell you more on why there was a problem opening the database.

However, if you are native programmer and using ISSCEMerge, ISSCERDA interfaces, then you need to do some extra steps in getting that information. We have written a sample to help you get the error information much like how our managed wrappers get it. Please have a quick look at our sample app attached to this blog post for more details.

Thanks,

Pawan Kumar