Microsoft Sync Framework

TechEd 2008 EMEA Presentations

Christian Liensberger and I will be presenting at TechEd EMEA in Barcelona Spain next week.  For those of you who will be attending, I hope we can get a chance to talk.  Although Christian and I will be spending a lot of time in the "Ask the Experts" section, please feel free to email me (using the Email link on the left sidebar) to set up a one-on-one meeting. 

These are the talks that we have lined up for the conference.  I hope to see you there!

WIN201 - Introduction to Microsoft Sync Framework (Liam Cavanagh)
Tuesday, November 11 09:00 - 10:15
In this session we will introduce Microsoft Sync Framework. Attendees will learn about the capabilities of the synchronization platform and see a demonstration of data collaboration across devices and services. We will use this demonstration as a guide in our walk-through of the key components involved in building an application using Microsoft Sync Framework that will enable any data store to exchange information.

WIN04-IS Sync Framework - Deep Dive (Christian Liensberger)
Tuesday, November 11 10:45 - 12:00
tive Session
Learn about the newest features that have been build into Microsoft Sync Framework version 1.0 and how they help you to synchronize your data with various devices, computers (PC to PC) or even services in the cloud. In the beginning of this session we are going to deliver a short overview of the Microsoft Sync Framework’s 101:

• What are the advantages of Microsoft Sync Framework
• How does synchronization work with the Microsoft Sync Framework
• How does Microsoft Sync Framework make synchronization easier and more efficient

After this short dive we will write a custom provider for the Microsoft Sync Framework to showcase some of the hooks that you might find interesting when getting started with writing own providers. After the session you should be able to write custom providers for the Microsoft Sync Framework to enable the synchronization scenarios in your own applications. 

- Liam Cavanagh

PDC Session Recordings on Sync Framework

We are just about finishing off day 3 of the Professional Developer Conference and, wow what a week it has been so far!  It has been great meeting with many of you at our Sync booth as well as at our various sessions.  During this week we have introduced Sync Framework v2 (CTP1) and a new SQL Services Labs site that incorporate two of our newest sync related projects (Codename "Huron" and Codename "Anchorage").

For those of you who were not able to come to the PDC, I wanted to point you to the recordings of our Sync Framework sessions that are now available for viewing.

Microsoft Sync Framework Advances (Presenter: Lev Novik)
This session shows you how the next version of the Microsoft Sync Framework makes it easier to synchronize distributed copies of data across desktops, devices, services, or anywhere else they may be stored.

Windows 7: Programming Sync Providers That Work Great with Windows (Presenter: Jason Roberts)
Learn how you can enable your application to synchronize with other applications that use the Microsoft Sync Framework. This session covers how to implement sync for contacts and other PIM data, how to package sync providers for distribution and installation, and how to register sync provider for use on Windows.

Sync Framework: Enterprise Data in the Cloud and on Devices (Presenter: Liam Cavanagh)
See how synchronization plays a pivotal role in transitioning to a managed cloud environment by creating a central hub of information in the cloud. Using synchronization, organizations can enable more efficient mobile and enterprise-to-enterprise scenarios.

We also had a number of related sessions that you might also find of interest (some of these will be posted over the next few days):

• SQL Server: Database to Data Platform - Road from Server to Devices to the Cloud
• Offline-Enabled Data Services and Desktop Applications
• SQL Services: Futures
• SQL Server Compact: Embedding in Desktop and Device

- Liam Cavanagh

New Sync Samples - Sync101 Updated for Sync Framework v2.0 CTP1

For those of you who have already downloaded the new Community Technology Preview (CTP) for Sync Framework v2.0, you may find it helpful to also check out the latest version of our Sync101 samples updated to reflect the new features of the release. 

One of the newest additions to Sync101 is a new sample that makes use of the Sync Framework Simple Provider API now part of v2.0.  As you will see in this sample ("Sync101 using Simple Sync Provider"), the Simple Provider API makes it simpler then ever to create providers as it gives developers a lot of the power and flexibility of the core Sync Framework APIs, but with less of a learning curve and less code, especially for replicas that have little or no change-tracking support.  

This and other samples can be viewed by going to the Sync Code Gallery at http://code.msdn.com/sync.

- Liam Cavanagh

Annoucing Sync Framework v2 CTP1

I am extremely happy to announce the availability of Sync Framework v2 CTP1 .  Sync Framework 2.0 expands on the capabilities offered by Sync Framework 1.0:

·         Adds features that cater to new scenarios or scenarios that were difficult to support.

·         Reduces the amount of work required to develop providers.

·         Supports more data sources with new built-in providers.

 

The major new features included in this initial CTP are:

·         Simple Providers

o   Reduces the amount of work required to develop providers, especially when the data source has very few synchronization-related capabilities, such as change-tracking.

o   The goal is to enable a developer to write a provider without having to become a synchronization expert. The majority of the code that is required for a simple provider is limited to that responsible for interacting with the data source.

·         Change Unit Filtering

o   Different data sources can have different representations of the same data types, with some data sources offering a richer and more complete representation than others. This is especially true when you compare data type representations on a PC and on a device; the device will often support only a subset of the data supported by the PC.

o   Sync Framework caters to scenarios in which a mixture of data sources is being synchronized, each with a different representation of the data type. Filtering support has been added to enable a provider to specify that the replica it serves is only able to store a subset of the data.  The synchronization metadata and synchronization process have both been enhanced to handle this filtering efficiently.

·         Filter negotiation

o   When a destination data source is only storing a subset of data, then it is more efficient if the source provider uses a filter to enumerate only the data that can be stored by the destination.

o   Filter negotiation allows a destination provider to specify it would like one or more filters to be used by the source provider; the source provider can accept or reject a filter. If a source provider does not support the requested filters then the destination provider can choose to receive all the data and do the filtering itself.

o   Sync Framework is responsible for calling the providers appropriately to negotiate filter usage.

 

For more details please visit the Microsoft download center at:

http://www.microsoft.com/downloads/details.aspx?FamilyId=109DB36E-CDD0-4514-9FB5-B77D9CEA37F6&displaylang=en

- Liam Cavanagh

Introducing SQL Services Labs

This week many of the Sync Framework team members are at the Professional Developer Conference in Los Angeles.  We have a number of exciting sessions and announcements that will be coming out this week. 

 

The first announcement is that the Microsoft Sync Framework team has teamed up with SQL Services Labs to bring you several sync related incubations that will soon be made available from www.SQLServicesLabs.com

 

Project Codename “Huron” – Leverage the power of SQL Data Services and Microsoft Sync Framework to build business data hubs in the cloud.  Using this cloud based data hub, “Huron” provides a simpler, more convenient and less expensive way to:

·         Publish databases to the cloud along with reports, forms and objects

·         Subscribe to published data and automatically configure the local database for sync

·         Make online changes through SQL Data Services and propagate those changes to subscribed users once they connect

·         Enable scheduled and background synchronization of data changes through SQL Data Services and then on to other subscribed users

·         Backup and restore of database applications to the cloud

Project Codename "Anchorage" – We’re evolving the popular SyncToy application to enable much more than just file/folder synchronization between PCs, devices, and services! With this project, we take content flow to a whole new level – enabling sync providers to register and be discovered in a variety of sync groups including contacts, calendars, files, favorites, videos, photos. Etc. And to make it more exciting – we’ve partnered with PhotoBucket.com and Smugmug.com to enable photo and video sync across services PCs, folders, and the Live Mesh.

 

For more details on these and other projects, check out the Labs portal at www.sqlserviceslabs.com

 

-Moe

Not an Updated Version of Microsoft Sync Framework

We have noticed a few blog and news posts pointing to what is being called an updated version of Microsoft Sync Framework.  In order to clear up the confusion, we wanted to let everyone know that this in fact is not an update to our previously released version 1.0 of Sync Framework. 

Last week we renamed a few of the download files to reflect the various architectures that are supported (ia64,x64,x86).  During this renaming, the Microsoft Download Center updated the published date, making it look like a new version of Sync Framework was available.

We want to appologize for any confusion this may have caused. 

Liam

Now Available - Sync Services for ADO.NET on Windows Mobile (Devices)

We are now announcing the availability of v1.0 Service Pack 1 of Sync Services for ADO.NET on Devices (Windows Mobile). "Microsoft Synchronization Services for ADO.NET 1.0 provides the ability to synchronize data from disparate sources. Rather than simply replicating a database and its schema, the Synchronization Services application programming interface (API) provides a set of components to synchronize data between data services and a local store. Applications are increasingly used on mobile clients, such as laptops and devices that do not have a consistent or reliable network connection to a central server. It is crucial for these applications to work against a local copy of data on the client".   "Equally important is the need to synchronize the local copy of the data with a central server when a network connection is available. The Synchronization Services API, which is modeled after the ADO.NET data access APIs, gives you an intuitive way to synchronize data. It makes building applications for occasionally connected environments a logical extension of building applications where you can count on a consistent network connection".

Synchronization Services for ADO.NET 1.0, which released with Visual Studio 2008, enabled synchronization between a server database and a SQL Compact 3.5 client database running on a desktop. With this download, you can also synchronize between a server database and a SQL Compact database on a Windows Mobile 5 or 6 device.

Webcast - Synchronizing Data: Inside the Microsoft Sync Framework

A few days ago, a number of people from the Sync Framework engineering team were interviewed by Charles Torre (from Channel 9). He shot a 1 hour video interview where we explained how the Microsoft Sync Framework works internally, some key concepts that developers need to take care of when creating new providers and how the Sync Framework team is focusing on both managed and native developer solutions. The video can be viewed at: http://channel9.msdn.com/posts/Charles/Inside-the-Microsoft-Sync-Framework/ or downloaded directly in the following formats:

• iPod (MP4)
• MP3
• PSP (MP4)
• WMA
• WMV
• WMV (High)
• Zune

I hope you enjoy watching the interview!

Liam

3 (more) New Sync Samples - Sync 101 for Sync Framework SDK v1.0

We have a couple more synchronization samples for you to take a look at.  These C# samples are based on Sync101 which is a sample template that allows us to show specific concepts relating to Sync Framework.  All of these samples have been updated to work with the latest v 1.0 release of Sync Framework SDK . I am particularly excited about the remote change application sample (RCA) as it has been a common request to understand how to integrate Sync Framework with technologies like WCF and enable a service to manage the server provider componets of the Sync Framework.   Each of the samples use the the SQL MetadataStore store for the metadata information for each replica database. This greatly simplifies the logic required to implement data synchronization between data stores. In these samples three in memory data stores are used to show how data collaboration can be accomplished and how the Sync Framework helps to minimize P2P complexities such as data conflicts / resolution and data deletions.

The following are the samples that were just added to the Sync Framework Code Gallery.

Sync101 with Metadata Store v1.0
This sample uses the Metadata store to show the basic concepts that are required when building a Sync Framework application and illustrates many of the core concepts such as synchronization providers and the synchronization orchestrator. 

Sync101 RCA with Metadata Store v1.0
Expanding on the previous sample, this samples outlines the "remote change application" (RCA) pattern used over WCF.

Sync101 with Change Units
Expanding on the first samples, this sample illustrates how a data store can use change units to partition data and send only data that has changed. This sample also illustrate explicit transactioning (batched instead of per-update).

Liam

Now Available - SyncToy 2.0 File Synchronization

SyncToy 2.0 has just been released to the Microsoft Download center. SyncToy 2.0 for Windows is available as a free download and is an easy to use, customizable application that helps you copy, move, rename, and delete files between folders and computers. SyncToy is powered by Microsoft Sync Framework and is yet another example of how Sync Framework helps to create end-to-end solution for enabling synchronization of any content or data.

Based on the feedback from the SyncToy Forum and Windows XP Professional Photography web site, the following features were added to this release of SyncToy:

• Dynamic Drive Letter Assignment: Drive letter reassignment will now be detected and updated in the folder pair definition. 
• True Folder Sync: Folder creates, renames and deletes are now synchronized for all SyncToy actions. 
• Exclusion Filtering Based on Name: File exclusion based on name with exact or fuzzy matching. 
• Filtering Based on File Attributes: The ability to exclude files based on one or more file attributes (Read-Only, System, Hidden). 
• Unattended Folder Pair Execution: Addressed issues related to running scheduled folder pairs while logged off. 
• Folder Pairs With Shared Endpoints: Ability for folder pairs associated with the same or different instances of SyncToy to share end-points. 
• Command line enhancements: Added the ability to manage folder pairs via the command line interface. 
• Re-Architect Sync Engine: The SyncToy engine has been rearchitected to provide scalability and the ability to add significant enhancements in future releases. 
• Sync engine is also more robust insomuch that many single, file level errors are skipped without affecting the entire sync operation. 
• Sync Encrypted Files: Sync of Encrypted files works when local folder and files are encrypted, which addresses the common scenario involving sync between local, encrypted laptop PC folder and remote, unencrypted desktop PC folder. 
• 64-Bit Support: SyncToy now has a native 64-bit build (x64 only) for 64-bit versions of Windows. 
• Folder pair rename 
• Sub-folder Exclusion Enhancements: Descendents created under excluded sub-folders are automatically excluded. Usability improvements for the sub-folder exclusion dialog. 
• Folder Pair Metadata Moved: Folder pair metadata removed from MyDocuments to resolve any issues with server-based folder pair re-direction setup. 
• Setup Improvements: Integrated setup with single self-extracting archive file and no extra downloads if you already have .NET Framework 2.0 installed. Enabled silent install for the SyncToy Installer file (see readme.txt file for more information).
• Removed combine and subscribe actions.

SyncToy 2.0 is available for download from: http://www.microsoft.com/downloads/details.aspx?familyid=C26EFA36-98E0-4EE9-A7C5-98D0592D8C52&displaylang=en

Liam

2 New Sync Samples - Download Only & Adapter Builder

We have just posted the following two new samples to the Sync Code Gallery today for Sync Services for ADO.NET v2: Download Only Sync This sample demonstrates two-tier download-only synchronization using Microsoft Sync Services for ADO.NET v2.0. The sample solution is composed of two projects: Client, which includes the UI form, data access code, and synchronization agent code for Sync Services; and SampleServerSyncProvider, which includes server synchronization provider code for Sync Services. For more information, see the code comments that are included in each file.

Sync Adapter Builder
This sample utility demonstrates how to create synchronization commands by using the SqlSyncAdapterBuilder in Microsoft Sync Services for ADO.NET v2. The sample solution is composed of one project that includes the UI form, data access code, and SqlSyncAdapterBuilder code. The SqlSyncAdapterBuilder is typically used at run time to generate synchronization commands, which are in the form of SQL queries. This provides an alternative to writing these commands manually at design time. In this sample, we use the SqlSyncAdapterBuilder to output synchronization commands. You can output commands that work with SQL Server change tracking (available in SQL Server 2008) or a custom change tracking system. Running this application enables you to see how the commands are structured, and to copy those commands for use in applicatons where you will specify commands manually.

Liam

Sync Framework for Windows Mobile (Devices) - CTP1

Right on the heels of our launch of Microsoft Sync Framework v1, we have just posted our first CTP for Microsoft Sync Framework for Devices. As of right now, you can download this CTP for Windows Mobile from the Microsoft Download Center.  As we have shown in previous webcasts and conference demos, this is great for enabling synchronization from devices to data stores such as SQL Server Data Services or even corporate web services.  Perhaps even to write an offline SmugMug application for synchronizing your device photos with your SmugMug account.

Not to be confused with the previous CTP release of Sync Services for ADO.NET for Devices (a component of the Sync Framework) that is specifically meant for synchronizing an ADO.NET enabled server database with a SQL Server Compact device database, this CTP release of Sync Framework for Devices includes the Core Sync Framework runtime that gives the ability to exchange information with any data store or file system on the device, and not just SQL Server Compact databases.

This release contains only the core sync engine you would find in “synchronization.msi” in the PC release. Other features such as the metadata store and file sync provider are not included: some of the omitted features don’t necessarily make sense on devices, others will appear in a future CTP. We are looking for feedback on which are which. In particular, we want to know: what are your scenarios for using the different aspects of the Sync Framework on a device?

We also decided to produce the CTP for a limited platform set: Windows Mobile 5 and 6 for ARM processors. Supporting this subset allows us to give a CTP usable by a very large percentage of mobile developers while still limiting the test matrix for this first CTP. We expect to expand the supported devices and platforms for RTM and we are considering the scope of that as we go forward.

So what exactly is in this release?

Microsoft Sync Framework core runtime

• Managed (.NET Compact Framework) and unmanaged/native (ARM)
• Supported platforms are Windows Mobile 5 and 6 (ARM processors only in CTP1)

Installer with Visual Studio integration

• Support for adding a reference to the Sync Framework in a mobile project using Visual Studio 2008

As always, we look forward to your feedback on the Sync Framework Forum.

Liam

Sync Case Study - ServiceU Corporation & Event Management

One of the best parts about getting our first version of Sync Framework out the door is that we can now start talking about some of the customers we have been working with over the past few months and some of the great solutions that they are building. 

Beyond the customers I mentioned earlier (SmugMug, InterScape, Fujitsu Siemens) I also wanted to point you to another customer called ServiceU Corporation.  This company is a leader in the field of supporting online and on-demand event management. As you will read in the case study:

"Its software services, created and deployed using the Microsoft® Application Platform, are used by churches, schools, universities, theaters, and businesses to manage concerts, conferences, and other events. As a Payment Card Industry Level 1 Service Provider, the company is interested in security and performance, which is why it is upgrading to Microsoft SQL Server® 2008. ServiceU is using a number of new SQL Server 2008 features including enhanced auditing, and Policy-based Management to enforce configuration policies. The company is also using the new Performance Data Collection and Plan Freezing features to enhance performance, and the Data Compression and Backup Compression features to reduce the cost of disk-based storage".

To help power this solution, ServiceU chose Sync Services for ADO.NET:

"Working from Visual Studio 2008, developers can use Microsoft Synchronization Services for ADO.NET 2.0 to synchronize data from disparate sources over multi-tier and service-based architectures. ServiceU likes this feature because it is interested in deploying more solutions that run on mobile and remote devices. “Whenever you have server-based, data-driven applications coordinating data with mobile devices, synchronization is essential,” says Smith. “Synchronization Services will be a great help as we develop new occasionally-connected solutions to offer to our customers.”

Over the next few months we will be talking more and more about some of the great companies that are using Sync Framework and Sync Services for ADO.NET.

Of course, given that our technology is free and readily downloadable from the web, we don't hear from every customer.  As such, if you are using any of these technologies please let me know (at the Email link on the sidebar) so we can share your success with the rest of the community.

For more details on this case study, please go to: http://www.microsoft.com/casestudies/casestudy.aspx?casestudyid=4000001258

Liam

Top New Features in Sync Services for ADO.NET v2

Today’s release of Microsoft Sync Framework includes the latest version of Sync Services for ADO.NET.  A great deal of work has gone into this release and I thought I would take some time to point out some of the top new features in version 2 since our previous release.

• Peer-to-peer synchronization support:

Beyone the traditional hub-and-spoke synchronization support built into Sync Services for ADO.NET.  v2 now supports peer-to-peer synchronization.  The focus with this part of the API is collaborative applications. For example, in an application that allows users to share project notes, project team members often require a local copy of data that they can work with. When they have made changes, they can synchronize with another team member to exchange changes. 

• Sync Services is now based on Microsoft Sync Framework:

Sync Services now uses Microsoft.Synchronization.dll. If you installed Sync Services by using SyncSetup.exe, this DLL is already installed. This enables Sync Services for ADO.NET to take advantage of all the features and capabilities built into the Sync Framework.

• Synchronize with devices: 

Sync Services for ADO.NET 1.0 enabled synchronization between a server database and a SQL Server Compact 3.5 database on the desktop. Now, you can also synchronize between a server database and a SQL Server Compact 3.5 database on a device.

• Support for SQL Server 2008 Integrated Change Tracking:

If you are using SQL Server 2008, Sync Services for ADO.NET has the ability to take advantage of the new SQL Server integrated change tracking feature. This feature addresses many of the issues of custom-tracking systems and provides a straightforward way to track changes. Change tracking is also supported by the Local Database Cache in Visual Studio 2008 SP1. Developers can now specify that the Configure Data Synchronization wizard should enable SQL Server change tracking on the server and generate the commands necessary to select and apply changes to the server database. Unlike custom change tracking systems, SQL Server change tracking does not require any schema changes in the server database.

• Support for the new data types in SQL Server 2008:

Sync Services for ADO.NET has added support for a number of new data types including those added in SQL Server 2008.

• The ability to trace the synchronization process: 

In distributed applications, tracing can be crucial because it enables you to troubleshoot issues that might otherwise be difficult to identify. Sync Services includes tracing for the client and server synchronization providers.

Liam Cavanagh

Announcing Availability of Sync Framework v1 and Sync Services for ADO.NET v2

I am happy to announce the general availability of Sync Framework v1 and Sync Services for ADO.NET v2.  As of today you can download the Sync Framework SDK in 11 languages including Chinese (Hong Kong), Chinese (Simplified), English, German, French, Italian, Japanese, Korean, Portuguese, Russian and Spanish for AMD64, IA64 and x86 processors from the Sync Framework Download Center. 

What is Sync Framework?

Imagine being able to build a solution that seamlessly exchanges contact information between Outlook, a database contact management application, your mobile device and your service based contact management system.  Or how about a mobile device that connects with other devices to exchange pictures and videos.  How about being able to take data from any of your enterprise databases, file or enterprise systems and make it available offline for users to modify and sync back up to the enterprise.  All of these capabilities are possible with the Sync Framework and best of all, it is free on Windows platforms and licensable on non-Windows platforms! 

Just a few examples of companies that are already using the Sync Framework include:

• SmugMug who has added support for the Sync Framework to enable developers to build rich offline applications that can allow photos to be easily shared on friends and families local computers. 
• InterScape have embedded Sync Framework into their Customer Relationship & Management (CRM) solution to enable sales people to synchronize enterprise files and data for offline access. 
• Fujitsu Siemens will embed the SyncToy file synchronization solution (powered by Sync Framework) on their STORAGEBIRD external drives to enable seamless file synchronization between devices and computers. 

Sync Framework Highlights 

Some of the highlights of this release include:

• Sync support to new and existing applications, services, and devices
• Collaboration and offline capabilities for any application
• Roam and share information from any data store, over any protocol, and over any network configuration
• Leverage sync capabilities exposed in Microsoft technologies to create sync ecosystems
• Extend the architecture to support custom data types including files

Sync Services for ADO.NET v2 Highlights 

Sync Services for ADO.NET is a Microsoft Sync Framework powered solution for synchronizing ADO.NET enabled databases in offline and collaboration scenarios. Sync Services for ADO.NET allows developers who are familiar with the concepts of ADO.NET to apply that knowledge to data synchronization through a very similar set of APIs to that of ADO.NET. Sync Services for ADO.NET provides the flexibility of a programming model like offline datasets and a richer synchronization feature set like that found in Merge replication. Sync Services for ADO.NET also supports synchronization over services, such as Windows Communication Foundation (WCF).

Some of the Sync Services for ADO.NET highlights include:

• Offline Database Synchronization: With Sync Services for ADO.NET you are able to build a solution where multiple remote clients connect and synchronize to a central ADO.NET database in a Hub-and-Spoke configuration. This enables occasionally connected devices to periodically connect and synchronize changes with a central ADO.NET database server. This topology is a common solution for remote workers such as sales reps or field service workers.
• Collaboration Between Databases: Sync Services for ADO.NET also includes Peer-to-Peer capabilities. Through a custom Peer provider, collaboration between two or more SQL Server databases can occur (support is not available for SQL Server Compact). Unlike a Hub-and-Spoke architecture, this provider enables a SQL Server database to communicate and exchange information with any other SQL Server database. This type of scenario is useful in group scenarios where users (such as auditors) need to update information and then collaborate those changes with other group members.
• SQL Server 2008 Integrated Change Tracking: If you are using SQL Server 2008, we recommend that you use the SQL Server change tracking feature. This feature addresses many of the issues of custom-tracking systems and provides a straightforward way to track changes. Change tracking is also supported by the Local Database Cache in Visual Studio 2008 SP1. Developers can now specify that the Configure Data Synchronization wizard should enable SQL Server change tracking on the server and generate the commands necessary to select and apply changes to the server database. Unlike custom change tracking systems, SQL Server change tracking does not require any schema changes in the server database. For more information, see the Visual Studio 2008 documentation.

Sync Framework Pricing

Sync Framework will be licensed free on Windows platforms.  In addition, we are also licensing the specifications and a source code porting kit to developers who want to implement Microsoft Sync Framework solutions on non-Windows platforms. 

To learn more, visit http://msdn.microsoft.com/sync

I also want to send a special thank-you to those of you who have provided feedback throughout our CTP's and have helped make this a great release!

Liam Cavanagh