Microsoft's Embedded Database - SQL Server Compact - Team Blog

SQL Server Compact 4.0 Tooling Support in Visual Studio 2010 SP1 and Visual Web Developer Express 2010 SP1

SQL CE Team — Tue, 15 Mar 2011 09:49:00 GMT

Quick Navigation Links:

Advantages of using Visual Studio 2010 SP1 or Visual Web Developer Express 2010 SP1 to develop applications for SQL Server Compact 4.0

Visual Studio Designers that do not work with SQL Server Compact 4.0

Fixing installation issues

Overview

The Ultimate, Premium and Professional editions of Visual Studio 2010 SP1, and the Visual Web Developer Express 2010 SP1 has support for tooling for SQL Server Compact 4.0. The Visual Studio 2010 SP1 and the Visual Web Developer Express 2010 SP1 can be installed using the Microsoft Web Platform Installer 3.0. After installing the Web Platform Installer 3.0, the screen as shown below will come up. Click on the Add button to install the Visual Studio 2010 SP1 and the Visual Web Developer Express 2010 SP1.

Screenshot 1: Web Platform Installer 3.0 with the option to install Visual Studio 2010 SP1 or Visual Web Developer Express 2010 SP1.

Note that selecting and installing the Visual Studio 2010 SP1 or the Visual Web Developer Express 2010 SP1 also installs the SQL Server Compact 4.0 components that are needed for the tooling support. These components are called as Visual Studio 2010 SP1 Tools for SQL Server Compact 4.0 and SQL Server Compact 4.0. The Web Platform Installer automatically detects whether these components are already installed on the machine, and if they are not installed, the Web Platform Installer will install them. There is no need to separately select the SQL Server Compact 4.0 components in the Web Platform Installer.

Back to the top

Advantages of using Visual Studio 2010 SP1 or Visual Web Developer Express 2010 SP1 to develop applications for SQL Server Compact 4.0

The Visual Studio 2010 SP1 can be used to develop ASP.NET web applications for Compact 4.0. The different designers that provide functionality for Compact 4.0 is as given below:

Server Explorer and Query Designer: After installing the Compact 4.0 components for Visual Studio 2010 SP1, a new option to connect to SQL Server Compact 4.0 databases will appear in the Change Data Source dialog. This option will be in-addition to the existing option to connect to the SQL Server Compact 3.5 databases. After connecting to a SQL Server Compact 4.0 databases, the different objects in it can be navigated through in the Server Explorer and queries can be run in the Query Designer window.

Screenshot 2: Change Data Source dialog for selecting SQL Server Compact 4.0 data source.

Transact-SQL Editor: Start the Transact-SQL Editor from the Data menu and connect to a SQL Server Compact 4.0 database in the Connect to Server dialog shown below by specifying the path to the database file or by selecting the New Database option in the Database file dropdown field. Note that when the path to the database file is specified, the version of file is auto-detected, and depending on the version of the file either SQL Server Compact 3.5 or SQL Server Compact 4.0 runtime engine is used to connect to the file. This behavior is different from that in the Add Connection and Change Data Source dialogs of the Server Explorer where the data source has to be explicitly set to Compact 3.5 or 4.0. The Create New SQL Server Compact Database dialog provides options for creating either Compact 3.5 or Compact 4.0 database file as shown in the screenshot below.

Screenshot 3: Connecting to a SQL Server Compact 4.0 database in the Transact-SQL Editor connection window.

Screenshot 4: Creating a new SQL Server Compact 4.0 database.

Designers in the VB and C# ASP.NET Web application or web site projects: Designers can be used in the VB and C# based ASP.NET Web Application or Web Site projects to generate datasets or entities from a Compact database and these can be bound to controls on the ASP.NET web page. The workflow is same as that used for SQL Server and more information about the controls are at the SqlDataSource and the EntityDataSource web control pages. In the Add New Item dialog box a new item template for SQL Server Compact 4.0 Local Database is available to add a new empty Compact 4.0 database file to the project.

Click once deployment: In the prerequisites dialog of the click once deployment there is a checkbox for Compact 4.0 and if the VB or C# Windows projects uses Compact 4.0 the click once publishing for Compact 4.0 will work if the Compact 4.0 entry is checked in the prerequisites window.

Screenshot 5: The prerequisites dialog for Click Once deployment.

Opening WebMatrix projects in Visual Studio 2010 SP1: A WebMatrix project can be opened in Visual Studio 2010 SP1 or in Visual Web Developer Express 2010 SP1 by clicking on the Visual Studio option in the WebMatrix IDE. In addition, if the required SQL Server Compact 4.0 components that are needed for Visual Studio 2010 SP1 are not installed on the machine, a message will pop-up to start the Web Platform Installer 3.0 and the Web Platform Installer 3.0 will install all the required components on the machine.

Back to the top

Visual Studio Designers that do not work with SQL Server Compact 4.0

The following wizards do not work with Compact 4.0 in the Windows project system. Developers can manually add reference to the ADO.NET provider for Compact 4.0 (System.Data.SqlServerCe) to develop programs for Compact 4.0 in the Windows projects:

- The Data Source Configuration Wizard that is used to configure datasets.

- The Configure Data Configuration wizard that is used to setup the syncing of data and schema with SQL Server using Sync FX

- The Entity Data Model wizard that is used to generate entities from a Compact database.

Back to the top

Fixing installation issues

If you face any issues in installing the SQL Server Compact 4.0 components do the following:

Go to Start à Control Panel à Programs à Programs and Features to see the list of all the applications and software installed on your machine. Another way to navigate is to open the Windows Explorer and copy in – Control Panel\Programs\Programs and Features – in the address bar at the top. Check to see whether the following are installed. On a 32-bit machine the name will show as Microsoft SQL Server Compact 4.0 x86 ENU:

Screenshot 6: Entries for SQL Server Compact in the Add/Remove Programs.

If the Microsoft Visual Studio 2010 SP1 Tools for SQL Server Compact 4.0 is not installed type in SP1 in the search bar at the top right in the Web Platform Installer. This will bring up all the components that have SP1 in their name. Click on the Add button in front of Microsoft Visual Studio 2010 SP1 Tools for SQL Server Compact 4.0 to install it. The Microsoft Visual Studio 2010 SP1 Tools for SQL Server Compact 4.0 can be only installed if there is an existing installation of Visual Studio 2010 SP1 or Visual Web Developer Express 2010 SP1. Similarly if you have trouble installing SQL Server Compact 4.0, type in SQL Server Compact 4.0 in the search bar and click on the Add button to install it. The information about the different executable files for installing different SQL Server Compact 4.0 components is at MSDN Online.

Screenshot 7: Searching for all products with SP1 in their name.

Screenshot 8: Highlighting Microsoft Visual Studio 2010 SP1 Tools for SQL Server Compact 4.0.

Screenshot 9: Searching for SQL Server Compact 4.0 and highlighting SQL Server Compact 4.0.

Back to the top

Regards,

Ambrish Mishra

Program Manager – SQL Server Compact

Microsoft SQL Server Compact 4.0 is available for download

SQL CE Team — Wed, 12 Jan 2011 14:03:00 GMT

The SQL Server Compact team is happy to announce that the next version of Microsoft SQL Server Compact 4.0 has been released, and is available for download with the Microsoft Web Platform Installer 3 or from the Microsoft Download Center.

SQL Server Compact 4.0 has been designed, developed, tested and tuned over the course of last year and the release has been also vigorously verified by the vibrant MVP and developer community. The feedback from the developer community has helped improve the quality of the SQL Server Compact 4.0 release and the Compact team would like to thank all community members who participated in the release.

The main value proposition of SQL Server Compact 4.0 is the ability to use it as a data store for ASP.NET web applications. Please take a moment to go through the scenarios in which SQL Server Compact 4.0 will not work. For these scenarios SQL Server Compact 3.5 SP2 is the right choice.

Reasons for using SQL Server Compact 4.0:

Database for ASP.NET Web Applications and Web Sites: Compact 4.0 has been optimized and tuned for use for ASP.NET web applications and has been field tested to ensure that Compact 4.0 can handle the load of starter websites and scale up to a database size of 4 GB.
1. Default database for Microsoft WebMatrix: Compact 4.0 is the default database for Microsoft WebMatrix, which is the web stack that encapsulates all the technologies like ASP.NET, IIS Express, Editor and SQL Server Compact that are needed to develop, test and deploy ASP.NET websites to third party website hosting providers.
2. Rapid website development with free, open source web applications: Popular open source web applications like mojoPortal, Orchard, Umbraco etc. support Compact 4.0 and can be used to rapidly develop, test and deploy websites.
3. Partial trust and virtual memory: Compact 4.0 works in partial trust that is the mainline scenario in which the ASP.NET web applications are deployed. The virtual memory used by Compact 4.0 has been optimized to ensure that that maximum allowed 256 connections can be opened for the database.
4. One click migration to SQL Server: As the requirements grow to the level of enterprise databases, the schema and data can be migrated from Compact to SQL Server using the migrate option in the WebMatrix IDE. This also adds a web.config xml file to the project that contains the connection string for the SQL Server. Once the migration completes, the website project seamlessly switches from using Compact to SQL Server.
5. Removes the soft block for use in ASP.NET web applications: In the earlier versions of Compact a flag, namely, SQLServerCompactEditionUnderWebHosting had to be flipped to use Compact in ASP.NET web applications. The flag has been removed in Compact 4.0.
Enhancements of the fundamentals (setup, deployment, reliability, and encryption algorithms): The bases of Compact 4.0 have been strengthened to ensure that it can be installed without any problems, and can be deployed easily, and works reliably while providing the highest level of security for data.
1. Consistent setup and install experience and small download: There are separate MSIs for x86 and x64 platforms, and the x64 MSI installs the Compact components in both the WOW mode and the native mode in - %Program Files (x86)%\Microsoft SQL Server Compact Edition\v4.0, and in %Program Files%\Microsoft SQL Server Compact Edition\v4.0. The x86 MSI has been blocked from installing Compact components on an x64 platform, and similarly x64 MSI has been blocked from installing on x86 platform. This leads to a consistent installation and setup experience and also avoids future issues in Service Pack releases, where a higher version of SP1 x86 MSI can overwrite the GACed assemblies with a higher version on an x64 platform. The issue was faced in the Compact 3.5 SP2 release and the knowledge base article no 974247 describes the issue in more detail. All these setup improvements have been done while keeping the small download size of Compact intact at 2.5 MB.
2. Easy private deployment: All the x86 and x64 Compact assemblies and DLLs that need to be privately deployed are in the folder - %Program Files%\Microsoft SQL Server Compact Edition\v4.0\Private. The contents of the folder can be copied in the application directory to privately deploy Compact. Both the x86 and x64 MSIs install the x86 and x64 Compact assemblies and DLLs in the Private folder and there is no need to install the x64 MSI on an x86 platform to get the x64 DLLs. Note that the native DLLs of SQL Server Compact need the Microsoft Visual C++ 2008 Runtime Libraries (x86 and x64) SP1 to function properly. The assemblies for the Visual C++ 2008 runtime are present in the Private folder, and deploying all the DLLs and folders in the Private folder deploys all the files needed for Compact to work properly.
3. Higher reliability: The ASP.NET web applications have a different workload than the desktop applications. Compact has been tested for handling the workload of starter websites, and this has made the product more reliable than the earlier versions.
4. Highly secure: Compact 4.0 uses SHA2 algorithm to secure data and provide a high level of security. Compact 4.0 is also FIPS compliant and can be used in applications that need FIPS compliance.
ADO.NET Entity Framework, API and T-SQL syntax enhancements: Compact 4.0 also has enhancements for Entity Framework, APIs and T-SQL syntax as described below:
1. ADO.NET Entity Framework 4 (.NET FX 4) code-first and server generated keys: Compact 4.0 works with the code-first programming model of ADO.NET Entity Framework. In addition, the columns that have server generated keys like identity, rowguid etc. are also supported in Compact 4.0 when used with ADO.NET Entity Framework 4.0 (this is the version of ADO.NET Entity Framework that released with .NET FX 4). Support for the code-first and for the server-generated keys rounds out the Compact support for ADO.NET Entity Framework and fulfills one of the main asks from the development community.
2. T-SQL syntax for OFFSET & FETCH: Compact 4.0 supports the T-SQL syntax for OFFSET & FETCH, and due to this paging queries can be run against the database file.
3. API enhancements: Compact adds in the support for two new APIs. The first one is the System.Data.SqlServerCe.SqlCeConnection.GetSchema() that can be used to get the schema from a Compact database file. The API is also used to provide the support for System.Data.Common.DbConnection.GetSchema. The second new API is the System.Data.SqlServerCe.SqlCeConnectionStringBuilder() that let developers programmatically create correct connection string for Compact 4.0, and to parse & rebuild existing connection strings. The API is also used to provide the support for System.Data.Common.DbConnectionStringBuilder.
Easy application development in Visual Studio 2010: Visual Studio 2010 provides for rich designers and easy debugging that can be used to develop ASP.NET web applications for Compact 4.0. See the Visual Studio 2010 SP1 Beta help page for information on installing the required components to develop applications for SQL Server Compact 4.0.

Scenarios not enabled by SQL Server Compact 4.0:

The SQL Server Compact was focusing on enabling the new scenarios for ASP.NET Web Applications, and due to time and resource constraints the following scenarios have not been enabled in SQL Server Compact 4.0. For these scenarios the SQL Server Compact 3.5 SP2 is the right choice.

Data replication with SQL Server: Compact 4.0 does not support data replication with SQL Server using Sync Framework, merge replication or remote data access (RDA).
LINQ to SQL: The LINQ to SQL does not work with Compact 4.0.
1. The ADO.NET Entity Framework 4 that ships in .NET FX 4 works with Compact 4.0, and additional features like support for code-first and server generated keys have been added in Compact 4.0 for EF 4.
Windows Mobile, Windows Phone and Windows CE devices: Compact 4.0 does not have a release for Windows Mobile, Windows Phone or for Windows CE devices.
Designers in the VB or C# Windows projects in Visual Studio 2010 SP1 Beta: The following wizards do not work with Compact 4.0 in the Windows project system. Developers can manually add reference to the ADO.NET provider for Compact 4.0 (System.Data.SqlServerCe) to develop programs for Compact 4.0 in the Windows projects:
1. The Data Source Configuration Wizard that is used to configure datasets.
2. The Configure Data Configuration wizard that is used to setup the syncing of data and schema with SQL Server using Sync FX.
3. The Entity Data Model wizard that is used to generate entities from a Compact database.

Known Issues:

The SQL Server Compact 4.0 Books Online and the updated read me is available on the Microsoft Download Center and at MSDN Online. The Books Online and the read me provides the required information for developing applications with SQL Server Compact 4.0 and the list of known issues and supported platforms. For more information about Microsoft WebMatrix refer to the links below:

Thanks,

Ambrish Mishra

Program Manager - SQL Server Compact

Downloads Available for SQL Server Compact 4.0 CTP2, Visual Studio 2010 SP1 Beta Tools for SQL Server Compact 4.0 CTP2, and CMS (mojoPortal, Orchard and Umbraco) that work with SQL Server Compact.

SQL CE Team — Wed, 22 Dec 2010 13:21:00 GMT

The second community technology preview release of SQL Server Compact 4.0 is available for download from the Microsoft Download Center or as part of the Microsoft WebMatrix Beta 3 in the Microsoft Platform Installer 3.0. To learn more about Microsoft WebMatrix Beta 3 see the WebMatrix Beta Book and video tutorials.

An update to the Microsoft Visual Studio 2010 SP1 Beta has been also released that provides for support for SQL Server Compact 4.0 CTP2 in Visual Studio 2010 SP1 Beta. Follow the steps below to enable support for SQL Server Compact 4.0 CTP2 in Visual Studio 2010 SP1 Beta. For more information refer to the Visual Studio 2010 SP1 Beta help page:

1. Download and install Microsoft Visual Studio 2010 SP1 Beta.

2. To use SQL Server Compact with the Visual Studio 2010 SP1 Beta, install the following packages from the Microsoft Web Platform Installer 3.0:

a. Microsoft SQL Server Compact 4.0 CTP2

b. Microsoft Visual Studio 2010 SP1 Tools for SQL Server Compact 4.0 CTP2

The following open source content management systems support SQL Server Compact as the database and can be used to rapidly build websites:

1. mojoPortal and more information is at mojoPortal’s site.

2. Orchard and more information is at Orchard’s site.

3. Umbraco and more information is at Umbraco’s site.

Thanks,

Ambrish Mishra

Program Manager

Introducing SQL Server Compact 4.0, the Next Gen Embedded Database from Microsoft

SQL CE Team — Wed, 07 Jul 2010 09:08:00 GMT

Microsoft SQL Server Compact 4.0 is the next generation of embedded database from the SQL Server family, and brings all of its goodness to the world of ASP.NET web application development. SQL Server Compact 4.0 will provide an easy and simple to use database storage for starter websites, and the features of Compact that wooed the mobile devices and desktop developers will now be available to the ASP.NET web application developers. In addition, the features and enhancements in SQL Server Compact 4.0 like the new Transact-SQL syntax for OFFSET & FETCH, higher reliability, seamless migration of schema and data to the higher SKUs of SQL Server, support for code-first & server generated keys in ADO.NET Entity Framework 4.0, and the ability to use Visual Studio 2010 to develop ASP.NET web applications for Compact 4.0 etc. will also excite the existing development community.

The first Community Technology Preview (CTP1) release of SQL Server Compact 4.0 is integrated with the WebMatrix Beta, which is a free ASP.NET web application development tool for Windows, and provides development & database support, and can easily publish/deploy websites to the web hosting providers. SQL Server Compact 4.0 CTP1 plays an important role in WebMatrix Beta, and enables WebMatrix Beta to be a small download, with a simple development experience, and contributes to the seamless end-to-end experiences from development through to publish. Scott Guthrie’s (ScottGu) blog has detailed information about the WebMatrix Beta and its different components. WebMatrix Beta can be installed in one-click through the Microsoft Web Platform Installer (Web PI) 3 Beta. In addition to the Web PI 3 Beta, SQL Server Compact 4.0 CTP1 is also available for download from the Microsoft download center.

The topics that are covered in the blog are listed below. SQL Server Compact will be referred to by its abbreviated form wiz. Compact throughout this blog for easy reading:

1. Specialty of SQL Server Compact – Highlights the reasons for using SQL Server Compact for application development. The section will be useful for the ASP.NET developers and will be a reaffirmation for the existing Compact developers.

2. New Features in SQL Server Compact 4.0 CTP1 – The section lists the new features in the first CTP release of Compact 4.0 and will be of interest to both the ASP.NET and the existing Compact development community.

3. Known Issues in SQL Server Compact 4.0 CTP1 – The known issues in the first CTP release of Compact 4.0 and the workarounds are listed in this section.

1. Specialty of SQL Server Compact:

SQL Server Compact has many specialties that are well liked by the developer community. The specialties of Compact, among others, are as listed below:

· Small Size: Easy to download with a small size of 2 to 2.5 MB and installs fast with no configuration needed.

· Program with .NET Framework APIs: Simple to use because of the familiar .NET Framework APIs, including ADO.NET and ADO.NET Entity Framework, and similar programming patterns.

· Supports Transact-SQL Syntax: Supports a rich subset of the Transact-SQL syntax supported by higher SKUs of SQL Server.

· Private (xcopy) Deployable: Xcopy deployable within the application folders and no admin privileges are needed. The database file contains no code and is safe to deploy along with the database.

· In-process Database: Loads in the application’s process memory space and requires no inter-process communication.

· Integrated with Visual Studio: Applications can be rapidly developed using the integrated development experience in Visual Studio.

· Easy Migration to SQL Server: Smooth on-ramp to SQL Server when the applications requirements grow and there is a need to handle large amounts of data and high concurrent loads due to multiple users.

· Free Product with Free Tools: Compact is a free product, and is integrated with the free Express versions of Visual Studio and WebMatrix Beta, and requires low investment to start building and deploying applications.

· Simple Solutions for Syncing data with SQL Server: Schema and data can be synced easily with SQL Server using inbuilt and programmable technologies like Sync Framework (Sync FX) and merge replication.

2. New Features in SQL Server Compact 4.0 CTP1

The first CTP release of SQL Server Compact 4.0 has a lot of features that will be useful to all of the development community of Compact, including the latest entrants, the ASP.NET developers. The different features of SQL Server Compact 4.0 CTP1 categorized separately are as described below:

2.1 Fundamentals

The set of features in this category are improvements that made Compact better, more reliable and secure. More details about the features are given below:

Higher Reliability – SQL Server Compact 4.0 CTP1 has been tested and certified to work with ASP.NET web applications and can handle the load of starter websites. Compact 4.0 CTP1 performs without any exceptions or access violations in scenarios where there are a number of concurrent connections and multiple requests are being made to the database.
Encryption Algorithm Update – Compact 4.0 CTP1 supports the SHA 2 encryption algorithms that provide a higher level of security for the encrypted Compact database files and are also needed for the FIPS compliance. The algorithms that were supported in the earlier versions of Compact are only supported in Compact 4.0 CTP1 for opening the database file to upgrade them to Compact 4.0 CTP1. Due to this, when Compact 4.0 CTP1 opens a Compact 3.5 database file, the Compact 3.5 database file will need to be upgraded to Compact 4.0 CTP1 using the System.Data.SqlServerCe.SqlCeEngine.Upgrade() API.
Setup Enhancements – SQL Server Compact 4.0 CTP1 setup and installation has been enhanced to ensure that the x86 and x64 components of SQL Server Compact install properly on a 64-bit machine. This is done to ensure that Compact 4.0 CTP1 supports both the WOW64 mode and the 64-bit native applications on the 64-bit machine without any problems, in both the conditions i.e. when Compact is deployed centrally using the exe installer or when it is privately deployed within the application folders.
1. In Compact 4.0 CTP1, the x86 exe will install on a 32-bit machine and the x64 MSI will install on a 64-bit machine only. The x64 MSI will install both the 32-bit and 64-bit Compact components on the 64-bit machine and due to this the x86 MSI is no longer needed to be installed on the 64-bit machine.
2. If one wants to do a private deployment of Compact 4.0 CTP1, all the assemblies (both 32-bit and 64-bit) for the private deployment will be available in the %Program Files%\Microsoft SQL Server Compact Edition\v4.0\Private and in the %Program Files (x86)%\Microsoft SQL Server Compact Edition\v4.0\Private folder.

2.2 Improvements for ASP.NET Web Applications

Support for ASP.NET – Compact 4.0 CTP1 supports ASP.NET without the need for any specific configuration. The flag SQLServerCompactEditionUnderWebHosting is not required in Compact 4.0 CTP1 and has been removed.
Virtual Memory Reduction – The virtual memory that Compact uses has been reduced in Compact 4.0 CTP1. The visible difference is that if an application tries to open 40 to 50 simultaneous connections in SQL Server Compact 3.5 SP2, an ‘out of virtual memory’ exception will be hit. With Compact 4.0 CTP1, the application can easily open as many connections as needed to the database file up to the full extent of 256 connections without running out of virtual memory.
Medium Trust – Compact 4.0 CTP1 assemblies have been attributed with Allow Partially Trusted Caller’s Attribute (APTCA) and Compact 4.0 CTP1 can run in medium trust.

2.3 Develop Compact 4.0 CTP1 Applications with WebMatrix Beta and Visual Studio 2010

Seamless Migration of Data and Schema from Compact 4.0 CTP1 into SQL Server – The WebMatrix Beta can be used to script data and schema from Compact 4.0 CTP1 into the higher versions of SQL Server. The WebMatrix Beta can be also used to script data and schema from Compact 4.0 CTP1 to SQL Azure. The script to SQL Azure is in an early (alpha) stage for this first CTP release.
Designer Support in WebMatrix Beta – The WebMatrix Beta can be used to develop ASP.NET web applications with Compact 4.0 CTP1 and to manage the Compact database including running T-SQL queries against Compact.
Designer Support in Visual Studio 2010 – An update to Visual Studio 2010 is also being worked upon that will provide support for using Compact 4.0 in Visual Studio 2010. Compact 4.0 will be also supported in Visual Studio 2010 Web Developer Express.

2.4 New Transact-SQL Syntax

Support for Paging Queries – Compact 4.0 CTP1 supports the T-SQL syntax for OFFSET & FETCH, and due to this paging queries can be run against the database file. A few sample T-SQL queries for OFFSET & FETCH that can be run against the sample Northwind.sdf database file are given below:

· The query below skips the first 10 rows and fetches all the other rows in the Employees table ordered by the Hire Date:

SELECT * FROM Employees ORDER BY [Hire Date] OFFSET 10 ROWS;

· The first query below skips the first 10 rows and fetches the next 10 rows in the Customers table and the rows are ordered by the Customer ID. The second query skips the first 2 rows and fetches the next one row in the Orders table ordered by the Shipped Date. This query a variation of the first query:

SELECT * FROM Customers ORDER BY [Customer ID] OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY;

SELECT * FROM Orders ORDER BY [Shipped Date] OFFSET 2 ROWS FETCH NEXT 1 ROW ONLY;

2.5 API Enhancement

Seamless Integration with ADO.NET Entity Framework 4.0 (.NET FX 4) – Compact 4.0 CTP1 will support the code-first programming model of ADO.NET Entity Framework. In addition, the columns that have server generated keys like identity, rowguid etc. will be also supported in Compact 4.0 when used with ADO.NET Entity Framework 4.0 (this is the version of ADO.NET Entity Framework that released with .NET FX 4). The ADO.NET Entity Framework is working on a separate release and that will provide the code-first and server generated keys support for Compact.
1. The Compact 4.0 CTP1 release only works with ADO.NET Entity Framework 4.0 that released with .NET FX 4. Using Compact 4.0 CTP1 with ADO.NET Entity Framework 3.5 SP1 that released with .NET Framework 3.5 SP1 will result in an error.

New API for GetSchema: Compact has a new API - System.Data.SqlServerCe.SqlCeConnection.GetSchema() – that can be used to get the schema from a Compact database file. The API is also used to provide the support for System.Data.Common.DbConnection.GetSchema. In the example below the GetSchema API is used to get the names of all the tables, columns, indexes, indexed columns and foreign keys from the sample Northwind.sdf database.

SqlCeConnection conn = new SqlCeConnection("DataSource=Northwind.sdf");

try

{

conn.Open();

DataTable dt = conn.GetSchema();

dt = conn.GetSchema("Tables");

dt = conn.GetSchema("Columns");

dt = conn.GetSchema("Indexes");

dt = conn.GetSchema("IndexColumns");

dt = conn.GetSchema("ForeignKeys");

}

catch (Exception e)

{

//Fail

}

New API for SqlCeConnectionStringBuilder: Compact is has new API - System.Data.SqlServerCe.SqlCeConnectionStringBuilder() – that let developers programmatically create correct connection string for Compact 4.0, and to parse & rebuild existing connection strings. The API is also used to provide the support for System.Data.Common.DbConnectionStringBuilder. In the example below the SqlCeConnectionStringBuilder performs checks for valid key/value pairs and creates a valid SQL Server Compact connection string.

System.Data.SqlServerCe.SqlCeConnectionStringBuilder builder =

new System.Data.SqlServerCe.SqlCeConnectionStringBuilder();

builder["Data Source"] = "Northwind.sdf";

builder["Mode”] = "Exclusive";

Console.WriteLine(builder.ConnectionString);

The result is the following connection string:

Data Source=Northwind.sdf;Mode=Exclusive

3. Known Issues in SQL Server Compact 4.0 CTP1:

The known issues in this CTP release of Compact 4.0 are listed below:

1. Compact 4.0 CTP1 needs Visual C++ 2008 Runtime Libraries (x86, IA64 and x64), Service Pack 1: The native DLLs of SQL Server Compact 4.0 CTP1 need the Microsoft Visual C++ 2008 Runtime Libraries (x86, IA64 and x64), Service Pack 1. Installing the SQL Server Compact 4.0 CTP1 using the Windows Installer (.exe) file, also installs the Visual C++ 2008 Runtime Libraries SP1. If SQL Server Compact 4.0 CTP1 is deployed privately in the application’s folder the following have to be present on the machine for SQL Server Compact to function properly:

a. Installing the .NET Framework 3.5 SP1 also installs the Visual C++ 2008 Runtime Libraries SP1.

b. Visual C++ 2008 Runtime Libraries SP1 can be downloaded and installed from the location given below: http://go.microsoft.com/fwlink/?LinkId=194827

Note that installing .NET Framework 2.0 or 3.0 or 4 does not install the Visual C++ 2008 Runtime Libraries SP1.

2. Uninstallation and repair of Compact 4.0 CTP1 from the command line does not work: Uninstallation of Compact runtime using command line options does not work in this CTP release. For uninstallation of SQL Server Compact runtime go to Add/remove programs (Start->Run->appwiz.cpl), and right click on the entry ‘Microsoft SQL Server Compact 4.0 ENU CTP1’ or ‘Microsoft SQL Server Compact 4.0 x64 ENU CTP1’ , and select uninstall from the menu.

3. Compact 4.0 development support in Visual Studio 2010: An update to the Visual Studio 2010 is being worked upon that will add the support for Compact 4.0. The update will also provide for the development support for Compact 4.0 in the Visual Studio 2010 Web Developer Express.

4. Code first programming model support for Compact 4.0 in ADO.NET Entity Framework: The Microsoft ADO.NET Entity Framework Feature Community Technology Preview 3 is an early preview of the code-first programming model for the ADO.NET Entity Framework 4.0. The code-first feature in the ADO.NET Entity Framework CTP3 release does not work properly with Compact 4.0. Future releases of code-first programming model for ADO.NET Entity Framework will provide support for code-first programming model for Compact 4.0.

Regards

Ambrish Mishra

Program Manager – SQL Server Compact

Troubleshooting: Privately deploying the SQL Server Compact 3.5 SP2 ADO.NET Entity Framework provider (System.Data.SqlServerCe.Entity.dll) in the application folder does not work

SQL CE Team — Wed, 12 May 2010 11:13:00 GMT

For complete instructions about deploying SQL Server Compact ADO.NET Entity Framework provider (System.Data.SqlServerCe.Entity.dll) privately in the application’s folder refer to the blog here. Note that from SQL Server Compact 3.5 SP2 release onwards if you are privately deploying SQL Server Compact 3.5 SP2 DLLs in your application folder you need to copy the managed DLLs (System.Data.SqlServerCe.dll and System.Data.SqlServerCe.Entity.dll) from the location %Program Files%\Microsoft SQL Server Compact Edition\v3.5\Private. Refer to the SQL Server Compact 3.5 SP2 Books Online for more information.

Symptoms

When an application uses System.Data.SqlServerCe.Entity.dll from SQL Server Compact 3.5SP2 in xcopy (i.e. private) deployment mode by copying the Compact’s ADO.NET provider (System.Data.SqlServerCe.dll) and the Compact’s ADO.NET EF provider (System.Data.SqlServerCe.Entity.dll) DLLs from the %Program Files%\Microsoft SQL Server Compact Edition\v3.5\Private, following exception is thrown:
"System.IO.FileLoadException: Could not load file or assembly 'System.Data.SqlServerCe, Version=3.5.1.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)"

Cause

System.Data.SqlServerCe.Entity.dll and System.Data.SqlServerCe.dll from the Private folder is of version 3.5.1.50. But unfortunately, the System.Data.SqlServerCe.Entity.dll from the Private folder (i.e. with version 3.5.1.50) is referring to System.Data.SqlServerCe.dll from the Desktop folder (i.e. with version 3.5.1.0). Due to this, an application that has only the dlls from Private folder fails as it doesn't find the 3.5.1.0 version of System.Data.SqlServerCe.dll.

Resolution

Add assembly binding redirection entry in app.config file.
Append following lines under the configuration section of the app.config:

- Vipul Hattiwale

Microsoft SQL Server Compact 3.5 with Service Pack 2 released with Visual Studio 2010 and on the Web.

SQL CE Team — Wed, 14 Apr 2010 13:17:00 GMT

The latest release of SQL Server Compact also referred to as SQL CE or SQL Server Mobile, released with Visual Studio 2010 on the 12th of April 2010. SQL Server Compact 3.5 SP2 is a SQL Server Compact 3.5 with SP2 release, meaning that installing it on a machine using the Windows Installer (MSI) file upgrades the existing installation of SQL Server Compact 3.5 or SQL Server Compact 3.5 SP1 on the machine to SP2.

The main features for SQL Server Compact 3.5 SP2 are as given below:

Supports working with a SQL Server Compact 3.5 database using the Transact-SQL Editor in Visual Studio 2010. The Transact-SQL Editor can be used to run free-text Transact-SQL queries against a SQL Server Compact 3.5 database. The Transact-SQL Editor also provides the ability to view and save detailed estimated and actual query show-plans for SQL Server Compact 3.5 databases. Previously, the functionality provided by the Transact-SQL Editor was only available through SQL Server Management Studio.
New classes and members named SqlCeChangeTracking have been added to the System.Data.SqlServerCe namespace to expose the internal change tracking feature used by Sync Framework to track changes in the database. The SQL Server Compact change tracking infrastructure maintains information about inserts, deletes, and updates performed on a table that has been enabled for change tracking. This information is stored both in columns added to the tracked table and in system tables maintained by the tracking infrastructure. By using System.Data.SqlServerCe.SqlCeChangeTracking one can configure, enable, and disable change tracking on a table, and also access the tracking data maintained for a table. The API can be used to provide functionality in a number of scenarios. For example it can be used to provide custom implementations of client-to-server or client-to-client sync for occasionally connected systems (OCS) or to implement a custom listener application.
The managed assemblies of SQL Server Compact for use by the applications that privately deploy SQL Server Compact are installed in the folder %Program Files%\Microsoft SQL Server Compact Edition\v3.5\Private. Using these assemblies ensure that the application uses the privately deployed version of Compact even when a lower version of SQL Server Compact 3.5 is installed in the GAC.
Visual Studio 2010 installs both the 32-bit and 64-bit versions of SQL Server Compact 3.5 SP2 on a 64-bit machine. If a SQL Server Compact application is deployed using Click Once in Visual Studio 2010 then both the 32-bit and the 64-bit version of SQL Server Compact are installed on a 64-bit machine
SQL Server Compact 3.5 SP2 supports Windows 7 and Windows Server 2008 R2, and can sync data using merge replication with SQL Server 2008 R2 November CTP.
A list of the bug fixes in SQL Server Compact 3.5 SP2 will be also released as a KB article similar to the KB article - http://support.microsoft.com/default.aspx?scid=KB;%5bLN%5d;955965 within two weeks. Apart from the bug fixes, the hot fixes that were released for SQL Server 2005 Compact Edition or SQL Server Compact 3.5 SP1 have been rolled up in SQL Server Compact 3.5 SP2. The list of hot fixes is as given below:
- http://support.microsoft.com/kb/953259: Error message when you run an SQL statement that uses the Charindex function in a database that uses the Czech locale in SQL Server 2005 Compact Edition: "The function is not recognized by SQL Server Compact Edition"
- http://support.microsoft.com/kb/958478: Error message when you run a "LINQ to Entities" query that uses a string parameter or a binary parameter against a SQL Server Compact 3.5 database: "The ntext and image data types cannot be used in WHERE, HAVING, GROUP BY, ON, or IN clauses"
- http://support.microsoft.com/kb/959697: Error message when you try to open a database file from a CD in SQL Server Compact 3.5 with Service Pack 1: "Internal Error using read only database file"
- http://support.microsoft.com/kb/960142: An error message is logged, and the synchronization may take a long time to finish when you use an application to synchronize a merge replication that contains a SQL Server 2005 Compact Edition subscriber
- http://support.microsoft.com/kb/963060: An error message is logged, and the synchronization may take a long time to finish when you synchronize a merge replication that contains a SQL Server Compact 3.5 subscriber: "UpdateStatistics Start app=<UserAppName>.exe"
- http://support.microsoft.com/kb/967963: Some rows are deleted when you repair a database by using the Repair method together with the RepairOption.RecoverCorruptedRows option in SQL Server 2005 Compact Edition and in SQL Server Compact 3.5
- http://support.microsoft.com/kb/968171: Error message when you try to create an encrypted database in SQL Server 2005 Compact Edition: "The operating system does not support encryption"
- http://support.microsoft.com/kb/968864: Error message when you run a query in SQL Server Compact 3.5: "The column name cannot be resolved to a table. Specify the table to which the column belongs"
- http://support.microsoft.com/kb/969858: Non-convergence occurs when you synchronize a SQL Server Compact 3.5 client database with the server by using Sync Services for ADO.NET in a Hub-And-Spoke configuration
- http://support.microsoft.com/kb/970269: Access violations occur when you run an application under heavy load conditions after you install the 64-bit version SQL Server Compact 3.5 Service Pack 1
- http://support.microsoft.com/kb/970414: Initial synchronization of a replication to SQL Server Compact 3.5 subscribers takes significant time to finish
- http://support.microsoft.com/kb/970915: Error message when you synchronize a merge replication with SQL Server 2005 Compact Edition subscribers: "A column ID occurred more than once in the specification. HRESULT 0x80040E3E (0)"
- http://support.microsoft.com/kb/971027: Error message when you upgrade a very large database to SQL Server Compact 3.5: "The database file is larger than the configured maximum database size. This setting takes effect on the first concurrent database connection only"
- http://support.microsoft.com/kb/971273: You do not receive error messages when you run a query in a managed application that returns columns of invalid values in SQL Server Compact 3.5
- http://support.microsoft.com/kb/971970: You cannot insert rows or upload changes into the SQL Server 2005 Compact Edition subscriber tables after you run the "sp_changemergearticle" stored procedure or you add a new merge publication article when another article has an IDENTITY column
- http://support.microsoft.com/kb/972002: Error message when you try to create an encrypted database in SQL Server Compact 3.5: "The operating system does not support encryption"
- http://support.microsoft.com/kb/972390: The application enters into an infinite loop when you run an application that uses Microsoft Synchronization Services for ADO.NET to synchronize a SQL Server Compact 3.5 database
- http://support.microsoft.com/kb/972776: When the application calls the SqlCeConnection.Close method or the SqlCeConnection.Dispose method in SQL Server Compact 3.5, the application may stop responding at the method call
- http://support.microsoft.com/kb/974068: Error message when an application inserts a value into a foreign key column in SQL Server Compact 3.5: "No key matching the described characteristics could be found within the current range"

Downloads for SQL Server Compact 3.5 SP2:

The Web downloads for SQL Server Compact 3.5 SP2 is as listed below:

SQL Server Compact 3.5 SP2 for Windows desktop (32-bit and 64-bit)

Note that the file available for download is a 6 MB self-extracting executable (exe) file that contains the 32-bit and the 64-bit Windows Installer (MSI) files for installing SQL Server Compact 3.5 SP2 on a 32-bit and a 64-bit Computer. It is important to install both the 32-bit and the 64-bit version of the SQL Server Compact 3.5 SP2 MSI on a 64-bit Computer. Existing SQL Server Compact 3.5 applications may fail if only the 32-bit version of the MSI file is installed on the 64-bit computer. Developers should chain both the 32-bit and the 64-bit MSI files with their applications and install both of them on the 64-bit Computer. Refer to the KB article for more information.

SQL Server Compact 3.5 SP2 for Windows mobile devices (all platforms & processors)

SQL Server Compact 3.5 SP2 Server Tools (32-bit and 64-bit)

SQL Server Compact 3.5 SP2 Books Online (The Books Online is available for download)

SQL Server Compact 3.5 SP2 Samples

Visual Studio 2010 and .NET Framework 4

Regards,

Ambrish Mishra

Program Manager

Merge replication vs. Sync Services for Compact

SQL CE Team — Mon, 09 Nov 2009 14:12:00 GMT

SQL Server Compact supports predominantly two different sync technologies. “Merge replication” and “Sync Services”. Users might be using one or the other, and interested in knowing the differences between the two technologies, from a use-case point of view. Still, there might be others interested in knowing, which technology better suites their need, by comparing and choosing the right one. One simple rule of thumb here: Merge replication is designed keeping enterprises in mind, where Sync Services is a framework for developer community/social networking/hobbyist programmers etc… (everything else). “Merge replication” is a solution, which is suitable for Enterprises mostly, in preference to Sync services. However, Sync services is an open-ended, extensible framework on top of which, one can do almost everything that Merge replication does. This article concentrates mostly on the merge replication angle in terms of where it makese more sense and where it doesn't.

Where does "Merge" make more sense?

In the enterprise scenario. In my opinion, enterprises like things faster, better (without any issues), and easily do-able/repeatable. If a solution (like “Merge replication”) allows them to do things like “setting up a publication, subscribers etc…”, although with significant coding, people are not going to like it. So, tools and wizards for enterprises are mandatory (Also give them a way to re-do the setups, typically outputting scripts from these tools/wizards). To avoid the setup/upgrade glitches, provide an out-of-the-box integration story, with other often used components (like Windows, IIS, SQL Server, HTTP etc…). Also, tighter integration helps performance. Usually, in enterprise contexts, setting up a “data server and clients” is a long-term commitment, so, tighter integration is fine there. All these are basic requirements for the “Enterprise” oriented “merge replication” technology.

Note: There are many ways in which a tighter integration with other components helps deployments.

1. Efficiency/performance

2. Automation possibility and tooling

3. Better end-2-end support/solution story for the customers

4. Ability to monitor and troubleshoot parts of the system

5. Essentially, you leave all the hardwork to us, and in the end, expect something that automatically works J (Means, reduced cost of development and testing for you)

Also, my take is that “Enterprises like rich (and relevant) features”. Give them something that can easily model/extend their business logic, and that could be of great value to them. Some of the features of “Merge replication”, in this category are,

1. Automatic partition management

2. Custom conflict resolution hooks

3. Business logic plugins to do custom processing, in the process of merge replication

a. Example is, if one wants to charge the client for synching

b. One wants to compute his share whenever a salesman syncs to the server

4. Retention cleanup, which can be used to (weakly) set a stringent policy about incremental syncs.

a. Like, every salesman must sync at least once in a week or every branch office, must sync at least once a day.

5. Incremental schema changes replication

6. Integration with SQL Server mirroring

7. Integration with SQL Server backup and restore.

Other valuable features are, “Automatic identity range management” etc. I hope now it is very clear as to, why “merge replication” technology is good for enterprise businesses. Enterprises do not mind, buying into a special architecture, if that provides value. Enterprises can setup a particular set (at the least) of windows machines, with IIS and SQL Server, and run merge replication. So, a tailored solution, like merge replication is best suited for them.

Where does Sync play better role compared to Merge?

Now, let’s examine another use-case. Incidentally, in this use-case, the features of merge replication, do not seem much relevant. You want to write a stock-alert application, and link it with a stock-tick source on a website. On closer examination, this is also a “data sync” scenario, so, merge replication could be used here. Trying to use merge replication here, elicits many incompatibilities/redundancies, that are not relevant to this use-case.

First of all, the nature of the data source is unknown, it can change while the app is running too. The web-site might use SQL backend now, and something else tomorrow. The transport is not known, it is over HTTP, but, can have any format (plain HTML/Json etc…). There is no need of rich data semantics here, as one is just comparing a single piece time-series data, and deciding to throw the alert or not. Also, one does not need a big database at client, all he does is read and discard (or read, alert and discard). Clearly, using merge replication here is an overkill, and unsuitable. So, you should use “Sync services”.

If you want to write a very quick sync app, and use it anywhere*, “Sync services” is the answer. Because Sync Services, is not tied to any particular, server architecture or transport mechanism, one can use it anywhere*. For quick development of sync apps, Sync Services is integrated with Visual Studio. But, this is only to develop the app quickly, not to deploy it. Visual Studio is not even needed for plain development of any sync app. It is just there to make it easy and fast.

Sync services is amazingly extensible, componentized, customizable. While merge replication answers enterprise use-case, which is a typical use-case for data synchronization, virtually all other use-cases are addressable by “Sync services”. If for some reason, people do not want to use a particular server/transport architecture, (or can’t use a single architecture), Sync services is the way to go. Besides, Sync services is a free platform to develop apps on. It is a great tool to enable non-(traditional)-enterprise related, businesses/users to realize the new models of interaction cropping up almost everywhere now. Sync services can be made to have feature-parity with “merge replication” solution through coding, although, I hope you would agree that it is not the intent of Sync services. I quoted that here, only to make you realize the potential of Sync services platform.

When an open-ended sync solution needs to be developed, Sync Services Framework is the one to use. Many times in usage, applications to sync data, need heterogeneous servers/transports etc. This is a very important capability, for example, for news generation and propagation web-site. News could be present in various web-sites, and also inside documents/databases of different formats. Also, the solution should be open-ended and extensible, so that, it can be easily tailored for a new data source. The importance of “Sync services” in enabling such scenarios, should not be underestimated. Sync Services, is coming out strong, and there is more to watch out in this field, going forward. With this, we conclude the explanations of the technologies, from the use-cases point of view.

The list of features of merge replication and Sync services are tabulated below:

Feature	Supported in merge replication	Supported in Sync Services
Enterprise-centric
Type of the technology	Solution	Framework
Target users	Enterprises with DBAs	Developers/Hobbyists/Social networking Communities
Integration with DBA tools (SQL Server)	Yes	No
More tooling support	Yes	No
Pluggable business logic hooks	Yes	No
Conflict resolution support	Built-in + support for custom***	Built-in + extensible**
Schema propagation support	Initial + any changes	Limited, available by extension**
Partitioning support	Built-in	Available by extension
Server type (State-full or State-less)	State-full	State-less
Integration with Server mirroring (SQL Server)	Yes	No
Integration with Server backup/restore (SQL Server)	Yes	No
Identity ranges management	Yes	No

Developer centric
Type of the technology	Solution	Framework
Target users	Enterprises with DBAs	Developers/Hobbyists/Social networking Communities
Developer platform support	No	Yes, Visual Studio integration provided
Pluggable transport	No (HTTP with IIS server only)	Yes (Web service model is possible, by extension)
Heterogeneous server	No (Only SQL Server)	Any server is good (by extension).
Network architectures	Fixed, 3-tier	Variable, 2-tier to N-tier
Supports web services model	No	Yes
Ability to work with other sync platforms/frameworks	No	Yes (by extension)
Exposed API surface for tracking****	No	Yes

Common functionality provided
Subscribed database deployment	Yes	Yes
Type of change tracking used	ROWGUID	PK or ROWGUID
Sync directions allowed	All	All
Programmability layer	Native & managed	Managed

Semantics provided
Auto-management of dependent tables	Yes	No
Sync granularity enforced	Yes (Publication level)	No (table level)
Schema propagation	Yes	Limited extent
Built-in conflict resolution strategies (also customizable)	Yes	Only some limited number of built-ins

* Well, almost everywhere.

** Whenever we use, “available by extension” or simply “by extension”, that means developer should write code to achieve the desired behavior.

*** (For this document,) the difference between customizable and extensible is the following: customization is a type of extension, where, the application architecture is not affected majorly. Like, register a COM dll, to do conflict resolution etc…

**** Tracking is a mechanism used to tag all changes. Changes could be Inserts/Updates/Deletes on tracked tables. This module is used to detect changes between successive syncs, so that the data can be forwarded to the other party. Exposing the tracking API (enable/disable tracking etc…), helps write applications like peer-peer sync, transaction notifications etc…

Wrap-up:

We have looked at the use-cases where merge replication technology is suitable and where it is not. There are scenarios where “Sync services” comes out as a candidate for solution, and there are scenarios for merge replication too. When considering the right technology for using, there is no silver bullet; there is no panacea that works for all needs. The requirements of your use determine the “right” technology for you. Merge replication provides a solution, but, there are many scenarios that it can’t help you with. Sync services can be made to work in “any” scenario (including the one that merge replication provides out-of-the-box J), but, are you fine with building such a (mammoth) architecture yourself, and incurring the various costs involved? In merge replication, we take the pains and give you a solution, although, for a special, tailored need, is also happens to be the most common and justified one. In sync services, you are on your own, but, it gives many benefits that you can’t do without in many situations.

Contributor: Udaya Bhanu Goteti

SQL CE Binaries - Details

Laxmi Narsimha Rao Oruganti MSFT — Wed, 07 Oct 2009 11:27:00 GMT

Many times developers get puzzled on which exact set of binaries one should consume in their application. There is also a misconception that SQL CE engine is too big in size. This may be because there was no effort from our side to clarify on “Which binary is for what purpose?”.

In an effort to get clarity in these aspects, I have published a blog post on my blog. Please do read and provide your comments.

Thanks,

Laxmi Narsimha Rao Oruganti

Troubleshooting: Access Violation exception while using SQL Server Compact database with ADO.NET provider

SQL CE Team — Wed, 06 May 2009 06:26:00 GMT

Possible causes for Access Violation exception in general:

An access violation occurs in unmanaged or unsafe code when the code attempts to read or write to memory that has not been allocated, or to which it does not have access. This usually occurs because a pointer has a bad value.

In programs consisting entirely of verifiable managed code, all references are either valid or null, and access violations are impossible. An AccessViolationException occurs only when verifiable managed code interacts with unmanaged code or with unsafe managed code.

For more information about this, please visit this link.

When this exception can be related to SQL Server Compact:

SQL Server Compact ADO.NET provider DLL [System.Data.SqlServerCe.dll] is a managed DLL which interacts with other native DLLs to provide the functionality.

There are two known issues related to the usage of SQL Server Compact provider which may lead to this Access Violation exception:

1. Version Mismatch [3.5 RTM and 3.5 SP1]:

Access Violation occurs when the loader loads different versions of Managed and Native SQL Server Compact 3.5 DLLs. This issue arises when different versions are available in the box and this happens with Private and Central deployment of SQL Server Compact 3.5.

To be specific, if SQL Server Compact 3.5 RTM is pre-installed [centrally deployed] in the box and application built on top of SQL Server Compact 3.5 SP1 privately deploys the SQL Server Compact DLLs.

In this case, the managed DLL [System.Data.SqlServerCe.dll v3.5.5692.0] will be loaded from private directory but the native DLLs [sqlceme35.dll, sqlceqp35.dll, etc. v3.5.5386.0] might get loaded from the central deployed directory.

If the application is built against SQL Server Compact 3.5 RTM and SQL Server Compact 3.5 SP1 is centrally deployed in the box then the application will start using SQL Compact 3.5 SP1 as there is a redirection entry which redirects an application (Compiled against 3.5 RTM) to use centrally deployed SQL Compact 3.5 SP1.

2. Thread Safety:

SQL CE objects are not thread-safe and are not thread affinitive either. If an instance of SqlCeConnection or SqlCeTransaction is shared across threads without ensuring thread safety, then that may lead to Access Violation exception.

Work-around if the cause is related to SQL Server Compact:

For the above two issues, there are workarounds available which should be followed to avoid any Access Violation exception.

1. Version Mismatch:

One should follow either of the given below approaches to avoid the Access Violation exception:

Scenario: Application built against SQL Server Compact 3.5 SP1 [to be deployed privately] and SQL Server Compact 3.5 RTM centrally deployed.

Solution:

1. There should be only one version of SQL Server Compact [v3.5 RTM or v3.5 SP1] installed in the box. The version with which the application is not built against [v3.5 RTM] should be un-installed.

2. Install SQL Server Compact [v3.5 SP1] centrally to avoid any future install mismatch.

Forum thread related to this issue can be found here.

2. Thread Safety:

It is recommended that each thread should use a separate connection than sharing across. If there is really a need for sharing SQL CE objects across threads, then the application should serialize access to these objects.

Thanks,

Mohammad Imran.

SQL Server Compact Garbage Collection – Whys and Hows

SQL CE Team — Tue, 05 May 2009 17:52:00 GMT

Some background about SQL Server Compact:

SQL Server Compact is an embedded database implemented in native and it can be accessed in application by either OLEDB or ADO.NET provider model.

ADO.NET provider for SQL Server Compact is a managed assembly which depends on native SQL Server Compact DLLs to provide the service. In general, the managed classes under ADO.NET provider model for SQL Server Compact are just wrappers around the native classes.

SQL Server Compact native objects are dependent. For instance, Cursors and Transactions are dependent on Connection object and are expected to be released in the right order.

What’s the problem then? In native application, the developer can dispose the cursors, commands, etc. before closing the connection but in managed application with .Net Garbage Collector there is no particular order in which these objects will be disposed.

So what do we do? Our SqlCeConnection object maintains weak references to all objects tied with it. Or to be specific, it maintains short weak references.

How does it help me in disposing the objects in order? When the connection object is getting disposed, the weak reference cache is iterated and all the objects there are disposed before the connection object getting disposed. Please note that when we say Dispose, we also mean Finalization.

How does .Net Garbage Collector comes into picture of SQL Server Compact

Simply employing short weak references to keep track of all related objects and dispose them in sequence does not solve all the problems.

What is the problem then? Sometimes the database files can remain locked in stress scenarios because not all the dependent objects are disposed upon dispose of SqlCeConnection. This happens because we use short weak refs to track object lifetime and we could end up in situations in which an object is in the finalization queue (hence the short reference is not longer alive) and isn’t in the freachable queue (hence isn’t guaranteed to be picked up by the GC immediately). More information about short weak references can be found here.

All this leads to a situation, where the database file is still locked even though customer application closed the connection. Hence, it is not able to get deleted the file using file explorer …etc means.

How did we solve the problem? When we are cleaning weak reference cache in dispose of SqlCeConnection object, we need to call GC.WaitForPendingFinalizers() before we return to the caller; in this way, we are guranteed that the GC will pick up all finalizable objects and hence all the native interfaces will be properly released even if we no longer have short references to them.

More information about this API can be found here.

FAQs

1. Why are we employing short weak references and not long weak references?

Ans.: SQL Server Compact ships for both Desktops as well as Devices. Long weak references are not supported in .Net CF. More information here.

2. Does call to GC.WaitForPendingFinalizers() has negative performance impact?

Ans.: No. Most of the time, the freachable queue is empty (the same is not true for finalization queue though). See here for more information about Garbage Collector.

3. Can this call to GC.WaitForPendingFinalizers() lead to deadlocks?

Ans.: Yes. If Dispose method of SQL CE objects are called in the finalization context than the explicit dispose context. The reason being when an object is getting finalized, it is not supposed to touch any other managed objects as the currently getting disposed object does not know the life-status of the object it is trying to refer.

4. Why did we choose this design?

Ans.: SQL CE was primarily designed for Devices where the memory, processing comes at very high cost. So, it has been designed to free up the resources as early as we can and hence it is calling GC.WaitForPendingFinalizers.

Thanks,

Mohammad Imran.