SQL Server Express WebLog : FAQ

Troubleshooting the corrupt file error when using the SQL Server 2008 Express Setup Package

sqlexpress — Thu, 23 Apr 2009 06:31:40 GMT

I've seen a number of people bringing this message up in the forums recently so I thought I would blog about it rather than type the answer over and over. (I'm lazy that way.)

Problem

When you run the installer you get the following error after the download of the SQL Server 2008 Express installation package:

An error occurred during the installation of SQL Server 2008 Express.

The downloaded file appears to be corrupt.

If you view the log you see an error that looks something like this:

Verbose: [T3][19:57:41. 91] ComponentInstaller.Install Start

Info: [T3][19:57:41. 92] ComponentInstaller.Install Preparing to install SQL Server 2008 Express with Advanced Services.

Verbose: [T3][19:57:41.136] ComponentInstaller.ValidateFile Start

Info: [T3][19:57:41.136] ComponentInstaller.ValidateFile Validating SQLEXPRADV_x64_ENU.exe.

Verbose: [T3][19:57:41.198] ComponentInstaller.GetFileCertificate Start - file=C:\ProgramData\Microsoft\SQL Server 2008 Express Install\Downloads\SqlExpressWithAdv\SQLEXPRADV_x64_ENU.exe

Warning: [T3][19:57:41.443] ComponentInstaller.GetFileCertificate Failed to parse certificate in SQLEXPRADV_x64_ENU.exe: System.Security.Cryptography.CryptographicException: Cannot find the requested object.

at System.Security.Cryptography.CryptographicException.ThrowCryptogaphicException(Int32 hr)

at System.Security.Cryptography.X509Certificates.X509Utils._QueryCertFileType(String fileName)

at System.Security.Cryptography.X509Certificates.X509Certificate.LoadCertificateFromFile(String fileName, Object password, X509KeyStorageFlags keyStorageFlags)

at Microsoft.SqlServer.SqlExpressWrapper.Controller.ComponentInstaller.GetFileCertificate(String file)

Verbose: [T3][19:57:41.948] ComponentInstaller.GetFileCertificate End - returned NULL

Warning: [T3][19:57:41.949] ComponentInstaller.ValidateFile File validation FAILED (certificate could not be obtained from file).

Verbose: [T3][19:57:41.949] ComponentInstaller.ValidateFile End - returned False

Verbose: [T3][19:57:41.991] ComponentInstaller.Install End - returned [ExitCode=0, Success=False, Action=RetryDownload, MessageStringId=]

Error: [T3][19:57:42.276] InstallController.InstallComponents Error: Microsoft.SqlServer.SqlExpressWrapper.Controller.InstallException was thrown: The downloaded file appears to be corrupt.

Source: SqlExpressWrapper

Stack: at Microsoft.SqlServer.SqlExpressWrapper.Controller.ComponentInstaller.Install()

at Microsoft.SqlServer.SqlExpressWrapper.Controller.InstallController.InstallComponents()

Cause

This is typically caused by a problem during the download process that results in the file saved to your disk being corrupt.

Resolution

Often this problem is resolved by simply hitting the Try Again button. If the problem was a dropped line or some noise on your connection it is usually transient and the next download will work fine.

In cases where you get the error a second time, or when you have a low bandwidth connection so you don't want to risk waiting through the second download just to get another error you should clear the cache to make sure that you don't have a partial download stuck on your hard drive. You will need to clear two folder locations to be sure you're starting clean:

Empty your Temp directory since this is where the interim pieces of the file download are stored during the download process. If your system has the Disk Cleanup Utility (It's at least part of Vista), you can use this tool to empty your Temp directory. If you don't have this utility you can just open up the Run dialog and type "%temp%" (without quotes) and then clean out the files manually. (I usually do it manually.)
Empty the Express Installation Package download cache. The download cache location changes depending on your operating system but I describe the location in this post introducing the Express Installation Package in the section on the architecture.
1. Navigate to the cache folder location appropriate for your system.
2. Find the .\Download folder.
3. Find the edition folder for the edition you're downloading. (eg. .\SqlExpressWithAdv if you're downloading SQL Express with Advanced Services).
4. Delete the installer package that is cached in this folder.

Once you've removed the files from the Temp directory and the Download Cache, run the Express Installation Package again and you'll be assured to get a new file from the Microsoft Download Center this time. This is one of those cases where Windows is sometimes too smart for its own good, it's trying to save you time by restarting a failed download where it left off, but in this case it can't restart correctly.

If you continue to get the corruption error after this you may be experiencing a different problem. If this is the case there are two things you can do:

Report the problem in the SQL Express forum.
You can try installing manually. You'll need to install the prerequisites first and then run the standard SQL Express package. But seriously, report the problem in the forum, if there is something weird going on with the download files, we need to know.

- Mike

FAQ: Error creating a Service-based database in Visual Studio 2008 SP1

sqlexpress — Fri, 03 Oct 2008 03:07:00 GMT

A few of you have run into problems when trying to add a new Service-based database to your project in Visual Studio 2008 SP1. This problem has been reported to Microsoft Connect and the Visual Studio team is working on a fix, but I thought I would provide some details on a workaround in the meantime.

What's the problem?

This issue occurs when you have Visual Studio 2008 SP1 (any edition) and SQL Server 2008 Express x64 installed. This won't happen if you have the 32bit version of SQL Express installed. You know you've hit the issue when you see this error:

Connections to SQL Server files (*.mdf) require SQL Server Express 2005 to function properly. Please verify the installation of the component or download from the URL: http://go.microsoft.com/fwlink/?LinkId=49251

The problem is caused by an error when checking the registry to verify that SQL Express is actually installed; it's installed, but Visual Studio doesn't think it's installed.

Two ways to work around the problem

The easy way out is to uninstall the 64bit version of SQL Express and replace it with the 32bit version installed in the WoW. You can download this version from the Express website. If you're fiercely devoted to the native 64bit installation, or you just don't like taking the easy way out, there is another option...

Congratulations on boldly accepting the challenge to stick with the native 64bit version of SQL Express in the face of this error. In order to "trick" Visual Studio into working with a database on this architecture of SQL while still preserving the expected behavior of how the database works, we're going to have to plumb the depths of working with a User Instance in order to create the database manually, add the existing database to the project and then "hook it up" once it's there.

Create the Database

To perform this part of you're task you'll need to connect directly to the User Instance with the client tool of your choice. I've already posted instructions on how to do this, so check those out first to get a connection to the User Instance. Once connected, we need to create a database in the same folder as the project you want the database to live in. You can do this using the standard UI dialogs in Management Studio by right-clicking on the Databases folder and selecting New Database. Once you have the dialog open, give the database a name and set the file path to the folder where your project lives. You can also do this in T-SQL if you'd like, here is a sample of the statement that would do this:

CREATE DATABASE database1

ON PRIMARY

(NAME = database1,

FILENAME = N'C:\Users\<username>\Documents\Visual Studio 2008\Projects\WindowsFormsApplication1\WindowsFormsApplication1\database1.mdf',

SIZE = 10MB,

MAXSIZE = 50MB,

FILEGROWTH = 10%)

LOG ON

( NAME = database1_log,

FILENAME = N'C:\Users\<username>\Documents\Visual Studio 2008\Projects\WindowsFormsApplication1\WindowsFormsApplication1\database1_log.ldf',

SIZE = 10MB,

MAXSIZE = 50MB,

FILEGROWTH = 10%)

As you can see, this is a normal CREATE DATABASE statement where I've specified the FILENAME argument to be the project folder for my project. Just replace the <username> place holder with your user name and your ready to go.

Important: Once the database has been created, you need to detach it from the User Instance before you go onto the next step or you're in for trouble later.

Adding the database to your VS project

Make sure you've detached the database after creating in the previous section. (I told you it was important.) Back in Visual Studio you can now add the database to the project.

On the Project menu, select Add Existing.
In the File Open dialog, navigate to the project directory select the database you created. (You may need to change the file type being shown so that you see the Data Files in the dialog.)
Select the database and click Add.

You'll get the same error that got us here in the first place. Ignore it! Hit OK as if all is right with the world and you'll see that the database was added to the project, take that foolish error message.

Hookin' the database up

At this point your database is in your project, but we want to take the final step to make it just like it would have been if you did this using the Add New Item | Service-based Database method. Add New Item, in this case, would have automatically dropped you into the DataSet wizard, so lets create one now.

On the Data menu, select Create New Data Source.
In the Data Source wizard, select Database and click Next.
Click the New Connection button.
In the Add Connection dialog, click the Browse button.
Select the database you created at the beginning and click Open. (If you didn't detach it like I told you to, this is where you're going to run into trouble. Go ahead and detach it now, I'll wait…)
If you don't trust me, click the Test Connection button.
Click OK.
Back in the Data Source Configuration Wizard, click Next.
Accept the option to save the connection string, feel free the change it to something that makes sense to you, and click Next.
After VS scans the database and discovers it's empty, click Finish.

You've now got a spankin' new DataSet in your project that's connected to your database. You can work with the database as you would expect, for example you could use the Server Explorer/Database pane to add new objects to the database which you could then add to your DataSet. Like I said, it's not the most straight forward workaround, but if you only need to create one database, this will take a lot less time than uninstalling the 64bit version and replacing it with the 32bit version. Hopefully VS will release a fix for this issue soon; go over to MS Connect and cast your vote to show that this issue has affected you. The more people affected, the better the chance for a fix.

- Mike

FAQ: "Installing" the SQL Server 2008 Express ClickOnce Bootstrapper for Visual Studio 2008 SP1

sqlexpress — Sat, 13 Sep 2008 03:01:00 GMT

Clearly, the names of our components are getting too long, but that's a matter for another post…

Many of you have noticed that the VS bootstrapper for SQL Express 2008 (and similarly the bootstrapper for Windows Installer 4.5) are installed by the Visual Studio 2008 Express SP1 products, but not by the VS SP1 patch that is applied to the non-Express versions. After noticing this, many of you have ask, "What's up with that?"

The short answer is that it's just one of those things that happens when two large development organizations get together, one group is using metric, the other is using standard and the next thing you know your Mars Probe is crashing; but I digress. The deal here is that we shipped SQL Express 2008 as part of VS Express 2008 SP1, so we included the necessary bootstrappers to support ClickOnce with the Express products. We didn't include SQL Express in the VS 2008 SP1 patch (non-Express) so we didn't include the bootstrappers. You can always install SQL Express 2008 for use with any SP1 edition of VS 2008, but the bootstrappers themselves are actually installed by VS, so we've got a gap. I've been talking to the VS team to get this sorted out, but in the mean time, here is a workaround to get the bootstrappers required for ClickOnce deployment of SQL Express 2008 onto your non-Express VS installations.

Install any flavor of Visual Studio 2008 Express SP1.
Go into the bootstrapper directory (C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bootstrapper\Packages) and rename both the SqlExpress2008 and WindowsInstaller4_5 directories.
Uninstall Visual Studio 2008 Express SP1.

OK, it's silly, but it works. Steps 2 & 3 are optional if you don't mind leaving the VS Express installation on your computer, but if you don't want to keep it around you need to rename the directories to prevent the uninstall action from removing them. You don't have to re-rename the directories because population of the ClickOnce Prerequisites doesn't have anything to do with the directory names, it just scans the sub-directories under Packages and builds the list based on the manifests it finds there. There you have it, have fun.

- Mike

FAQ: Switching between SQL Server 2008 edtions

sqlexpress — Sun, 24 Aug 2008 20:26:00 GMT

There is an extra step that you need to take when upgrading from one edition of SQL Server 2008 to another, for example when upgrading from Express to Express with Tools. This extra step is called Edition Upgrade (sometimes SKU Upgrade) and the net result is that you have to run the install wizard twice when upgrading an existing instance to a different edition.

Why did we do it?

I don't have all the technical details here (I'm not a setup guy) but the basic deal is that the feature tree you see when installing is based on information about the edition that has already be put on the machine. When you're upgrading from one edition to another, the information on the machine is about the old edition, so you have to first update that information to specify the new edition before you can add new features related to the new edition.

In general Edition Upgrade is a rare occurance, but it happens more often for Express users since it's likely that you've installed some other product such as Visual Studio, that included only the core Express edition and you want to add a feature from one of the more complete editions such as Express with Tools.

What do we do?

Edition Upgrade is really just another trip through the install wizard in which you provide the Product Key for the new edition. When you're moving to a paid edition, this would involve actually typing the new Product Key that was provided with your new edition, but since all Express editions have pre-populated Product Keys, this new step is really just a matter of downloading the package for the new edition (I suggest you save it to your Hard Drive since you'll need to run it twice) and running through it twice.

The first time you run the package you'll need to select the option to upgrade to a different edition. This will run through a set of checks and give you the option to provide the new Product Key, or in the case of Express product you'll see the page where the new Product Key is supplied for you, you don't type it in yourself. Once you've completed the Edition Upgrade, you run the install wizard again, this time choosing the option to Add Features to an existing Instance. Several more wizard pages and you'll get to the feature tree that will have all the features available in the new instances listed.

There's always an exception, what is it?

Yes, there's always an exception. In this case the exception is that we were able to elimnate the Edition Upgrade step between SQL Express with Tools and SQL Express with Advanced Services. If you already have SQL Express with Tools installed and you'd like to add a feature from Express Advanced (say Full Text Search) you can go directly to Add Feature option and skip the Edition Upgrade.

Hopefully this helps clarify things when you're upgrading your edition.

- Mike

What's up with SQL Server 2008 Express editions

sqlexpress — Fri, 08 Aug 2008 02:49:00 GMT

Microsoft has announced the release of SQL Server 2008 and that means it time for another post about SQL Server Express.

The press release indicated that SQL Express is already available, but those of you who have followed the link have found that it is not actually there. Nothing to worry about, has a dependency on the .NET Framework 3.5 SP1 and we need to coordinate the release to the web for both of these which will take a few more days. I'll blog about it when we release it and you can watch the Express web site for updates; when we release it, it will be available from that site.

There will be three editions of SQL Express, each one adding to the functionality of the previous one; you simply pick the edition that includes the set of functionality you need and install it. The information I posted about SQL Express RC0 is still valid, but I'm reproducing the feature comparison table here to include the third edition:

Feature	SQL Server 2008 Express	SQL Server 2008 Express with Tools	SQL Server 2008 Express with Advanced Services
Management
PowerShell Integration	Y (Separate installation)*	Y	Y
Policy Based Management	Y (manual only)**	Y (manual only)*	Y (manual only)**
Management Studio Basic	N	Y	Y
SQL Engine
Integrated Full Text Search	N	N	Y
Merge & Upsert	Y	Y	Y
New Data type support
Filestream support	Y	Y	Y
New Date & Time data types	Y	Y	Y
Geodetic data types	Y	Y	Y
Advanced Spatial Libraries	Y	Y	Y
Support for Spatial Standards	Y	Y	Y
New Tools
Import/Export Wizard	Y	Y	Y
Replication
Change Tracking	Y	Y	Y
Synchronization Services	Y (Separate installation)***	Y (Separate installation)***	Y
Reporting Services
Increase RS Memory Limit	N	N	Y
RS Word/Rich Text Export	N	N	Y
IIS Agnostic Report Deployment	N	N	Y
Enhanced Gauges & Charting	N	N	Y
Business Intelligence Developer's Studio	N	N	Y

* The SqlPS command line tool can be enabled in SQL Express by installing Windows PowerShell 1.0 before installing SQL Express.

** Policies can be created in SQL Express and run manually. There is no support for automated policy based management.

*** Synchronization Services support in SQL Express requires that you install the component separately from the SQL Server 2008 Feature Pack.

As you can see, SQL Express with Tools includes the core database engine and the basic version of Management Studio; this is the ideal edition for people who want the right tools for developing relational database applications. The advanced features such as Integrated Full Text Search, Reporting Services and BIDS are available in SQL Express Advanced. SQL Express with Tools will be delivered in the same architectures and with the same prerequisites as SQL Express Advanced, which are as follows:

SQL Express with Tools architecture

32-bit only installation package (x86 platforms only)

64-bit native installation packages (x64 platforms only)

SQL Express with Tools Prerequisites

.NET Framework 3.5 SP1

Windows Installer 4.5

Windows PowerShell 1.0

As with previous releases (SQL Server 2005) we will be releasing SQL Express 2008 in stages. SQL Express core will be released first, along with Visual Studio 2008 SP1, and the other two editions will follow about a month later. As I said, watch the Express web site and this blog for information about the release of these two additional editions.

- Mike

Understanding SQL Express behavior: Idle time resource usage, AUTO_CLOSE and User Instances

sqlexpress — Sat, 23 Feb 2008 04:07:00 GMT

I've gotten a number of questions recently asking how SQL Express uses resources during idle time, and how that behavior impacts the way SQL Express behaves "on first connection" after being idle for a period of time. While many of these questions have been related to web hosting, the behavior is a general behavior for SQL Express, so I'll answer in general terms.

SQL Express: It's SQL Server with a twist

SQL Express is SQL Server. This may seem an odd thing to say, but many people get confused on this point, so I'll say it again: SQL Express is SQL Server. We have introduced some scale limitations to SQL Express and removed some features from SQL Express, but it is the same code base as all the other SQL Server editions. There are some specific places where we have changed the way SQL Express behaves compared to other editions and one case where we've introduced a feature that is unique in SQL Express (more on that later). I won't go into the details of which features are in which editions, you can find that information on the Microsoft web site: SQL Server 2005 Feature Comparison. This post is about those behavioral differences that lie just beneath the surface.

Idle thoughts about Idle time memory usage

SQL Server, in general, tries to return query results fast. This makes sense in terms of being a centralized, multi-user data service. When there are a whole bunch of users simultaneously asking questions, it's good to be able to answer those questions fast. One of the many ways SQL Server manages to do this is by caching frequently used information in memory. (That's as technical I'll get, you'll have to look elsewhere for a deep, technical discussion of the ins and outs of memory usage in SQL Server.) With SQL Express we wanted to target a different scenario, specifically, we wanted to support being a data store for single-user applications. This presents a different set of challenges, for one thing, there are long periods of time (call it idle time) when no one is asking any questions. The idea of holding a whole bunch of data in memory doesn't make as much sense if no one is going to be using it, and it might even be bad, because there may be other ways the computer could use that memory. So we made a change in the way SQL Express behaves:

When SQL Express is active it works just like any instance of SQL Server, data is cached in memory in order to improve the performance of subsequent queries.
When SQL Express is idle it aggressively trims back the working memory set by writing the cached data back to disk and releasing the memory. This frees up memory for other applications to use.

This behavior makes sense for a single user database engine - it supports higher performance when the application using the data is running, but once that application is shut down, memory is released to allow other applications to use it. I like to think of this as "being a good citizen" in the application community. There is always a catch when making this kind of trade-off though, and SQL Express is no different in this aspect. When SQL Express transitions from idle to active some of the memory that was released needs to be reclaimed; this results is a slight lag during "startup time" when you're first connecting to SQL Express. You can actually see this happening in the Widows Application Log, when you first activate SQL Express after some idle time, an entry is written to the log that reads like this:

Server resumed execution after being idle 16056 seconds: user activity awakened the server. This is an informational message only. No user action is required.

This is just telling you that SQL Express was idle and is now awake because some user activity woke it up. (When you think of it, it's kind of rude for users to keep waking up SQL Express without consideration for SQL Express's feelings. Maybe it's had a rough day and needs some rest!) This behavior is not configurable, it's just the way SQL Express is written. As mentioned above, this behavior change is unique to SQL Express, if you have need for your SQL Server to stay awake, SQL Workgroup or higher would be a more appropriate choice for your database engine.

This (data)store will AUTO_CLOSE in 300 milliseconds

Another minor change in SQL Express that has a lesser impact on perceived performance is the way the AUTO_CLOSE property of a database is handled. The AUTO_CLOSE property allows a database to be managed more like any other file in Windows by releasing the lock that SQL Server would normally hold on the file and allowing it to be copied, backed up, etc. This type of behavior is very important to supporting XCopy type deployment, which is a core piece of functionality needed to support Visual Studio ClickOnce™ Deployment. Supporting ClickOnce is something we wanted to do with SQL Express, so we modified the behavior of the CREATE DATABASE functionality in SQL Express in order to set AUTO_CLOSE to True for all databases. The results of this is that after 300 ms of inactivity for a database, SQL Express will close the database and release the lock on the file. It's important to recognize that Close and Detach are two different things. SQL Express still maintains metadata information about the closed databases, we just don't lock the file. When a request comes in that requires the use of a Closed database, we Open the database using the information stored in metadata. Opening a database that has been closed does not have a significant performance impact, but it can have some interesting side effects (and by interesting I mean negative) for certain kinds of operations:
If you are running an iterative process that includes a cross-database query and the period of the iteration exceeds 300 ms, the external database your process access would go through a Close/Open cycle during every iteration. This can add up over a large number of iterations and become a large component of the process.
If you have a process that polls a database at intervals, and that interval is larger than 300 ms, the database will go through a Close/Open cycle with every poll. In this case, the direct performance impact is negligible, but the fact that SQL Server writes an entry to the Windows Application Log every time a database is opened can result in your log filling up unexpectedly. (One example of this is if you have installed Reporting Services Express; it polls one of it's databases every couple of minutes.)

Happily, you have some flexibility here to address the behavior within SQL Express. The behavior of CREATE DATABASE in SQL Express is not configurable, it will always set AUTO_CLOSE to True, but AUTO_CLOSE is just another database property, so you can change that property once the database has been created. Read the BOL topic ALTER DATABASE (Transact-SQL) for more information about changing the AUTO_CLOSE property programmatically. You can also change this property using SMO and in the Database Properties dialog in management studio. (Interesting Trivia: When you create a database in management studio, there is some post-creation processing that happens to set the database properties to match what ever is in the model database. Since all system database have AUTO_CLOSE set to False, even in SQL Express, this results in databases created in management studio to have AUTO_CLOSE set to False, even for SQL Express, unless you've changed the properties on your model database.)

Net/Net - If you're using SQL Express as a standard multi-user server, you'll probably want to consider changing the AUTO_CLOSE property of your databases to False after you create them.

What is a RANU?

An electric car built in Seattle and powered with recycled Starbuck's coffee cups.
A small marsupial native to Redmond, Washington that lives in the beards of SQL developers.
A user specific process of SQL Express designed to allow non-administrative users to use SQL Server as the data store for Windows Forms based applications.

As much as I wish the answer were either 1 or 2, it is actually 3. RANU stands for Run As Normal User and is a feature that is unique to SQL Express. You've probably heard this feature referred to as User Instances. I'm not going to go into a detailed, technical description of User Instances, but will rather point you to the white paper on the topic, SQL Server 2005 Express Edition User Instances. I'll stick with the theme for this post and just discuss how RANU behavior impacts perceived performance and resource usage.

If you didn't read the white paper (feel free to do so now, I'll wait) it is important to understand that RANU is a separate process of SQL Express from the parent instance and that it has some additional limitations beyond SQL Express:

RANU supports only local connections via Shared Memory. (i.e. It's designed for single-user applications.)
RANU supports only Windows authentication.
RANU is a user specific instance, each user gets their own RANU instance that is not shared with other users. (i.e. The databases can not be shared between RANU instances.)

There are three basic issues to be aware of when consider how RANU impacts resources and perceived performance:
Since RANU is starting a separate user Process of SQL Express when an application launches, you have multiple copies of SQL Express running and using system resources. Most commonly this means there are two instances running, the parent instance and the RANU instance, for a computer with one user. (This could change, for example, on Terminal Services where you can have more than one user of the computer simultaneously.) RANU is tied to the User, not to the application, so while an application starting can result in a RANU instance being started, it does not shutdown the RANU instance automatically when the application closes. A RANU instance has a defined timeout value of 60 minutes; after 60 minutes without activity, RANU shuts itself down. The RANU timeout is configurable at the parent instances through the 'User Instance Timeout' setting.
The first time you start RANU for a specific user, SQL Express has to make copies of all the system database that will be used by the RANU instance. Making copies of these files can take longer than the default connection timeout value (30 seconds if you're wondering) and result in a timeout error. Behind the scenes, the file copy will finish and the RANU instance will starte, it's only the connection that fails. If you try the connection again, it will succeed. I find that changing the connection timeout to 60 seconds in your RANU connection strings handles this "first use" delay. (Your mileage may vary.)
On all subsequent connections to RANU for the same user (see #2 for the first time connection issues), there are two states, RANU may already be running having been started previously and not timed out yet or RANU may not be running, in which case it must be started. If RANU is not running, there is a short delay while the instances is started. I've not found this delay to be large enough to be noticeable within the context of other application load operations, and have rarely seen problems with the connection failing because of the lag, other claim there is a delay. Again, your mileage may vary, but changing the connection timeout to 60 seconds in RANU connection strings usually ensures a successful connection. The "start up lag" is part of the territory with RANU. You can consider the trade-offs of setting the User Instance Timeout to a longer period, thus reducing the number of occurrences of "start up lag", with the viability of keeping the dormant RANU instances running when it's not being used. (Note: RANU will trim it's memory usage when idle just like the parent instance of SQL Express, so you're really splitting hairs at this point.

That's quite enough for now, it's been awhile since I've posted anything and I guess I got carried away.

- Mike Wachal
- SQL Express team

DIY SQL Express SP2 Bootstrapper

sqlexpress — Fri, 24 Aug 2007 02:41:37 GMT

I've waited far too long to communicate this disappointing news to everyone, so here it is…

I'm not going to be releasing the SQL Express SP2 Bootstrapper manifests that I had discussed in a long ago blog posting. I'll spare you the sob story of my efforts to create this; suffice it to say that some requirements arose that complicated an official release and priorities changed over time. If you work in the software industry you know how it goes. So that brings us to…

My feeble backup plan

Most of the complications (ok, all of them) that I ran into were related to packaging up the manifests so they could be downloaded and installed. (Seriously, it's a long story, so don't ask.) The manifests themselves are pretty straight forward and only required some minor tweaks of the existing RTM manifests. There is nothing to stop me from sharing those updates with you in this blog and giving instructions on how to use them.

DIY SQL Express Bootstrapper

Technically, the bootstrapper, or the VS Generic Bootstrapper as I like to call it, refers to the technology created by the Visual Studio team to allow any developer to deliver specific sets of functionality as prerequisites that can be checked for and automatically installed by both ClickOnce and MSI based installers built with VS. The framework for building your own prerequisite is documented on MSDN in the topic Adding Custom Prerequisites. A more end to end treatment of the technology was printed in MSDN magazine back in 2004 in the article Use the Visual Studio 2005 Bootstrapper to Kick-Start Your Installation. These are both recommended reading for anyone delving into the land of creating a bootstrapper manifest. Feel free to take a look at those article now if you want, I'll wait until you're done.

The SQL Express bootstrapper is composed of five pieces:

The product manifest that describes the non-localized pieces of the package
The package manifest that describes the localized pieces of the package, such as the messages associated with specific error codes.
A license file containing the SQL Express EULA.
A custom executable that Microsoft created to check to see if SQL Express is installed or not.
The actual SQL Express installation package.
All five of these pieces is available to you, so you can create your own SP2 bootstrapper by creating a few directories and copying the files into the right place. These instructions assume a default installation of Visual Studio 2005 and also that you have not redirected VS to an alternate bootstrapper package directory.
Creating the directory structure
The default directory for bootstrapper packages is at C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\BootStrapper\Packages. Do the following:

Create a parent directory for the package named SqlExpressSP2.
Under SqlExpressSP2, create the localized resource directory named EN.
Files in the SqlExpressSP2 directory
Copy the contents of the product manifest that I've modified for SP2 (located here) and create a file in this directory named product.xml.
Copy the file named SqlExpressChk.exe from the SQL Express 2005 RTM bootstrapper package (located at C:\Program Files (x86)\Microsoft Visual Studio 8\SDK\v2.0\BootStrapper\Packages\SqlExpress) into your SP2 version.
Localized files in SqlExpressSP2\EN
Download (or copy if you've got it already) the installer package for SQL Express SP2 from the Microsoft Download Center. (Note: This particular flavor of the manifest is specific for the 32-bit only install of SQL Express, which is named SQLEXPR32.EXE.)
Copy the EULA from the SQL Express install location and save it to the name license.txt in the EN directory. By default it is at C:\Program Files (x86)\Microsoft SQL Server\90\EULA and is named License_EXPR_ENU.txt.
Copy the contents of the manifest I've modified for SP2 (located here) and create a file in this directory named package.xml.
Test your spankin' new bootstrapper package
That's really all there is to do. You should now be able to launch Visual Studio 2005 and see SQL Server 2005 Express Edition SP2 listed in the prerequisites dialog.
There's always a catch
There are a few known limitations with the package that is created using this method, here are the gotchas:

This set of manifests does not handle upgrades, only fresh installs.
SqlExpressChk.exe is limited to only the default instance name, SQLEXPRESS. It won't detect other named instances.
This set of manifests is specific to the 32-bit only installer package. If you want to use the WoW enabled package (for installing on 64-bit platforms), you would need to make some changes to the manifest. (This would be a good point to go back and read those MSDN articles about the Generic Bootstrapper if you haven't already.)
Other bootstrapper solutions for SQL Express
I know of at least one other description of a DIY SQL Express SP2 bootstrapper that was mentioned in a comment of the original post on the topic. Check out IGrocholski' blog post on this same topic. There are some difference between the approaches, particularly in discovery. You may also find other implementations by searching on the MSDN forums.
I'm always happy to see community contribution, so feel free to share you own implementations with the rest of us by posting comments.

- Mike

Connecting to SQL Express User Instances in Management Studio

sqlexpress — Thu, 23 Nov 2006 06:33:00 GMT

Many of you are working with User Instances, sometimes without even realizing it. Briefly, a User Instances is special process of SQL Express that can be started by an application at runtime and allows SQL Express to emulate some of the characteristics of embedded databases. User Instances were added to SQL Express in order to better server developers creating ClickOnce applications in Visual Studio 2005.

As long as you work completely within Visual Studio, you'll likely never even notice that anything special is going on. Visual Studio gives you the data tools you need to handle most tasks required for creating a database that sits behind an application. In some cases, you might want to move to a richer tool set, such as SQL Server Management Studio or SQL Server Management Studio Express, which is when you're going to hit a road bump. What road bump you ask?

You can't find your database in Management Studio.
You can't attach to your database in the location where Visual Studio creates it.
If you manage to attach the database in Management Studio, your application can not longer user it.

There are a myriad of solutions to each of those problems individually, but the real solution to all of them is to learn how to connect Management Studio to your User Instance so that you can work with the database using the same instance of SQL Server that your Visual Studio application is using. As always, there are a couple things you have to be aware of when working with User Instances.

User Instances are only supported in SQL Express.
You can not start a User Instance from Management Studio. A User Instances is spawned by the parent instance of SQL Express when an application makes a special connection to the server.
If the User Instance isn't running, you will not be able to connect to it from Management Studio.
User Instances are only accessible through the Shared Memory connection protocol, and are only exposed through a Named Pipe.
Each user on a computer has their own User Instances that is unique to them.

The rest of this post describes how to start a User Instance, how to determine the State and Pipe Name for a User Instances and how to connect to a running User Instances using Management Studio.

Starting a User Instance

A User Instances is automatically created and started by SQL Express when an application makes a connection to the parent instances using a special connection string. Specifically, the connection string must contain the keyword "User Instance=TRUE". Read the white paper if you're interested in the gory details, for this post all you need to know is that SQL Express knows what to do with that connection string keyword. When you add a database to your Visual Studio project, a connection string is automatically generated that contains the right information to start the User Instance when the application is run. Since it is most common that you will want to work with your database while you are working with your application, the easiest way to start your User Instances is to start the application using Start Debugging or the F5 key. If you don't want to bother opening Visual Studio and loading your application, you can download a tool called the SQL Express Utility that is capable of starting your User Instances. SSEUtil is a tool written by the Visual Studio team to help troubleshoot User Instance issues, you can read more about it in the read me file that is installed with the utility.

Determine the State and Pipe Name of a User Instance

You can get information about the User Instances on a machine that are associated with a specific instance of SQL Express by using the dynamic management view sys.dm_os_child_instances. I say "associated with a specific instance" because User Instances are unique to both the user than starts them and the instance of SQL Express where they are started. For example, if I have two instances of SQL Express on my computer named SQLEXPRESS and MIKESOTHERINSTANCE, I could actually have two separate User Instances running for me, one for each parent instance. Back to the DMV. There are a number of columns available from this View, but I'll focus on three of them and let you explore the rest yourself. The three columns of interest are owning_prinicipal_name, instance_pipe_name and heart_beat, as in:

SELECT owning_principal_name, instance_pipe_name, heart_beat FROM sys.dm_os_child_instances

Here is the purpose of each column:

owning_principal_name: The name of the Windows User account that owns the User Instance in the form MacineName\UserName or DomainName\UserName.

instance_pipe_name: The Named Pipe reference to the User Instance, which will be important in the next section.

heart_beat: Denotes the State of the User Instance as Alive if it's running or Dead if it is not running.

Connecting to a User Instance in Management Studio

Now to the point, go ahead and start Management Studio and connect to your parent instance of SQL Express, it's probably named SQLEXPRESS. Open a new query windows and run the query from the section above. If you've never created a User Instance before you won't see any results returned, but if you have, you'll see something like this:

Domain\mike	\\.\pipe\191E06DB-55EE-41\tsql\query	dead
Computer\OtherUser	\\.\pipe\B7FA2FCD-4C3E-47\tsql\query	dead

As you can see, I have two User Instances on my computer, neither of which is running at the moment. I can see that the first User Instances belongs to me, while the second User Instance belongs to OtherUser, which is a local user on my computer that I use for testing. The Pipe Name is interesting because that is how I'm going to connect to the User Instance once I start it. For this example, I'd just use SSEUtil by opening a Command Prompt to the directory where I've saved the tool and running SSEUtil -L. If you've read the read me file, you know this command line will start the current users User Instances and list all the databases currently attached. The important thing is that my User Instances is now started, and if I ran the DMV again, it would now look like this:

Domain\mike	\\.\pipe\191E06DB-55EE-41\tsql\query	alive
Computer\OtherUser	\\.\pipe\B7FA2FCD-4C3E-47\tsql\query	dead

Now that my User Instance is alive, I can connect to it in Management Studio just like any other SQL Server.

Copy the Pipe Name out of the query results.
On the File menu, click Connect Object Explorer to open the Connect to Server dialog.
Paste the Pipe Name from step 1 into the Server Name text box.
You have to use Windows Authentication, that's the only authentication supported by User Instances.
Click Connect.

You will see a new Server show up in your Object Explorer that represents the User Instance. You can work with this User Instance just like any other SQL Server. Since this is the same User Instance that is being used by your Visual Studio applications, you won't have any conflicts between having your embedded database open in the application and opened in Management Studio.

Rather than running the DMV every time you want to connect to your User Instance, it's a real time saver to add your User Instance to the Registered Servers list once you discover it's Pipe Name. When SQL Express creates the User Instance the first time a random Pipe Name will be created, but once it's created, that same Pipe Name will be used every time the same user starts the User Instance. Once you've added your User Instance to the Registered Servers list, you can connect to it by just double clicking it, as long as it is already running. Have fun!

Mike

SQL Express SP2 and Windows Vista UAC

sqlexpress — Thu, 16 Nov 2006 06:38:00 GMT

Now that SQL Server 2005 SP2 has been released I thought I should make good on my promise to explain more about how we handle Windows Vista User Access Control in SQL Express SP2. I explained earlier that UAC works by removing the administrator ACEs from your User Token when you are a member of the Builtin\Administrators group. This effectively limits your permission to connect to an manage SQL Server because we assign permissions based on membership in the Builtin\Administrators group. To address this in SQL Server SP2, we created two ways to add a user directly to the SysAdmin Server role during the installation process. One of these ways is unique to SQL Express, the other is available for all editions of SQL Server SP2, but is exposed differently for SQL Express.

Adding the "setup user" to SysAdmin

In order to address the unique needs of SQL Express, we added a new option into the setup process that allows you to automatically add the user who is running setup into the SysAdmin Fixed Server Role. This option is turned off by default, so you have to take a specific action to make this happen. If you don't take this action, the default install of SQL Express SP2 will behave exactly the same as SP1 does with regards to UAC. If you miss setting the option during installation, you can use the provisioning tool that I describe later to accomplish the same thing.

Using the Setup UI - We've added a new checkbox to the Configuration Options page of the Setup UI with the caption 'Add user to the SQL Server Administrator role'. The checkbox is unchecked by default. Checking this box will result in a Login being created for the user account that is running Setup.exe and that Login being associated with the SysAdmin role. This is how we provide permissions for the Builtin\Administrators group already.

Watch out! When you start the installer for SQL Express SP2 on Windows Vista with UAC enabled you will be asked to "elevate" the process and give it administrative privileges. If you are already an administrator on the computer, you can elevate the process using your own account with your Admin Token restored. Windows Vista also supports the ability to allow normal users (e.g. non-administrative users) to provide the credentials of a different user who has administrative privileges in order to elevate the installer. If you do this, Setup will be running as the administrative user, not as the normal user who is logged in. In this case, a Login would be created for the administrative user, not the normal user.

Using the Setup command line - To support embedding, we've added a new command line argument that results in the same behavior. The argument is ADDUSERASADMIN with the following usage:

To add a Login for the user running setup:

ADDUSERASADMIN=1

To not add a Login for the user running setup:

ADDUSERASADMIN=0 (default)

Using the SQL Server User Provisioning on Vista tool

All editions of SQL Server SP2 will include the SQL Server User Provisioning on Vista tool. There are two ways to launch this tool:

On non-Express editions, the tool can be launched from a link on the last page of the Setup UI.
For all editions, including SQL Express, you can launch the tool from within the SQL Surface Area Configuration tool. I'll discuss this part since it applies to SQL Express.

When you launch SQL SAC in SQL Express SP2, you should see a link that reads 'Add New Administrator'. Clicking this link should launch the provisioning tool and allow you to specify the user you want to add to the SysAdmin role and the SQL instances that you want to make the addition to. You can specify more than once instance at a time, but only a single user at a time. For SQL Express, you will only see SQL Server instances listed, but for non-Express editions other SQL services will be listed as well.

Note: I've found in recent builds that the link in SQL SAC is not working correctly, I'm not sure if this is the case in the November CTP or not. If the link in SQL SAC does not work, you can launch the provisioning tool manually by double-clicking the file located at:

C:\Program Files\Microsoft SQL Server\90\Shared\sqlprov.exe

When not to use this

If you're application is already creating specific Logins (whether SQL or Windows Authentication) as part of your installation process then you will likely not be impacted by Windows Vista UAC. Use of SQL Express provisioning mechanism or the provisioning tool is targeted at installing SQL Express for use in an administrative environment, in fact, both SQL Setup and the provisioning tool require the user to have administrative privileges on the computer. When you are installing SQL Express to be used as part of an application, it is a best practice to create specific Logins and Users with appropriate permissions for the use of your application.

- Mike

Getting things working on Vista (aka - Dealing with User Account Control)

sqlexpress — Wed, 11 Oct 2006 21:26:00 GMT

Update: After a recent article on CNNMoney.com (see my post with the link to the article) I thought I would add to this post to clarify that SQL Express SP1 is supported for use on Windows Vista. As discussed in this post, there are some issues that SP2 addresses, so I still say deploying SP2 is the way to go for those who can wait, but if you can't, deploying SP1 is supported. - Mike

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

One of my co-workers called me the other night and said, "I've got a question about SQL Express SP1 on Vista." My response, of course, was "Doesn't work right, does it?" Having had a little laugh at my friends expense, I proceeded to tell him how to get SQL Express SP1 running on Vista. Since he's asking, I'm sure others are too, so I thought I'd share it with the world.

First, a disclaimer: Full Vista support will happen starting with SQL Express SP2. We'll be releasing a Community Technology Preview real soon and you should target using SP2 for Vista compatibility in your applications. The workaround here is to tide you over until you can start testing with the CTP.

Note: SQL Express SP2 will be released to Microsoft Update, so existing installations will be upgraded automatically as I'm mentioned before.

Note: You should not use SQL Express RTM on Windows Vista, there are some known problems that impact functionality. SQL Express SP1 is reasonable for starting to test your applications once you're handled the UAC issues per the rest of this post.

What happened?

Three little words are changing the way you work on Windows in Vista, those words are User Account Control or UAC. I'm not going to go into the details of UAC here, rather I'll direct you to the UAC Overview page for more information. UAC changes the way Windows treats administrative users, in short, it makes you into a normal user, but gives you the ability to "elevate" your permissions when you need to be an Administrator. For those of you using Vista already, UAC is responsible for all those "Allow" dialogs that pop up. UAC is a good thing (no, really) and promises to make Windows a more secure place to play.

Why does this impact SQL?

In short, SQL Express assumes you're an administrator on the computer in order to give you permissions to do things. As part of the default installation, we create Logins for the Builtin\Administrator user and the Builtin\Administrators group and assign both of these Logins to the SysAdmin Fixed Server Role. This way, everyone on the computer who is part of the Administrators group, is automatically a SysAdmin. Neat and clean, yeah right…

UAC functions by removing the administrator ACEes from your User Token, which effectively makes you a normal user. When you log into SQL Express it takes a look at your Token to see if it knows anything about you. First, it sees that there is no specific Login for your User Account. (Strike One) Next, it checks if you're part of any group it has a Login for, because of UAC, you're not part of Builtin\Administrators unless you've elevated, which you didn't. (Strike Two) Finally, and this is unique to SQL Express, you are part of Builtin\Users, which does have a Login, so you're not quite out. (Hooray for the home team) Unfortunately, Builtin\Users doesn't have many (any?) permissions in a default installation of SQL Express, so you can log in, but that's pretty much it. If you try to do anything, you'll get an error indicating a failure, usually do to a permissions error.

Another Note: Like I said, this is unique to SQL Express, other Editions of SQL Server 2005 don't actually create a Login for Builtin\Users, so you're not part of any existing Login and you fail to even connect to the non-Express Server. (Strike Three, You're outta there and my metaphor is complete.)

What do we do?

To get the same functionality from a default install of SQL Express on Windows Vista, you will need to create a Login explicitly for your User Account. This is where it's important that you have the ability to elevate yourself to get administrative rights when you need them. I'll give the steps to do this using Management Studio Express, but you can do the same with SQLCmd and T-SQL statements, you just have to launch the command prompt (cmd.exe) as elevated.

On the Start menu, right-click on SQL Sever 2005 Management Studio Express and click Run as administrator.
When prompted, choose to Continue. (This causes SSMSE to run with your Admin Tokens intact so that SQL Express recognizes you as an Administrator.)
Once SSMSE is opened, connected to the Instance of SQL Express that you're interested in and open the Security folder.
Right-click on Logins and click New Login…
Fill in the information to create a Login for your Windows Account. Be sure to select the SysAdmin role from the Server Roles tab.
Click OK and you're ready to go.

The next time you launch SSMSE, you don't have to go through all this because you are now explicitly a SysAdmin on your Instance of SQL Express. You will need to do this for every Instance you have installed.

A note about data access and security

You should not provide data access via the SysAdmin Server Role, rather, explicit Logins and Users should be created to grant users the permissions they need in the databases they need to access. This is a common practice, and if you're doing this already, you will find that data access in your applications will continue to work without modification when running on Windows Vista with UAC enabled. If your application has been relying on the SysAdmin Role, you should move to a model of explicit Logins and Users that provide data access using the least privileges required by the user. This is just good security design for an application. Read the BOL topic Security Consideration for Databases and Database Applications to get more details about recommended security designs for applications.

A note about User Instances

If you developer applications that rely on User Instances to provide data access, they should just work. User Instances were designed to enable non-administrative use of SQL Server, which is exactly what happens in Windows Vista with UAC turned on.

What's next?

We've made some changes to the installer for SQL Express SP2 to help manage the UAC behavior. I'll discuss these in a future post once the CTP is released.

- Mike

"Windows is searching for devenv.exe" when launching BIDS

sqlexpress — Wed, 27 Sep 2006 23:30:00 GMT

A number of people have been experiencing a problem launching the Business Intelligence Developer's Studio after installing SQL Server 2005 Express Edition Toolkit. The problem is that the shortcut is pointing to a file named devenv.exe that isn't there.

Resolution

Redirect the BIDS shortcut to the location where devenv.exe actually exists.

From the Start menu, choose Search.
Search for "devenv.exe"
Record the location where you find it.
On the Start menu, right-click the shortcut from BIDS.
Enter the correct path to devenv.exe in the Target text box.
Click OK, to update the shortcut.

You should be able to successfully launch BIDS now.

Note: When you launch a shortcut to a file that is missing, Windows will try to find a matching file somewhere else on the computer. When I was testing this issue, Windows XP actually found devenv.exe and offered to fix the link for me. I clicked OK, the shortcut was fixed and BIDS launched. I guess it pays to be patient (or distracted).

Cause

This happens when you have installed any Edition of Visual Studio 2005 into a non-default location onto you computer prior to installing BIDS. BIDS makes some (bad) assumptions about where the VS development environment is located so when you install VS to a non-default location the BIDS shortcut is actually pointing to the default location for VS installations.

This all happens because BIDS is really just a set of Project Templates that install into the normal Visual Studio environment (devenv.exe).

Configuring SQL Express During Installation

sqlexpress — Thu, 21 Sep 2006 00:32:00 GMT

In this post I'm going to explore the various ways you can configure SQL Server Express during installation. The easiest way is to just use the Setup UI, but that isn't ideal when you want to embed SQL Express into your own application installer.

Command Line Installation

The SQL Setup program supports a number of command line parameters that can configure many aspects of SQL Express directly during the installation process. The compressed installer file used by SQL Express (SQLEXPR_ADV.EXE for example) supports the exact same set of command line parameters by passing the parameters to Setup.exe once it has been extracted onto the hard drive. The full details of the command line parameters can be found in Books Online and in the install.ini file that is included in the installation package. Some of the parameters that are interesting for SQL Express are:

Quite mode - SQL Express support two different quite mode installations. You can show a status UI, but not require any User input using the /qb parameter. To install without any UI at all, use the /qn parameter.

Naming your server - You specify the Named Instance for your server using the INSTANCENAME parameter, as in INSTANCENAME=SQLEXPRESS. If you want to install SQL Express as the default instance of SQL Server, i.e. you refer to it simply by the machine name, you specify INSTANCENAME=MSSQLSERVER.

Handling Components - You specify which components you want to install using the ADDLOCAL parameter. You can do a full install by specifying ADDLOCAL=ALL, or you can specify specific components, as in ADDLOCAL=SQL_Engine,SQL_FullText. You can also use ADDLOCAL during maintenance mode to add additional features to an installed server. The REMOVE and UPGRADE parameters work in a similar way only for removing features and upgrading features respectively.

Setting Security - SQL Express uses Windows Integrated Security by default. You can switch to SQL Security by using the SECURITYMODE and SAPWD parameters. When you specify SECURITYMODE=SQL you must also specify a strong password for the SA account using the SAPWD parameter.

Enabling Network Connections - By default, SQL Express does not allow network connections. This can be changed using the DISABLENETWORKPROTOCOLS parameter. SQL Express supports connections through Shared Memory (a local only protocol), TCP/IP and Named Pipes. Shared Memory is always available, the others are controlled as follows:

DISABLENETWORKPROTOCOLS=0 - All protocols are enabled.
DISABLENETWORKPROTOCOLS=1 - Only Shared Memory is enabled.
DISABLENETWORKPROTOCOLS=2 - Shared Memory and TCP/IP are enabled, Named Pipes are disabled.

There are a number of other options you can configure from a command line install such as the directory where SQL Express is installed and the Service Account that SQL Express runs as. Explore the BOL topic for more information on other command line parameters.

Wrapping the SQL Express Installer

SQL Express is only available as an MSI based installer, so it can not be embedded inside another MSI, such as the MSI used to install your custom application. In order to install both your application and SQL Express, you need to create a wrapper that handles installing both. The Visual Studio Bootstrapper is one implementation of this kind of a wrapper. The VS Bootstrapper is pre-configured to allow you to build and install your application along with a set of pre-requisites, such as SQL Server and the .NET Framework 2.0. The VS Bootstrapper is used for both ClickOnce deployment and for VS Setup projects.

If the VS Bootstrapper isn't flexible enough for you, you can write a custom wrapper of you own. Wrapper can be as restricted or as dynamic as you like, but the ultimate goal is to create the correct command line to install SQL Express in the desired configuration. Luckily, there is already a white paper that describes how to do this, thus saving me from having to document it here.

A note on the SQL Express installer package - You might be asking how a file named SQLEXPR.EXE (The SQL Express installer) is an "MSI based installer" as I stated earlier. SQL Server is actually a set of MSI installers that each install specific components. The collection of MSI files for a given SQL Edition are installed using our own wrapper program named Setup.exe. (pretty clever name if you ask me) The fact that the individual components are MSI base causes the inability to embed SQL Express directly into another MSI, which is a limitation of Windows Installer.

Since SQL Express is a web download rather than a CD based installation, we actually compress all the files into a special kind of CAB archive that understands how to pass the command line parameters into the Setup.exe process once the archive has been extracted. You can actually extract the SQL Express installation files manually by typing SQLEXPR.EXE /x and then specifying the directory where you want the file copied.

Post-install Configuration using the SAC Utility

The SQL Surface Area Configuration tool (SqlSAC) is a UI tool that allows you to configure a number of aspects of SQL Server, for example, whether CLR is enabled or not. (CLR is disabled by default in SQL Express.) In addition to using the UI, SqlSAC supports a command line interface we call the SAC Utility that allows you to provide configuration settings as a file and apply them to any server. The SAC Utility can be used to configure a number of servers identically using the same configuration settings file or can be called once SQL Server has been installed to apply certain post-install configurations. You could call the SAC Utility as part of a wrapper, or from a Custom Action in your MSI package.

Post-install Configuration using sp_configure

The most common way to configure SQL Server is probably sp_configure, so I'll end the post with this one. You can call sp_configure at any time to change server configuration options, you just need to be connected to the server. This is another technique you can use to configure SQL Express once it's been installed. Just like the SAC Utility, you can use a wrapper or Custom Action to run code that connects to the server and runs the appropriate T-SQL statements. This can be done by simply including a T-SQL script in your installation and then calling SQLCmd to run the script at the end of your installation procedure.

Mike

FAQ: Detecting SQL Server 2005 using WMI

sqlexpress — Sat, 29 Jul 2006 10:55:00 GMT

I've gotten a number of e-mails asking how to detect if SQL Express is installed on a computer. I'm finally getting around to putting this in the Blog so I don't have to keep typing the answer. We've made discovery much easier in SQL Server 2005 with the addition of the SQL Server 2005 WMI Provider. This method will work for all editions of SQL Server 2005, but the example is tailored to SQL Express.

Why can't I just use the registry?

Do a little research on this topic and you'll find a number of examples on how to use the registry to detect SQL Server. Don't do it this way!

The problem with using the registry to detect SQL Server is that Microsoft doesn't make any promises that we won't change the registry without warning. (It's our registry, we can change it.) We changed the registry between SQL 2000 and SQL 2005 and it's a good bet well do the same between 2005 and what ever comes next. Enter the SQL WMI Provider, this provider abstracts the registry and allows you to discover information about SQL Server 2005. We will be maintaining the SQL WMI provider in future versions so that your detection code will continue to work on later versions.

Do I detect a catch?

Yes, you do. The SQL WMI Provider doesn't work for SQL Server versions prior to 2005. If you need to detect earlier versions, say 2000, you're pretty much back to your old registry tricks.

What else can the SQL WMI Provider do?

What, you want more? Fine. The SQL WMI Provider actually covers a bunch of different functionality beyond detection and is broken into two separate providers:

WMI Provider for Server Events - With this provider you can monitor events for a given instance of SQL Server.

WMI Provider of Configuration Management - With this provider you can detect and manage a given instance of SQL Server.

(If the links don't work for some reason, say MSDN completely restructures its content, just search for the topics in Books Online.)

Why not use SMO?

(Added in response to Jens' comment.)

Many observent readers (OK, it was just Jens) pointed out that SMO contains the ManagementServer namespace which offers similar functionality. I chose not to use SMO for a couple reasons:

WMI does not require managed code or the .NET Framework. Yes, this is a C# sample, but the same WQL would work from VBScript on a computer without the framework installed. Sometimes it's important not to have a dependency on managed code.
The SQL WMI Provider calls seem to handle being run on computers without SQL Server 2005 installed where SMO did not in my hands. I just had problems getting SMO based code to fail correct when SQL wasn't installed. I'm no SMO genious, so it's likely this was pilot error.

WMI is the detection mechanism recommend by our Servicing and Lifecycle Platform team and it does not have as many prerequisite as SMO. If you can count on the correct prerequisites being on the computer where you're wanting to detect SQL Express, feel free to check out the SMO ManagementServer namespace. If you want to use the recommended method, stick with WMI.

Hey Explanation Boy, how about a sample?

Ok already, I'll get to the sample. It’s in C# if you didn’t notice.

using System;
using System.Management;

namespace WMISample
{
    // The WMI query for this class was created using the WMI Code Creator tool
    // that is available from
    // http://www.microsoft.com/downloads/details.aspx?FamilyID=2cc30a64-ea15-4661-8da4-55bbc145c30e&DisplayLang=en
    public class MyWMIQuery
    {
        public static void Main()
        {
            bool foo = isExpressInstalled();
            if (foo)
            {
                Console.WriteLine("You have SQL Express SP1. Sweet!!");
            }
            else
            {
                Console.WriteLine("No instances named SQLEXPRESS exists on this computer.");
            }
            Console.WriteLine("Hit Enter to continue.");
            Console.Read();
        }

        public static bool isExpressInstalled()
        {
            const string edition = "Express Edition";
            const string instance = "MSSQL$SQLEXPRESS";
            const int spLevel = 1;

            bool fCheckEdition = false;
            bool fCheckSpLevel = false;

            try
            {
                // Run a WQL query to return information about SKUNAME and SPLEVEL about installed instances
                // of the SQL Engine.
                ManagementObjectSearcher getSqlExpress =
                    new ManagementObjectSearcher("root\\Microsoft\\SqlServer\\ComputerManagement",
                    "select * from SqlServiceAdvancedProperty where SQLServiceType = 1 and ServiceName = '" 
                    + instance + "' and (PropertyName = 'SKUNAME' or PropertyName = 'SPLEVEL')");

                // If nothing is returned, SQL Express isn't installed.
                if (getSqlExpress.Get().Count==0)
                {
                    return false;
                }
                
                // If something is returned, verify it is the correct edition and SP level.
                foreach (ManagementObject sqlEngine in getSqlExpress.Get())
                {
                    if (sqlEngine["ServiceName"].ToString().Equals(instance))
                    {
                        switch (sqlEngine["PropertyName"].ToString())
                        {
                            case "SKUNAME":
                                // Check if this is Express Edition or Express Edition with Advanced Services
                                fCheckEdition = sqlEngine["PropertyStrValue"].ToString().Contains(edition);
                                break;

                            case "SPLEVEL":
                                // Check if the instance matches the specified level
                                fCheckSpLevel = int.Parse(sqlEngine["PropertyNumValue"].ToString()) >= spLevel;
                                //fCheckSpLevel = sqlEngine["PropertyNumValue"].ToString().Contains(spLevel);
                                break;
                        }
                    }
                }

                if (fCheckEdition & fCheckSpLevel)
                {
                    
                    return true;
                }
                return false;
            }
            catch (ManagementException e)
            {
                Console.WriteLine("Error: " + e.ErrorCode + ", " + e.Message);
                return false;
            }
        }
    }
}

isExpressInstalled() is the worker bee here. This method runs a WQL query that specifically looks for instances of the SQL Server Engine (SQLServiceType = 1) where the Instance Name is SQLEXPRESS (ServiceName = MSSQL$SQLEXPRESS). In the sample, I happen to pass the Instance Name as a variable instance, but you can do it any way you want. I choose to look specifically for SQLEXPRESS for a couple reasons: It's the default Instance Name, Visual Studio Express and ClickOnce deployment use this Instance Name, and we'd like to see more applications pointing to a single Instance Name, so the default makes sense. Once it's established that an instance named SQLEXPRESS exists, a further check is made to ensure that it is actually one of the SQL Express Editions and that it is running at the correct service pack level, in this case, SP1

That's pretty much it, a few checks to see what happened, and the function returns either True or False to the calling routine. You can call this from where ever you want and adjust the parameters as appropriate. One final note, check out the WMI Code Creator as mentioned in the code comment. This is the tool that will help you explore the SQL Server 2005 WMI Provider to find other ways to use it and other properties that you can query.

Mike

Addendum

Barry Sumpter converted this code into VB.NET in a forum post and I thought I'd add it here so it's easier to find.

Imports System
Imports System.Management

Namespace WMISample

    Public Class MyWMIQuery

        Public Shared Sub Main()

            Dim foo As Boolean = isExpressInstalled()

            If foo Then

                Console.WriteLine("You have SQL Express SP1. Sweet!!")

            Else

                Console.WriteLine("No instances named SQLEXPRESS exists on this computer.")

            End If

            Console.WriteLine("Hit Enter to continue.")

            Console.Read()

        End Sub

        Public Shared Function isExpressInstalled() As Boolean

            Const edition As String = "Express Edition"

            Const instance As String = "MSSQL$SQLEXPRESS"

            Const spLevel As Integer = 1

            Dim fCheckEdition As Boolean = False

            Dim fCheckSpLevel As Boolean = False

            Try

                Dim getSqlExpress As ManagementObjectSearcher = New ManagementObjectSearcher("root\Microsoft\SqlServer\ComputerManagement", "select * from SqlServiceAdvancedProperty where SQLServiceType = 1 and ServiceName = '" + instance + "' and (PropertyName = 'SKUNAME' or PropertyName = 'SPLEVEL')")

                If getSqlExpress.Get.Count = 0 Then

                    Return False

                End If

                For Each sqlEngine As ManagementObject In getSqlExpress.Get

                    If sqlEngine("ServiceName").ToString.Equals(instance) Then

                        Select Case sqlEngine("PropertyName").ToString

                            Case "SKUNAME"

                                fCheckEdition = sqlEngine("PropertyStrValue").ToString.Contains(edition)

                            Case "SPLEVEL"

                                fCheckSpLevel = Integer.Parse(sqlEngine("PropertyNumValue").ToString) >= spLevel

                        End Select

                    End If

                Next

                If fCheckEdition And fCheckSpLevel Then

                    Return True

                End If

                Return False

            Catch e As ManagementException

                Console.WriteLine("Error: " + e.ErrorCode + ", " + e.Message)

                Return False

            End Try

        End Function

    End Class

End Namespace

FAQ: My data doesn't get saved in Visual Studio

sqlexpress — Tue, 18 Jul 2006 06:56:00 GMT

The problem of data not being saved is usually the result the Copy to output directory property of the database file. The Copy to output directory property controls if and when a file is copied during deployment of a VS application, including when you debug (F5) your application. (Debugging an application copies all the project files into a Debug directory as part of the build process.)

The default value for the Copy to output directory property is Copy Always which results in the original database from your project directory being copied every time you debug or deploy your application. Because the original copy of the database is copied every time, it appears that data is not being saved. What is really happening is that data is being saved, but the file with the data is being copied over.

Proposed Solutions

Change the Copy to output directory property

A quick solution is to set the Copy to output directory property to Copy if Newer. Using this setting results in the database only being copied if it is newer than the database in the output directory. This solution is not complete for two reasons:

When your database is newer, it will still be copied over the database in the deployed folder, for example Debug, and will result in your database being over written.
There are a number of things that can happen in VS that will make your database appear to be newer and cause a file copy. This can happen even if you have not modified the structure of your database.

Write a custom deployment Class

A better solution is to write a custom deployment Class that will create your database at first run but use the existing database once it has been created. This same Class could be used to update your database by applying T-SQL scripts to the existing database.

An example of a custom database deployment class is demonstrated in the Embedded Database web cast.

Mike Wachal
SQL Express team

How to: Configure Express to accept remote connections

sqlexpress — Thu, 05 May 2005 23:32:00 GMT

Updated

The information in this posting is superseded by the following KB Article:

914277 How to configure SQL Server 2005 to allow remote connections
http://support.microsoft.com/default.aspx?scid=kb;EN-US;914277

-----------------------------------------------------------------------------------

Some people have been having issues when trying to make remote connections
to SQL Express. This document will hopefully clarify most of the issues
around remote connections.

First, networking protocols are disabled by default in SQL Server Express.
Thus, if someone simply installs Express and chooses all the defaults, SQL
Server Express will only be able to have connections originating on the
local machine where SQL Server is installed.

To enable SQL Server Express to accept remote connections we need to perform
the following steps:

STEP 1: Enabling TCP/IP

First we must tell SQL Server Express to listen on TCP/IP, to do this
perform the following steps:

1. Launch the SQL Server Configuration Manager from the "Microsoft SQL
Server 2005 CTP" Program menu
2. Click on the "Protocols for SQLEXPRESS" node,
3. Right click on "TCP/IP" in the list of Protocols and choose, "Enable"

STEP 2: To Browse or not to Browse

Next, we have to determine if we want the SQL Browser service to be running
or not. The benefit of having this service run is that users connecting
remotely do not have to specify the port in the connection string. Note: It
is a security best practice to not run the SQLBrowser service as it reduces
the attack surface area by eliminating the need to listen on an udp port.

OPTION A: If you want to always specify a TCP port when connecting (Not
using SQL Browser service) perform the following steps else skip these
steps:

1.      Launch the SQL Server Configuration Manager from the "Microsoft SQL
Server 2005 CTP" Program menu

2.      Click on the "Protocols for SQLEXPRESS" node

3.      Click on the "TCP/IP" child node

4.      You will notice an entry on the right panel for "IPAll", right click
on this and select, "Properties"

5.      Clear out the value for "TCP Dynamic Ports"

6.      Give a TcpPort number to use when making remote connections, for
purposes of this example lets choose, "2301"

At this point you should restart the SQL Server Express service. At this
point you will be able to connect remotely to SQL Express. A way I like to
check the connection is my using SQLCMD from a remote machine and connecting
like this:

SQLCMD -E -S YourServer\SQLEXPRESS,2301

The "," in the server name tells SQCMD it's a port.

So you've tried this and still get an error. Take a look at Step 3, this
should address the remaining issue.

OPTION B: If you want to use SQL Browser service perform these steps:

            Note:
            You will need to make this registry key change if you are using the April
            CTP or earlier versions:

            To enable sqlbrowser service to listen on the port 1434, the following
            registry key must be set to 1

            HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\SQL
            Browser\Ssrplistener

            Next, restart the sqlbrowser service.

      1. Start the SQL Browser Service

STEP 3: Firewall..?

      At this point you should be able to remotely connect. If you still
can't chances are you have a firewall configured on the computer where SQL
Express is running. The instructions below are for Windows XP SP2's
firewall settings.

      To enable the firewall to allow SQL Server Express traffic:

1.      Launch the Windows Firewall configuration tool from the control
panel.

2.      Click the Exceptions Tab

3.      Click the "Add Programs." button and select "sqlservr.exe" from the
location where you install SQL Server Express

You should be able to remotely connect. Note, you can get more restrictive
by just specifying the port number that will be allowed (used best when
configured with Option A).

Note: If you chose to use the SQL Browser service, you must also add
sqlbrowser service executable to the exception list as it listens on udp
port 1434.