SQL Developer - Connecting Databases : OLEDB API

OLEDB API Calls

selvar — Sat, 10 Nov 2007 18:22:48 GMT

OLE DB is a specification. It is not a set of components or files. It defines the way that a consumer, the client who is retrieving the data, talks to a provider, the server who is supplying the data. The provider is a COM object which is created by calling CoCreateInstance() . After creating an instance of the provider, the consumer than talks to the provider using the OLE DB defined interfaces. These interfaces are acquired by calling QueryInterface().

Here is a diagram showing the objects exposed by OLE DB:

Figure 1 – OLE DB Object Model

· Enumerators. Enumerators search for available data sources and other enumerators. Consumers that are not customized for a particular data source use enumerators to search for a data source to use.

· Data Source Objects. Data source objects contain the machinery to connect to a data source, such as a file or a DBMS. They are a factory for sessions.

· Sessions. Sessions provide a context for transactions and can be implicitly or explicitly transacted. A single data source object can create multiple sessions. Sessions are a factory for transactions, commands, and rowsets.

· Transactions. Transaction objects are used when committing or aborting nested transactions at other than the lowest level.

· Commands. Commands execute a text command, such as an SQL statement. If the text command specifies a rowset, such as an SQL SELECT statement, the command is a factory for the rowset. A single session can create multiple commands.

· Rowsets. Rowsets expose data in tabular format. A special case of a rowset is an index. Rowsets can be created from the session or the command.

The only object which gets created using the OLE function CoCreateInstance() function is the Data Source object which is classified as the OLE DB Provider. It is given a CLSID and ProgID. No other OLE DB objects shown above have a CLSID or ProgID as they are not creatable COM Classes (CoClasses). The remaining OLE DB objects are created by calling methods of an interface like IDBCreateSession::CreateSession() and IDBCreateCommand::CreateCommand().

A Walk Through Using OLE DB

To better understand how OLE DB works and how the objects are created and used, this section will walk through the steps of traversing data supplied by an OLE DB Provider.

1. Create an instance of the OLE DB Provider by calling the appropriate COM functions such as CoCreateInstance(). Request one of the interfaces of the data source object such as IDBInitialize or IDBProperties.

2. Get information about the data source or set the data source properties. Finally, call IDBInitialize::Initialize().

3. Create a Session object from the data source object and possibly start a transaction.

4. Create a rowset immediately if there are no commands needs to access the data or create a command object. One example where a rowset could be created immediately is when a data source object is fed the property of a text file and no other information is needed to read the file. The provider could simply support the creation of the rowset immediately without having to receive a command. This is done through the OLE DB interface IOpenRowset which is a interface of the Session object. If a command is necessary, the Session object is queried for the IDBCreateCommand interface and IDBCreateCommand::CreateCommand is called. In the call to CreateCommand() the programmer specifies which interface of the command object to retrieve.

5. In the command object, set the command text by calling ICommandText::SetCommandText().

6. Set the properties of the rowset that will be retrieved by using the ICommandProperties::SetProperties(). Some of these properties determine updatability, scrollablity, and whether or not the cursor will be a server-side cursor.

7. Create OLE DB accessors for any parameters which will be used in the command.

8. Call ICommand::Execute() to retrieve the rowset and get back an interface to the rowset.

9. Create an accessor for the returned rowset

10. Move through the rowset using GetNextRows() to retrive the handles to the rowset rows.

11. Call IRowset::GetData() using the accessor to retrieve the data of the current record.

12. Retrieve all of the data and be sure to free all of the handles (row and accessor).

13. Release all of the interfaces of the OLE DB objects. This will unload the OLE DB Provider.

OLE DB Resource Pooling

selvar — Sat, 10 Nov 2007 15:04:02 GMT

OLE DB resource pooling, also known as OLE DB session pooling, is handled by the OLE DB core components. To take advantage of resource pooling, an OLE DB consumer must invoke the data provider by using either the IDataInitialize or the IDBPromptInitialize interface. (ADO will attempt to do this automatically.) OLE DB resource pooling can be turned on for one provider and off for another.

Note Performing CoCreateInstance on IDBInitialize is traditionally used in OLE DB to open a data source object. To use resource pooling, you perform CoCreateInstance on either IDataInitialize or IDBPromptInitialize. Both interfaces are part of the OLE DB service components and not your OLE DB data provider. You can use IDataInitialize and IDBPromptInitialize to retrieve an instance of IDBInitialize from your data provider so that the service components can be used. The two interfaces make it possible to use service component features such as pooling, transaction enlistment, and the Client Cursor Engine.

When the application creates an OLE DB data source object (via ADO or an OLE DB consumer), OLE DB services query the data provider for supported information and provide a proxy data source object to the application. To the consuming application, this proxy data source object looks like any other data source object, but in this case setting properties merely caches the information in the local proxy. When the application calls IDBInitialize::Initialize in OLE DB or opens a connection in ADO, the proxy data source object checks whether any connections already exist that match the specified connection information and are not in use. If so, rather than creating a new object, setting properties, and establishing a new connection to the database, the proxy data source object uses the existing initialized data source object. When the application releases the data source object, it is returned to the pool. Any data source object that is released by the application and not reused after 60 seconds is automatically released from the pool.

Figure 4 shows how OLE DB resource pooling works.

Figure 4. The Benefits of OLE DB Resource Pooling

Note The term resource pooling is actually something of a misnomer. It implies that any kind of resource can be pooled, when in fact OLE DB resource pooling is just for the pooling of an OLE DB data source proxy object (DPO). Resource pooling does use the resource dispenser found with Microsoft Transaction Server, which is a more generic form of pool manager. More than a few OLE DB users have been confused by the term "resource pooling" into thinking that it pools more than it actually does.

Note Most of the functionality of Microsoft Transaction Server has been incorporated into Windows Component Services for Microsoft Windows 2000 and XP. References to Microsoft Transaction Server in this article may not reflect the equivalent behavior with Windows Component Services. If you are using Windows 2000 or XP, see your operating system documentation for more information.

Configuring OLE DB Resource Pooling

By default, service components are enabled for all Microsoft OLE DB providers if the provider is invoked by IDataInitialize or IDBPromptInitialize and if the provider is marked to work with pooling by using the OLEDB_Services registry key.

You can control pooling from your application in the following three ways:

You can disable pooling for an individual provider through the registry.
If you write directly to the OLE DB API, you can control pooling through the properties you set when connecting to the database.
You can control pooling from your connection string.

The time-out value is set to 60 seconds; this value cannot be configured in OLE DB resource pooling prior to the release of MDAC 2.5.

Retry Wait works much differently in resource pooling than in ODBC connection pooling. After the server has been determined to be unavailable, resource pooling blocks the pool. The next retry for a valid connection on the server occurs after one minute. If this attempt fails, the next retry occurs after two minutes, and again after five minutes. Thereafter, the retry occurs every five minutes until a valid connection is returned.

Configuring Resource Pooling from the Registry

To determine whether individual core components can be invoked to satisfy extended functionality requested by the consumer, OLE DB compares the properties specified by the consumer to those supported by the provider. Table 2 lists values that can exist in the OLEDB_Services registry entry, which should be created during installation of the provider.

Table 2. Setting OLE DB_Services Registry Entry Under Provider's CLSID

Default services enabled DWORD value

All services (the default) 0xffffffff

All services except pooling 0xfffffffe

All services except pooling and auto-enlistment 0xfffffffc

All services except client cursor 0xfffffffb

All services except client cursor and pooling 0xfffffffa

No services 0x00000000

No aggregation, all services disabled No OLEDB_Services registry entry

With the release of MDAC 2.5, you can configure both connection time-out and the Retry Wait values by using the registry.

When set under a provider's CLSID entry in the registry, the SPTimeout value controls the length of time, in seconds, that unused sessions are held in the pool. SPTimeout should be entered as a DWORD value under HKEY_CLASSES_ROOT/CLSID/ClassID where ClassID is the CLSID of the OLE DB provider. The default SPTimeout value is 60 seconds.

The Retry Wait value controls the length of time, in seconds, to wait between connection attempts. It is entered as a DWORD value under HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/DataAccess/Session Pooling. The default SPTimeout value is 60 seconds.

Configuring resource pooling from your application

If you write directly to the OLE DB API, you can set the DBPROP_INIT_OLEDBSERVICES property to enable or disable various core components, including OLE DB resource pooling. There is no way to configure connection time-out or Retry Wait programmatically in your application except by manipulating the registry entries directly. It is best not to have any services or applications using OLE DB when you make these changes, or the effects might not show up right away.

Table 3 lists values and the services enabled or disabled by the value settings. Code Example 2 is an excerpt of code that uses resource pooling in an OLE DB consumer.

Table 3. Setting OLE DB Services by Using the DBPROP_INIT_OLEDBSERVICES Property

Services enabled Property value

All services DBPROPVAL_OS_ENABLEALL

All services except pooling (DBPROPVAL_OS_ENABLEALL & ~DBPROPVAL_OS_RESOURCEPOOLING)

All services except pooling and auto-enlistment (DBPROPVAL_OS_ENABLEALL
& ~DBPROPVAL_OS_RESOURCEPOOLING & ~DBPROPVAL_OS_TXNENLISTMENT)

All services except client cursor (DBPROPVAL_OS_ENABLEALL
& ~DBPROPVAL_OS_CLIENTCURSOR)

All services except client cursor and pooling (DBPROPVAL_OS_ENABLEALL
& ~DBPROPVAL_OS_RESOURCEPOOLING & ~DBPROPVAL_OS_CLIENTCURSOR)

No services ~DBPROPVAL_OS_ENABLEALL

Code Example 2: OLE DB Consumer Code Using Pooling

long OLEDBConnect( char   *lpszInitString,
                   bool    bUsePooling,         // Flag to set pooling
                   ULONG   nCount )
{
   IDataInitialize   *pIDataInitialize = NULL;
   IDBProperties     *pDBProperties    = NULL;
   IDBInitialize     *pIDBInit         = NULL;
   HRESULT            hr               = S_OK;
   WCHAR              wszInitString[1024];

   AtoU( lpszInitString, &wszInitString[0], 1024 );
   hr = CoCreateInstance(CLSID_MSDAINITIALIZE,
                         NULL,
                         CLSCTX_INPROC_SERVER,
                         IID_IDataInitialize,
                         (void**)&pIDataInitialize);
   for (ULONG i=0;i<nCount && SUCCEEDED( hr ); i++)
   {
      hr=pIDataInitialize->GetDataSource(NULL,
                                         CLSCTX_INPROC_SERVER,
                                         wszInitString,
                                         IID_IDBInitialize,
                                         (IUnknown**)&pIDBInit);
      if (pIDBInit)
         hr=pIDBInit->Initialize();
      if ( bUsePooling == false )
         if (pIDBInit)
            pIDBInit->Uninitialize();   // Disables Pooling(!)
      if (pIDBInit) pIDBInit->Release();
         pIDBInit = NULL;
   }
   if (pIDataInitialize)
      pIDataInitialize->Release();
   return 0;
}

Note One of the key points to note in the preceding code is that if you call IDBInitialize::Uninitialize, you will turn off pooling! To release a connection, use IDBInitialize::Release instead.

The OLE DB property DBPROP_INIT_OLEDBSERVICES maps directly to a connection string attribute, OLE DB Services, as shown in Table 4. Code Example 3 demonstrates this.

Table 4. Setting OLE DB Services by Using ADO Connection String Attributes

Services enabled Value in connection string

All services (the default) "OLE DB Services = -1;"

All services except pooling "OLE DB Services = -2;"

All services except pooling and auto-enlistment "OLE DB Services = -4;"

All services except client cursor "OLE DB Services = -5;"

All services except client cursor and pooling "OLE DB Services = -6;"

No services "OLE DB Services = 0;"

Code Example 3: ADO Consumer Code Using Pooling

'This will take advantage of resource pooling.
Dim i As Integer
Dim cnn As New ADODB.Connection
Dim rst As ADODB.Recordset

cnn.Open "DSN=LocalServer;UID=MyUserName;PWD=MyPassword; " & _
         "OLE DB Services=-1"
For i = 1 To 100
      Set rst = New ADODB.Recordset
      rst.Open "SELECT * FROM Authors", cnn
      Set rst = Nothing
Next i

cnn.Close
Set cnn = Nothing

Enabling OLE DB Resource Pooling

Resource pooling can be enabled in several ways:

Automatically, when a consumer accesses a provider from within Microsoft Transaction Server or Internet Information Services 4.0 or later.
For an OLE DB-based consumer, by using the IDataInitialize or IDBPromptInitialize interface. If you use IDataInitialize::CreateDBInstance, you can enable or disable services components by using the DBPROP_INIT_OLEDBSERVICES property.
For an ADO-based consumer, by keeping one open instance of a Connection object for each unique user and using the OLEDB_SERVICES connection string attribute to enable or disable pooling. By default, ADO attempts to use pooling, but if you do not keep at least one instance of a Connection object open for each user, there will not be a persistent pool available to your application. (However, Microsoft Transaction Server keeps pools persistent as long as the connections in them have not been released and have not eventually timed out.)

Note Pooling will not be enabled if you call CoCreateInstance directly on the CLSID of the data provider.

Determining the Number of Available Pools

In OLE DB resource pooling, the following formula determines the number of pools (where N is the number of pools, P is the number of processors, and C is the number of distinct sets of connection attributes on the system):

N = (P + 1) * C

Using this formula, OLE DB resource pooling eliminates lock contentions on the pools.

Note P+1 is a default value which can be overridden by setting the following registry key:
CLSID\{2206CDB0-19C1-11D1-89E0-00C04FD7A829}\Holders.

OLE DB resource pooling keeps a map of which pools contain which users. From there, it is possible to jump to the right pool and start looking for an available connection. With OLE DB resource pooling, a given pool contains connections only for a single provider with a specific set of connection attributes. In other words, each pool contains only connections to one set of credentials and one data store. As a result, a new pool will be created for each user if a user is connecting using Windows Authentication.

Writing Data Consumers That Work With OLE DB Services

When developing an application using either ADO or native OLE DB, the following tips will help you to enable resource pooling or to avoid inadvertently disabling it. These rules apply whether you use ADO, OLE DB consumer templates, or native OLE DB code.

Tips for ADO Users

The ADO Connection object implicitly uses IDataInitialize. However, this means your application needs to keep at least one instance of a Connection object instantiated for each unique user—at all times. Otherwise, the pool will be destroyed when the last Connection object for that string is closed. (The exception to this is if you use Microsoft Transaction Server. In this case, the pool is destroyed only if all of the connections in the pool have been closed by client applications and are allowed to time out.)
Note If you open a Connection object, remember to close it. Do not rely on garbage collection to implicitly close your connections.
By avoiding connection creep (discussed in section "Connection Creep and Effective Server Tracing" later in this article), you can also help your application use pooling.

Tips for OLE DB Users

Use IDataInitialize or IDBPromptInitialize to create an instance of your data provider. As long as one instance of either interface exists, a pool is available to your application. Otherwise, the pool will be destroyed.
Do not request provider-specific interfaces prior to connecting.
Do not request prompting.
Do not use IDBInitialize::Uninitialize. Use IDBInitialize::Release instead. If you call Uninitialize, the connection you were using will be flushed from the pool. This is because the user can change the properties of the connection after calling Uninitialize.
Do not release your last instance of IDataInitialize or IDBPromptInitialize while pooling is needed by your application.
Use only one resource per connection.
By avoiding connection creep (discussed below), you also help your application use pooling.

Writing Data Providers That Work with OLE DB Services

As mentioned above, ADO attempts to use or enable services implicitly by referencing the OLEDB_Services registry value when a connection is established. However, some older providers may not be able to support services. Simply creating this registry entry is not enough to enable services for your application. The data provider must support certain features or services will not work. Attempting to use services with a data provider that does not offer this functionality can create unexpected behavior in your application. Following is a brief list of the functionality a data provider must support for your consumer application to utilize services:

The provider must support aggregation of all objects.
Providers must support the free-threaded model or, at a minimum, the rental-threaded model. Services will determine the data provider's thread model by using the DBPROP_DSOTHREADMODEL property.
If the provider has a global connection state that may change while the data source object is in an initialized state, it should support the DBPROP_RESETDATASOURCE property.
Providers that connect to a remote database and can detect whether that connection has been lost should support the DBPROP_CONNECTIONSTATUS property. This allows pooling to detect dead connections and to ensure that they are not returned to the pool.
Providers should support both pooling and transaction enlistment by using the DBPROP_INIT_OLEDBSERVICES property. The values used with this property also correspond to the OLEDB_Services registry key.

For more information, consult either the OLE DB Readme file that shipped with the Data Access 2.0 SDK or the OLE DB Services section in the documentation with version 2.1 and later of the SDK.

ADO Overview and Architecture

selvar — Sat, 10 Nov 2007 14:53:36 GMT

ADO is the most popular and well-known of Microsoft’s Data Access API’s. ADO is, like OLEDB, a COM object model for accessing, retrieving and updating data. Unlike OLE DB, however, ADO objects expose OLE Automation interfaces, allowing them to be accessed not only from C++, but also languages such as Visual Basic, and even from scripting languages like VBScript on ASP pages. ADO is also the easiest of the API’s to program against, as it provides a great deal of flexibility in how to do many operations, and the object model is relatively simple.

Behind the scenes, the ADO objects act as an OLE DB consumer. When you use ADO in your application, you are using OLE DB implicitly. ADO applications therefore must specify what OLE DB provider they wish to use. ADO uses the concept of connection strings similar to ODBC. The connection strings created through OLE DB data links can be used in ADO.

As with any OLE DB consumer, ADO can load any provider it wishes (although some providers may not support interfaces which ADO requires). This means that ADO can access data not only through data source specific providers, but also through ODBC drivers via MSDASQL. In fact, if in your ADO connection string you only specify what DSN to use, you will by default load the MSDASQL provider and connect to the data source specified in the DSN. For example, your connection string could look like:

“DSN=MyTestDSN1”

Remember, however, that going through MSDASQL to reach a data source for which an OLE DB provider exists is not the best solution. Therefore, for instance, if you are using ADO to talk to SQL Server you should not specify a DSN in the connection string as this implies you want to load MSDASQL, when in fact you should be using the SQL Server OLE DB provider.

The following diagram shows the basic flow of an application connecting to SQL Server via ADO:

MSDASQL - OLEDB Provider for ODBC drivers

selvar — Sat, 10 Nov 2007 14:43:52 GMT

As mentioned before, a provider is needed for any data source that is to be exposed through OLE DB. Microsoft releases providers for several common data sources, including SQL Server, Oracle, and Jet. In addition, Microsoft ships a special provider called the “OLE DB Provider for ODBC Drivers”. This was the very first OLE DB provider ever released, and is also known as “MSDASQL”, due to the fact that it is contained in “MSDASQL.DLL”.

MSDASQL allows OLE DB consumer applications to utilize ODBC drivers to connect to a data source. The provider converts incoming OLE DB calls into ODBC calls, and passes them on to the specified ODBC driver. It then retrieves results from the ODBC driver and formats those into OLE DB specific structures which the consumer then has access to.

The following diagram shows the flow of connecting to a SQL Server database through the MSDASQL provider and SQL Server ODBC driver.

OLE DB Flow with MSDASQL

MSDASQL was written because at the time OLE DB was first released, very few data sources were exposed through OLE DB providers. The MSDASQL provider allowed OLE DB applications to be able to talk to data sources for which providers had not yet been written. However, as with any scenario where you increase the length of the code path to complete an operation, talking to a data source via MSDASQL is slower than talking directly with a data source specific provider. We should recommend that customers use data source specific providers wherever possible. In addition, MSDASQL is becoming deprecated. It will not be ported to 64 bit platforms, and will not receive any feature enhancements in future releases.

OLE DB Programming Overview and Architecture

selvar — Sat, 10 Nov 2007 14:36:50 GMT

OLE DB is a COM based object model used for access to various relational data sources as well as structured, not relational data (such as folders in an email store, or a file system). This ability to access both relational and structured data is one of the major features that sets OLE DB apart from ODBC. However, there are many more differences between the two API’s. OLE DB was developed as a successor to ODBC, and has continued to gain increasing importance as the core of Microsoft’s data access strategy. (Although with the release of .NET, OLE DB’s importance has diminished some to being only the core of the unmanaged, or native, environment.) Both ODBC and OLE DB API’s provide very fast access to data. Although benchmark results vary from test to test, generally OLE DB and ODBC perform similarly well in most tasks.

The fundamental concepts of OLE DB are that of the Consumer and the Provider. In a general sense, consumers are applications or components which take and use data from a data source. A provider is a piece of software which works to present data to consumers. The ODBC equivalent of an OLE DB provider is the driver. In OLE DB, providers are written for each individual data source, as ODBC drivers are also written for each data source. An OLE DB provider exposes data to an OLE DB consumer. The architecture of the consumer-provider system is quite flexible, and in some cases a consumer can talk to a provider, which itself acts as a consumer to another provider, and so on. However, most of the time a data source is exposed by a single provider with which the consumer, usually a client application, communicates. The diagram below shows the standard flow of communication in OLE DB, in this example from a client application to a SQL Server database.

The OLE DB Service Components is a set of services provided in OLEDB32.DLL. The service components are frequently loaded along with a data provider by OLE DB consumers. The OLE DB Service Components (also known as Core Services) provide such facilities as pooling and the data links, which is discussed next. Loading the service components is also necessary to use client side cursors, which is also discussed later in this course. (The client cursor engine, however, is not in OLEDB32.DLL, but rather in MSDACE.DLL).

Data Links

Similar to the ODBC DSN facility, OLE DB does provide a way for connection information to be stored separate from the compiled client code. This mechanism, however, is entirely file-based, and does not use the registry as ODBC DSN’s can. The mechanism which allows OLE DB consumers to use connection information stored in files is called the Data Links Service, and is provided in the OLE DB Core Services. The files used to store this information are known as “UDL” files, and are saved with a “.UDL” extension.

The following demonstration will show how a UDL file can be created, and used as a troubleshooting tool to test OLE DB connectivity.

Demonstration:

1. Make sure in Windows Explorer, Tools->Folder Options, View Tab, that “Hide file extensions for known file types” is not checked.

2. Right click on Windows desktop, and select New->Text File.

3. Name the file “Test.udl”. The icon for the file should now be the special “UDL” icon.

4. Double click the file to open the Data Links dialog.

5. Click on the “Provider” tab. Select “Microsoft OLE DB Provider for SQL Server”.

6. Click on the “Connection” tab. In the first dialog box (server name) type “(local)”.

7. Click the radio button for “Use NT Integrated Security”.

8. Click on this “Advanced” and “All” tabs. These are additional options available to set on the provider at the time you connect.

9. Go back to the “Connection” tab and click “Test Connection”. You should see “Test Connection Succeeded”. This is a good way to verify if basic OLE DB connectivity works to a given data source.

10. Now click ‘OK’ to close the dialog. Then open the “Test.udl” file with notepad. What you see is the “Connection String” for the information specified in the dialogs. The Data Links interfaces in OLEDB allow a consumer to pass connection strings formatted such as this to establish a connection to a data source. You can also use these connection strings in ADO. It is a helpful trick to cut and paste the connection string from a UDL file such as this into your ADO code. (we will cover ADO in much more detail later).

Another benefit Data Links provides is that consumer applications can launch this pre-built dialog to allow users to enter connection information for a data source at run time.