WCF LOB Adapter SDK and BizTalk Adapter Pack : OracleDB

Using Oracle Database or Oracle EBS adapter for Oracle AQ (Advanced Queuing)

manasg — Mon, 25 May 2009 14:31:00 GMT

Here is a simple way to get started on Oracle AQ (Advanced Queuing) using the Oracle Database and/or Oracle E-Business Suite adapters. Here is what I did, right from defining my queue - one step at a time.

Step 1 - Define the queue payload type
CREATE TYPE AQ_MESSAGE_TYPE AS OBJECT (ID VARCHAR2(30), INFO VARCHAR2(200));

Step 2 - Define the queue table
EXEC DBMS_AQADM.CREATE_QUEUE_TABLE(QUEUE_TABLE => 'INFO', QUEUE_PAYLOAD_TYPE => 'AQ_MESSAGE_TYPE');

Step 3 - Create the queue
EXEC DBMS_AQADM.CREATE_QUEUE(QUEUE_NAME => 'AQ_MESSAGE_QUEUE', QUEUE_TABLE => 'INFO');

Step 4 - Start the queue
EXEC DBMS_AQADM.START_QUEUE(QUEUE_NAME => 'AQ_MESSAGE_QUEUE');

Step 5 - Create a function to enqueue
CREATE OR REPLACE FUNCTION ENQUEUE_AQ_MESSAGE_QUEUE(PAYLOAD IN AQ_MESSAGE_TYPE) RETURN RAW
AS
PROPERTIES DBMS_AQ.MESSAGE_PROPERTIES_T;
MSGID RAW(16);
OPTIONS DBMS_AQ.ENQUEUE_OPTIONS_T;
BEGIN
DBMS_AQ.ENQUEUE('AQ_MESSAGE_QUEUE',
    OPTIONS,
    PROPERTIES,
    PAYLOAD,
    MSGID);

RETURN MSGID;
END;

Step 6 - Create a function to dequeue
CREATE OR REPLACE FUNCTION DEQUEUE_AQ_MESSAGE_QUEUE(PAYLOAD OUT AQ_MESSAGE_TYPE) RETURN RAW
AS
PROPERTIES DBMS_AQ.MESSAGE_PROPERTIES_T;
OPTIONS DBMS_AQ.DEQUEUE_OPTIONS_T;
MSGID RAW(16);
NOTHING_TO_DEQUEUE EXCEPTION;
PRAGMA
EXCEPTION_INIT (NOTHING_TO_DEQUEUE, -25228);
BEGIN
OPTIONS.WAIT := DBMS_AQ.NO_WAIT;
DBMS_AQ.DEQUEUE('AQ_MESSAGE_QUEUE',
    OPTIONS,
    PROPERTIES,
    PAYLOAD,
    MSGID);
RETURN MSGID;
EXCEPTION
WHEN NOTHING_TO_DEQUEUE
  THEN RETURN NULL;
END;

A word on dequeue and enqueue functions - these are simplistic in this example. You might want to modify them to take more inputs (for example - more message properties or enqueue/dequeue properties as input). The dequeue function returns immediately if a message is not available for dequeue and returns null in that case. You could tweak that too.

Step 7 - Call the functions using the adapter!
It is easy calling these from the adapter now. The reason why the DBMS_AQ.ENQUEUE and DBMS_AQ.DEQUEUE cannot be called directly from the adapter is that they use complex data types that are not supported by ODP.NET. One can now even call the dequeue function in a polling context, and get messages delivered to them as they are pushed into the queue! The links to portions of the Oracle Database adapter help that will help you do this are:

Developing BizTalk applications http://msdn.microsoft.com/en-us/library/dd788030.aspx
Support for Oracle User Defined Types http://msdn.microsoft.com/en-us/library/dd788045.aspx

Thanks to Mustansir for suggesting this blog idea!

Oracle EBS/DB Adapter Changes in Beta

Sai Shankar — Wed, 28 Jan 2009 08:48:00 GMT

As the Beta release for the Adapter pack approaches, here's a list of changes in the Oracle EBS/DB Adapters so that you can prepare for it:

· Now support Windows Vista and Windows Server 2008

· Based on Oracle ODP.Net 11.1.0.7

Both adapters now _require_ ODP.Net 11.1.0.7 installed on the client as compared to ODP.Net 11.1.0.6 which was the base for CTP 5 and earlier. Doing this enables us to deliver x64 support.

· Added Synonym support

Both adapters now support synonyms. Synonyms show up in the object tree wherever the underlying artifact (table/view etc.) of that type can occur. Synonyms are supported for the following types.

Tables
Views
Materialized views
Sequences
Procedures
Functions
Packages

Synonyms are not supported for:

Types
Synonyms

In addition, synonyms pointing to DB links are not supported.

· Added performance counters

We’ve added performance counters to both Adapters which are described in the online help.

· Added notification support, support for User Defined Types (UDTs) in the DB Adapter

Please note that this is not a complete list of changes. That list is available in the online documentation.

BizTalk Adapter Pack v2 CTP

Vivek Krishna — Mon, 15 Dec 2008 09:47:00 GMT

BizTalk Adapter Pack team has released a CTP. This is the 5^th CTP and the first one available publicly for everyone to try. The specific enhancements in this CTP are mostly to the OracleDB adapter(included in the BAPv1) to which we added features like support for polling stored procedures and composite operations. We also added 64 bit support for the Oracle EBS adapter.In addition we have addressed some feedback we got from the other adapters .

To re-iterate here is the complete list of features BAPv2 is adding . The downloads are available at the connect site for public download(requires sign up with live id)

BizTalk Adapter Pack V2 - Themes

New Adapters

· Oracle EBS Adapter

Ø Browse and search metadata for each Oracle application

Ø Support Interface tables, Interface views

Ø Support for PL/SQL APIs

Ø Concurrent Programs, Request Sets

Ø Support for polling tables, Stored Procedures

Ø Notification support

Ø App context initialization

Ø Supports REF cursors, User Defined Types, LOB types, RECORD types

Ø Support for PL/SQL Tables, boolean data types

Ø Support for MLS EBS installations

Ø Microsoft Office SharePoint Server Integration

Ø Versions supported – ODP.NET 11.1.0.6.21 (available only on 32bit), Oracle EBS 11.5.9, 11.5.10 and 12

· SQL Server Adapter

Ø Insert/Update/Delete/Select operations on tables and views

Ø Invoke (SQL and CLR) Stored procedures, scalar valued functions , table value functions

Ø Supports execution of generic T-SQL statements, via the ExecuteReader(), ExecuteNonQuery() and ExecuteScalar() operations

Ø Configurable polling – polling statement, polling interval, receive multiple result sets

Ø Support for x86 and x64 platforms

Ø Support for new data types in SQL2005 and SQL2008 (notably FILESTREAM, varbinary (max))

Ø Support for UDTs

Ø Ability to pass table value parameters

Ø Composite operations - invoke operations on multiple tables and call any number of stored procedures in the same transaction

Ø SQL query notifications

Ø Ability to directly call stored procedures which used the ‘for xml’ style supported by the BizTalk 2006 SQL adapter to ease backward compatibility

Ø Versions supported – SQL 2000, SQL 2005, SQL 2008

Enhancements to the existing V1 Adapter Pack

Include hot fixes built since the V1 Adapter Pack released
Support new Microsoft Platform releases – BTS 2009, VS 2008, Windows Server 2008
Oracle DB Adapter:

Ø Polling support for Stored procedures and Functions

Ø UDTs, PL/SQL tables

SAP Adapter:

Ø Support for Table Types

Ø Customization of adapter behavior when “special” values obtained for DATS/TIMS/NUMC

Ø SAP ADO Provider Enhancements:

o Support for SSRS

o Support for SAP Queries

o Ability to specify the result set to give to SSIS when executing BAPIs

Siebel Adapter:

Ø MVG operations (associate, dissociate, query) in design time enabled

Ø Bounded pick list fields supported per recommended approach

Ø Context specific custom messages generated for exceptions

Enhancements to existing Oracle DB Adapter

Murali Narumanchi — Sat, 31 May 2008 12:34:00 GMT

For the next release of adapter pack, we are adding 2 new adapters -- Oracle EBS and SQL. In addition, we are also enhancing support for the existing OracleDB adapter. We are planning to add support for the following:
           -       UDTs
           -       Table Types
           -       Boolean types

We would like to hear from you about any other specific features that you may want in OracleDB adapter. Please do let us know if you’d like to participate in the TAP program.

I see errors (or warnings) in the event log when I disable the Receive Location, for small Polling Intervals (especially 0)

tusharma — Fri, 04 Jan 2008 17:41:00 GMT

The Symptoms!

Occasionally, you may see errors/warnings in the event log when you disable a WCF-Custom receive location which was configured to use the polling feature of the oracle adapter, especially if the polling interval is set to a low value. You would see these errors almost every time you disabled the location, if Polling Interval = 0.One of the errors would be:

"The adapter "WCF-Custom" raised an error message. Details "System.InvalidOperationException: The reply is invalid because the underlying transaction object has been disposed. This occurs if the polling interval expires before receiving a reply. Use a higher value for polling interval."

Another would be:

"The adapter "WCF-Custom" raised an error message. Details "System.Transactions.TransactionAbortedException: The transaction has aborted."

In some cases you may see another error as well: "Connection must be open for this operation".

What's going on?

To understand what's going on, let’s take a closer look at how polling happens in the Oracle Adapter. I'll avoid the gory code level details and present a relatively high level picture.

Simply speaking, at any instant during a running polling scenario, the Adapter may be considered to be in either of the following two states (Figure 1):

Actively Polling:
The adapter is polling for a message, or waiting for the message to be consumed. In fact this state may itself be considered to be made up of the following sub-states (the above state transition diagram may be redrawn to accomodate this detail):-

Poll Msg -> Msg Available -> Msg consumed -> Polling Transaction Committed (or aborted) -> (Go to 'Poll Msg' or 'Sleeping' depending on the remaining polling interval)

The reason for combining these steps into a single state is that they are executed as part of a single transaction.
Sleeping
If the Polling Interval specified is greater than the time required to complete all the tasks mentioned above in the 'Actively Polling' state, the adapter switches to the 'Sleeping' state for the remaining period of the Polling Interval, after which it transitions back to the 'Actively Polling' state ('Poll Msg' to be specific) and polls for the next message.

Clearly, if the polling interval is small (compared to the time required to complete the tasks in 'Actively Polling' state), at any random point of time the adapter is more likely to be in the 'Actively Polling' state than in the 'Sleeping' state.

Note that Polling Interval = 0 is a special case which tells the adapter to take as much time as it needs to complete the set of steps in the 'Actively Polling' state and poll for the next message as soon as previous transaction has been completed. Thus, in this case, the adapter never goes to the 'Sleeping' state.

Also note that when Polling Interval > 0, there’s no guarantee that the adapter will be able to complete all the above steps within the specified interval. If it is unable to do so, the polling transaction will be aborted and the adapter will go back to poll the next message (if Retry Count > 0). In this case too, the adapter will never go to the ‘Sleeping’ state.

Now if the 'Receive Location' is disabled when the adapter is in 'Sleeping' state, everything would be fine. However, if the adapter happened to be in the 'Actively Polling' state at that point, it may lead to problems. In most cases the following sequence of events will take place at the instant when you disable the receive location:-

The Receive Location (WCF-Custom adapter) is consuming an available message (it sends an acknowledgement/reply to the Oracle Adapter to indicate whether or not it has successfully consumed the message)
The WCF-custom adapter aborts its own transaction and you see a 'Transaction Aborted' exception.
The Oracle adapter receives the instruction to stop polling and aborts the current transaction.
If the message had a part which was being streamed by the adapter (e.g. a blob column), you might see an exception saying "Connection must be open for this operation".
By the time the last reply from the WCF-Custom adapter reaches the Oracle adapter, the corresponding transaction has already been aborted and the transaction object disposed. This is when you see the exception saying "System.InvalidOperationException: The reply is invalid because the underlying transaction object has been disposed. This occurs if the polling interval expires before receiving a reply. Use a higher value for polling interval."

The above picture is rather simplified (the thread level details are missing) and may not be completely accurate but I hope it gives some insight into why you see those errors while disabling the receive location and how they are related to the Polling Interval.

Ok, but what are the implications? What do I do if I see this error?

You don’t need to do anything. As explained above, the error is a result of how polling happens and is benign. The only thing you need to keep in mind when you see this error is that the last transaction which the adapter was executing has been aborted. (So, effectively, the last post-poll statement was not executed).

BizTalk Oracle Adapter Vs WCF-based Oracle DB Adapter

mahadevanv — Mon, 29 Oct 2007 08:11:00 GMT

The following is a list of key feature differences between the BizTalk Oracle adapter and the WCF-based Oracle DB adapter in the BizTalk Adapter Pack.

1. LOB access: While the BizTalk Oracle adapter uses an ODBC driver to connect to Oracle DB, the WCF-based Oracle DB adapter uses Oracle’s ODP.NET

2. Inbound processing: The BizTalk Oracle adapter supports a “Polling Query” and a “TableChangedEvent” mechanism for receiving data from Oracle into BizTalk. The WCF-based Oracle DB adapter replaces these mechanisms with a single polling based mechanism that that supports the following key features:

a. Users can generate a strongly typed schema for the inbound message from a polling statement which is typically a SELECT statement

b. Users can specify a post poll statement that can be a PL/SQL block of code that would be executed by the adapter after the polling statement

c. The polling statement and the post poll statement are executed in a transaction context

d. Users can specify the transaction isolation level required to be used for the transaction

e. The polling statement, post poll statement and the transaction isolation levels are typically specified via binding properties

3. Design-time: The WCF-based Oracle DB adapter uses a standardized GUI for browsing, searching and retrieving metadata for LOB artifacts. This GUI is accessible as part of BizTalk design time and Visual studio. The WCF-based Oracle DB adapter also provides some design-time enhancements including browsing of procedures, functions and packages including the procedures and functions constituting the package in a categorized and hierarchical manner

4. Metadata retrieval: Unlike the BizTalk adapter where metadata can only be retrieved in the form of XML schemas in BizTalk, the WCF-based Oracle DB adapter exposes programming access to retrieving metadata. This includes WCF’s standard MEX interfaces and WCF LOB adapter SDK’s special MetadataRetrievalContract that supports both browse and search operation in addition to metadata retrieval

5. Programming Interface: The WCF-based Oracle DB adapter, being a WCF custom binding element, naturally exposes WCF’s programming interfaces including the WCF service model and WCF channel model in addition to the WCF web service model (when hosted in IIS) as opposed to the BizTalk Oracle adapter that exposes XML message schemas for BizTalk users for integration

6. Transactions: The WCF-based Oracle DB adapter supports transactions in WCF channel/service model programming modes and from BizTalk. Transactions are supported in outbound (adapter to LOB) and inbound scenarios (LOB to adapter)

7. REF cursors: The WCF-based Oracle DB adapter supports Oracle IN, OUT and INOUT REF cursors including the strongly typed and weakly typed variants unlike the BizTalk Oracle adapter.

8. LOB types: The WCF-based Oracle DB adapter provides rich support for BLOB, CLOB and BFILE types in tables and stored procedures/functions. The adapter also exposes special operations for streaming BLOB/CLOB/BFILE data in the WCF service model interface.

RECORD types: Enhancing on top of ODP.NET, the WCF-based Oracle DB adapter supports RECORD type parameters in stored procedures/functions even though these types are not natively supported by ODP.NET (version 10.2.0.20)

Streaming in OracleDB adapter

kdas — Fri, 26 Oct 2007 11:15:00 GMT

OracleDB adapter supports streaming of data for both inbound (wrt adapter) and outbound messages where it is possible to do so. Streaming can be at record level (in case of Insert operation), xml node level or xml node value level.

For inbound messages, the adapter consumes it using Message.WriteBodyContents(XmlDictionaryWriter). In general for any operation the adapter keeps on reading the message until it has accumulated the minimum amount of data that can be passed to Oracle database using ODP.NET. This is true for all operations except UpdateLOB (will come to this in a short while). Also in most cases what this means is that the message is fully read before sending any data. Insert operation (inserting array of rows into a table) is slightly different. In this case one row/record is read and inserted into Oracle (in future version the number of records to be read before doing an insert will be customizable through a binding property).

One problem with the above approach is that it is very difficult to stream data into Oracle large object types like BLOB, (N)CLOB. So the UpdateLOB operation was introduced on tables/views having these columns. Input to UpdateLOB operation is the large object column name, unique filter giving the row that is to be updated and the actual data. The data is read in small chunks and passed to Oracle. This actually has xml node value level streaming. This method works fine in WCF channel scenario (and BizTalk which is an extension to that) but not in the WCF proxy scenario. In proxy the actual data becomes a byte array and defeats streaming. This is a limitation of WCF proxy model which doesn't allow a combination of streamed (actual data) and non-streamed (column name and filter) parameters in the same direction.

For outbound messages, the adapter always tries to stream if possible. For e.g. in case of Select operation response, one row/record is read from Oracle and output. If the row has large object types then the values for those are streamed as well (xml node value streaming). The same approach is also taken for stored procedures/functions that has output parameters of large object and REF CURSOR types. But this works only in WCF channel scenario when the message is consumed using Message.WriteBodyContents(XmlDictionaryWriter) (in BizTalk this is the case as well). In proxy scenario there is no streaming as the message is fully de-serialized into objects. In order to support streaming for large object types in proxy, ReadLOB operation was added on tables/views having large object type columns. The method signature looks like Stream ReadLOB(string column_name, string filter). This is only meant to work in proxy. In channel the same thing can be achieved using a Select operation.

Getting LOB metadata from SAP, OracleDB and Siebel adapter using svcutil

Jayanthi S — Tue, 23 Oct 2007 11:56:00 GMT

The WCF based adapters that ship in the Biztalk Adapter Pack are WCF bindings. The svcutil tool can be used to get metadata for a LOB method using these bindings. The scheme in the URI determines which adapter binding is loaded by svcutil. However when using the svcutil tool, there is no easy way to specify the LOB credentials that should be used to get metadata from the LOB. One way to specify the LOB credentials to our adapters is to set it in the URI. However, by default (for security reasons) the adapters do not accept credentials in the URI.

To get around this, you need to set the binding property acceptCredentialsInUri to true and then specify the LOB credentials in the URI. To be able to set binding properties or other custom settings when using svcutil.exe, you should use the svcutil.exe.config file. This article describes how to use it.

Attached is a sample svcutil.exe.config which has the endpoint and binding configurations for all the three adapter bindings. Place this in the same folder as svcutil.exe.

The following are sample command lines to use to get metadata from our adapters.

SvcUtil.exe "sap://user=youruser;passwd=yourpassword;Client=800;lang=EN@A/adapsap47/00?op=http://Microsoft.LobServices.Sap/2007/03/Rfc/RFC_GET_FUNCTION_INTERFACE"

SvcUtil.exe "oracledb://User=MYUSER;Password=MYPASSWORD@ADAPTER/?op=http://Microsoft.LobServices.OracleDB/2007/03/SCOTT/Table/AEMP/Insert"

SvcUtil.exe "siebel://Username=MYUSER;Password=MYPASSWORD@mssebldemo:2321?SiebelObjectManager=SSEObjMgr_enu&SiebelEnterpriseServer=ent771&Language=enu&op=http://Microsoft.LobServices.Siebel/2007/03/BusinessObjects/Account/Agreement/Query"

Note the quotation mark around the URI. In the configuration for Siebel (in the attached config file) also note the idleConnectionTimeout without which it will take the default 1 minute for svcutil to terminate. Look for more details on the idleConnectionTimeout in a separate post on Siebel.

Note also that if you use the Add Adapter Service Reference tool (that gets installed with the WCF LOB Adapter SDK) to get metadata for a LOB method for use in a C# or VB project, the AcceptCredentialsInUri binding property will not be visible or settable. You can set the LOB credentials using the credentials tab in the tool.