How the connector works

The connector is responsible for communicating with an application to pull data out and put data into the application.

At startup, the Retek connector logs into the integration broker and retrieves its initial configuration properties, supported business object definitions, and subscription list from the repository. It connects to the application database by using the JDBC Connect mechanism. One connector-specific configuration parameter (DatabaseURL) allows you to specify the name of the database server to which the connector should connect. For information on the configuration parameters, see "Configuring the connector properties".

When the connector is started, it establishes a connection pool with the database. It uses connections from this pool for all database transactions. Connections are created up to a maximum limit of maxDBConnections. If no connections are available because the maximum number of connections has been reached, the connector will wait for a connection to become available. On termination of the connector, all connections in the pool are closed.

For more information on starting the connector, see "Starting the connector" and one of the following documents, depending on the integration broker you are using:

Processing application events

Application events are processed according to the stages described in the following sections:

Event detection and notification

When an "event of interest"---a create, update, or delete operation--occurs in a Retek application database table, database triggers detect the event and record it in a message family staging table using the ADDTOQ procedure and formatting the data in XML. The "event" is "of interest" because it affects the data of an application entity that is associated with a business object definition. Each new row that appears in the table represents an event notification.

The message family staging table queues the events for pick up by the connector. Each event contains enough information so that the connector can determine the name and verb of the business object instance that it will create to represent an event. Each message family has exactly one staging table, which is set up at installation and which is administered by a Message Family Manager (MFM) set of stored procedures.

Event publishing

The connector looks for events by polling (pollForEvents) the message family staging tables. The polling thread that is configured to service a specific message family calls the appropriate MFM GETNXT-stored procedure, which varies from one message family to another. In so doing, it retrieves the header parameters and XML message data from the message family staging table. A message family is processed by a single thread.

Multithreading feature

When parallel processing is required, due to the high volume of data to be retrieved, the connector may create multiple threads, each thread configured to handle the events of one or more message family staging tables. Each thread iterates through its configured MFMs and calls the appropriate GETNXT method to retrieve the next message.

The pollQuantity is configured at the message family level. Each MFM keeps track of the number of events that are retrieved. It returns NO_MORE_MESSAGES once the pollQuantity has been reached. If a thread runs out of events to retrieve from the staging tables it services, it will end polling and enter a wait state until the next polling cycle is initiated.

Based on the value of the BOSupportMode property (generic, message-specific, or mixed), the connector either creates a Retek generic (message-independent) business object using the retrieved XML message to populate the message attribute or it passes the retrieved XML message to the XML data handler for conversion to a message-specific business object.

In the latter case, the data handler creates a business object, modeling the Retek message structure. The business object is populated with the data contained in the XML message. The data handler returns the Retek message-specific business object to the connector.

The connector calls gotApplEvent to deliver the Retek business object to the broker (InterChange Server, WebSphere MQ Integrator Broker, or WebSphere Application Server). Each publishing application can publish messages for multiple message families. For more information about the role of the XML data handler, see the Data Handler Guide.

The following diagram summarizes application event processing:




Event status

The connector provides status information for various event processing scenarios:

For information on the GETNXT-stored procedure, see "Creating business objects". For more information on error handling, see Troubleshooting and error handling.

Processing verbs

The connector processes business objects passed to it by a broker, based on the verb for each business object. The connector uses one business object handler and its doForVerb method to process the business objects and verbs that the connector supports. The connector supports the following business object verbs:

Processing service call requests and responses

When the connector receives a service call request from an integration broker to perform an application operation, such as creating, updating, or deleting some data in an application, the request takes the form of a Retek message business object.

If the message is generic, the connector retrieves the XML payload from the supplied business object and invokes the appropriate CONSUME-stored procedure. If not, it passes the business object to its configured XML data handler to generate the XML message payload.

The business object handler builds the SQL statement for the CONSUME call, adds any other extra parameters configured in the CONSUME metaobject, and calls the CONSUME-stored procedure to perform the required operations on the Retek application. The connector sends a response to the broker, telling whether it was successful or not at processing the service call request.

Notes:

  1. Every delivered message is successfully executed exactly once; no messages are lost or processed twice.

  2. The order of messages within a particular family with a particular ID is preserved, but messages with different IDs are allowed to flow independently so that one error does not hold up the entire family.

  3. In case of failure, the connector returns an error to the broker.

  4. If the processing is successful, all database updates are committed and the message is removed from the queue. If the processing ended in error, the entire transaction is rolled back and the connector returns an error to the caller.

The following diagram summarizes service call request processing:




For more information on using the CONSUME-stored procedure, see "Creating business objects".

Copyright IBM Corp. 1997, 2003