Warehouse node

Purpose

Use the Warehouse node to interact with a database in the specified ODBC data source. The Warehouse node is a specialized form of the Database node that stores the entire message, or parts of the message, or both, in a table within the database. You define what is stored by defining mappings that use the data from the input message to identify the action required.

You can use the message warehouse:

  • To maintain an audit trail of messages passing through the broker.
  • For offline or batch processing of messages that have passed through the broker (data mining).
  • As a source from which to reprocess selected messages in the broker.

You can retrieve messages that you have stored in the warehouse using standard database query and mining techniques. No explicit support is provided by WebSphere Message Broker.

You must have created (or identified, if someone else created them for you):

  • Input data in the form of a message set and message.
  • An ODBC connection to the database.
  • A database and database table to store the message.
  • At least two columns in the table: one for the binary object (the message), one for the timestamp.

The Warehouse node is represented in the workbench by the following icon:

Warehouse node icon

Using this node in a message flow

When you use the Warehouse node, you can choose to store in the database associated with the node:

  • The entire message, optionally with an associated timestamp. The message is stored as a binary object, with the timestamp in a separate column. There are two advantages to this:
    1. You do not have to decide beforehand how you will use the warehoused data; because you have stored it all, you can retrieve all the data and apply data mining tools to it later.
    2. You do not have to define a specific database schema for every type of message that might pass through the broker. In a complex system, there might be many different message types and the overhead of defining a unique schema for each message type can become prohibitive. You can precede each Warehouse node with a Compute node that converts each message into a canonical warehouse format with a common schema, or you can store the whole message as a binary object.
  • Selected parts of the message, optionally with an associated timestamp. Doing this requires a defined database schema for that message type. The message is mapped to true type so, for example, a character string in the message is stored as a character string in the database.

Configuring the Warehouse node

When you have put an instance of the Warehouse node into a message flow, you can configure it. Right-click the node in the editor view and click Properties. The node's basic properties are displayed.

All mandatory properties for which you must enter a value (those that do not have a default value defined) are marked with an asterisk on the properties dialog.

Configure the Warehouse node as follows:

  1. Specify in Data Source the name by which the appropriate database is known on the system on which this message flow is to execute. The broker connects to this database with user ID and password information that you have specified on the mqsicreatebroker, mqsichangebroker, or mqsisetdbparms command.

    On z/OS systems, the broker uses the broker started task ID, or the user ID and password that were specified on the mqsisetdbparms command JCL, BIPSDBP in the customization data set <hlq>.SBIPPROC.

  2. In Field Mapping, identify the mapping routine that is to be executed in this node. By default, the name assigned to the mapping routine is identical to the name of the mappings file in which the routine is defined, and the default name for the file is the name of the message flow concatenated with the name of the node when you include it in the message flow (for example, MFlow1_Warehouse.mfmap for the first Warehouse node in message flow MFlow1). You cannot specify a value that includes spaces.

    If you click Browse next to this entry field, a dialog is displayed that lists all available mapping routines that are accessible by this node. Select the routine that you want and click OK. The routine name is set in Field Mapping.

    To work with the mapping routine associated with this node, right-click the node and select Open Mappings. If the mapping routine does not exist, it is created for you with the default name in the default file. If the file already exists, you can also open file <flow_name>_<node_name>.mfmap in the Navigator view.

    The content of the mapping routine determines what is stored in the database, and in what format. You can choose, for example, to store all or just a part of each message. You can also choose to store the data as binary data, or to store each field in the same format as it is in the message (for example, a character field in the message is stored as character in the database).

    A mapping routine is specific to the type of node with which it is associated; you cannot use a mapping routine that you have developed for a Warehouse node with any other node that uses mappings (for example, a DataInsert node). If you create a mapping routine, you cannot call it from any other mapping routine, although you can call it from an ESQL routine.

  3. Select the Transaction setting from the drop-down menu. The values are:
    • Automatic (the default). The message flow, of which the Warehouse node is a part, is committed if it is successful. That is, the actions that you define in the mappings are taken and the message continues through the message flow. If the message flow fails, it is rolled back. Therefore choosing Automatic means that the ability to commit or roll back the action of the Warehouse node on the database depends on the success or failure of the entire message flow.
    • Commit. If you want to commit any uncommitted actions taken in this message flow on the database connected to this node, irrespective of the success or failure of the message flow as a whole, select Commit. The changes to the database are committed even if the message flow itself fails.
  4. Select Basic in the properties dialog navigator and select or clear the two check boxes:
    • If you want database warning messages to be treated as errors, and the node to propagate the output message to the failure terminal, select the Treat Warnings as Errors check box. The box is initially cleared.

      When you select the box, the node handles all positive return codes from the database as errors and generates exceptions in the same way as it does for the negative, or more serious, errors.

      If you do not select the box, the node treats warnings as normal return codes, and does not raise any exceptions. The most significant warning raised is not found, which can be handled as a normal return code safely in most circumstances.

    • If you want the broker to generate an exception when a database error is detected, select the Throw Exception on Database Errors check box. The box is initially selected.

      If you clear the box, you must handle the error in the message flow to ensure the integrity of the broker and the database: the error is ignored if you do not handle it through your own processing, because you have chosen not to invoke the default error handling by the broker. For example, you could connect the failure terminal to an error processing subroutine.

  5. Select Description in the properties dialog navigator to enter a short description, a long description, or both.
  6. Click Apply to make the changes to the Warehouse node without closing the properties dialog. Click OK to apply the changes and close the properties dialog.

    Click Cancel to close the dialog and discard all the changes that you have made to the properties.

Terminals and properties

The Terminals of the Warehouse node are described in the following table:

Terminal Description
In The input terminal that accepts a message for processing by the node.
Failure The output terminal to which the input message is propagated if a failure is detected during the computation. If you have selected Treat Warnings as Errors, the node propagates the message to this terminal even if the processing completes successfully.
Out The output terminal that outputs the message following the execution of the database statement.

The following tables describe the node properties; the column headed M indicates whether the property is mandatory (marked with an asterisk on the properties dialog if you must enter a value when no default is defined), the column headed C indicates whether the property is configurable (you can change the value when you add the message flow to the bar file to deploy it).

The Warehouse node Basic properties are described in the following table.

Property M C Default Description
Data Source No Yes   The ODBC data source name of the database that contains the tables to which you refer in the mappings associated with this node (identified by the Field Mapping property).
Field Mapping Yes No Warehouse The name of the mapping routine that contains the statements that are to be executed against the database or the message tree. The routine is unique to this type of node.
Transaction Yes No Automatic The transaction mode for the node. This can be Automatic or Commit.
Treat Warnings as Errors Yes No Cleared Treat database SQL warnings as errors. If you select the check box, the action is performed.
Throw Exception on Database Error Yes No Selected Database errors cause the broker to throw an exception. If you select the check box, the action is performed.

The Description properties of the Warehouse node are described in the following table.

Property M C Default Description
Short Description No No   A brief description of the node.
Long Description No No   Text that describes the purpose of the node in the message flow.