WebSphere Message Brokers
File: ac00660_
Writer: Karen Cameron

Concept topic

This build: July 31, 2007 21:16:58

Message flow aggregation

Aggregation is the generation and fan-out of related requests that are derived from a single input message, and the fan-in of the corresponding replies to produce a single aggregated reply message.

The initial request that is received by the message flow, representing a collection of related request items, is split into the appropriate number of individual requests to satisfy the subtasks of the initial request. This process is known as fan-out, and it is provided by a message flow that includes aggregation nodes.

Replies from the subtasks are combined and merged into a single reply, which is returned to the original requester (or another target application) to indicate completion of the processing. This process is known as fan-in, and it is also provided by a message flow that includes aggregation nodes.

A message aggregation is initiated by a message flow that contains the AggregateControl node followed by an AggregateRequest node. The responses are aggregated back together using a flow that contains the AggregateReply node. The aggregation nodes work correctly only for transports that use a request/reply model; for example, WebSphere MQ Enterprise Transport.

If you use WebSphere MQ Enterprise Transport, the responses that are received by the fan-in flow must be valid reply messages that contain the reply identifier. You must set the reply identifier to the value of the message in the request message's message descriptor (MQMD), then store the reply identifier in the correlation identifier field (CorrelId) of the MQMD. If the CorrelId is set to MQCI_NONE, the AggregateReply node issues an error because the reply message is not valid, and cannot be matched to a request message.

WebSphere Message Broker provides three message flow nodes that support aggregation:

When you include these nodes in your message flows, the multiple fan-out requests are issued in parallel from within a message flow. The standard operation of the message flow is for each node to perform its processing in sequence.

You can also use these aggregation nodes to issue requests to applications outside the broker environment. Messages can be sent asynchronously to external applications or services; the responses are retrieved from those applications, and the responses are combined to provide a single response to the original request message.

These nodes can help to improve response time because slow requests can be performed in parallel, and they do not need to follow each other sequentially. If the subtasks can be processed independently, and they do not need to be handled as part of a single unit of work, they can be processed by separate message flows.

You can design and configure a message flow that provides a similar function without using the aggregation nodes, by issuing the subtask requests to another application (for example, using the HTTPRequest node), and recording the results of each request in the LocalEnvironment. After each subtask has completed, merge the results from the LocalEnvironment in a Compute node, and create the combined response message for propagating to the target application. However, all the subtasks are performed sequentially, and they do not provide the performance benefits of parallel operation that you can achieve if you use the aggregation nodes.

Examples of aggregation flows that use the aggregation nodes are provided in the following samples: The Aggregation sample demonstrates a simple four-way aggregation, and the Airline Reservations sample simulates requests that are related to an airline reservation service, and illustrates the techniques that are associated with aggregation flows. You can view samples only when you use the information center that is integrated with the Message Brokers Toolkit.

In previous releases of WebSphere Message Broker, the aggregation nodes used a table in the broker database to persist aggregation requests. From WebSphere Business Integration Message Broker Version 5.0 onwards, you can use WebSphere MQ instead. The external functioning of aggregation nodes is unchanged, but you can configure an execution group to use WebSphere MQ queues for storing aggregations, instead of a database table. Using WebSphere MQ in this way improves performance and means that you can run aggregation in a non-persistent mode when persistence of aggregation requests is not required. For details of how to migrate and configure an execution group to use WebSphere MQ, see Using WebSphere MQ to store state in aggregation nodes. If you change from using a database table to using WebSphere MQ to store aggregation state, no migration of existing aggregations is performed. Therefore, you must ensure that no outstanding aggregations exist, because these aggregations are not migrated.

The aggregation nodes store state for aggregations on WebSphere MQ queues. If you do not specify a timeout on the AggregateControl node, or if you leave it set to zero, aggregation requests that WebSphere MQ stores internally are never cleaned up unless all reply messages return. This situation could lead to a gradual build up of messages on the internal queues. Set the timeout to a value greater than zero to ensure that requests are cleaned up and queues do not fill with redundant requests. It is good practice to set the timeout value to a large value, for example, 864000 seconds (24 hours), so that the queues clear old aggregation messages even if timeouts are not required or expected.

The aggregation nodes use WebSphere MQ message expiry to manage timeout of messages. For message expiry to work, the aggregation nodes must browse the message queues. The aggregation nodes browse the queues automatically to ensure that expired messages are processed.

z/OS platform On z/OS, you can configure WebSphere MQ to run a scavenger process that browses the queues instead of the aggregation nodes. To enable the scavenger, set the broker's queue manager property EXPRYINT to 5 seconds. If you do not set EXPRYINT, or set it to a value higher than 10 seconds, the aggregation nodes revert to browsing the aggregation queues automatically.

Related concepts
Message flow nodes
Related tasks
Designing a message flow
Defining message flow content
Configuring aggregation flows
Related reference
Built-in nodes
AggregateControl node
AggregateReply node
AggregateRequest node
Notices | Trademarks | Downloads | Library | Support | Feedback

Copyright IBM Corporation 1999, 2007Copyright IBM Corporation 1999, 2007. All Rights Reserved.
This build: July 31, 2007 21:16:58

ac00660_ This topic's URL is: