Message flow aggregation

Aggregation is the generation and fan-out of related requests derived from a single input message and the fan-in of the corresponding replies to produce a single aggregated reply message.

The initial request received by the message flow, representing a collection of related request items, is split into the appropriate number of individual requests to satisfy the subtasks of the initial request. This process is known as fan-out and is provided by a message flow that includes aggregation nodes.

Replies from the subtasks are combined and merged into a single reply that is returned to the original requester (or another target application) to indicate the completion of the processing. This process is known as fan-in, and is also provided by a message flow that includes aggregation nodes.

You can initiate aggregation by sending a message to a message flow from a client application that is connected to the broker across any supported protocol; the aggregated response can also be sent to a client application across all these protocols. The messages issued by the fan-out message flow, and the responses received by the fan-in message flow, must be request/reply messages. You are therefore limited to client applications that connect using the WebSphere MQ Enterprise Transport (sending and receiving messages to and from the MQInput and MQOutput nodes) or to clients that use another protocol supported by user-defined input and output nodes that conform to the request/reply communication model.

WebSphere Message Broker provides three message flow nodes that support aggregation:

When you include these nodes in your message flows, the multiple fan-out requests are issued in parallel from within a message flow. This is in contrast to the standard operation of the message flow in which each node performs its processing in sequence.

You can also use these nodes to issue requests to applications outside the broker environment; messages can be sent asynchronously to external applications or services, the responses retrieved from those applications, and the responses combined to provide a single response to the original request message.

These nodes also provide an opportunity for an improvement in response time, because slow requests can be performed in parallel and do not have to follow each other sequentially. If the subtasks can be processed independently, and do not have to be handled as part of a single unit of work, they can be processed by separate message flows.

You can design and configure a message flow that provides a similar function without using the aggregate nodes, by issuing the subtask requests to another application (for example, using the HTTPRequest node) and recording the results of each in the LocalEnvironment. After each subtask has completed, you can merge the results from the LocalEnvironment in a Compute node, and create the combined response message for propagating to the target application. If you do this, all the subtasks are performed sequentially, and do not provide the performance benefits of parallel operation that you can achieve using the aggregate nodes.

Examples of aggregation flows using the aggregate nodes are provided in the Aggregation sample and Airline Reservations sample. The Aggregation sample demonstrates a simple four-way aggregation, and the Airline Reservations sample simulates requests related to an airline reservation service, and illustrates the techniques associated with aggregation flows.

Start of changeIn previous releases of WebSphere Message Broker, the aggregation used a table in the broker database to persist aggregation requests. From WebSphere Business Integration Message Broker Version 5.0 onwards, you can use WebSphere MQ instead. The external functioning of aggregation nodes is unchanged, but you can configure an execution group to use WebSphere MQ queues for storing aggregations, instead of a database table. Using WebSphere MQ in this way improves performance and means that aggregation can be run in a non-persistent mode when persistence of aggregation requests is not required. For details of how to migrate and configure an execution group to use WebSphere MQ, see Using WebSphere MQ to store state in aggregation nodes.End of change

Start of changeWhen you change from using a database table to using WebSphere MQ to store aggregation state, there is no migration of existing aggregations so it is important to ensure that there are no outstanding aggregations, because these aggregations will not be migrated.End of change

Start of changeThe new aggregations nodes use WebSphere MQ message expiry to manage time outs of messages. For message expiry to work, the message queues must be browsed. The aggregation nodes browse the queues automatically to ensure that expired messages are processed. On z/OS, you can configure WebSphere MQ to run a scavenger process that performs this browsing instead of the aggregation nodes. To enable the scavenger, set the broker's queue manager property EXPRYINT to 5 seconds. If EXPRYINT is either not set, or set to a value higher than 10 seconds, the aggregations revert to browsing the aggregation queues automatically.End of change

Related concepts
Message flow nodes
Related tasks
Designing a message flow
Defining message flow content
Configuring aggregation flows
Related reference
Built-in nodes
AggregateControl node
AggregateReply node
AggregateRequest node