Creating the aggregation fan-out flow

The aggregation fan-out flow receives the initial input message and restructures it to present a number of requests to a number of target applications.

Before you start:

To complete this task, you must have completed the following task:

You are also advised to read the overview about Message flow aggregation, before completing this task.

It is possible to include the fan-out and fan-in flow within the same message flow. However it is advisable to create two separate flows. For more information about the benefits of configuring separate message flows, see Associating fan-out and fan-in aggregation flows.

To review an example of a fan-out flow, see the Airline Reservations sample that is supplied with WebSphere Message Broker.

To create the fan-out flow:

  1. Switch to the Broker Application Development perspective.
  2. Create a new message flow to provide the fan-out processing.
  3. Add the following nodes in the editor view and configure and connect them as described:
    Input node
    The input node receives an input message from which multiple request messages are generated. This can be any one of the built-in nodes, or a user-defined input node.
    1. Right-click the input node and click Properties.
    2. Specify the source of input messages for this node. For example, specify the name of a WebSphere MQ queue in the Basic property Queue Name from which the MQInput node retrieves messages.
    3. Optional: specify values for any other properties that you want to configure for this node. It is recommended that you set the Advanced property Transaction mode, to the default, Yes, to ensure that aggregate request messages are put under syncpoint. This will avoid a situation where the AggregateReply node receives response messages before it has received the control message informing it of the aggregation instance. By putting the fan-out flow under transactional control you ensure that the fan-out flow completes before any response messages get to the AggregateReply.
    4. Connect the input node's out terminal to the in terminal of an AggregateControl node.

      This represents the simplest configuration; if appropriate, you can include other nodes between the input node and the AggregateControl node. For example, you might want to store the request for audit purposes (in a Warehouse node), or add a unique identifier to the message (in a Compute node).

    5. Optional: If your fan-out and fan-in flows are combined within one message flow, it is advisable to modify the Order Mode property on the Advanced tab. Select the By Queue Order option and ensure that the Logical Order option is also selected. This forces the input node to be single threaded so as to maintain the logical order of the messages that arrive on the queue. This results in any additional instance threads that you make available, being shared amongst only the fan-in input nodes to improve the performance of aggregation. If your fan-in and fan-out flows are in separate message flows this step is not necessary as you can make additional threads available specifically to the fan-in flow.
    AggregateControl node
    The AggregateControl node updates the LocalEnvironment associated with the input message with information required by the AggregateRequest node.
    1. Right-click the AggregateControl node and click Properties.
    2. Set the Aggregate Name property of the AggregateControl node to identify this particular aggregation. It is used later to associate this AggregateControl node with a specific AggregateReply node. The Aggregate Name that you specify must be contextually unique within a broker.
    3. Optional: set the Timeout property to specify how long the broker waits for replies to arrive before taking some action (described in Setting timeouts for aggregation).

      If a timeout is not set on the AggregateControlNode then aggregate requests stored internally will not be removed unless all aggregate reply messages return. This might lead to a gradual build up of messages on the internal queues. To avoid this situation, set the timeout to a value other than zero (zero means never timeout) so that when the timeout is reached the requests are removed and the queues will not fill up with redundant requests. Even if timeouts are not required or expected, it is good practice to set the timeout value to a large value (for example: 864000 seconds which is 24 hours) so that the queues occasionally get cleared of old aggregations.

    4. Connect the out terminal of the AggregateControl node to the in terminal of one or more Compute nodes that provide the analysis and breakdown of the request in the input message that is propagated on this terminal.
    Note: The control terminal of the AggregateControl node has been deprecated at Version 6.0 and by default any connections from this terminal to the AggregateReply node (either direct or indirect), will be ignored. This is to maximize the efficiency of aggregation flows and does not damage the reliability of aggregations. This is the optimum configuration.

    However, if you do want a control message to be sent from the AggregateControl node to the AggregateReply node, you must connect the control terminal to the corresponding AggregateReply node on the fan-in flow (either directly or indirectly, as described in Associating fan-out and fan-in aggregation flows). If you connect it indirectly to the AggregateReply node, for example through an MQOutput node, you must include a Compute node to add the appropriate headers to the message to ensure that it can be safely transmitted.

    In addition, for the Control terminal and connections from it to be recognized, you must enable the environment variable MQSI_AGGR_COMPAT_MODE. However, choosing this option has implications regarding the performance and behavior of message aggregations. For a full description of these implications and the environment variable, see Using control messages in aggregation flows.

    Compute node
    The Compute node extracts information from the input message and constructs a new output message.

    If the target applications that handle the subtask requests can extract the information that they require from the single input message, you do not need to include a Compute node to split the message. You can pass the whole input message to all target applications.

    If your target applications expect to receive an individual request, not the whole input message, you must include a Compute node to generate each individual subtask output message from the input message. Configure each Compute node in the following way, copying the appropriate subset of the input message to each output message.

    1. Right-click the Compute node and click Properties.
    2. Select a value for the Basic property Compute Mode. This property specifies which sections of the message tree are modified by the node.

      The AggregateControl node inserts elements into the LocalEnvironment tree in the input message that the AggregateRequest node reads when the message reaches it. Ensure that the LocalEnvironment is copied from the input message to the output message in the Compute node. This happens automatically unless you specify a value that includes LocalEnvironment (one of All, LocalEnvironment, LocalEnvironment and Message, or Exception and LocalEnvironment).

      If you specify one of these values, the broker assumes that you are customizing the Compute node with ESQL that writes to LocalEnvironment, and that you will copy over any elements within that tree that are required in the output message.

      If you want to modify LocalEnvironment, add the following statement to copy the required aggregate information from input message to output message:

      SET OutputLocalEnvironment.ComIbmAggregateControlNode = 
                  InputLocalEnvironment.ComIbmAggregateControlNode;
    3. Optional: specify values for any other properties that you want to configure for this node.
    4. Connect the out terminal of each Compute node to the in terminal of the output node that represents the destination of the output request message that you have created from the input message in this node.
    Output node
    Include an output node for each output message that you generate in your fan-out flow. Configure each node as described below, with the appropriate modifications for each destination.

    This must be an output node that supports the request/reply model, such as an MQOutput node, or a mixture of these nodes (depending on the requirements of the target applications).

    1. Right-click the output node and click Properties.
    2. Specify the destination for the output messages for this node. For example, specify the name of a WebSphere MQ queue in the Basic property Queue Name to which the MQOutput node sends messages. The target application must process its request, and send the response to the reply destination indicated in its input message (for example the WebSphere MQ ReplyToQueue).
    3. Click Request in the left view and set values for these properties to specify that replies are sent to the fan-in flow's input queue.
    4. Optional: specify values for any other properties that you want to configure for this node.
    5. Connect the out terminal of the output node to the in terminal of an AggregateRequest node. When the message is propagated through the output node's out terminal, the built-in output node updates the WrittenDestination folder within the associated LocalEnvironment with additional information required by the AggregateRequest node.

      The information written by the built-in nodes is queue name, queue manager name, message ID and correlation ID (from the MQMD), and message reply identifier (set to the same value as message ID).

    AggregateRequest node
    Include an AggregateRequest node for each output message that you generate in your fan-out flow.
    1. Right-click the output node and click Properties.
    2. Set the Basic property Folder Name to a value that identifies the type of request that has been sent out. This value is used by the AggregateReply node to match up with the reply message when it is received in the fan-in flow. The folder name that you specify for each request that the fan-out flow generates must be unique.

    The AggregateRequest node writes a record Start of changein WebSphere MQEnd of change for each message that it processes. This enables the AggregateReply node to identify which request each response is associated with. If your output nodes are non-transactional, the response message might arrive at the fan-in flow before this database update is committed. Refer to Setting timeouts for aggregation for details on how you can use timeouts to avoid this.

    CAUTION:
    Although the use of timeouts can help to avoid this situation described above, it is preferable to ensure that it is not possible for response messages to get to the fan-in flow before the corresponding AggregateRequest nodes have committed their database updates, by making your fan-out flow transactional.
  4. Press Ctrl-S to save the message flow and validate its configuration.
To collect the aggregation responses initiated by your fan-out flow, create you fan-in flow, see Creating the aggregation fan-in flow.
Related concepts
Message flows overview
LocalEnvironment tree structure
Message flow aggregation
User-defined input nodes
User-defined output nodes
Related tasks
Configuring aggregation flows
Creating the aggregation fan-in flow
Using control messages in aggregation flows
Associating fan-out and fan-in aggregation flows
Setting timeouts for aggregation
Using multiple AggregateControl nodes
Handling exceptions in aggregation flows
Designing a message flow
Creating a message flow
Defining message flow content
Generating multiple output messages
Developing user-defined extensions
Related reference
AggregateControl node
AggregateReply node
AggregateRequest node
Compute node
MQeInput node
MQeOutput node
MQInput node
MQOutput node