WebSphere Message Brokers
File: ac00350_
Writer: Bill Oppenheimer

Task topic

This build: July 31, 2007 21:16:28

Optimizing message flow throughput

Each message flow that you design must provide a complete set of processing for messages received from a certain source. However, this might result in very complex message flows that include large numbers of nodes and cause a performance overhead and potential bottlenecks. Increasing the number of message flows that process your messages provides the opportunity for parallel processing and therefore improved throughput.

You can also consider the way in which the actions taken by the message flow are committed, and the order in which messages are processed.

Consider the following options for optimizing message flow throughput:

Multiple threads processing messages in a single message flow
When you deploy a message flow, the broker automatically starts an instance of the message flow for each input node that it contains. This is the default behavior. However, if you have a message flow that handles a very large number of messages, the input source (for example, a WebSphere MQ queue) might become a bottleneck.

You can update the Additional Instances property of the deployed message flow in the bar file: the broker starts additional copies of the message flow on separate threads, providing parallel processing. This is the most efficient way of handling this situation, if you are not concerned about the order in which messages are processed.

If the message flow receives messages from a WebSphere MQ queue, you can influence the order in which messages are processed to some extent by setting the Order Mode property of the MQInput node:

  • If you set Order Mode to By User ID, the node ensures that messages from a specific user (identified by the UserIdentifier field in the MQMD) are processed in guaranteed order. A second message from one user is not processed by an instance of the message flow if a previous message from this user is currently being processed by another instance of the message flow.
  • If you set Order Mode to By Queue Order, the node processes one message at a time to preserve the order in which the messages are read from the queue. Therefore, this node behaves as though you have set the Additional Instances property of the message flow to zero.

For publish/subscribe applications that communicate with the broker over any supported protocol, messages for any given topic are published by brokers in the same order as they are received from publishers (subject to reordering based on message priority, if applicable). This normally means that each subscriber receives messages from a particular broker, on a particular topic, from a particular publisher, in the order that they are published by that publisher.

However, it is possible for messages, occasionally, to be delivered out of order. This can happen, for example, if a link in the network fails and subsequent messages are routed by another link.

If you need to ensure the order in which messages are received, you can use either the SeqNum (sequence number) or PubTime (publish time stamp) parameter on the Publish command for each published message, to calculate the order of publishing.

For more information about the techniques recommended for all MQI and AMI users, see the WebSphere MQ Application Programming Guide for programs written to the MQI, and WebSphere MQ Application Messaging Interface (available as SupportPac MA0F from the WebSphere MQ SupportPacs Web page) for programs written to the AMI.

WebSphere MQ Everyplace and SCADA applications use a different method of message ordering as described in WebSphere MQ Mobile Transport and WebSphere MQ Telemetry Transport respectively. The broker does not provide message ordering for messages received across WebSphere MQ Web Services Transport, WebSphere MQ Real-time Transport, or WebSphere MQ Multicast Transport.

Multiple copies of the message flow in a broker
You can also deploy several copies of the same message flow to different execution groups in the same broker. This has similar effects to increasing the number of processing threads in a single message flow, although usually provides less noticeable gains.

This option also removes the ability to determine the order in which the messages are processed. This is because, if there is more than one copy of the message flow active in the broker, each copy can be processing a message at the same time, from the same queue. The time taken to process a message might vary, and multiple message flows accessing the same queue could therefore read messages from the input source in a random order. The order of messages produced by the message flows might not correspond to the order of the original messages.

Ensure that the applications that receive message from these message flows can tolerate out-of-order messages.

Copies of the message flow in multiple brokers
You can deploy several copies of the same message flow to different brokers. This solution requires changes to your configuration, because you must ensure that applications that supply messages to the message flow can put their messages to the right input queue or port. You can often make these changes when you deploy the message flow by setting the message flow's configurable properties.
The scope of the message flow
You might find that, in some circumstances, you can split a single message flow into several different flows to reduce the scope of work that each message flow performs. If you do this, be aware that it is not possible to execute the separate message flows in the same unit of work, and if there are transactional aspects to your message flow (for example, the updating of multiple databases), this option does not provide a suitable solution.

The following two examples show when you might want to split a message flow:

  1. In a message flow that uses RouteToLabel, the input queue has become a bottleneck. You could use another copy of the message flow in a second execution group, but this is not appropriate if you want all messages to be handled in the order in which they appear on the queue. You can consider splitting out each branch of the message flow that starts with a Label node by providing an input queue and input node for each branch. This might be appropriate, because when the message is routed by the RouteToLabel node to the relevant Label node, it has some level of independence from all other messages.

    You might also need to provide another input queue and input node to complete any common processing that the Label branches connect to when unique processing has been done.

  2. If you have a message flow that processes very large messages that take a considerable time to process, you might be able to:
    1. Create other copies of the message flow that use a different input queue (you can set this up in the message flow itself, or you can update this property when you deploy the message flow).
    2. Set up WebSphere MQ queue aliases to redirect messages from some applications to the alternative queue and message flow.

    You could also create a new message flow that replicates the function of the original message flow (but only processes large messages that are immediately passed on to it by the original message flow) that you modified to check the input message size and redirect the large messages.

The frequency of commits
If a message flow receives input messages on a WebSphere MQ queue, you can improve its throughput for some message flow scenarios by modifying its default properties after you have added it to a bar file. (These options are not available if the input messages are received by other input nodes; commits in those message flows are performed for each message.)

The following properties control the frequency with which the message flow commits transactions:

  • Commit Count. This represents the number of messages processed from the input queue before an MQCMIT is issued.
  • Commit Interval. This represents the time interval that elapses before an MQCMIT is invoked.
Related concepts
Message flows overview
Deployment overview
Related tasks
Optimizing message flow response times
Creating a message flow
Defining message flow content
Editing configurable properties
Related reference
Built-in nodes
Configurable message flow properties
Notices | Trademarks | Downloads | Library | Support | Feedback

Copyright IBM Corporation 1999, 2007Copyright IBM Corporation 1999, 2007. All Rights Reserved.
This build: July 31, 2007 21:16:28

ac00350_ This topic's URL is: