Coding ESQL to handle errors

Introduction

When processing messages in message flows, errors can be due to:
  1. External causes. For example, the incoming message is syntactically invalid, a database used by the flow has been shut down, or the power supply to the machine on which the broker is running fails.
  2. Internal causes. For example, an attempt to insert a row into a database table fails because of a constraint check, or a character string read from a database cannot be converted to a number because it contains alphabetic characters.

    Internal errors can be caused by programs storing invalid data in the database or by a flaw in the logic of a flow.

The message flow designer must give errors serious consideration and decide how they are to be handled.

Using default error-handling

The simplest strategy for handling ESQL errors is to do nothing and use the broker's default behavior. The default behavior is to cut short the processing of the failing message and to proceed to the next message. Input and output nodes provide options to control exactly what happens when processing is cut short.

If the input and output nodes are set to transactional mode, the broker restores the state prior to the message being processed:
  1. The input message that has apparently been taken from the input queue is put back.
  2. Any output messages that the flow has apparently written to output queues are discarded.
If the input and output nodes are not set to transactional mode:
  1. The input message taken from the input queue is not put back.
  2. Any output messages that the flow has written to output queues remain there.

Each of these strategies has its advantages. The transactional model preserves the consistency of data, while the non-transactional model maximizes the continuity of message processing. Remember that in the transactional model the failing input message is put back on to the input queue and the broker will attempt to process it again. The most likely outcome of this is that the message continues to fail until the retry limit is reached, at which point it is placed on a dead letter queue. The reason for the failure to process the message is logged to the system event log (Windows) or syslog (UNIX). Thus the failing message holds up the processing of subsequent good messages for a while and is then left unprocessed by the broker.

Most databases operate transactionally, so that all changes made to database tables are committed if the processing of the message succeeds and rolled back if it fails, thus maintaining the integrity of data. An exception to this is if the broker itself, or a database, fails. (For example, the power to the machines they are running on could be interrupted.) It is possible in these cases for changes in some databases to be committed and changes in others not; or for the database changes to be committed but the input and output messages not to be committed. If these possibilities concern you, the flow should be made coordinated and the databases involved configured for this way of working.

Using customized error handling

Here are some general tips about creating customized error handlers:
  1. If you require something better than default error handling, the first step is to use a handler (see DECLARE HANDLER statement). Create one handler per node, to intercept all possible exceptions (or as many exceptions as can be foreseen).
  2. Having intercepted an error, the error handler can use whatever logic is appropriate to handle it. Alternatively, it can use a THROW statement or node to create an exception, which could be handled higher in the flow logic or even reach the input node, causing the transaction to be rolled back. See Throwing an exception.
  3. If a node throws an exception that is not caught by the handler, the flow is diverted to the failure terminal, if one is attached, or handled by default error-handling if not.

    Use failure terminals to catch unhandled errors. Attach a simple logic flow to the failure terminal. This logic flow could consist of a database or compute node that writes a log record to a database (possibly including the message's bit-stream) or writes a record to the event log. It could also contain an output node that writes the message to a special queue.

    The full exception tree is passed to any node connected to a failure terminal. (The exception tree is described in ExceptionList tree structure.)

  4. Your error handlers are responsible for logging each error in an appropriate place, such as the system event log.

For a detailed discussion of the options that you can use to process errors in a message flow, see Handling errors in message flows. The following topics provide examples of what you can do:

Coding to detect errors

The following sections assume that it is the broker that detects the error. It is quite possible, however, for the logic of the flow to detect an error. For example, when coding the flow logic you could use:
  • IF statements inserted specifically to detect situations that should not occur
  • The ELSE clause of a case expression or statement to trap routes through the code that should not be possible
As an example of a flow logic-detected error, consider a field that has a range of possible integer values indicating the type of message. In such a case it would not be good practice to leave to chance what would happen if a message were to arrive in which the field's value did not correspond to any known type of message. One way this could happen is if the system is upgraded to support extra types of message but one part of the system is not upgraded.

Using your own logic to handle input messages that are not valid

Input messages that are syntactically invalid (and input messages that appear to be not valid because of erroneous message format information) are difficult to deal with because the broker has no idea what the message contains. Probably the best way of dealing with them is to configure the input node to fully parse and validate the message.

Note, however, that this applies only to predefined messages, that is, MRM or IDOC.

If the input node is configured in this way, the following is guaranteed if the input message cannot be parsed successfully:
  • The input message never emerges from the node's normal output terminal (it goes to the failure terminal).
  • The input message never enters the main part of the message flow.
  • The input message never causes any database updates.
  • No messages are written to any output queues.

To deal with a failing message, connect a simple logic flow to the failure terminal.

The only disadvantage to this strategy is that, if the normal flow does not require access to all the message's fields, the forcing of complete parsing of the message affects performance.

Using your own logic to handle database errors

Database errors fall into three categories:
  1. The database isn't working at all (for example, it's off line).
  2. The database is working but refuses your request (for example, a lock contention occurs).
  3. The database is working but what you ask it to do is impossible (for example, to read from a non-existent table).

If you require something better than default error handling, the first step is to use a handler (see DECLARE HANDLER statement) to intercept the exception. The handler can determine the nature of the failure from the SQL state returned by the database.

Database not working
If a database isn't working at all and is essential to the processing of messages, there is probably not much you can do. In this case the handler, having determined the cause, might do any of the following:
  • Use the RESIGNAL statement to re-throw the original error, thus allowing the default error handler to take over.
  • Use a different database.
  • Write the message to a special output queue.

    However, take care with this sort of strategy. Because the handler absorbs the exception, any changes to other databases or writes to queues are committed.

Database refuses your request
The situation when a lock contention occurs is similar to the "Database not working" case. This is because the database will have backed out all the database changes you have made for the current message, not just the failing request. Therefore, unless you are sure this was the only update, it is unlikely that there is any better strategy than default error-handling, except possibly logging the error or passing the message to a special queue.
Impossible requests
The case where the database is working but what you ask it to do is impossible covers a wide variety of problems.
If, as in the example, the database simply doesn't have a table of the name the flow expects, it is again unlikely that there is any better strategy than default error-handling, except possibly logging the error or passing the message to a special queue.
Many other errors may be handled successfully, however. For example, an attempt to insert a row might fail because there is already such a row and the new row would be a duplicate. Or an attempt to update a row might fail because there is no such row (that is, the update updated zero rows). In these cases, the handler can incorporate whatever logic you think fit. It might insert the missing row or utilize the existing one (possibly making sure the values in it are suitable).
Note: For an update of zero rows to be reported as an error the node property "treat warnings as errors" must be set to true. This is not the default setting.

Using your own logic to handle errors in output nodes

Start of changeErrors occurring in MQ output nodes report the nature of the error in the SQL state and give additional information in the SQL native error variable. Thus, if something better than default error handling is required, the first step is to use a handler (see DECLARE HANDLER statement) to intercept the exception. Such a handler typically surrounds only a single PROPAGATE statement. End of change

Using your own logic to handle other errors

Besides those covered above, a variety of other errors can occur. For example, an arithmetic calculation might overflow, a cast might fail because of the unsuitability of the data, or an access to a message field might fail because of a type constraint. The broker offers two programming strategies for dealing with these.
  1. The error causes an exception that is either handled or left to roll back the transaction.
  2. The failure is recorded as a special value that is tested for later.

In the absence of a type constraint, an attempt to access a non-existent message field results in the value null. Null values propagate through expressions, making the result null. Thus, if an expression, however complex, does not return a null value, you know that all the values it needed to calculate its result were not null.

Cast expressions can have a default clause. If there is a default clause, casts fail quietly; instead of throwing an exception, they simply return the default value. The default value could be an innocuous number (for example, zero for an integer), or a value that is clearly invalid in the context (for example, -1 for a customer number). Null might be particularly suitable, because it is a value that is different from all others and that will propagate through expressions without any possibility of the error condition being masked.

Handling errors in other nodes

Exceptions occurring in other nodes (that is, downstream of a PROPAGATE statement) might be caught by handlers in the normal way. Handling such errors intelligently, however, poses the special problem that, as another node was involved in the original error, another node, and not necessarily the originator of the exception, is very likely to be involved in handling it.

To help in these situations the database and compute nodes have four new terminals called out1, out2, out3, and out4. In addition the syntax of the PROPAGATE statement has been extended to include target expression, message source and control clauses to give more control over these extra terminals.