This topic describes the FileInput node.
Use the FileInput node to receive messages from clients that connect to the broker by creating files in a directory in the broker server's file system.
The node removes the file from the directory when the flow successfully completes.
The FileInput node looks in the directory for files matching a pattern in a process known as polling. It then processes all matching files in the directory except those files that are locked by another process or broker flow. The node then waits for a period of time, called the polling interval, before repeating processing any new files that arrive in the directory. In normal use, the node processes files faster than they arrive, so execution consists of repeatedly waiting for the polling interval to expire and then processing all the files that have arrived in the directory. Locked files remain unprocessed in the directory until the lock is released. When the node next processes the files in the directory, it includes the files now unlocked.
The FileInput node processes matching files in sequence according to the file system time stamp for when the file was last modified, oldest first. Do not code flows that rely on this sequence because, for example, files may arrive while one or more flow instances are already processing files and your sequence will become out of step.
After successfully processing a file's contents, the FileInput node can delete the input file, or move it to the archive subdirectory in the same file system. If the flow fails, the node can move the file to a back out subdirectory defined in the same file system, move the file to the archive subdirectory, or delete the file.
If you include an output node in a message flow that starts with a FileInput node, it can be any of the supported output nodes, including user-defined output nodes; you do not need to include a FileOutput node. You can create a message flow that receives messages from files and generates messages for clients that use any of the supported transports to connect to the broker because you can configure the message flow to request that the broker provides any conversion that is necessary.
When the FileInput node propagates a message, it stores the information in the LocalEnvironment.File message tree. If the input file is empty, the message is empty (assuming it passes any validation), and the Size element is zero. The following table lists the LocalEnvironment.File message tree structure.
Element Name | Element Data Type | Element Attributes |
---|---|---|
Directory | CHARACTER | Absolute directory path. Type the path separator character ("/" or "\") according to the operating system on which your flow is executing. The final path separator character is suppressed. On Windows®, start the directory path with the drive letter prefix, for example, "C:". |
Name | CHARACTER | File name and extension. |
Size | INTEGER | Size of the file in bytes. |
IsDirectory | BOOLEAN | Reserved for future use. Always False. |
LastModified | TIMESTAMP | Date and time the file was last modified. |
TimeStamp | CHARACTER | Date and time the input node started processing the file in the coordinated universal time (UTC) zone, as a character string because this is the formatted form used to create archive and backout file names. This is used to timestamp the input file if you configure the Add Timestamp and Move to Archive Subdirectory and Add Timestamp and Move to Backout Subdirectory properties. |
The following elements contain data about the detected record if present: | ||
Offset | INTEGER | Start of the record with the file. The first record starts at offset 0. |
Record | INTEGER | Number of the record within the file. The first record is record number 1. |
Delimiter | CHARACTER | The characters to be used to separate the record from the subsequent record, if Delimited is specified in Record Detection. The delimiter is either the default of a line end sequence, or a character string that you have specified in Custom delimiter. |
The FileInput node is contained in the File drawer of the message flow node palette, and is represented in the workbench by the following icon:
You can view samples only when you use the information center that is integrated with the Message Brokers Toolkit.
When you have put an instance of the FileInput node into a message flow, you can configure it. The properties of the node are displayed in the Properties view. All mandatory properties for which you must enter a value (those that do not have a default value defined) are marked with an asterisk.
Configure the FileInput node:
Wildcard character | Description | Example |
---|---|---|
* | Indicates any sequence of zero or more characters | *.xml matches all files with an xml extension |
? | Indicates any single character | f??????.csv matches all files with the letter f followed by six characters and a .csv extension |
A lower polling interval limit can be set on a message broker for this value. If the value on the node is below the lower limit set on the message broker, the value is raised to the broker limit. Set an execution group property on the broker with the following values:
The FileInput node will scan the input directory when the flow starts and thereafter according to the above interval.
The value set for the polling interval here can be overridden by the Scan delay property on the FTP tab.
The FileInput node considers each occurrence of the delimiter in the input file as the start of a new record. If the first record of the file begins with a delimiter, the node treats the missing file contents preceding that delimiter as a record and propagates an empty record. The delimiter is never included in the propagated message.
The default is $Body denoting the whole custom-parsed message. This is equivalent to specifying Whole File on Record Detection.
The FileInput node terminals are described in the following table.
Terminal | Description |
---|---|
Failure | The output terminal to which a message is routed if a failure is detected when the message is propagated to the Out flow (such as a message validation failure). Failures routed to this terminal include those caused by the retry processing occurring before the retry propagates the message to the Out flow. |
Out | The output terminal to which the message is routed if it has been successfully put to the output queue, and if further processing is required within this message flow. If no errors occur within the input node, a message received from an external resource is always sent to the Out terminal first. |
Catch | The output terminal to which the message is routed if an exception is thrown downstream and caught by this node. |
The following tables describe the node properties. The column headed M indicates whether the property is mandatory (marked with an asterisk if you must enter a value when no default is defined); the column headed C indicates whether the property is configurable (you can change the value when you add the message flow to the bar file to deploy it).
The FileInput node Description properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Node name | No | No | The node type | The name of the node. |
Short Description | No | No | None | A brief description of the node. |
Long Description | No | No | None | Text that describes the purpose of the node in the message flow. |
The FileInput node Basic properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Input directory | Yes | Yes | None | The full directory path scanned on each polling interval. |
File name or pattern | Yes | Yes | * | A file name or simple string containing optional wildcard characters (* and ?) identifying files to process from the input directory. |
Action on successful processing | Yes | No | Move to archive subdirectory | The action the node takes on the file after successfully processing the contents. |
Replace duplicate archive files | Yes | No | False | This property controls whether the node replaces existing archive files with the same name as the input file. |
The FileInput node Polling property is described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Polling Interval | Yes | Yes | 5 | The polling interval in seconds |
The FileInput node Retry properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Retry mode | Yes | No | Failure | How the node handles a flow failure. |
Short retry count | Yes | Yes | 0 | The number of retries. |
Short retry interval | No | Yes | 0 | The interval in seconds between each retry if the short retry count is not zero. |
Long retry interval | No | Yes | 300 | The interval between each retry, if the mode is Short retry then long retry and the short retry count is exhausted. |
Action on failing file | Yes | Ye | Move to back-out subdirectory | The action the node takes with the input file after all attempts to process its contents fail. |
The FileInput node Records and Elements properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Record detection | Yes | No | Whole file | The mechanism used to identify records in the input file. |
Length | Yes | No | 80 | The length of records when Fixed length record detection is selected. |
Delimiter | Yes | No | Line end | Record separator bytes when Delimited record detection is selected. |
Custom delimiter | No | No | The delimiter in hexadecimal when Delimited record detection is selected. | |
Delimiter type | Yes | No | Postfix | The position of the delimiter when Delimited record detection is selected. |
Propagated record element | No | No | $Body | The element to propagate as a record when Parse Embedded Records record detection is selected. |
The FileInput node Validation properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Validate | Yes | Yes | None | This property controls whether validation takes place. Valid values are None, Content and Value, or Content. |
Failure action | Yes | No | Exception | This property controls what happens if validation fails. You can set this property only if you set Validate to Content or Content and Value. Valid values are User Trace, Local Error Log, Exception, and Exception List. |
The FileInput node FTP properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
FTP | No | No | Cleared | This property defines whether the node uses the File Transfer Protocol (FTP) properties listed below and reads files from an FTP server. |
Server identification | Yes | Yes | None | The name of the configurable service in which the FTP server connection is configured, or the IP address of remote FTP server if no such resource. |
Client user identification | No | Yes | None | The name of the user identification used to access the FTP server. If left blank, the system uses the identification defined in the server identification configurable service. |
Client user password | No | Yes | None | Password for the client user identification. If left blank, the system uses the password associated with the client user identification. |
Server directory | No | Yes | "." | The directory in the FTP server. |
Transfer mode | No | Yes | Binary | The mode for transfer of files. Select Binary or Text. |
Scan delay | No | Yes | 60 | The time, in seconds, to wait before repeating the scan of the directory. |
The FileInput node Transactions properties are described in the following table:
Property | M | C | Default | Description |
---|---|---|---|---|
Transaction mode | No | Yes | Yes | This property defines how the node controls transactions. When you include a FileInput node in a message flow, the value that you set for Transaction mode defines whether messages are sent under sync point. The FileInput node is not transactional, and does not itself process files under sync point. If you set this property to Yes (the default), the message is received under sync point (that is, within a WebSphere® MQ unit of work). Any messages subsequently sent by an output node in the same instance of the message flow are put under sync point, unless the output node has explicitly overridden this. If the File Input node backs-out the transaction, all changes made under sync point are backed-out. If you set this property to No, the message is not received under sync point. Any messages subsequently sent by an output node in the flow are not put under sync point, unless an individual output node has specified otherwise. Changes made under sync point can be backed-out if the FileInput node backs-out the transaction. When FTP is enabled, the file transfer is performed outside of sync point, and no attempt is made to restore any file or files that have been transferred, whether the message flow is successful or not. |
Property | M | C | Default | Description |
---|---|---|---|---|
Additional instances pool | No | Yes | Use Pool Associated with Message Flow | The pool from which additional instances are obtained.
|
Additional instances | No | Yes | 0 | The number of additional instances that the node can start if the Additional instances pool property is set to Use Pool Associated with Node. By default, no additional instances are given to the node. |