|
|
Deltas and Compute ChangesDeltasWarningThe Delta mechanism should be avoided if you can. Not because it does not work or is buggy, but because it introduces a new local repository in order to manage the synchronization process. The data source which is being scanned for changes becomes the master in a master-slave relationship, and it is then vital that all changes made to the slave be done such that the Delta mechanism is informed of them. Otherwise, the temporary Delta database which Integrator maintains will become inconsistent and the Delta function will fail. However, there are some situations where you will either want to integrate as non-intrusively as possible, or be forced to; For example, with a legacy application that cannot be interfaced directly. In cases like this you may be forced to read a change log generated by the target system, or even a periodic dump of all registered entries. Here is were the Delta feature can used so that Integrator can determine which Entries are new, modified or deleted. Whenever possible, use the Compute Changes functionality of the Update connector mode instead of the Delta feature. OverviewDeltas are used to synchronize a slave data source with a master. The Delta mechanism will know whether Entries or Attributes have been added, changed or deleted in the master and replicate the changes to the slave. However, if the slave is changed by anyone but the Delta mechanism, the Delta mechanism will never discover it. You can set Delta settings on Iterator Connectors . The effect of this is that the first time the Connector iterates through the data, it records each Entry in a local high-speed B-tree database. Then, on successive runs, the Connector will look up each Entry read to see if it has been changed (requiring a modify), is new (resulting in an add), or has gone missing from the input data source (signalling a delete in the slave repository). Note that this functionality could be implemented by hand by creating a separate Lookup Connector for any database of choice, and using it to check whether add, modify or delete was necessary. The Integrator Delta feature simply wraps all this into a checkbox. Process workflowThe Delta functionality uses a local B-tree database for storing all Entries handled by the Iterator Connector during its runs. During a run of the AssemblyLine each Entry is compared to its record in the local B-tree database (if it exists). Thus each Entry is tracked for existence and changes in its Attributes' values. All the Entry's Attributes are stored in the local B-tree database. One of the Entry's Attributes is selected as the unique key Attribute - this is configured in MI Admin, Connector's "Configure Delta" dialog. Note that the value of the key Attribute must be unique for each Entry. This Attribute is used to retrieve the local copy of the Entry retrieved by the Iterator Connector in order to determine its state - unchanged, new, modified or deleted. This state is set in the Entry, and the Entry is delivered to the other Connectors in the AssemblyLine. You can get an Entry's state from within your scripts through the "getOperation()" Entry method (in an Output Connector you will actually have to filter Entries according their state and this means you have to examine the value of "work.getOperation() "). Below is the sequence of actions performed by MI to determine the state of each Entry obtained from the data source: 1. The Iterator Connector retrieves the Entry from the data
source. If you want the Delta mechanism to process the Entries that have been deleted then you have to turn on the "Read Deleted" option in the Connector's "Configure Delta" dialog. With this option turned on, the following processing occurs after the Iterator Connector has read all Entries from the data source: 1.
The local B-tree database is
scanned. Another important Delta option concerning the processing of the deleted Entries is "Remove Deleted". When this option is turned on the deleted Entries, once determined, are removed from the local B-tree database. When the option is turned off, these Entries stay in the B-tree database and will be processed for deletion in further executions of the AssemblyLine. If you delete Entries in the target system, you will typically want to check "Remove Deleted" so the Entries disappear from the B-tree as well. How it worksIn the B-tree every Entry will get a counter as it is iterated through. This is an internally generated number, which is incremented for each run of the AssemblyLine. For the sake of this explanation, we'll say that it has a value of n. Next time the Iterator connects to the source and starts to iterate through the data again, one of the following three operations takes place:
ExampleA ready to run MI configuration that demonstrates the use of Deltas is included here. Note, however that this demo uses a MS Access database, so in order to run it you must have access to ODBC/Access. Computed ChangesComputed Changes option is a switch on Update Connectors. It is used to make sure an existing record has changed before the Connector tries to update it. Description of how is this option used can be found in the Connectors concept page, in its "Update mode" section.
|
|
|