*
Metamerge logo
Search

Advanced Search
*
*
*
* HOME DOCUMENTS & RESOURCES DOWNLOADS EARLY TECH ACCESS SUPPORT FAQ KNOWN ISSUES OLD VERSIONS
*

Connectors

Connectors are used to access and update information sources in a variety of formats and protocols. There are basically two types of connectors. The first is where both transport and content is known to the connector using a well known API (like ODBC, LDAP). The second type is where the transport mechanism is known but not the content. The latter requires a Parser to interpret or generate the content in order to function properly.

A list of all Connectors included with the Metamerge Integrator is included here. But you can also write your own Connector, see an example here.

Connectors operate in the following modes: Iterator, Lookup, AddOnly, Update, Delete and Passive. Each Connector mode determines a specific Connector's behavior. Note however, that it is not necessary that every Connector support all of the modes available. When you use a Connector you should first consult its manual for the modes supported.

How do Connectors share data?

Connectors not in Iterator mode have their data passed through the work object.  If no work object exist because you have no Iterator, the non-iterator Connectors will not be called. You can create your own Working Entry in the Prolog by:

init_work = system.newEntry(); // Create a new Entry object
init_work.setAttribute("uid", "cchateauvieux"); // populate it
task.setWork(init_work); // make it known as work to the Connectors

Iterator mode

Connectors in Iterator mode are used to scan a data source and extract its data. The Iterator Connector actually iterates through the data source entries, reads their attributes' values and delivers each entry to the AssemblyLine and its non-Iterator connectors. Note that it does not matter exactly what the data source is (database, LDAP directory, XML document, etc.) and how its data is actually stored. Each Connector presents an abstract layer over the particular data source and you will access and process data through instances of the Entry and Attribute classes.

In fact, each AssemblyLine (except those called with an initial Working Entry) should contain at least one Connector in Iterator mode. Iterators (if we call the Connectors in Iterator mode this way) supply the AssemblyLine with data. If an AssemblyLine does not have an Iterator it will be useless because its Connectors will be passed nothing to process.

Using the Iterator mode

The most common pattern for using a Connector in Iterator mode should be:

1. Add a Connector in Iterator mode to your AssemblyLine.
2. Choose the "configure..." link from MI Admin user interface.
    2.1. Set values for the Connector's parameters (usually specifying the data source to be used).
    2.2. Go to the "Attributes" section (tab) of the "Configure Connector" dialog.
        2.2.1. Push the "Connect" button to connect to the data source.
        2.2.2. Push the "Query Schema" button (or the "Next" button if "Query Schema" is not supported) to retrieve an Entry's Attributes.
3. In the "Attribute Map" Connector's section push the "Select" button and mark all Attributes you will need in your integration process.
These Attributes will be retrieved from the data source and passed to the non-Iterator Connectors in your AssemblyLine.

Lookup mode

Connectors in Lookup mode are used to retrieve particular Entry's attributes from a data source by other (most often key) Entry's attributes (already retrieved from another data source or set by the Administrator). Lookup Connectors give the possibility to join data from different data sources using the relations among the entities from these data sources.

The main action in setting up and using a Lookup Connector is the configuration of its "Link Criteria". It is always configured through the MI Admin user interface - each Lookup Connector has an associated "Link Criteria" section. In this section the rule for finding entries is described.  The Link Criteria rule is itself a conjunction of simple conditions required for certain Entry's attributes.

Note that the "Link Criteria" may identify more than one Entry (these Entries are called Duplicates). Lookup Connectors have an option that allows/disallows Duplicates. You can control this option from the "Allow Duplicates" check-box available.

o If "Allow Duplicates" is turned off (default state) and Duplicates appear then:
    - if the "Multiple Entries Found" event is not enabled the work Entry being processed is skipped (i.e. not passed to the other Connectors in the AssemblyLine).
    - if the "Multiple Entries Found" event is enabled, the custom script placed there should handle the situation.
o If "Allow Duplicates" is turned on and Duplicates appear then automatically the first of the Duplicates is used. If no additional processing is performed in any of the events available the rest of the Duplicate Entries are ignored. However they can  be handled according user-defined logic by scripting in the "Lookup OK" event.

Example package with ready to run MI configuration that handles Duplicates is included here.

Using the Lookup mode

The most common pattern for using a Connector in Lookup mode should be:

1. Add a Connector in Lookup mode to your AssemblyLine.
2. Choose the "configure..." link from MI Admin user interface.
    2.1. Set values for Connector's parameters (usually specifying the data source to look up in).
    2.2. Go to the "Attributes" section (tab) of the "Configure Connector" dialog.
        2.2.1. Push the "Connect" button to connect to the data source.
        2.2.2. Push the "Query Schema" button (or the "Next" button if "Query Schema" is not supported) to retrieve Entry's Attributes.
3. In the "Attribute Map" Connector's section (tab) push the "Select" button and mark all Attributes you will need in your integration process (these include the Attributes used for looking up).
4. In the "Link Criteria" Connector's section (tab) add all criterions for looking up.
All new Attributes added from the Lookup Connector along with the Attributes added from previous Iterator and Lookup Connectors will be available for the other Connectors in the AssemblyLine.

AddOnly mode

Connectors in AddOnly mode are used for adding new data in a data source. This Connector mode requires almost no configuration - just point out the attributes of the Entry and the data source parameters. The AddOnly Connector receives Entries (most often from an Iterator) and writes them in its data source.

Using the AddOnly mode

The most common and simple pattern for using a Connector in AddOnly mode should be:

1. Add a Connector in AddOnly mode to your AssemblyLine.
2. Choose the "configure..." link from MI Admin user interface.
    2.1. Set values for Connector's parameters (usually specifying the output data source to be used).
3. In the "Attribute Map" Connector's section push the "Select" button and mark all working Attributes you will need in your integration process. You can also create your own Attributes and calculate their values through scripting.
Entries with the Attributes you have selected will be added in the data source during AssemblyLine's execution.

Update mode

Connectors in Update mode are used for adding and modifying data in a data source. For each Entry passed from the AssemblyLine, the Update Connector tries to locate an Entry from the data source to modify with the Entry's attributes values received. The rule for finding the Entry for modification is configured in the "Link Criteria" section in the same format as in the Lookup mode. If no such Entry is found, a new Entry is added to the data source. If one such Entry is found, it is modified. If more than one entry matches the "Link Criteria",  the "Multiple Entries Found" hook is called. Furthermore, each attribute can be tuned whether to be modified or added, when respectively Modify or Add operation is performed.

An important feature that Update Connectors offer is the "Compute Changes" option. When turned on, the Connector first checks the new values against the old ones and updates only if and where needed. Thus you can skip unnecessary updates which can be really valuable if the update operation is a heavy one for the particular data source you are updating.

Using the Update mode

The most common and simple pattern for using a Connector in Update mode should be:

1. Add a Connector in Update mode to your AssemblyLine.
2. Choose the "configure..." link from MI Admin user interface.
    2.1. Set values for Connector's parameters (usually specifying the output data source to be used).
    2.2. Go to the "Attributes" section (tab) of the "Configure Connector" dialog.
        2.2.1. Push the "Connect" button to connect to the data source.
        2.2.2. Push the "Query Schema" button (or the "Next" button if "Query Schema" is not supported) to retrieve Entry's Attributes.
3. In the "Attribute Map" Connector's section push the "Select" button and mark all Attributes you will need to update in your integration process. Through the "Add" and "Modify" check-boxes you can specify for each Attribute whether it should be modified or added when respectively Modify or Add operation is performed.
4. In the "Link Criteria" Connector's section (tab) add all criteria for fixing the Entry to be updated (or finding out that it is missing and has to be added).
Entries' Attributes you have selected will be updated in the data source during AssemblyLine's execution.

Delete mode

Connectors in Delete mode are used for deleting data from a data source. For each Entry passed from the AssemblyLine, the Delete Connector tries to locate an Entry from the data source. If one Entry is found, it is deleted, otherwise nothing happens. The rules for finding the Entry for deletion is configured in the "Link Criteria" section in the same format as in the Lookup mode.

Using the Delete mode

The most common and simple pattern for using a Connector in Delete mode should be:

1. Add a Connector in Delete mode to your AssemblyLine.
2. Choose the "configure..." link from MI Admin user interface.
    2.1. Set values for Connector's parameters (usually specifying the output data source to be used).
3. In the "Link Criteria" Connector's section (tab) add all criteria for locating the Entry to be deleted. If exactly one Entry match the "Link Criteria", it will be deleted.
4. The "Attribute Map"  section is ignored for a Connector in Delete mode.

Passive mode

Connectors in Passive mode are not passed Entries during the AssemblyLine execution. They are configured by the Administrator and initialised by the AssemblyLine, but never used in the standard AssemblyLine's execution process. They can be invoked by script code from any of the control points for scripting provided by Metamerge Integrator (see Scripting concept).

Using the Passive mode

The most common and simple pattern for using a Connector in Passive mode should be:

1. Add a Connector in Passive mode anywhere in your AssemblyLine.
2. Set values for the Connector's parameters (usually specifying the output data source to be used). The name of the Connector is used to refer to it from other Connectors.
3. Add scripting code to use the Connector, usually in Hooks in other Connectors. If you have named the Passive Connector MyPassive, the code may look like this:

work.setAttribute ( "ERROR", "Failed to lookup" );
MyPassive.connector.putEntry ( work ) ;

The instructions above assume that you only want to use the Raw Connector and Parser components of the Passive mode Connector. If you also want to use the Attribute Mapping and Hooks that the Connector provides, you should do this instead of the first item in the instructions above.

1. Add the Connector in the mode that matches the way you are going to use it,.
2. Specify the Attribute Mapping and the Hooks you want.
3. Change the mode of the Connector to Passive. The admin tool does not show the Hooks anymore, but they are still present
4. You may now use the calls available to the AssemblyLine Connector, e.g. MyPassive.add(work).

Example

Demo package containing ready to run configurations that demonstrate each of the above Connector modes through the JDBC Connector is included here.

 

*
  Metamerge Integrator version 4.5 ©Copyright Metamerge AS 2000-2002 Last edited 2002-04-30 contact us