The delta load process extracts only that data which has changed
since the last time a build was run. The delta load process is used for extracting
data for the operational data store of IBM® Rational® Insight
data warehouse.
This topic is an overview of the delta load implementation.
To run the delta load process, you need to store the date and time of the
last successful build of the extract, transform, and load (ETL) process. The
CONFIG.ETL_INFO table
in the data warehouse is defined for this purpose. Every time an ETL job is
run, some variables are initialized. For the delta load process, the following
two variables are used:
- The MODIFIED_SINCE variable.
The ETL job searches
the CONFIG.ETL_INFO table to get date and time for the
last successful ETL run and sets that value to the MODIFIED_SINCE variable,
which will be used in the later ETL builds later to determine if there are
changes to the data since the last run.
- The ETL_START_TIME variable
The ETL job gets the
system date and time and stores that value to the ETL_START_TIME variable.
After the ETL job is over, the value stored in this variable is used for updating
the CONFIG.ETL_INFO table.
Whether the delta load process works for a specific product or not depends
upon the data service through which the product data is extracted.