Extracting data
The source data is extracted through the XML ODBC driver from data services or XML data files. Data from data services is extracted through staging: small ETL builds extract the data, and store them into temporary tables from where other ETL builds pick up the data, transform the data, and map it to the operational data store area of the data warehouse. For products that support delta loads, extraction from data services is through delta loads. The structure of source data is specific to the data source. The attributes are extracted according to the measurement objectives. Therefore, not all attributes available in the data sources are loaded to the data warehouse.
Transforming data
Delivering data
A control attribute provided by Data Manager, this attribute is for a unique number for each row and must be mapped to the surrogate key column in the data warehouse table.
A control attribute provided by Data Manager, this attribute is for the date on which an existing row was updated and must be mapped to the REC_TIMESTAMP column in the data warehouse table.
This column in the data warehouse must be used to store the GUID of the data source, which can be used for differentiating data of different sources. For data sources where the data is extracted through the XML ODBC driver, a GUID is automatically assigned to each resource group and the value put in each table in column DATASOURCE_ID, which must be mapped to the SOURCE_ID column in the data warehouse table. For other data sources where the XML ODBC driver is not used, the value needs to be given manually.
An attribute to store the integer or character type of the natural key from the data source.
An attribute to store a user-visible identifier, if the data source has one.
An attribute to store the URL of an XML resource of a data source
An attribute for some commonly used artifacts such as projects, requests, requirements, tasks, activities, and components. This attribute can be used for further classifying the data in these tables. For each of these artifacts, a table with _CLASSIFICATION in the name is defined in the data warehouse and the IDs and values are predefined when the data warehouse is created. The ETL builds that deliver these artifacts into the data warehouse must specify the value of the classification ID and map to the corresponding column with _CLASS_ID in the name.