By Topic

Incremental data feed maintenance of a data warehouse system derived from multiple autonomous data sources

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

7 Author(s)
Wei Xu ; Dept. of Autom., Xiamen Univ., China ; Maoqing Li ; Shunxiang Wu ; Shunzhi Zhu
more authors

The data acquisition process, in which the data warehouse and operational data store (ODS) are populated from operational sources, represents the most technically challenging part of any business intelligence (BI) environment. Some industry experts estimate that 60 to 80 percent of a Bl project's effort is spent on this process alone. Nevertheless, most of the previous development work is to trigger visual GUIs tools such as Informtica manually, enter properties and drive ETL process. However, the purpose of this article is to introduce a better on-demand means to pull data from modern heterogeneous data sources with the integration of Informatica, Oracle and Korn shell script. We introduced a practical production instance on how to accomplish an efficient, scalable, controllable and maintainable ETL (extract, transform, loading) architecture. Within this infrastructure, we adopt two new techniques: process synchronization control (PSC) and time range control (TRC).

Published in:

2005 International Conference on Control and Automation  (Volume:2 )

Date of Conference:

26-29 June 2005