Many company’s viewpoint of data integration is blinkered, they see it as something that is purely IT department related and is involved with the mythical subjects of ETL and data warehousing. They are missing, and therefore not utilising the power behind these tools to harness all the information held by the company and use it productively.

If you ignore the advantages of integration, warehousing and ETL you are paying a high price for integrating data, thereby encouraging the creation of data silos and forces the spending of an inordinate amount of time reconciling data to fill the information gaps.

Seen simply data integration involves the ability to gather information, transform it into something understandable and then present it in a way that is useful to the stakeholder. Now that is ETL in a nutshell. ETL tools allow the automation of these tasks and give developers a toolkit to enhance their hand-coding abilities. These tools, for example, include transformations for the basic tasks of converting data and performing lookups to complex processes of change data capture. These prebuilt transformations increase the productivity of the developer productivity and allow a greater consistency in the final results..

The integration is chiefly associated with batch processing (ETL) that then send the transformed data to data warehouse or some other storage based technology such as EAI, EII or SOA. The reality of working this way is that companies had to start from scratch every time they had a new data integration task. This resulted in data silos being built using different technologies, therefore producing inconsistent business intelligence.

Many vendors are now combining all the relevant technologies into data integration suites which enable companies to integrate data in a consistent manner with the appropriate transport technology.

As more powerful suites, it has evolved and now includes such initiatives as: data migration, application consolidation, business information and and product information management.

About the Author:
Mike has more than 15 years of experience designing and implementing data warehouses based on Oracle, MS SQL Server, MySql, PostgreSQL and more.
He is currently working for DB Software Laboratory

Mike Rewnick

Article Source