Is ETL solution for all problem!!

Many time I come across a situation where management team want to nail down the issue by using a key world "ETL solution". For them ETL is solution for all data transfer performance, data quality etc etc. I hope one day this will be possible to solve most of issue with one single solution.

ETL is not the solution for all Data integration. Yes ETL tool can help and enable data integration. But the business community has to take the ownership to define the business rule and provide regress data cleanising and data integration rules. And is not the end. They have to keep updating the business rule and keep them self on the top of any ERROR which are posted in the process.

ERROR from the data integration create major challenge to IT and business community. Most of time, there is no specific owner assigned to resolve the ERROR. Each group try to push the bug on other side.

ERROR handling ( correcting ) is one of the most in important component to make successful data integration. In the process, we should try to get managable set of error. Means we should not create ERRORs which is almost unrealistic to correct or manage. This again boils down to effective business rule definition. If at initial test we accept huge number of ERROR records, then data has to cleansed before bringing to Data Integration process. There are many tool available in market to do data profiling and data cleanising.

Next topic we will look at different ERROR handling approach.