Generalized Big Data Test Framework for ETL migration

ETL (Extract, Transform and Loading) is a data warehousing process that migrate the data from source database by performing certain transformation rules over the extracted data. This transformed data is loaded back to the target database. With emergence of Big Data various organization are moving toward big data technology like Hadoop, Hadoop Ecosystem Projects like Hive and HBase to store their data. Organization uses data migration process to migrate data from RDBMS to Hadoop and further these data is used for various analysis purpose. But data migration task some time causes discrepancy in data due to various reasons which may lead to inaccurate data analysis. This paper talks about the generalize data validation framework between RDBMS and Hadoop.