Evolution of ETL Tools: Trends and Insights from On-Premises to Cloud Solutions

Data preparation, including extraction, transformation, and loading (ETL), is a critical yet resource-intensive process in modern data-driven systems, particularly with the increasing volume of heterogeneous, high-velocity data from AI, cloud computing, and IoT. Traditional ETL tools often struggle with performance bottlenecks and scalability issues when processing large datasets from multiple sources. This study presents a high performance, multi-threaded ETL tool designed to address these challenges by optimizing local file ingestion and enabling parallel processing. The tool integrates flexible data cleaning and transformation mechanisms, enhancing data preparation efficiency for AI models and cloud-based systems. Through comprehensive evaluation across domains like healthcare and cloud computing, this research contributes a scalable, domain-agnostic solution that streamlines data preparation and supports modern data pipelines.