Comparison of a FaaS- and SaaS-Based ETL Process in a Cloud Environment

The goal of this paper is to analyze experiences from a FaaS-based (function as a service) cloud-native implementation of an ETL process (extract, transform, load) based on AWS Lambda functions. The actual implementation is outlined and the experiences from that implementation are evaluated. This results in an overview of pros and cons of a cloud-native implementation in general and determines best practices for other implementations in other cloud environments or business domains.

I. Introduction

Traditional ETL (extract, transform, load) processes and their implementations stem from data warehouse approaches. Data are extracted from one or several source systems and then transformed into either an intermediate format (e.g. Parquet [13]) or into the format of the target system. After this transformation, when the data are in the adequate format, they are loaded into the target system. Still, current data ware-houses [20] [29] [7] offer possibilities to implement such ETL processes. But, although these implementations also include cloud-based capabilities, they often result in big monoliths.