Designing ML-based Approximate Query Processing Services on Time-Varying Large Dataset for Distributed Systems

Approximate query processing (AQP) has been well established for big data analytics to complement performance degradation due to the ever-increasing size of datasets. The evolution of machine learning technologies creates another opportunity for improving the AQP. There are two architectural aspects that should be considered for AQP-based data analytics services to embrace the trends. First, the services should support rapidly changing data. Second, the systems should manage the life-cycle of the machine learning process and accommodate the diversity of ML technologies. This paper discusses the requirements and design considerations for ML-based AQP services on time-varying large datasets for distributed environments in a system-independent way.