A+ Tuning: Architecture+Application Auto-Tuning for In-Memory Data-Processing Frameworks

Processing big data eventually leads to an upsurge in datacenters' power consumption, which is one of the pivotal concerns to be addressed. Many of the existing works focus on optimizing either power or performance, which is not the best parameter to consider for achieving high energy efficiency with low operational costs. Furthermore, the existing works require profiling of big data applications exhaustively and only consider tuning of either architectural or software parameters, often leading to sub-optimal settings. To cope up with the above-mentioned drawbacks of the existing works, we propose a system, A+ Tuning (Architecture + Application Auto-tuning) which enables us to determine a close to optimal settings by simultaneously optimizing for Energy Delay Product(EDP), representing energy efficiency. The proposed A+ Tuning involves a) profile the incoming unknown applications to different types (compute-bound, memory-bound and etc.) based on known applications classification result; b) co-locate the applications, and c) employs a machine learning-based model to determine the optimal settings and tune from both architectural and application settings for the co-located applications. By applying the proposed A+Tuning system, datacenters achieve up to 4×EDP improvement compared to fairshare methodology and 2.5×compared with recent works such as BestConfig.