A Comprehensive Review of the Main Heavyweight and Lightweight SDLC for Big Data Analytics Systems (BDAS)

Big Data Analytics Systems (BDAS) are software systems developed with descriptive, predictive, or prescriptive decision-making purposes, and currently are implemented in diverse domains – i.e. Healthcare, Finance, and Manufacturing, among others -. BDAS emerged by the joint availability of Analytics techniques and affordable sources of massive internal and external datasets. However, despite the advanced technological progress – i.e. on BDAS algorithms, languages and platforms-, their development has been mainly through uncomplete ad-hoc or old rigorous heavyweight System Development Life Cycles (SDLC). Nowadays, the business competitive environment demands lightweight and agile SDLC, and initial ones have emerged for BDAS. However, studies comparing heavyweight vs lightweight BDAS SDLC are still scarce in the literature. This research, thus, addresses this knowledge gap, and report a comparative review between the main heavyweight (KDD, SEMMA, CRISP-DM, BDPL) and emergent lightweight (ASUM, TDSP, DDSL) SDLC for BDAS, selected through a Selective Systematic Literature Review (SSLR) research method for the 2000-2023 period. For this aim, a Pro Forma of the ISO/IEC 29110 standard – Basic Profile –is used to examine their conceptual structure – i.e. roles, phases-activities, products -. This comparative review provides theoretical and practical insights for discriminating both approaches for BDAS development and calls for further conceptual and empirical research.