AI Governance in the System Development Life Cycle: Insights on Responsible Machine Learning Engineering

In this study we explore the incorporation of artificial intelligence (AI) governance to system development life cycle (SDLC) models. We conducted expert interviews among AI and SDLC professionals and analyzed the interview data using qualitative coding and clustering to extract AI governance concepts. Subsequently, we mapped these concepts onto three stages in the machine learning (ML) system development process: (1) design, (2) development, and (3) operation. We discovered 20 governance concepts, some of which are relevant to more than one of the three stages. Our analysis highlights AI governance as a complex process that involves multiple activities and stakeholders. As development projects are unique, the governance requirements and processes also vary. This study is a step towards understanding how AI governance is conceptually connected to ML systems’ management processes through the project life cycle. CCS CONCEPTS • Software and its engineering $^{\rightarrow}$ Software creation and management.