Questioning the Questions We Ask About the Impact of AI on Software Engineering : MSR 2024 Keynote

The recent advent and wide diffusion of generative AI has initiated a fundamental change in how software is developed. This technology is just one innovation along a long arc of disruptions in software engineering that include the internet, high-level programming languages, integrated development environments, open source, agile development, and social coding environments. Disruptive technologies such as these show the potential to augment and accelerate development activities along many socio-technical dimensions, while altering fundamental business processes and paradigms. Yet paradoxically, these innovations have the potential to eventually undermine the very advancements they seek to promote, rendering technologies and methods obsolete [1].When any new disruptive technology emerges, successful software companies that traditionally respond well to incremental innovations often fail when they suffer from inertia to change or don’t anticipate how people will interact with the new technology. Similarly, researchers constrained by rigid research discipline can be slow to react, and may fail to recognize important and urgent societal and industrial needs. Researchers and companies alike may struggle in knowing which metrics to use and even how to measure the impact of change, further misleading their efforts to adapt.In this talk, I question the way we select research questions in software engineering and how we study them, particularly in the face of innovations such as generative AI. To provoke a change in our research, I introduce a disruptive playbook to steer us towards broader and more novel research directions. This step-by-step playbook is first illustrated by applying it to a prior disruptive technology, Stack Overflow. I will discuss how the playbook provides a new lens for reflecting on this body of research and how doing so reveals new insights. I then use the playbook, assisted with a customized research playbook GPT, to brainstorm and frame new research directions about the emerging disruptive innovations in software engineering that are being built on top of generative AI.