Towards standardized benchmarks for Dynamic Software Updating systems

Dynamic Software Updating (DSU) has been an active topic of research for at least the last 30 years. However, despite many recent advances, DSU has yet to see widespread adoption and deployment in practice. In this paper, we review a slice of the history of DSU research to study how DSU for C has evolved over the last two decades. We examine the ways DSU systems are evaluated in the research literature. We identify several shortcomings of the evaluation criteria that have been used, and propose key improvements. We believe that using better evaluation criteria can guide DSU research to produce systems that will be more practical, flexible, and usable.