An Empirical Study of Release Note Production and Usage in Practice

The release note is one of the most important software artifacts that serves as a communication bridge between development teams and users. Release notes contain a set of crucial information, such as descriptions of enhancements, improvements, potential issues, development, evolution, testing, and maintenance of projects throughout the whole development life cycle. A comprehensive understanding of the characteristics of release notes and how to best document one for different targeted users would be highly beneficial. However, the release note is often neglected and has not to date been systematically investigated by researchers. In this paper, we conducted a descriptive case study to investigate release note production and usage in practice. We first performed a large scale empirical study of 32,425 release notes in 1,000 GitHub projects to understand the characteristics of real-world release notes, and eight categories of information identified that are normally documented in release notes. We then conducted interviews with 15 professionals and an online survey with 314 respondents to investigate their opinions on release notes in practice. Our results show that both release note producers and users consider that well-formed release notes impact software activities (e.g., software evolution) positively. We summarised 27 statements about release notes grouped into eight topics based on participants’ opinions. Our study uncovers significant discrepancies between release note producers and users in perceiving release notes. Based on these findings, we provide a set of release note production and usage guidelines for practitioners and highlight future research directions.