Crash Report Prioritization for Large-Scale Scheduled Launches

Software crashes are high-impact bugs that cause applications to terminate unexpectedly. Crash prioritization focuses the attention of maintenance teams on incoming types of crashes that are likely to have a large impact. Prior approaches have been applied to software that is continuously released, whereas, in the context of video game development, large releases are often rolled out following a schedule. A game studio will work for months or years on a new title before releasing it on a scheduled date. In that context, crash data from live players is not available until after release, which is often too late to react. In this study, we analyze post-release game crashes to identify temporal patterns that can inform strategies for prioritization at Ubisoft-a multinational video game publisher. Our analysis shows that most types of post-release crashes impact few players; however, a subset goes “viral,” quickly impacting many players. Those viral crashes may escalate immediately (i.e., outbreaks) or lie dormant before propagating to many players (i.e., time bombs). We use data from a previously released title to detect such viral crashes in a new title by leveraging stack trace similarity and Machine Learning (ML). We found that prioritizing crash types based on stack trace similarity outperforms prioritization using ML models, successfully identifying over half of the viral crash types in the new title. Moreover, the crash types detected by our similarity-based prioritization account for a significantly larger number of crash occurrences than the viral crash types that are misclassified as non-viral. The findings of our study inform game development teams about proactive monitoring of mild crash types that can potentially impact many players, as well as approaches for early detection of potential viral crash types.