An Interview Study of how Developers use Execution Logs in Embedded Software Engineering

Execution logs capture the run-time behavior of software systems. To assist developers in their maintenance tasks, many studies have proposed tools to analyze execution information from logs. However, it is as yet unknown how industry developers analyze logs in embedded software engineering. In order to bridge the gap, we study how developers analyze logs by interviewing 25 software developers from ASML, which is a leading company in developing lithography machines. In particular, we explore the type of logs developers analyze, the purposes for which developers analyze logs, the information developers need from logs and their expectation on tool support. As the main contribution, we observed that the lack of domain knowledge, lack of familiarity with code base and software design, and presence of concurrency, raise major challenges in log analysis for such complex and multidisciplinary systems. Particularly, we observed that inspecting execution information at different levels of abstraction is useful to develop comprehension of such complex systems. However, obtaining the abstraction is difficult with current tools. Our study has several implications. The empirical evidence provided by our study implies the need to support log inspection and comparison with multiple levels of abstraction, categorize log differences, and recover links between different types of logs.