Assessing the Real Worth of Software Tools to Check the Healthiness Conditions of Automotive Software

There are a number of software-controlled features in today's automotive vehicles making them comfortable, safer, entertaining, informative and even greener! The number of features is rapidly growing and so is the software content of automotive vehicles to meet these requirements. The software code that realises any one feature is, nowadays, often distributed across several Electronic Control Units (ECUs) as well. In order to produce highly reliable automotive vehicles, their increasingly complex software has to be of high-quality and this requires sophisticated tools and techniques within the automotive industry. One such category of tools statically check whether the software developed hold certain properties (healthiness conditions), such as checking that a variable is set before it is read, and that arithmetic operations do not lead to overflow. These tools typically generate a list of issues, which highlight potential areas of the code, where the healthiness conditions being checked might fail. In practice, the list of generated issues typically contains a significant number of false positives; i.e. issues that cannot lead to a genuine failure of a healthiness condition. This paper discusses the design of objective experiments and the initial stages of an ongoing automotive industry study to assess the real worth of such tools. Towards this end, relevant concepts such as healthiness conditions for software are explained and the various criteria used for the objective experiments are defined giving their rationale.