The Worst Case Execution Time Tool Challenge 2006: The External Test

The first worst case execution time (WCET) tool challenge, performed in 2006, attempted to evaluate the state-of- the-art in timing analysis for real-time systems and so to encourage research and activities in the WCET community. It applied two evaluation approaches to the tools submitted, self-evaluation and external evaluation. In order to balance users' effects and to achieve a comparable evaluation, an independent test person has been assigned by the WCET Challenge Working Group to perform the external tool evaluation. The test person visited the tool developers and evaluated the tools entered. This paper describes the testing procedures applied, the results obtained, and the evaluation made in the Challenge.