An Empirical Framework for Evaluation of Spectral-based Fault Localization Techniques

Spectral-based fault localization is a statistical technique that aims at helping software developers to find programming faults quickly. In this technique fault locations are found by analyzing abstractions of program traces to provide a ranked list of most probable faulty components (e.g., program statement). Although the semantic structure of code is not considered in Spectral-based fault localization methods, they are widely known as high-performance and state-of-the-art methods. Many studies have been conducted in this area that have led to the creation of various methods, with different accuracy and efficiency. Nevertheless, the relative performance of these methods are neither examined nor discussed in the literature. In this paper, we introduce an empirical framework for implementation of SBFL methods in large-scale real bug java programs. Implementation results are reported using evaluation metrics to provide a clear comparison of the performance of the most effective methods.