Excvate: Spoofing Exceptions and Solving Constraints to Test Exception Handling in Numerical Libraries

Testing a numerical library's exception handling is often left to its regression tests. However, designing floating-point inputs that exercise exceptional behavior is difficult. Further-more, existing input generation techniques are designed with the view that any exception-causing input is of interest when most are unremarkable from the standpoint of exception handling: most functions should handle exceptions correctly by construction, i.e., by returning exceptional values (NaN, $+\text{Inf})$ due to IEEE 754's mandate that most operations propagate such values. To test library exception handling, we propose an approach which - given only a set of regression test executables and a set of function prototypes - cheaply identifies potential failures via exception-spoofing and then reifies these failures using an off-the-shelf SMT solver that generates concrete inputs inducing buggy behavior. We implement this approach in a prototype tool Excvate,present an evaluation that targets 26 BLAS functions across three implementations, two compilers, and multiple compiler optimizations, and ultimately identify exception-handling failures in five functions in multiple BLAS versions.