Automated Black Box Testing Tool for a Parallel Programming Library

As computers with multi-core CPUs become the norm, software development professionals must confront the challenge of developing software that takes advantage of this hardware trend. API providers, in turn, must provide useful parallel programming libraries. PLINQ, a parallel version of Microsoft's LINQ-to-Objects, is one such API aimed at providing a simple, functional style programming model for parallelism. At Microsoft, we have multiple testing strategies for the software components we develop. We present in this paper one approach to testing PLINQ using an automated test generation, execution, and evaluation strategy named SLUG (systematic LINQ unit-test generator). This approach enables us to not only generate unit tests, but performance, stress, and security tests as well. We show that by using this approach to testing, we can achieve high code coverage; we analyze how SLUG allowed us to gain a higher confidence in the quality of our code.