Abstract:
Most techniques to detect program errors, such as testing, code reviews, and static program analysis, do not fully verify all possible executions of a program. They leave...Show MoreMetadata
Abstract:
Most techniques to detect program errors, such as testing, code reviews, and static program analysis, do not fully verify all possible executions of a program. They leave executions unverified when they do not check certain properties, fail to verify properties, or check properties under certain unsound assumptions such as the absence of arithmetic overflow. In this paper, we present a technique to complement partial verification results by automatic test case generation. In contrast to existing work, our technique supports the common case that the verification results are based on unsound assumptions. We annotate programs to reflect which executions have been verified, and under which assumptions. These annotations are then used to guide dynamic symbolic execution toward unverified program executions. Our main technical contribution is a code instrumentation that causes dynamic symbolic execution to abort tests that lead to verified executions, to prune parts of the search space, and to prioritize tests that cover more properties that are not fully verified. We have implemented our technique for the .NET static analyzer Clousot and the dynamic symbolic execution tool Pex. It produces smaller test suites (by up to 19.2%), covers more unverified executions (by up to 7.1%), and reduces testing time (by up to 52.4%) compared to combining Clousot and Pex without our technique.
Date of Conference: 14-22 May 2016
Date Added to IEEE Xplore: 03 April 2017
ISBN Information:
Electronic ISSN: 1558-1225
References is not available for this document.
Contents Select All
1.
S. Anand, P. Godefroid, and N. Tillmann. Demand-driven compositional symbolic execution. In TACAS, volume 4963 of LNCS, pages 367–381. Springer, 2008.
2.
T. Avgerinos, A. Rebert, S. K. Cha, and D. Brumley. Enhancing symbolic execution with veritesting. In ICSE, pages 1083–1094. ACM, 2014.
3.
T. Ball, B. Hackett, S. K. Lahiri, S. Qadeer, and J. Vanegue. Towards scalable modular checking of user-defined properties. In VSTTE, volume 6217 of LNCS, pages 1–24. Springer, 2010.
4.
T. Ball, R. Majumdar, T. D. Millstein, and S. K. Rajamani. Automatic predicate abstraction of C programs. In PLDI, pages 203–213. ACM, 2001.
5.
T. Ball and S. K. Rajamani. Boolean programs: A model and process for software analysis. Technical Report MSR-TR-2000-14, Microsoft Research, 2000.
6.
M. Barnett, M. Fähndrich, K. R. M. Leino, P. Müller, W. Schulte, and H. Venter. Specification and verification: The Spec# experience. CACM, 54 : 81–91, 2011.
7.
D. Beyer, T. A. Henzinger, M. E. Keremoglu, and P. Wendler. Conditional model checking: A technique to pass information between verifiers. In FSE, pages 57–67. ACM, 2012.
8.
P. Boonstoppel, C. Cadar, and D. R. Engler. RWset: Attacking path explosion in constraint-based test generation. In TACAS, volume 4963 of LNCS, pages 351–366. Springer, 2008.
9.
C. Cadar and D. R. Engler. Execution generated test cases: How to make systems code crash itself. In SPIN, volume 3639 of LNCS, pages 2–23. Springer, 2005.
10.
O. Chebaro, N. Kosmatov, A. Giorgetti, and J. Julliand. The SANTE tool: Value analysis, program slicing and test generation for C program debugging. In TAP, volume 6706 of LNCS, pages 78–83. Springer, 2011.
11.
M. Christakis. Narrowing the Gap between Verification and Systematic Testing. PhD thesis, ETH Zurich, 2015.
12.
M. Christakis, P. Müller, and V. Wüstholz. Collaborative verification and testing with explicit assumptions. In FM, volume 7436 of LNCS, pages 132–146. Springer, 2012.
13.
M. Christakis, P. Müller, and V. Wüstholz. An experimental evaluation of deliberate unsoundness in a static program analyzer. In VMCAI, volume 8931 of LNCS, pages 336–354. Springer, 2015.
14.
E. M. Clarke, O. Grumberg, S. Jha, Y. Lu, and H. Veith. Counterexample-guided abstraction refinement. In CAV, volume 1855 of LNCS, pages 154–169. Springer, 2000.
15.
P. Cousot and R. Cousot. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In POPL, pages 238–252. ACM, 1977.
16.
C. Csallner and Y. Smaragdakis. JCrasher: An automatic robustness tester for Java. SPE, 34 : 1025–1050, 2004.
17.
C. Csallner and Y. Smaragdakis. Check ‘n’ Crash: Combining static checking and testing. In ICSE, pages 422–431. ACM, 2005.
18.
C. Csallner, Y. Smaragdakis, and T. Xie. DSD-Crasher: A hybrid analysis tool for bug finding. TOSEM, 17 : 1–37, 2008.
19.
M. Czech, M.-C. Jakobs, and H. Wehrheim. Just test what you cannot verify! In FASE, volume 9033 of LNCS, pages 100–114. Springer, 2015.
20.
E. W. Dijkstra. Guarded commands, nondeterminacy and formal derivation of programs. CACM, 18 : 453–457, 1975.
21.
M. D. Ernst, J. H. Perkins, P. J. Guo, S. McCamant, C. Pacheco, M. S. Tschantz, and C. Xiao. The Daikon system for dynamic detection of likely invariants. Sci. Comput. Program., 69 : 35–45, 2007.
22.
M. Fähndrich, M. Barnett, and F. Logozzo. Embedded contract languages. In SAC, pages 2103–2110. ACM, 2010.
23.
M. Fähndrich and F. Logozzo. Static contract checking with abstract interpretation. In FoVeOOS, volume 6528 of LNCS, pages 10–30. Springer, 2010.
24.
C. Flanagan, K. R. M. Leino, M. Lillibridge, G. Nelson, J. B. Saxe, and R. Stata. Extended static checking for Java. In PLDI, pages 234–245. ACM, 2002.
25.
X. Ge, K. Taneja, T. Xie, and N. Tillmann. DyTa: Dynamic symbolic execution guided with static verification results. In ICSE, pages 992–994. ACM, 2011.
26.
P. Godefroid. Compositional dynamic test generation. In POPL, pages 47–54. ACM, 2007.
27.
P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In PLDI, pages 213–223. ACM, 2005.
28.
P. Godefroid, M. Y. Levin, and D. A. Molnar. Automated whitebox fuzz testing. In NDSS, pages 151–166. The Internet Society, 2008.
29.
P. Godefroid, A. V. Nori, S. K. Rajamani, and S. Tetali. Compositional may-must program analysis: Unleashing the power of alternation. In POPL, pages 43–56. ACM, 2010.
30.
S. Graf and H. Saïdi. Construction of abstract state graphs with PVS. In CAV, volume 1254 of LNCS, pages 72–83. Springer, 1997.
