An empirical study of static call graph extractors

Informally, a call graph represents calls between entities in a given program. The call graphs that compilers compute to determine the applicability of an optimization must typically be conservative: a call may be omitted only if it can never occur an any execution of the program. Numerous software engineering tools also extract call graphs, with the expectation that they will help software engineers increase their understanding of a program. The requirements placed on software engineering tools when computing call graphs are typically more related than for compilers. For example, some false negatives-calls that can in fact take place in some execution of the program, but which are omitted from the call graph-may be acceptable, depending on the understanding task at hand. In this paper we empirically show a consequence of this spectrum of requirements by comparing the C call graphs extracted from three software systems (mapmaker, mosaic, and gee) by five extraction tools (cflow, CIA, Field, mk-functmap, and rigiparse). A quantitative analysis of the call graphs extracted for each system shows considerable variation, a result that is counterintuitive to many experienced software engineers. A qualitative analysis of these results reveals a number of reasons for this variation: differing treatments of macros, function pointers, input formats, etc. We describe and discuss the study, sketch the design space, and discuss the impact of our study on practitioners, tool developers, and researchers.