Evaluating C++ Design Pattern Miner Tools

Many articles and tools have been proposed over the years for mining design patterns from source code. These tools differ in several aspects, thus their fair comparison is hard. Besides the basic methodology, the main differences are that the tools operate on different representations of the subject system and that the pattern definitions differ as well. In this paper we first provide a common measurement platform for three well-known pattern mining systems, Columbus, Maisa and CrocoPat. Then we compare these tools on four C++ open-source systems: DC++,WinMerge, Jikes and Mozilla. Columbus can discover patterns from the C++ source code itself, while Maisa and CrocoPat require the representation of a software system in a special textual format, so we extended Columbus to provide the common input for the two other tools. We compared these tools in terms of speed, memory consumption and the differences between the hits. While the first two aspects showed comparable results, the recognition capabilities were quite diverse. This is probably due to the significant difference in how the patterns to be recognized and formalized by the tools. Therefore we conclude that a more precise and formal description of design patterns would be desirable.