Processing software source text in automated design recovery and transformation

Software source text is the raw material of program understanding and transformation systems. In order to share the results of source analyses, both between phases of a design recovery process, and between tools and systems in different processes, a source text interchange format is needed. The paper describes a simple technique, 'source factoring', by which a common structural decomposition of source text can address the many issues of preprocessing, macro processing, lexical analysis, design recovery, and automated transformation. Above all, source factorization allows the results of design analysis to be attached to source, and the results of source transformation to be reinstalled cleanly into the code base. This view of source text underlies the architecture of a successful software maintenance system which has processed billions of lines of legacy code in all major programming languages.