A structured demonstration of program comprehension tools

This paper describes a structured tool demonstration, a hybrid evaluation technique that combines elements from experiments, case studies and technology demonstrations. Developers of program understanding tools were invited to bring their tools to a common location to participate in a scenario with a common subject system. Working simultaneously the tool teams were given reverse engineering tasks and maintenance tasks to complete on an unfamiliar subject system. Observers were assigned to each team to find out how useful the observed program comprehension tool would be in an industrial setting. The demonstration was followed by a workshop panel where the development teams and the observers presented their results and findings from this experience.