Toward a Systematic Approach to Program Evaluation Design

The evaluation of programs in the public sector has been problematic and often invalid, primarily due to the fact that the evaluation plans or designs are lacking. The broad field of program evaluation is reviewed along with attempts to synthesize and systematize the steps necessary to develop valid and comprehensive evaluation designs. First, a design framework is identified which links program characteristics to design elements through an expanded set of threats to validity. Second, the various design elements are grouped into five systematically convenient components, including test hypotheses, selection scheme, measures framework, measurement methods, and analytic techniques. Third, it is proposed that the different types of evaluation can be contained in an evaluation taxonomy composed of eight measures-related classifications. Finally, the paper should be regarded as only an initial step toward a systematic approach to program evaluation design; a basis for further discussion is provided.