The Use of Decision-Making Methods to Ensure Assessment Validity

In this paper, we present the problem of ensuring a course assessment's validity from the pedagogical and decision-making perspective. The assessment's validity is one of the five criteria considered in the Van der Vleuten utility formula. It is vital to improve the validity of the whole course by clearly linking the assessment plan to the intended learning outcomes (LOs). The first step is to identify the criteria by which learning outcomes will be evaluated. We used a focus group to identify the following four criteria: importance of the topic or context for the future profession; required level of the LO based on chosen taxonomy; contribution to the development of the 21st-century generic skills and student workload needed to fulfill the LOs. In the second phase, we used the Analytic Hierarchy Process (AHP) and the Analytic Network Process (ANP) to determine the weights of evaluation criteria and the consequent relative importance of LOs of a course we used as a case study. The problem is interesting from the decision-making point of view as well because we consider prioritization (not selection) of alternatives (here LOs), and alternatives (LOs) as not independent of each other.