Measuring the Impact of HPC Training

The demand for computational power is rising rapidly in science and engineering and is now expanding into areas from the social sciences and humanities. This is met with the use of large-scale, and increasingly complex, High-Performance Computing (HPC) systems. Programming such heterogeneous and quickly evolving systems efficiently has always been challenging, particularly for novice HPC programmers, but is continually increasing in complexity. HPC service providers are therefore growing their training offerings as part of their overall service provision. However, there is no standard for understanding the impact and effectiveness of this training. Traditional training and education metrics, such as examination, are typically not applicable. Additionally, the key criteria for service providers are not only knowledge gained but the impact that the knowledge has on HPC usage. To capture and evaluate the effectiveness of training, a novel methodology with two key components is proposed: progress productivity and training productivity. The methodology includes a process for deriving an activity-specific weighted selection of the metrics that comprise the productivity measures. The proposed productivity criteria could then be applied to improve the training of HPC application programmers taking the participant's pre-knowledge and focus areas into account.