Serious Games Assessment: Analytics, Measurement, and Visualization of Nursing Competencies

It is crucial for practicing nurses to have comprehensive knowledge on the safety and management of blood transfusion administration (BTA), which is a common core competency for registered nurses worldwide. Skill competency assessments are regularly conducted at healthcare institutions to ensure practicing nurses meet standards. However, direct (face-to-face) observations of nursing competencies have many inconsistency issues, including quality of assessment (personal judgment), suitability in matching cases (due to complications), manpower and time availability (among assessor, nurses, and cases). The advent of serious games analytics makes it possible to deploy serious games as remote (psychometric) assessment tools. This study assessed the (psychometric) validity and reliability of a Blood Transfusion Serious Game (BTSG) as an assessment tool for measuring nursing competencies in blood transfusion administration. Empirical data (serious games analytics) were collected in situ the game. We evaluated the following psychometric properties, namely: action analysis (using difficulty and discrimination measures), construct validity (through an exploratory factor analysis with principal factor analysis and a Promax rotation), and internal-consistency reliability (via Cronbach's alpha). After removing game actions that were not very discriminating or did not load on a single factor, we found the resulting game to be both internally reliable and valid in measuring six remotely related latent sub constructs. Suggestions to visualize the analytics as individual and group competency profiles were provided to assist management and administration with recommendations for the (re)training of underperformed nurses (for remediation or probation) and improvement of the measurement for future serious games assessment.