Learning Analytics Perspective: Evidencing Learning from Digital Datasets in Cybersecurity Exercises

Cybersecurity exercises are gaining in popularity in university curricula and professional training paths and are seen as an effective teaching method. Such exercises provide digital datasets to facilitate a learning analytics approach such as by using the traces that learners leave behind to improve the learning process and environment. While there are various learning measurement efforts from digital datasets in the existing literature, a holistic learning analytics approach incorporated into cybersecurity exercises is still lacking. We propose a practical reference model for incorporating a learning analytics approach into the cybersecurity exercise life-cycle. To facilitate this application, we have performed an extensive review of existing academic research on applying learning analytics in the context of cybersecurity exercises. We specifically focus on the learning indicators used to measure empirical impact and training effectiveness that could indicate achievement of defined learning outcomes. This reference model and overview of existing learning analytics use cases and learning metrics in various types of exercises can help educators, organisers and cyber range developers. This results in more adaptive exercise design and measurement using evidence-based data and connects digital learning traces to skills and competencies.