Mirror Mirror On The Wall - What Are Cybersecurity Educational Games Offering Overall: A Research Study and Gap Analysis

Cybersecurity educational games are capable of meeting a variety of goals, including learning of fundamental cybersecurity concepts, exposure to cybersecurity literacy, basic awareness, experiential learning plus situational awareness, and being a medium for K-12 outreach, college level learning plus corporate training. Existing literature shows that there have been several prior research surveys focused on analyzing cybersecurity educational games from various viewpoints. However, to our knowledge, there has been no previous work that has analyzed cybersecurity educational games in terms of their alignment with the current benchmarks in academic and industry standards that include the cybersecurity curriculum plus assessment guidelines in higher education, the K-12 standardized cybersecurity concepts, and the cybersecurity job-related functions. In an effort to address this research gap, we use the CSEC2017 curricular guidelines, the Cybersecurity Assessment Tools (CATS) model, the National Security Agency (NSA) GenCyber concepts and the National Initiative for Cybersecurity Education (NICE) framework for performing a unique analysis of a list of popular cybersecurity educational games, plus a few Capture the Flag (CTF) instances. The list of cyber educational games, which we study, comprises of Anti-Phishing Phil, Cyber Awareness Challenge, Cyber CIEGE, Cyber Protect, Nsteens, NOVA Labs, What.Hack, PASDJO, CyberStart Go, OnGuardOnline, Safe Online Surfing, and Interland. This non-traditional survey-based research work demonstrates a novel, multi-faceted approach for analyzing popular opensource cybersecurity educational games in terms of their alignment with the standard academic and industry benchmarks. Our survey results include conceptual mapping of these cyber educational games to (1) the CSEC2017 curricular knowledge areas (Data Security, Software Security, Component Security, Connection Security, System Security, Human Security, Organizational Security and Societal Security), (2) the CATS concept inventory topics, including the Cybersecurity Concept Inventory (CCI), which pertains to basic first year college concepts, and the Cybersecurity Curriculum Assessment (CCA), which contains core concepts and learning outcomes for a college graduate, (3) the NSA GenCyber concepts (Defense in Depth, Confidentiality, Integrity, Availability, Think Like an Adversary and Keep It Simple), and (4) NICE framework-based specialized skill sets (Analyze, Collect & Operate, Investigate, Operate & Maintain, Oversee & Govern, Protect & Defend and Securely Provision). Our research provides a first-of its kind study and a user-friendly analysis of open-source cybersecurity educational games that can serve as an insightful reference for cybersecurity educators and other audiences for using these games. Additionally, in this full research paper, we exhibit how the results of our study can be used for performing an overall gap analysis with cybersecurity educational games in terms which benchmarks they cover and which they do not.