Situational Awareness as a Measure of Performance in Cyber Security Collaborative Work

Cyber defense competitions arising from U.S. service academy exercises offer a platform for collecting data that can inform research that ranges from characterizing the ideal cyber warrior to describing behaviors during certain challenging cyber defense situations. This knowledge in turn could lead to better preparation of cyber defenders in both military and civilian settings. We conducted proof-of-concept experimentation to collect data during the Pacific-rim Regional Collegiate Cyber Defense Competition (PRCCDC) and analyzed it to study the behavior of cyber defenders. We propose that situational awareness predicts performance of cyber security professionals, and in this paper we focus on our collection and analysis of competition data to determine whether it supports our hypothesis. In addition to normal cyber data, we collected situational awareness and workload data and compared it against the performance of cyber defenders as indicated by their competition score. We conclude that there is a weak correlation between our measure of situational awareness and performance. We hope to refine and exploit this correlation in further studies.