Measuring performance of Cyber situation awareness systems

Enabling situation awareness necessitates working with processes capable of identifying domain specific activities. This paper addresses metrics developed to assess research level systems and to measure their performance in providing those processes. The metrics fall into four dimensions; confidence, purity, cost utility, and timeliness. The bulk of the discussion will provide an overview of each of these dimensions, the anticipated usefulness of measures within each dimension, and a discussion of observations resulting from the use of each measure in building Cyber Network Defense capabilities. The paper concludes with a brief mention of ongoing activities and identifies ideas for future research.