A Methodology for Developing & Assessing CTI Quality Metrics

Since its first steps in the cybersecurity field, Cyber Threat Intelligence (CTI) has gained recognition and increased its importance in the daily operations of cybersecurity teams. However, the many forms of CTI exchanged, the vast amount of CTI products, and the plurality of the sources have raised doubts about the CTI quality. This paper discusses the problem of CTI quality, focusing on the quality factors that better evaluate the products of CTI and how we measure them. Consequently, we propose a methodology for developing and assessing CTI quality metrics and demonstrate the application of this methodology by developing the relevance (
$RE$
) and weighted completeness (
$WC$
) metrics for unstructured and structured CTI products, respectively. We created two sets of structured and unstructured CTI data for this demonstration, utilizing them as benchmark datasets for estimating
$RE$
and
$WC$
.The proposed methodology introduces a systematic approach for developing and assessing quantitative CTI quality metrics for evaluating CTI data and CTI sources.