Towards Transparent Intrusion Detection: A Coherence-Based Framework in Explainable AI Integrating Large Language Models

Intrusion Detection Systems (IDS) are essential for maintaining cybersecurity by identifying potential threats within networks. However, the black-box nature of the Artificial Intelligence models used in many IDS deployments hinders their effectiveness. Explainable AI (XAI) methods have emerged to provide transparency, but their evaluation metrics—such as faithfulness, completeness, and stability—are generic and fail to measure coherence with domain-specific knowledge transparently. Domain-specific knowledge is typically expressed as natural language rules, yet manually applying these rules in real-time cybersecurity is labor-intensive, costly, and error-prone due to the high volume of incidents. This paper introduces a coherence evaluation metric designed to ensure XAI explanations align with IDS domain knowledge, aiding cybersecurity analysts in understanding and responding to incidents more effectively. We propose a framework that integrates Generative AI (GAI) with XAI to automate the coherence evaluation process, leveraging Large Language Models (LLMs) for optimal performance. Using the NSL-KDD and CICIDS2017 datasets, we demonstrate the effectiveness of our framework in improving the interpretability and trustworthiness of IDS predictions. Our findings revealed that the chain-of-thought (COT) prompting method achieved an 86% correctness rate in LLM responses for automating written rules towards the coherence metric. Additionally, our framework models written rules using colors, provides coherence or incoherence recommendations, improves the integration of XAI with LLMs, and ultimately advances human-computer interaction in security decision-making.