XAI-IoT: An Explainable AI Framework for Enhancing Anomaly Detection in IoT Systems

The exponential growth of Internet of Things (IoT) systems inspires new research directions on developing artificial intelligence (AI) techniques for detecting anomalies in these IoT systems. One important goal in this context is to accurately detect and anticipate anomalies (or failures) in IoT devices and identify main characteristics for such anomalies to reduce maintenance cost and minimize downtime. In this paper, we propose an explainable AI (XAI) framework for enhancing anomaly detection in IoT systems. Our framework has two main components. First, we propose AI-based anomaly detection of IoT systems where we adapt two classes of AI methods (single AI methods, and ensemble methods) for anomaly detection in smart IoT systems. Such anomaly detection aims at detecting anomaly data (from deployed sensors or network traffic between IoT devices). Second, we conduct feature importance analysis to identify the main features that can help AI models identify anomalies in IoT systems. For this feature analysis, we use seven different XAI methods for extracting important features for different AI methods and different attack types. We test our XAI framework for anomaly detection through two real-world IoT datasets. The first dataset is collected from IoT-based manufacturing sensors and the second dataset is collected from IoT botnet attacks. For the IoT-based manufacturing dataset, we detect the level of defect for data from IoT sensors. For the IoT botnet attack dataset, we detect different attack classes from different kinds of botnet attacks on the IoT network. For both datasets, we provide extensive feature importance analysis using different XAI methods for our different AI models to extract the top features. We release our codes for the community to access it for anomaly detection and feature analysis for IoT systems and to build on it with new datasets and models. Taken together, we show that accurate anomaly detection can be achieved along with understanding top features that identify anomalies, paving the way for enhancing anomaly detection in IoT systems.