Comparative Evaluation of Machine Learning-Based Controller Area Network Intrusion Detection Systems

The controller area network (CAN) protocol is widely used in vehicle networks. However, it lacks essential security features like confidentiality and authentication. To enhance vehicle security, researchers have proposed and developed various methods, including intrusion detection systems (IDSs). These IDSs play a key role in early detection of potential cyberattacks, safeguarding vehicles, passengers, and road users. Machine learning (ML) algorithms are increasingly employed in CAN IDSs but they face challenges due to diverse experimental settings, datasets, and metrics used for evaluation, impeding progress and comparison of different approaches in this field. This study aims to benchmark well-recognized ML algorithms, including gaussianNB, k-nearest neighbors (KNN), decision trees (DT), random forest (RF), long short-term memory (LSTM), and convolutional neural networks (CNN), for intrusion detection in CAN. We evaluate the performance of these algorithms using seven evaluation metrics on the real ORNL automotive dynamometer (ROAD) dataset, a state-of-the-art benchmark in this field. Experimental results demonstrate that KNN, DT, RF, LSTM, and CNN can detect various types of attacks (e.g., Fuzzing, targeted ID, and masquerade attacks) with high accuracy. However, the computational and time efficiency of tree-based classifiers, namely DT and RF, makes them a potentially appealing choice for real-time intrusion detection tasks in CAN.