LOCoCAT: Low-Overhead Classification of CAN Bus Attack Types

Although research has shown vulnerabilities and shortcomings of the controller area network bus (CAN bus) and proposed alternatives, the CAN bus protocol is still the industry standard and present in most vehicles. Due to its vulnerability to potential intruders that can hinder execution or even take control of the vehicles, much work has focused on detecting intrusions on the CAN bus. However, most literature does not provide mechanisms to reason about, or respond to the attacks so that the system can continue to execute safely despite the intruder. This letter proposes a low-overhead methodology to automatically classify intrusions into predefined types once detected. Our framework: 1) groups messages of the same attacks into blocks; 2) extracts relevant features from each block; and 3) predicts the type of attack using a lightweight classifier model. The initial models depicted in this letter show an accuracy of up to 99.16% within the first 50 ms of the attack, allowing the system to quickly react to the intrusion before the malicious actor can conclude their attack. We believe this letter lays the groundwork for vehicles to have specialized runtime reactions based on the attack type.

LOCoCAT: Low-Overhead Classification of CAN Bus Attack Types Caio Batista de Melo and Nikil Dutt , Fellow, IEEE Abstract-Although research has shown vulnerabilities and shortcomings of the controller area network bus (CAN bus) and proposed alternatives, the CAN bus protocol is still the industry standard and present in most vehicles.Due to its vulnerability to potential intruders that can hinder execution or even take control of the vehicles, much work has focused on detecting intrusions on the CAN bus.However, most literature does not provide mechanisms to reason about, or respond to the attacks so that the system can continue to execute safely despite the intruder.This letter proposes a low-overhead methodology to automatically classify intrusions into predefined types once detected.Our framework: 1) groups messages of the same attacks into blocks; 2) extracts relevant features from each block; and 3) predicts the type of attack using a lightweight classifier model.The initial models depicted in this letter show an accuracy of up to 99.16% within the first 50 ms of the attack, allowing the system to quickly react to the intrusion before the malicious actor can conclude their attack.We believe this letter lays the groundwork for vehicles to have specialized runtime reactions based on the attack type.
Index Terms-Attack type classification, controller area network, embedded systems.

I. INTRODUCTION
M ODERN vehicles are a distributed system by design, where all their functionality is spread out to different electrical component units (ECUs) that have specific goals.Due to this distributed design, some vehicles need over 70 ECUs to operate correctly [11].In order to enable the many ECUs to communicate necessary information with each other, most vehicles use the controller area network bus (CAN bus).Although the industry standard since 1996 [3], the CAN bus is still vulnerable to attacks from bad actors.Since messages are neither encrypted nor signed, if a malicious agent can gain access to the CAN bus, it can read all shared messages and send messages that other ECUs will receive [2], [7], [9], [11].Such attacks happen in our day-to-day lives outside of academic and industry experiments, e.g., [5].
Much work has been proposed in the literature to detect intrusions in the CAN bus, e.g., [2], [3], [7], [8], [9], and [12].However, most intrusion detection systems (IDSs), e.g., [2], [7], [9], and [12], stop the analysis at the detection stage, so there is no analysis after an intruder has been detected, and thus do not present pathways to facilitate safe responses.These proposed frameworks block the messages from being processed at ECUs but do not analyze if there is a better response for the ongoing attack.Attack type classification is vital to safety-critical systems as it allows systems to better react to ongoing attacks and prepare for future ones [6].The few works that have approached the issue of attack classification on the CAN bus [3], [8], do not analyze how much overhead their complex classification models add to the system runtime; and high overheads can compromise safe responses.
In this letter, we propose LOCoCAT, an attack-type classification system that can identify what type of attack the intruder is executing with very little overhead.Our proposal can extend any state-of-the-art IDS and adds minimal power, memory, and processing requirements to the overall system.It works by: 1) grouping detected malicious messages into attack blocks; 2) calculating discrete features for each attack block; and 3) using a lightweight classifier model to identify the type of each attack.We evaluate the applicability of our proposal using three of the most popular available CAN bus datasets in [2], [7], and [9], observing excellent classification results with low latency in an energy-and resource-constrained environment.
The main contributions of this letter are twofold.1) We propose a way to combine multiple CAN bus messages into blocks that can be classified together, reducing the amount of inferences required.2) We evaluate our approach with related work for accuracy and overhead when running on a low-power platform using multiple datasets from the literature.

II. BACKGROUND AND RELATED WORK A. CAN Bus Attacks
The CAN bus is the industry standard for communication between the different ECUs in vehicles [11].However, due to its design limitations, the ECUs connected to the CAN bus are vulnerable to malicious agents that gain access to the bus.Therefore, much work has been done in 1) creating datasets with CAN bus attacks and 2) devising IDSs for the CAN bus.
Hanselmann et al. [7]   up to eight signal values per message and contains four different attack types, where each individual attack ranges from 3 to 5 s.Kukkala et al. [12] proposed LATTE, an IDS based on a combination of neural networks and support vector machines to detect CAN bus attacks with an average improvement of 18.94% in accuracy compared to other previous IDSs for CAN bus data.Additionally, Kukkala et al. provide a complete overhead analysis of LATTE using an NVIDIA Jetson TX2 as their platform, where it achieves a latency of 193 µs.The Jetson TX2 is a powerful GPU-equipped embedded device often used as an additional ECU in the literature, showing the applicability of LATTE in a real-world system.

B. CAN Bus Attack Types
In addition to gaining access to the bus, an attacker can craft malicious messages with different intents and outcomes.If the attacker has domain knowledge about the vehicle, they can spoof the RPM gauge and drive gear [2], make the vehicle malfunction [9], or even steal it by opening its doors and turning it on to drive away [5].Furthermore, even if the attacker does not have domain knowledge, they can still affect the vehicle by, e.g., flooding the CAN bus to achieve a denial of service attack [2], [7], [9] or sending messages with random values to make actuation fuzzy [2], [9].Thus, it is essential to detect CAN bus attacks and identify the attacker's intent so the system can react appropriately [6].Existing literature that identifies the types of CAN bus attacks is limited.To the best of our knowledge, [3] and [8] are the only works that analyze CAN bus attacks as a multiclass classification problem.
Amato et al. [3] proposed an IDS for car hacking that uses a neural network model to classify car-hacking attacks in multiclass labels achieving a weighted F1-score of 96.8%.However, the authors do not mention the platform where experiments were run or analyze their approach's overhead.Hossain et al. [8] proposed an IDS for survival-IDS that uses a neural network model to classify survival-IDS attacks in multiclass labels achieving a weighted F1-score of 98.84%.Their model uses a powerful GPU-equipped desktop computer, but no overhead analysis is provided.Since neither work provides a complete overhead analysis and [8] uses a powerful platform, it is unknown whether their approaches are feasible in the context of a real-time safety-critical system.In Section IV, we compare our approach's accuracy results with [3] and [8] and provide a complete overhead analysis.
III. METHODOLOGY Fig. 1 outlines the entire process for type classification of attacks on the CAN bus.Fig. 1(a) shows an attacker that has gained access to the bus and can send malicious messages.Next, Fig. 1(b) depicts a state-of-the-art IDS from the literature that can detect malicious messages among normal data in the CAN bus.As a proof of concept, we use LATTE [12] as the framework's IDS due to its high accuracy and low latency; however, replacing it in the overall framework would be easy as new models appear in the literature.Finally, Fig. 1(c) shows our contribution-a framework that can classify the attack type of groups of malicious messages detected by the IDS into previously known types.As Fig. 1(a) and (b) are well covered in the literature (see Section II), the rest of this section will focus on Fig. 1(c), which is our main contribution.
The LOCoCAT (Low-Overhead Classification of CAN bus Attack Types) aims first to quickly, accurately, and nonintrusively identify the type of an ongoing attack so that the system can choose the best reaction available.Additionally, since our approach requires an IDS to flag malicious messages and the most accurate IDSs in the literature (e.g., [7] and [12]) use complex models that require a GPU-equipped board to function (e.g., NVIDIA Jetson TX2), LOCoCAT does not significantly affect the overall overhead added to the system.
We achieve both of these goals by exploring models that 1) have good accuracy results; 2) can run on low-end hardware; 3) are lightweight; and 4) have an acceptable latency when considering the usual length of CAN bus attacks.
Our proposed classification methodology analyzes messages flagged as malicious by an existing IDS.First, we group malicious messages with timestamps close to each other into attack blocks.Second, we choose a window at the start of the attack blocks to analyze, as we want a classification result before the ongoing attack ends.Third, we calculate discrete features for the attack block window, resulting in a finite set of features for all attack blocks, regardless of how many messages were a part of the attack.Finally, we feed this discrete featureset into a classifier model to get the attack type.The rest of this section describes our approach in detail.

A. Attack Blocks and Features
Each CAN bus attack consists of a sequence of messages that the attacker sends during a short time among the normal messages.Since LOCoCAT already knows which messages are malicious due to the IDS, we group the malicious messages into blocks while an attack occurs.An attack block will then have a finite number of features.We calculate three metrics for each signal in the messages: 1) median; 2) average; and 3) standard deviation.We also count how many messages are a part of this attack block.Overall, we have a total of 13 features for each attack block of the SynCAN dataset [7], as it has up to four signals per message, and 25 features for each attack block of the Car-Hacking [2] and Survival-IDS [9] datasets, as they each have up to eight signals per message.

B. Window Selection
One of the goals of identifying the type of attack is to give the system more information on how to best react.Thus, it is crucial to have a classification result before the attack is over.Since the length of each CAN bus attack on the literature datasets [2], [7], [9] range from 3 to 8 s, we decided to use at most 1s of the attack for classification.This gives enough time for the system to decide and execute a reaction to the ongoing attack.Therefore, we explore models that consider messages up to 50, 100, 200, 500, and 1000 ms from the start of the attack.

C. Model Selection
To develop our framework, we use Python3 with the scikitlearn [4] library to train all models and run all inferences.Scikit-learn provides an easy way to train and export models that can run on CPU-only platforms.We consider the following multiclass classification models available in the library: Adaptive Boosting, Decision Tree, Gaussian Naive Bayes, k-Nearest Neighbors, Quadratic Discriminant Analysis, and Random Forest.We perform a parameter grid search for each model to find the best results considering each window length option.The results shown in Section IV are obtained from the best parameter combination of each model.

IV. EXPERIMENTAL RESULTS
All experiments were executed on a Raspberry Pi Zero W with a 1-GHz single-core ARMv6 CPU and 512 MB of memory, and all scripts and results are made available online at https://github.com/cbdm/LOCoCAT.We use three datasets with malicious CAN bus data from the literature for all experiments: 1) SynCAN [7]; 2) car hacking [2]; and 3) survival-IDS [9].All three datasets are available online. 1  We prepare all datasets for our experiments by: 1) filtering out the normal messages; 2) creating attack blocks with malicious messages that are part of the same attack; 3) shuffling the attack blocks; and 4) splitting these blocks into 80% for training and 20% for testing.In the following two subsections, we analyze two different aspects of our approach.First, in Section IV-A, we analyze our proposal's performance in correctly identifying the ongoing attack.Then, in Section IV-B, we analyze the added overhead of the models.

A. Correct Classification
Tables I and II depict the weighted F1-scores for the different models and windows explored for the Car-Hacking and SynCAN datasets, respectively.We chose to omit the complete results for survival-IDS as there were too few attack blocks 1 SynCAN is available at https://github.com/etas/SynCAN,car hacking is available at https://ocslab.hksecurity.net/Datasets/car-hacking-dataset,and survival-IDS is available at https://ocslab.hksecurity.net/Datasets/survival-ids. to make a thorough analysis.When grouping the malicious messages of each dataset in attack blocks, the Survival-IDS dataset had only 29 separate attacks, while car hacking (1200) and SynCAN (545) had considerably more.However, it is still worth mentioning that a decision tree model achieves a weighted F1-score of 100% with a 100-ms window.
As Table I shows, our proposal works exceptionally well for the Car-Hacking dataset, achieving an F1-score of over 99% with a Gaussian Naive Bayes model considering only the first 50 ms of each attack block.These results are extremely promising to the applicability of our approach in a real-world system, as using a resource-constrained platform with very low power requirements, we can still achieve outstanding classification results for a 4-class problem.Although the scores for the SynCAN dataset shown in Table II are not as high as the ones observed for car hacking, the results are also promising, as the SynCAN dataset contains one extra attack.

B. Framework Overhead
First, we look at the inference latency, as we want the classification process to conclude before the attack is over to allow the system to react.Table III shows the average latency in ms for each model-window combination over a total of 174 500 inferences.The two models that perform most accurately, Gaussian Naive Bayes and Decision Tree (see Section IV-A), had very low inference times with 9.3 and 3.3 ms.For the Car-Hacking dataset, this would account for a total classification time of 59.3-50 ms of receiving messages and 9.3 ms of inference, leaving at least 2.94 s for the system to react.For Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE IV AVERAGE TRAINED MODEL SIZE (KB)
the SynCAN dataset, this would account for a total classification time of 503.3-500 ms of receiving messages and 3.3 ms of inference, leaving least 3.49 s for the system to react.For the Survival-IDS dataset, this would account for 103.3-100 ms of receiving messages and 3.3 of inference, leaving at least 4.896 s for the system to react.In all cases, the system should have enough time to choose and deploy the appropriate reaction for the ongoing attack type.
Next, we analyze the space requirements.Table IV shows the average sizes in KB for each considered model.The most accurate models, Gaussian Naive Bayes (50 ms) and Decision Tree (100 and 500 ms), would only require 1.9, 4.3, and 5.8 kB, respectively.These sizes are much smaller than more complex neural network models in the literature, e.g., LATTE requires 1439 kB [12].Additionally, by using a low-power platform, such as the Raspberry Pi Zero W, we can significantly reduce the power consumption we add to the system.A Raspberry Pi Zero W operating at its maximum CPU usage consumes 1.512 W [1], whereas an NVIDIA Jetson TX2 can use nearly ten times that at 15 W [10].

C. Comparison With Existing Literature
To compare our results to previous proposals, we trained models inspired by their description in each respective paper.
To replicate [3], we use a 3-hidden-layer multilayer perceptron (MLP-3) using scikit-learn to classify attack blocks.To replicate [8], we train a single-layer 512-unit long short-term memory (LSTM) model to classify malicious messages, and use TensorFlow Lite 2 to run it on the Pi Zero.Additionally, we added a DummyClassifier that chooses the attack type randomly as a baseline for comparison.Table V shows the results we obtained with these three different approaches in comparison to the best performing LOCoCAT models.The Car Hacking, Survival-IDS, and SynCAN columns show the F1-score for each model for the respective dataset.Latency shows the average model latency over all datasets, and Size shows the average model size over all datasets.
These results show that the LSTM, MLP-3, and LOCoCAT approaches perform exceptionally well in the Car-Hacking and Survival-IDS datasets; however, LOCoCAT achieves much better results in classifying attacks in the SynCAN dataset, showing an improvement of over 1.75× in comparison to the other methods.Additionally, LOCoCAT and MLP-3 show a considerably lower latency to the LSTM model.Finally, LOCoCAT shows great improvements in memory size, 10× to MLP-3 and 1000× to LSTM.These results show the feasibility of running LOCoCAT under strict resource constraints to accurately perform CAN bus attack classification.

V. CONCLUSION AND FUTURE WORK
In this letter, we propose LOCoCAT, a framework that can identify the type of ongoing attacks on the CAN bus with very little overhead.We evaluate the efficacy of LOCoCAT using three existing datasets used in the literature to develop IDSs for the CAN bus, and we can accurately classify attacks with better results than related works.Additionally, the classification latency is appropriate for the attack lengths, as it allows the system plenty of time to react, and the memory and power requirements are much lower than related works.In the future, we want to approach two paths.
1) Expand LOCoCAT to work with attack types that are either unknown to the system or are a mix of existing attacks.2) Explore the feasibility of running our approach on a platform with even lower resource requirements, such as the Raspberry Pi Pico.

Fig. 1 .
Fig. 1.Step-by-step process for classifying CAN bus attacks.(a) Attacker gains access to the CAN bus and sends malicious messages.(b) State-of-the-art IDS flags the malicious messages on the bus.(c) LOCoCAT analyzes the flagged messages and identifies the attack type.
2 https://www.tensorflow.org/lite [2] et al. [9]introduced survival-IDS, a dataset containing real CAN bus data from three different vehicles-Hyundai Sonata, Kia Soul, and Chevrolet Spark.Survival-IDS has up to eight signal values per message and contains three different attack types, where each individual attack is 5-s long.Seo et al.[2]introduced car hacking, a dataset containing real CAN bus data from a Hyundai YF Sonata.Car hacking has c 2023 The Authors.This work is licensed under a Creative Commons Attribution 4.0 License.For more information, see https://creativecommons.org/licenses/by/4.0/Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.