A Cyber Security Evaluation Framework for In-Vehicle Electrical Control Units

Modern vehicles are equipped with more than 100 Electrical Control Units (ECUs) with over 2500 signals to transmit internally. The application of advanced electronics and communication techniques helps a vehicle transform from an information island into a powerful distribution center. However, a large number of ECUs have introduced a wider range of security threats for vehicles. The attackers can compromise a vehicle remotely through a vulnerable ECU. How to evaluate the cyber security of in-vehicle ECUs has become an important issue. Current Threat Analysis and Risk Assessment (TARA) only carries out theoretical analysis on the potential threats and risks faced by the vehicle in the conceptual design phase of the lifecycle, but lacks the details of actual security evaluation. In this paper, we proposed a Cyber Security Evaluation Framework (CSEF) to independently evaluate the security of the in-vehicle ECUs, which is composed of the asset identification, the threat analysis, the risk assessment, and the security test. The proposed CSEF is applied to a pre-installed On-Bord Unit (OBU) to provide a use case. The use case show that the proposed CSEF is able to figure out assets, threats, risks behind threats, and vulnerabilities of OBU, playing an important role in guiding others to conduct security evaluation. Moreover, CSEF can be extended to evaluate the cyber security of other critical ECUs, such as the Telematic Box, the infotainment units, and the gateway.


I. INTRODUCTION
Modern vehicles are not only mechanical tools for transportation, but also mobile smart devices for autonomous driving, audio-visual entertainment, and information sharing, etc [1]. The advancement of network communication and electronics techniques has improved the level of automotive intelligence, informatization, and automation. Automobiles, sensors, mobile terminals, road traffic infrastructures, internet and other smart devices form an information sharing network, which is called Internet of Vehicles (IoV) [2]. ECUs with computing and storage capabilities are responsible for processing massive information received, instructing the actuator to control the vehicle based on the intelligent driving instruction [3]. the in-vehicle ECUs are all connected to the bus network, if an attacker compromises an ECU, they may use it as a springboard to control the vehicle, which will cause privacy disclosure, financial damage, functionality failure, and personal injury. In addition, privacy in IoV services has become an increasingly important security hazard [10], [11]. Attackers can analyze the behaviors and habits of target vehicle users based on private data such as identity and location to lay the foundation for subsequent attacks.
In order to evaluate cyber security of the in-vehicle ECU to fully grasp the security status of the ECU in the vehicle, we propose a Cyber Security Evaluation Framework (CSEF) to comply with the ISO/SAE 21434 standard [12]. CSEF consists of the asset identification [13], the threat analysis [14], the risk assessment [15], and the security test. The asset identification gives the asset that attackers are interested in.The threat analysis systematically exposes the potential threats that the assets may face. The risk assessment rates the level of risk according to the potential consequences of threats. And the security test case set is designed and conducted to verify that if the target object meets the security objectives. The major contributions of this paper are as follows: First, unlike the existing Threat Analysis and Risk Assessment(TARA) frameworks that only conduct theoretical threat analysis in the conceptual design phase, CSEF combines the TARA and the security test, which makes it can be applied to both the development phase and the security evaluation phase when the vehicle is released.
Second, the framework can be applied to critical in-vehicle ECUs, such as the Telematics Box, the in-vehicle gateways, the infotainment systems, the navigation systems, the domain controllers, and the On-Board Unit (OBU), etc. It has a wide range of applications and good evaluation effect. We applied the proposed cyber security framework to an actual preinstalled in-vehicle OBU to show how to use it. According to the use case, CSEF is able to expose assets, threats, risks, and several actual vulnerabilities of ECU, such as Serial Wire Debug (SWD) [16] information leakage, SWD debugging privilege, and firmware logic leakage, which proves the values of CSEF.
Third, CSEF complies with the ISO/SAE 21434 standard and has been deeply optimized on to be more suitable for the IoVs. Compared with the existing TARA framework, CSEF has more evaluation details to help the evaluator fully understand the security status of the target ECUs.
The rest of the paper is organized as follows. Section II gives the related work. Section III provides the basic knowledge related to modern vehicles and In-Vehicle Networks. Section IV describes the proposed cyber security evaluation framework. An usage case and the effectiveness of the framework is provided in Section V. Finally, section VI draws conclusions.

II. RELATED WORK
The vehicle has a very complex supply chain, both horizontally and vertically, which make it is a difficult tack to manage the supply chain. From the cyber security perspective for the lifetime of a modern vehicle, there will be new threats and attack vectors introduced constantly.To address the security issue of the in-vehicle ECUs, researchers mainly focuses on three directions: threat modeling, risk assessment, and security testing. In 2016, Zhendong Ma et al. [17] proposed a practical approach to conduct threat modeling for automotive security analysis during the development lifecycle, but it did not explain how to model and analyze the threat items of in-vehicle systems. Mohammad et al. [18] tried to revise the existing threat modeling efforts in the vehicular domain with only four categories of attack objective, which lacked fine-granularity for complex in-vehicle threat analysis. In 2017, Karahasanovic et al. [19] demonstrated that the threat modeling process in the computer industry can be adapted and applied in the automotive industry. They did not propose a new approach or optimization to make existing approaches more efficient. Recently, Mohammad et al. [20] proposed an approach that combines different existing threat modeling approaches to create a comprehensive and hybrid threat model to support security analysis for in-vehicle systems. However, the approach had no in-depth analysis of vehicular security, and the optimization for vehicular system was till not enough. Since the security evaluation of the vehicle is performed after product development, it requires additional testing on the basis of threat analysis and risk assessment for the security evaluation. The paper [21]- [23] proposed the security evaluation framework for in-vehicle CAN bus. The paper [24]- [27] proposed the test framework for the subsystem in the vehicle. Unfortunately, existing test frameworks only partially examined a certain aspect or a particular sub-system independently. On one hand, there is no universal test framework that can be used for various in-vehicle ECUs. On the other hand, the threat modeling has not been deep optimized to improve the efficiency for the modern vehicles.
In addition, different approaches have been proposed to manage massive threats and risks of complex vehicle systems. Typically, the existing TARA methods recommended in SAE J3061 [28] just focus on the concept design phase in the lifetime of vehicles. In 2020, the ISO/SAE 21434 standard proposed a security framework to guide security practices in the lifecycle of the modern vehicles and the in-vehicle ECUs without the actual implementation process. The two most commonly used tools, EVITA [29] and HEAVENS [30], provide the TARA framework, but lack the details of actual threat analysis. EVITA only describes the threat from the attack probability and HEAVENS considers too few factors in the threat modeling process.

A. EVITA
The EVITA method considers four cyber objectives, ''Operational'', ''Safety'', ''Privacy'', and ''Financial''. For each of the cyber objectives, the EVITA method identify potential threats with the help of the attack tree approach and classify threat risk level based on severity of the threat outcome VOLUME 9, 2021 and probability of a successful attack. The probability of a successful attack in EVITA is based on the concept of ''attack potential'' used in IT security evaluation, and considers both the attacker and the system. The severity and attack probability are then combined using a ''risk graph'' approach to identify the risk associated with each threat.
However, in the EVITA model, assets are not strictly defined, which makes it difficult for security assessors to accurately and comprehensively determine threatened target assets when conducting TARA. The indicators for evaluating the probability of an attack are not comprehensive. Although the evaluation indicators in the IT security field have been introduced, they have not been optimized for the automotive field to fully describe the probability of attacks in the automotive field. automotive field.

B. HEAVENS
The HEAVENS security model adopt Microsoft's STRIDE [31] approach for identification of the threats associated with the assets of the evaluated objects. The STRIDE extends the original CIA model by correlating threats with security attributes, which can be used to discover and enumerate threats present in a software system. The threat level based on the attack complexity and the impact level based on the severity of the attack jointly determine the risk level.
The threat level based on the attack complexity and the impact level based on the severity of the attack jointly determine the risk level. Similar to the EVITA model, when evaluating the impact of an attack, HEAVENS considers four parameters,''Safety'', ''Financial'', ''Operational'', ''Privacy and legislation''. But HEAVENS has more detailed attack impact severity scoring rules, which can guide security assessors to conduct more detailed security assessments. However, HEAVENS oversimplifies the evaluation criteria while evaluating the complexity of the attack. It only considers the knowledge level of the attacker, the knowledge of the attack target, the window of opportunity for attack, and the complexity of the equipment used for the attack.

C. OCTAVE
The OCTAVE [32] is a process-driven threat assessment and risk assessment methodology and is best suited for enterprise information security risk assessments. OCTAVE is especially good at bringing together stakeholders with system experience and subject matter experts with security experience through a progressive series of workshops to develop a thorough organizational and technological view of the problem domain. A series of detailed worksheets are completed in the workshops to identify assets, current practices, Cyber security requirements, threats, and vulnerabilities and then to develop a strategy and plan for mitigating risks and protecting assets.
The OCTAVE method focuses on bringing together stake holders of security through a progressive series of workshops, which is best suited for enterprise information security risk assessments but not readily applicable for embedded automotive systems [33].

III. SECURITY CONCERNS OF MODERN VEHICLES
The advances in automotive electronics lead to the rapid increase in complexity and diversity of the in-vehicle ECUs. Each ECU relies on a set of sensors and actuators to serve one or more of the Electrical and electronic(E/E) [34] systems or subsystems in the vehicle. These ECUs are interconnected through various bus protocols, forming complex In -Vehicle Networks(IVNs), which play a critical role to improve road safety and driving conditions in the Intelligent Transportation Systems(ITSs) [16], [35].With numerous sensors, actors, and processors being connected on IVNs, a vehicle provides drivers and manufacturers with the data-processing services, including infotainment, vehicle diagnostics, Firmware Over-The-Air [36], and automatic driving [37]. Fig.1 is the typical IVNs based on CAN bus. The original CAN protocol is designed under the assumption that all ECUs are legitimate, trustworthy, and operating according to their specifications [38]. Without considering security, this results in several intrinsic vulnerabilities for the CAN protocol. Moreover, external interfaces such as the Second On-Board Diagnostic (OBD-II) [39], Bluetooth, Wi-Fi, and the Global Positioning System (GPS) provide opportunities for adversaries to break into the open in-vehicle systems through the unprotected CAN bus [40]. Once attackers compromise the CAN bus through numerous external interfaces, they can indirectly control the ECU, thereby realizing vehicle control, leading to serious hazards [41]. According to the automotive electronic and electrical architecture, the universal security evaluation framework for various in-vehicle ECUs involves ten security fields in four levels.

A. PHYSICAL LEVEL 1) HARDWARE SECURITY
The hardware security includes cyber security problems that may be caused by ECU hardware, including Printed Circuit Board security, bus security, hardware interface security, and chip security. Malicious attackers can obtain the information about the chip, hardware interface and bus protocol used in the evaluated object through PCB. Through the hardware interface, the attacker can obtain the startup information about target ECU, debugging privilege, interactive data, firmware, internal storage data, and other information. Through the bus, the attacker can obtain communication data, internal storage data and other sensitive data. By analyzing the chip, the attacker can obtain the chip power consumption, running time, electromagnetic information, which is helpful to deduce key, random number and other critical sensitive data.

2) IN-VEHICLE BUS SECURITY
The ECUs in the vehicle are connected by the buses. And the data is exchanged among ECUs with the help of various bus protocols. There are five bus protocols in modern vehicle for communication, CAN bus, Media Oriented System Transport(MOST) bus [42], FlexRay bus [43], Local Interconnection Network(LIN) bus [44] and vehicle-mounted Ethernet bus [45]. The bus protocols have various performance and are used in different communication scenarios. And the bus protocols were primarily designed for reliable communication without considering cyber security.The lack of encryption, authentication, and integrity checking introduces vulnerabilities for bus protocol making the vehicle vulnerable to cyber-attacks. So, the evaluation framework needs focus on the cyber security of in-vehicle bus to figure out potential cyber security threats.

3) SENSORS SECURITY
For achieving self-driving, the modern automobile is equipped various sensors to perceive surroundings. There are several kinds of sensors embedded in the modern intelligent vehicle, camera, Lidar, ultrasonic radar, millimeterwave radar, GPS, Beidou and so on [46]. Sensors offer the possibility of autonomous driving, while also introducing the serious cyber security threat to the automobile that has self-driving capability. The malicious attacker can blind or disturb the optical camera to launch the deny of service attack. Besides, the attacker can forge the surrounding information that will be caught by the camera, Lidar and radar to deceive the sensor data processing algorithms, resulting in dangerous autopilot decisions.

B. SYSTEM LEVEL 1) FIRMWARE SECURITY
It is very important for malicious attackers to obtain firmware of ECUs. The attacker can expose the operating logic of the target ECU through reverse engineering technology. The negligence on the part of developers can lead to the disclosure of sensitive data such as passwords, web addresses, accounts, and email addresses that are hard coded in the firmware.

2) SYSTEM SECURITY
The modern vehicle has more and more ECUs that are composed of several embedded systems, controlling the actuators in the vehicle that provide the vehicular functionalities. As for the infotainment system, the ECU is very complex, consisting of an operation system and various applications, which introducing a number of cyber security concerns into the modern automobiles. The system security requires the evaluator to conduct the cyber security evaluation of the vehicle ECU operating system, the system services and applications running on it to identify potential cyber security threats.
The modern vehicle is no longer an information island. With the development of the IoV technology, vehicles need to communicate with surrounding vehicles, cloud devices, mobile devices, sensors and road infrastructures. And the network architecture of the vehicle becomes more and more complex. The modern vehicle uses Bluetooth, WiFi, NFC/RFID, cellular and other wireless communication technology that occupy various radio frequencies. Because the radio channel is open, any attackers can eavesdrop and manipulate the wireless communication traffic. If the traffic is not encrypted and no integrity check mechanism is adopted to protect the traffic, the malicious attackers can obtain the secrets in the traffic and temper the traffic at will.

2) NETWORK SECURITY
The radio communication security is more concerned about the physical layer and the evaluator is more concerned about the security of the wireless protocol itself. The network security focus on the security of network traffic between traffic the vehicle and the cloud. Compared with the radio communication security, the network security involves a higher level of communication protocol, such as network layer, transport layer and application layer. The malicious attacker may exploit the vulnerabilities of network communication procedure to invade the target evaluated object.

D. APPLICATION LEVEL 1) WEB SECURITY
For more efficient management and more convenient service, there are some automotive management and platform based web applications. The web application likely introduces vulnerabilities that are exploited by the malicious attackers to access the private data, leading to serious information disclosure.

2) APPLICATION SECURITY
For convenience, the vehicular manufacturers provide the functionality that users can control specific automotive behavior through applications of mobile phone. And the applications have potential cyber security threat against the vehicle. The hackers may use the application as a springboard to launch attacks against the target vehicle.

3) PRIVACY SECURITY
In the IoV service, vehicles need to constantly broadcast their real-time information to surrounding vehicles, including VOLUME 9, 2021 speed and position, to improve traffic efficiency. The speed, position and other personal information are the privacy that need to be protected. Once the malicious attackers obtain the critical private data, they may pose threats to the drivers' life and property.

IV. CYBER SECURITY EVALUATION FRAMEWORK
The proposed Cyber Security Evaluation Framework (CSEF) is described in Fig.2. The ''Object'' means the evaluated object in detail, including hardware and software composition, logic functionalities, security features. The ''Hypothesis'' specifies the supporting environment that the evaluated object needs to run properly, avoiding system errors introduced by abnormal operations. The ''Object'' and ''Hypothesis'' are the premise of CSEF that restricts the scope of cyber security evaluation. Based on the ''Object'' and ''Hypothesis'', the framework conduct the asset identification, threat analysis, and the risk assessment, which derives the ''Asset'', ''Threat'', and ''Risk''. The ''Asset'' indicates the data, service, and privilege assets the evaluated should protect. The ''Threat'' comprehensively describes the potential attack scenarios from the target assets, attack path, and attack impact on the assets. The ''Risk'' considers financial damage, personal injury, operational failure, and privacy disclosure that a certain threat may cause. Since the ''Threat'' and ''Risk'' are only theoretical. We should conduct the practical ''Test'' case to verify if the threat will actually causes the corresponding risk. And the ''Security Objective'', derived from the ''Threat'' and ''Risk'', is the practical test criteria. The ''Security Objective'' describes the requirements that the evaluated object needs to protect the target ''Asset''. When the test result show that the evaluated object does meet the security requirements, the evaluated does have the ability to protect the target assets. And the test passes.

A. OBJECT
The ''Object'' describes the target object that will be evaluated in details. The object defines the physical and logical boundaries of the evaluated target, distinguishing it from the external entities that do not need to be evaluated. With the help of the ''Object'', the evaluator can have a detailed knowledge about the hardware, software, logical functionalities, and security capabilities of the evaluated target object, so as to a deeper understanding of the evaluated object and the more sophisticated cyber security evaluation scheme.

B. HYPOTHESIS
The ''Hypothesis'' explains the assumption we should make about the environment such that the evaluated object is able to provide functionality properly. If the object under evaluation is placed in an operating environment that does not meet these assumptions, the evaluated object may no longer provide all of its functionalities. The hypothesis can be about the physical, human, and runtime aspects of the environment.
With respect to the physical aspects of the physical operating environment, we may assume that the evaluated object is placed in a room designed to minimize electromagnetic radiation, or the admin console of the evaluated object is placed in a restricted access area. In terms of operator, we assume that users are sufficiently trained to operate the evaluated object, or users will not write down their passwords. On the assumption of runtime environment, we may assume that a PC workstation has at least 10 GB of available disk space to run the evaluated object, or the evaluated object will not connect to an untrusted network.
We should note that these assumptions are considered to be true during the assessment: they will not be tested in any way. For these reasons, we can only make assumptions about the running environment and the behavior of the evaluated object must never be assumed.

C. ASSETS
The cyber security is related to the assets that need to be protected. The malicious attackers will try to acquire assets to obtain financial interests or to destroy the target system. Assets come in a variety of forms, from the contents of files or servers, to the availability of instant messaging programs, to the operating privileges of confidential facilities, and so on. Many assets are stored, processed, and transmitted by IT products in the form of information to meet the requirements of the information owner. To avoid the subjective assessment leading to the subjective assessment to situation where almost anything can become an asset, we classify assets into three categories: data, service, and privilege. All attacks against the evaluated object must be aimed to manipulate the three kinds of assets.

1) DATA:
The data is the asset that is stored, processed and transmitted in the evaluated object. The attacker can obtain and tamper the data asset, and can also manipulate the data asset by impersonating the authorized user or the communication entity of the target data. Attributes of the data asset consist of confidentiality, integrity, and availability.

2) SERVICE:
A service is a function provided externally by the evaluated object. Attackers can take corresponding actions to force the target to lose the ability of external service delivery, which will affect the availability of the target service. The attribute of the service asset is availability.

3) PRIVILEGE:
The evaluated object usually sets different levels of permissions for system resources based on different roles.The privilege of different level can access different resource. Malicious attackers often want to gain higher privileges to access more important resources. The attribute of the privilege asset is availability.

D. THREATS
A threat consists of a hostile behavior to the asset, which will affect one or more attributes of the asset, and the asset reflects its value through these attributes. A threat agent can be described as a single entity, but in some cases it may be better described as an entity class or group of entities. Threat agents can be attackers, users, computer processes, unexpected events, and so on. The threat can be further described in terms of expertise, resources, opportunities, and motivations.
In the proposed framework, we represent the threat with several attributes, including name, description, type, entry, path, connectivity, threatened asset, risk, vulnerability and countermeasure. According to the STRIDE threat model, there are six types of threats: spooling, tampering, repudiation, information disclosure, Deny of Service (DoS), and elevation of privilege. The entry, the path, and the threatened asset describe in detail how potential attacks pose a threat to targeted assets. The potential attack behind the threat may access the evaluated object through physical connectivity, near-field wireless connectivity and remote wireless connectivity. The risk describes the consequences that a potential attack might have on the system and the likelihood of a successful execution of the potential attack. The vulnerability is the attack event that is actually occurred. And the countermeasure is the security mechanism we should take to protect the target system against known threats.
The proposed framework adopts the attack tree model to model the threat.The framework will model the threat with ten aspects based on the hardware architecture, software architecture, and the external interactive entity of the evaluated object. As for the evaluated object that is related with the vehicle, the threat model should consider hardware security, firmware security, system security, In-Vehicle bus security, radio communication security, network security, web security, sensors security, and privacy security.

E. RISKS
The risk describes the serious consequences that a potential attack might have on the target object and the external world, the threat severity, and the likelihood of a successful execution of the potential attack. The level of the risk is determined by the matrix of threat severity and impact severity.

1) THREAT SEVERITY (TS)
The automotive industry is similar to the traditional computer industry, but there are some differences. The automotive industry has stringent security requirements. The harm caused by security accidents in the computer industry may only be financial loss or privacy leakage, but security accidents in the automotive industry may cause personal injury. Secondly, in the long lifecycle of automobiles, the architecture of in-vehicle ECU changes slowly, making the security evaluation more valuable. Based on the above characteristics, the framework optimized the Common Vulnerability Scoring System (CVSS) [47] to propose a threat severity assessment mechanism.
In general, the threat severity takes into account a number of different factors, including attack vector, attack scope, attack method, time window, expert knowledge, target information, attack equipment, privileges required, user interface, confidentiality impact, integrity impact, availability impact, and weight coefficient. The 13 metrics are divided into three groups: exploitability metrics, knowledge metrics, and impact metrics. The exploitability metrics represent the specific details of the actual attack of the threat. The knowledge metrics represent the information, knowledge, and authorization before the attack of the threat. And the impact metrics describe the impact on the evaluated object introduced by the threat. The parameters and scores are shown in TABLE 1. The attack vector measures how an attacker connects to a target system. There are three kinds of attack vectors. The remote wireless connectivity means that an attacker can launch a remote attack on a target via either LTE or 5G cellular networks. The near-field wireless connectivity means that an attacker can launch a certain degree of remote attack to the target only through WiFi, Bluetooth and other short-range wireless networks. The physical attack indicates that an attacker must physically access the target in order to launch an attack.

a.2) Attack Scope(AS):
The attack scope refers to the range that the threat affects. Some threats affect only one target, some affect multiple targets, and some affect all targets.    c.1) Confidentiality Impact(CI): The metric indicates the impact on the confidentiality of data. The attacker may obtain critical data that has serious and direct impact on the target. For example, if the hackers obtain the key material of the system manage, they can access most resource of the target system.

c.2) Integrity Impact(II):
The metric indicates the impact on the integrity of data. An attacker can modify the data as he wishes to make it completely incomplete or completely unprotected.

c.3) Availability Impact(AI):
The metric indicates the impact on the availability of data. An attacker may cause a complete denial of service or degrade the performance of the target data system.
We define the Score TS as the score of the threat severity, the Score EM as the score of the exploitability metrics, the Score KM as the score of knowledge metrics, and the Score IM as the score of impact metrics.The setting of the score can qualitatively indicate the importance of a certain metric, and the value refers to CVSS. If necessary, other values can also be used completely, as long as it can reflect the difference between different parameters of a certain metric. And then, the scores of metrics are calculated as follows.
Score TS = 10 * Score EM * Score KM * Score IM The α 1 , . . . , α 5 are weight coefficients of the metrics in the exploitability metrics group. The β 1 , . . . , β 4 are weight coefficients of the metrics in the knowledge metrics group. And the γ 1 , γ 2 , γ 3 are weight coefficients of the metrics in the impact metrics group. The S with various subscripts represent the value of metric parameter. To restrict the score within 10, we use the coefficient 10 in the formula for calculating the Score TS . According to the score of the threat severity, the threat can be classified into several levels by the proposed evaluation framework, as depicted in TABLE 2.

2) ATTACK PROBABILITY(AP)
The probability that a potential attack will succeed is called attack probability. The attack probability varies according to the threat analysis and risk assessment methods used in the evaluation framework. The framework measures the attack probability of a special risk item from five impact factors, namely professional knowledge, auxiliary tools, target knowledge, target environment and time cost, as shown in TABLE 3. Professional Knowledge(PK): The professional knowledge indicates the level of expertise required by the attacker to carry out the attack. The attacker may be an amateur, skilled person, expert, or expert in a variety of fields.

b: AUXILIARY TOOLS(AT)
The auxiliary tool indicates whether the tool is special tool or the tool that can be obtained publicly.

c: TARGET KNOWLEDGE(TK)
The target knowledge indicates whether the information about the target object can be obtained publicly. In some case, we cannot obtain any information about the target object because of the private information.

d: TARGET ENVIRONMENT(TE)
The metric tells us whether the attack environment is simple or complex. In the simple attack environment, it is easier for an attacker to access the target and carry out an attack.

e: TIME COST(TC)
The time cost indicates how long the attack will take. The factors affecting the time cost include the construction of attack environment, the collection of target object information, the development of attack tool, vulnerability mining, vulnerability verification and so on.
We define δ as the vulnerability conversion factor that indicates the probability of a successful potential attack in the proposed evaluation framework. The S with various subscripts represent the value of metric parameter. And α 1 , α 2 are weight coefficients of the metrics.

3) IMPACT SEVERITY(IS)
The impact severity indicates the potential harm of the threat. The proposed evaluation framework takes financial damage, personal injury, operational failure and privacy disclosure as the reference indexes to evaluate the potential impact level of a certain threat.

a: FINANCIAL DAMAGE(FD)
The financial damage considers all financial losses that can be either direct or indirect. Direct financial damages may include product liability issues, legislation issues and product features. For example, the attack may lead to product recalls and significant losses. And the attack may result in loss of sales due to product defects. On the other hand, indirect financial damages include damage to reputation, loss of market share, intellectual property infringement and etc. To summary, the financial damage is the sum of direct and indirect costs for the manufacturer and the root cause may originate from any of the stakeholders.

b: PERSONAL INJURY(PI)
The personal injury indicates the damage to the person caused by the attack against the evaluated object. And the safety to the passengers and pedestrian is the highest priority. According to ISO 26262 [48], the injury can be classified as no injury, light and moderate injuries, severe injuries, and life-threating injuries and fatal injuries.

c: OPERATIONAL FAILURE (OF)
The operational failure includes operational damages caused by unexpected loss or control of a vehicular function. The attacker may control the vehicular function or make the vehicular function deny of service. Examples of such operational damages include critical and secondary functionalities loss. However, in certain situations, operational damages may cause safety and financial damages. If the safety-related vehicle functionalities are controlled by malicious attackers, the personal safety of passengers and road users will not be guaranteed.

d: PRIVACY DISCLOSURE(PD)
The privacy disclosure considers damages caused by privacy violation of stakeholders, such as vehicle owner, driver and passengers. Usually, the privacy disclosure do not have direct injury, financial and operational dimensions. However, in certain situations, privacy violations may lead to the loss of access to certain market and operational damages to the stakeholders. The proposed evaluation framework refers to the HEAV-ENS Security Model putting forward the impact severity assessment method. The parameters and scores are shown in TABLE 4. We define the Score IS as the score of the impact severity. The S with various subscripts represent the value of metric parameter. And then, the score of impact severity is calculated as follows. According to the score of the VOLUME 9, 2021   impact severity, the impact of a certain threat can be classified into several levels by the proposed evaluation framework, as depicted in TABLE 5.

4) RISK MATRIX
As depicted in TABLE 6, the proposed evaluation framework combines the threat severity and the impact severity level to derive the risk security level. The higher the threat severity level and the higher the impact severity level, the higher the possibility that the threat will be transformed into an actual attack and the greater the potential harm caused, the higher the risk level.

F. SECURITY OBJECTIVES AND TESTS
Threat analysis and risk assessment are usually conducted in the concept design phase of the life cycle to derive the security objectives that will be achieved through technology in subsequent development phase. However, the products in the markets are in the operation phase. After the threat analysis and risk assessment, the evaluation framework just presents the theoretical cyber security threats and the corresponding risk, which is not enough for the products that have been finalized and marketed. Given that, we need to design the conduct the test to verify whether the product meet the security objectives derived from the threat analysis and risk assessment. The security objectives are the test criteria of the evaluated object. If the evaluated object does not meet the security objectives, there may be vulnerabilities. The test case is designed based on the threat and risk and is conducted based on the security objectives. According to the cyber security framework of the vehicle, the test case in our evaluation framework is designed based on following ten aspects.

G. COMPARISONS
The comparisons between CSEF and other TARA methods are shown in Table 7. Compared with other methods that can only be used in the conceptual design stage, CSEF can not only reduce the cyber security risk of the in-vehicle ECUs in the conceptual design stage, but also guide testers to conduct cyber security testing after the ECUs are released. In addition, CSEF uses the attack tree method to identify important assets, which is more comprehensive than brainstorming.Compared with the traditional TARA method, threat analysis and risk assessment method in CSEF are deeply optimized for the automotive field.

V. USE CASE A. OBJECT AND HYPOTHESIS
The OBU under the experiment in our work is an active dual-chip pre-installed OBU product, which is an important in-vehicle ECUs in the ETC system. The ETC system is an internationally recognized effective technology to solve the problem of automatic charging at road, bridge, parking lot and other toll station. In the ETC system, the OBU communication with the RSU adopting the DSRC protocol. And the wireless communication technology effectively improve traffic efficiency and alleviate traffic congestion in Bridges and tunnels, expressway entrances and exits, urban trunk roads and other places with large traffic flow. As shown in Fig.3, the ETC system mainly includes the lane control system, background database system, RSU, OBU and Integrated Circuit(IC) card. The lane control system serves as road gate control, passage light control, vehicle capture, etc. The background database system assists the completion of registration, settlement, and related operations. And the OBU, RSU, IC card are used to automatic vehicle identification, automatic cost collection and other functionalities. The OBU stores the vehicle identification information such as the prepaid amount, vehicle model, vehicle color, license plate number and owner information. The RSU installed in the gantry frame at the road edge of toll station or above the lane use RF antenna and microwave technology to read the relevant information in the on-board OBU, identifying the vehicle and calculating the charge amount, thus completing the automatic non-parking charge.
The ETC system adopts 5.8GHz frequency band as communication frequency band, mainly including physical layer, data link layer and application layer. The physical layer specification provides data transmission, synchronization and timing functions to realize the physical connection of data transmission. The ETC technical national standard specifies the physical layer parameters and performance stan- dards for OBU devices. The data link layer specifies the key parameters, data frame format, communication link establishment,Media Access Control(MAC) and Logical Link Control(LLC) sub-layers in ETC communication process, which are used to regulate the communication between RSU and OBU. The application layer describes the core framework of ETC system and the basic services provided by the kernel.
The OBU plays a role of both microwave communication and information storage in ETC system. On the one hand, the OBU stores vehicle information, road information, owner information and other key information used for road and bridge cost settlement. On the other hand, the 5.8GHz microwave frequency band is used for DSRC communication between OBU and RSU. According to the power supply mode, the OBU can be divided into active OBU and passive OBU. After awaken, the active OBU will actively send the vehicle information stored in the OBU. The passive OBU transmits relevant information through induced current. According to the presence or absence of IC card, the OBU can be divided into single-chip type and dual-chip type. Singlechip OBU has no IC interface, which is of high risk. Dualchip OBU has IC card interface, which can fuse the functions of OBU and IC card with higher security and relatively high cost.
The chip and hardware interface of the OBU are shown in Fig.4. From the printed word in the printed circuit board, there are SWD and Universal Asynchronous Receiver/Transmitter(UART) interfaces for debug in the development phase. And there are BLE and CAN interfaces for communication with BLE devices and CAN bus network. The main chips used in the OBU are SE chip, BLE chip, ETC chip, CAN controller, and s32k118 Micro Controller Unit(MCU). The interfaces and chips in the OBU can help to infer the functionalities provided form the OBU. In summary, the information obtained from the OBU hardware board is as follows.
1) The Bluetooh Low Energy(BLE) chip and BLE communication ability.
2) The security element chip with secure storage and the data encryption/decryption ability.  3) The CAN controller and communication functionality with vehicle-mounted CAN network. 4) The MCU to coordinate and control the operation of each chip module of the OBU. 5) The ETC Radio Frequency(RF) chip to process DSRC communication data and the capability to communicate wirelessly with RSUs. 6) The UART, SWD, CAN, BLE, ISO7816 and other external interfaces. 7) The Inter Integrated-Circuit(I2C) and Serial Peripheral Interface(SPI) bus protocols for data transmission between chip modules. Based on the proposed security evaluation framework and the information of the OBU to be evaluated, we conducted the following experiment. The experiment analyzed the cyber security risks of the OBU products from five perspectives: asset, hypothesis, threat, risk and test. The asset is the valuable data, privilege, and functionality service in the target OBU. The hypothesis is a prerequisite for the cyber security evaluation of the target OBU, which ensures that the OBU to be evaluated can operate normally in the environment constrained in the hypothesis. The threat is a potential cyber security threat to target OBU. The risk is the potential harm that cyber security threat may bring. And the test assesses the probability that the risk will translate into an actual attack.
The hypothesis specifies the preconditions for stable and normal operation of the target OBU. The use of hypothesis in the cyber security evaluation framework can eliminate cyber security threats caused by manufacturing, improper operation, improper management, etc. The hypothesis of the OBU is as follows.

1) SECURE MANUFACTURING
During the manufacturing integration phase of the OBU life cycle, it is assumed that appropriate technologies and measures have been taken to ensure the security of the OBU assets, including the generation, installation and import of materials such as the initial key.

2) STANDARDIZED OPERATION
At each stage of the OBU life cycle, it is assumed that the operation of the target OBU is performed in accordance with the standard procedure and will not infringe the OBU asset due to improper operation.

3) STANDARDIZED MANAGEMENT
It is assumed that all documents, data, keys and other materials related to the target OBU are under the control of the management, and no material leakage event will occur.

4) TRUSTED STAFF
It is assumed that all the technicians and managers who come into contact with the OBU before it is put on the market are credible and will not pose a security threat to the OBU assets.

5) PHYSICAL PROTECTION
Suppose that the OBU product has a corresponding physical protection mechanism.

6) RUNNING NORMALLY
It is assumed that the OBU is running normally during the running phase of the life cycle and will not run incorrectly with no reason.

B. ASSETS OF OBU
According to the ''Object'' and '' Hypothesis'', the framework conduct the asset identification, threat analysis, and the risk assessment. Based on the knowledge of the OBU in section ''Object and Hypothesis'', the firmware in the OBU is the binary codes without operating system. And the OBU does not communicate with the cloud server. So, according to the security concerns in section II and the characteristics of the OBU, the proposed evaluation framework identifies the valuable assets in the OBU from four dimensions of hardware security, firmware security, in-vehicle bus security, and radio communication security.
The assets of the target OBU are listed in TABLE 8. The CIA (Confidentiality, Integrity, and Availability) security model is adopted in the evaluation framework to indicate that we should protect which security characteristic of the target asset.
According to the information obtained from the hardware board and the functionalities inferred, we can identify the assets of OBU. The data assets include the data that needs to be transmitted during the communication process and the data stored by the device. The Privilege assets include system code execution permissions that can be leaked through device hardware and wireless interfaces. The service assets include the service provided form the OBU.

C. THREATS AND RISKS
In order to apply the security evaluation framework to the automotive field, we uses the attack tree method to conduct threat analysis under the automotive security framework. The threats described in TABLE 9 are classified as follows. All assets in the TABLE 8 have threats and potential risks. All threats will be analyzed in order to grasp the potential attack point of the target OBU. In addition, the risk level will be provided to indicate the severity of the impact to person, environment, and vehicle.

1) HARDWARE SECURITY THREATS a: PCB DATA LEAKAGE
The printing word on the PCB board of OBU and other in-vehicle ECUs will reveal information such as hardware debugging interface, chip, communication protocol, etc., which is of great significance for information collection before malicious attacks.

b: UART DATA LEAKAGE
In addition to being used for serial communication, the UART interface is often used as a debugging interface. A malicious attacker can physically contact the UART interface to obtain system startup information, including key information such as u-boot version, chip name used by the system, memory layout, and so on.

c: UART PRIVILEGE
Malicious attackers can not only obtain sensitive data of the ECU and system through the UART interface, but also obtain system privileges. If the developer reserves a command line debugging interface with super authority in the UART interface, the attacker can crack the login password to obtain the system super authority, which will bring great harm.

d: SWD DATA LEAKAGE
If the debugging function is not turned off before the ECU is re-released, the attacker can start the hardware debugging function of the target ECU through the Joint Test Action Group(JTAG) or SWD debugging protocol. This functionality allows the attacker to read the data inside the CPU and the firmware of the ECU.

e: SWD PRIVILEGE
Besides the firmware and the sensitive data inside the CPU, the SWD interface may allow the attacker to load the firmware into the ECU to achieve the purpose of controlling the ECU.

f: I2C/SPI/ISO7816 DATA INTERCEPTION
With the help of an oscilloscope and digital logic analyzer, the attacker can monitor and parse protocol data to obtain communication content. If the time is right, the attacker can even execute the attack within the firmware transfer window to obtain the complete firmware.
g: I2C/SPI/ISO7816 DATA TAMPERING On the basis of monitoring and parsing the protocol data, the attacker can change the protocol data transmitted on the bus within a specific time window to interfere with normal data transmission.

h: SE CHIP DATA LEAKAGE
Although it is a difficult task to compromise a security chip, it is still a potential security hazard to attack the security chip through side channel analysis, fault injection attacks to access the data inside the chip. Attackers may obtain sensitive information such as keys stored in the security chip.
i: SE CHIP DATA TAMPERING Furthermore, an attacker can tamper with the data stored inside the chip if the security chip have been compromised.

2) FIRMWARE SECURITY THREATS a: FIRMWARE LOGIC LEAKAGE
After obtaining the firmware, the attacker can reveal the functional logic of the firmware through reverse engineering which helps understand the operation of the target ECU and discover potential security vulnerabilities that can be exploited.

b: FIRMWARE DATA LEAKAGE
In addition to the function logic, due to the negligence of the developer, there may also be plaintext data in the firmware, which may expose sensitive information such as key IP addresses, email addresses, and security keys.

c: FIRMWARE LOGIC TAMPERING
It is obvious that the attacker can tamper the content of the firmware to precisely control the target ECU.

3) IN-VEHICLE BUS SECURITY THREATS a: CAN DATA INTERCEPTION
The CAN bus protocol lacks identity authentication, and attackers can forge malicious nodes to connect to the vehicular CAN bus network. Based on the broadcast transmission mechanism of the CAN bus, an attacker can receive all the data transmitted by the CAN bus. Due to the lack of security encryption mechanism, an attacker can obtain the content of the data transmitted by the CAN bus.

b: CAN DATA TAMPERING
Not only the interception, the CAN bus protocol does not have a data integrity check mechanism, and malicious VOLUME 9, 2021 attackers can tamper with the data content without the receiver's awareness.

c: CAN ENTITY SPOOFING
As mentioned above, without the protection of an identity authentication mechanism, a malicious attacker can fake a CAN controller as a legitimate node to connect to the vehicular CAN bus network.

d: CAN BUS DoS
In addition, the CAN bus protocol uses a priority-based blanking mechanism to solve the problem of bus competition. The attacker can construct CAN bus data frames with higher priority and send them to the CAN bus to occupy CAN bus resources for a long time, causing denial of service effect.

4) RADIO SECURITY THREATS a: DSRC/BLE DATA INTERCEPTION
The openness of wireless communication channels allows any attacker to capture electromagnetic waves transmitted in the air through software define radio equipment and parse them into data bit streams according to protocol specifications. Without an encryption algorithm to protect the transmission content, an attacker can obtain the transmitted message content.

b: DSRC/BLE DATA TEMPERING
In the absence of a data integrity check mechanism, an attacker can tamper with the data content transmitted in the wireless channel.

c: DSRC/BLE ENTITY SPOOFING
If the wireless communication protocol does not use an identity authentication mechanism or uses a weak identity authentication mechanism to verify the identity of the communicating entity, an attacker can break through the identity authentication and forge an arbitrary communicating entity.

d: BLE DoS
Like other wireless technologies, Bluetooth is also vulnerable to DoS attacks, making the ECU's Bluetooth interface unusable and draining the ECU's battery.
In summary, the attacker may launch an attack on the target device through any potential attack path, in order to steal the target device's data, destroy its service, or obtain the execution privilege of the target device. And in terms of the impact of attacks on target assets, threats can be classified using the stride model.
According the threat analysis, we assign the attributes of the threat to calculate the threat severity level. And the impact severity level can be calculated based on the risk assessment. Furthermore, the risk level can be determined by the risk matrix that comprehensively considers the effects of threat severity level and impact severity level on risk level. The specific risk level values are shown in TABLE 10.

D. SECURITY OBJECTIVES AND TESTS
After TARA, we have systematically grasped the threats faced by the target OBU and the potential impact that the threats may cause. In order to reduce the security risk of the target OBU, theoretically, corresponding security measures need to be taken to protect the OBU from threats. These security measures are the security goals. However, whether the target OBU has actually taken security measures requires security testing to verify. Only by passing a security test with clear inspection standards, does it show that the target OBU has a certain degree of security protection capability, which can be regarded as achieving the security goal.
The security objectives against threats are shown in TABLE 11. Every security threat has a corresponding security goal as a security protection measure. We need to design test sets to verify whether the OBU actually meets the security goal. If a certain security goal is not met, the OBU faces security threats, and there are corresponding security risks introduced by the threat. The test cases are following.

1) PCB TEST
In the PCB test, the security assessor needs to carefully examine the printed information on the PCB with the help of tools such as a microscope and a magnifying glass to see if it has leaked information such as UART interface information and chip model information.

2) UART TEST
In uart test, we connect the test machine to the uart interface in the OBU with the USB to Transistor Transistor Logic (TTL) tool that supports uart communication protocol. The test machine runs a Serial debugging software tool such as minicom [49]. With the appropriate baud rate, we observe whether the OBU outputs startup information when power up. If there is startup information from uart, we test whether attacker can enter the bootloader shell by pressing any key. And we need to confirm whether it will automatically enter the system shell after the startup process is completed.

3) SWD TEST
The SWD test is similar to the uart test, but the tools used are different. In the SWD test, we connect the test machine to the swd interface with the JTAG debuger such as J-Link. With the help of openOCD [50], a debug software in the Linux platform, we can read data from the MCU and write data to the MCU, which can help to test whether the SWD interface introduce risk to the OBU. If the OBU permit hardware debug through SWD interface, there is security risk.

4) FIRMWARE TEST
After obtaining the firmware, we should analysis the firmware to confirm that the firmware will not disclose sensitive information such as passwords and URLs. With the help of reverse engineering, we also can ensure whether the attacker can easily figure out and tamper the functionality logic of the OBU.

5) I2C/SPI/ISO 7816 TEST
In the I2C, SPI, and ISO 7816 test, We use a digital logic analyzer to capture and analyze the electrical signals on the chip's I2C pins, SPI pins, and ISO 7816 pins to verify whether the data can disclose critical information.

6) DSRC TEST
We capture the radio signal between OBU and RSU with the help of USRP B210 [51] from Ettus, an soft defined radio(SDR) tool. After analyzing the signal based on the DSRC protocol specification, we can obtain the transmitted data. And we can simulate RSU and OBU by replaying data or sending fake data.

7) BLE TEST
With the Ubertooth one [52], a 2.4G Bluetooth data sniffer, we can capture the data between Bluetooth devices to verify whether the data is encrypted. And we can simulate Bluetooth data packets to test whether we can fake a Bluetooth device. Send a large number of junk data messages on the Bluetooth communication channel to test whether it can block the Bluetooth communication and cause a denial of service.

8) CAN TEST
The CAN bus protocol lacks security mechanisms such as encryption, integrity verification, and identity authentication.  We can use the CAN transceiver to intervene in the CAN network to monitor and tamper with the bus transmission data and counterfeit the transmission node. Based on the arbitration mechanism of the CAN bus protocol, a higher priority data message can be sent to the CAN bus to preempt bus resources, resulting in a denial of service.

9) SE CHIP TEST
The SE chip provides cryptographic calculation and stores key keys. The side channel attack can be used to analyze whether the SE chip has leaked the key during operation.
In summary, as depicted in TABLE 11, the OBU under our evaluation has three kinds of security threats: information disclosure, spoofing, and elevation of privilege. Although the data leakage of I2C and SPI bus allows the attacker to obtain the corresponding data, it is very difficult to parse and utilize the data due to the high coupling between bus communication and running time window. In addition, although the BLE module will expose the subservices provided by the OBU, they are all public services and cannot be effectively utilized. For the DSRC communication, the communication data between the OBU and RSU can be intercepted by software radio tools. However, since the DSRC communication data is encrypted, the RSU will verify the OBU's validity, and it becomes very difficult to effectively utilize the intercepted data. Although the CAN bus have no security mechanism, it is difficult to attack the CAN bus because it requires physical access.
Different from information disclosure, the elevation of privilege in this OBU evaluation has a high level security risk.The target OBU exposed the SWD debugging interface. Unfortunately, the vendor did not turn off SWD debugging functionality, but instead opened debugging permissions. Attackers can use OpenOCD, ST-Link, and other tools to debug the target OBU obtaining the internal stored data, firmware, and malicious code execution privilege of the target object.
The target firmware data obtained in the experiment is shown in Fig.5. The firmware contains two pieces of main logic, which are used in different phases of the OBU's life cycle. The SWD debugging permission is very dangerous and can cause significant property damage. In addition, in this experiment, the OBU opens debugging permission by default, so attackers do not need to bypass SWD read and write protection mechanism, which greatly reduces the difficulty of attack.

VI. CONCLUSION
In order to better apply the security assessment to the IoV, we analyzed the security architecture of the IoV in detail, and proposed a security framework for the IoV. The security framework focuses on ten security aspects of smart vehicles and in-vehicle ECUs at four levels, which regulates the scope of security evaluation. In addition, we proposed CSEF that can be applied to in-vehicle ECUs to evaluate the cyber security of in-vehicle ECUs.The CSEF is designed based on the ISO/SAE 21434 standard and is optimized to have richer security assessment details, which can be better applied to the field of automotive security. The framework aims to solve five main problems:(1) identifies the assets of the evaluated objectives from ten aspects. (2) Identifies the cyber security threats faced by in-vehicle ECU through threat analysis that uses the attack tree method. (3) Rates the security risks faced by ECUs in the vehicle based on financial damage, personal injury, operational failure, and privacy disclosure. (4) Defines the security requirements of the identified asset (5) Confirms whether the evaluated target has security vulnerabilities that does not meet the cyber security target through the security test set. To show how to apply CSEF in the security evaluation of in-vehicle ECUs, we provide a use case of on-board OBU. The use case showed that the evaluation framework can be used to expose most threats and the potential vulnerabilities introduced by inappropriate design or coding. With the help of CSEF, OBU developers and other roles can have a deep grasp of the security status and potential security risks of the designed products. Security goals can be proposed to effectively protect the target products according to the threats and risks, and test cases can be used to verify whether the evaluated object meet the security goals. CSEF can plays an important role in guiding relevant personnel to conduct security evaluation activities.Based on the universal security framework proposed in section II, CSEF can be extended to other ECUs in the vehicle.