Safety Evaluation of Autonomous Vehicles for a Comparative Study of Camera Image Distance Information and Dynamic Characteristics Measuring Equipment

Currently, the stage of technological development for commercialization of autonomous driving level 3 has been achieved. However, the legal and institutional bases and traffic safety facilities for safe driving on actual roads in autonomous driving mode are insufficient. Therefore, in this study, a measurement method using a camera(monocular or dual) was used to evaluate autonomous vehicles. In addition, integrated scenarios was proposed wherein the scenarios proceeded continuously. The precision of the autonomous vehicle safety evaluation method using cameras was verified via comparisons and analyses with the results of real vehicle tests. As a result of the test, the difference in the average error rate of inter-vehicle distance between the monocular camera and the dual camera was 0.34%. The difference in the average error rate of the distance to the lane was 0.3 to 0.5%, showing similar results. It is judged that it will be possible to compensate for each other’s shortcomings if they are used at the same time rather than the independent use of monocular cameras and dual cameras.


I. INTRODUCTION
Recently, both the Korean government and private companies are focusing on policies and developmental research to realize autonomous driving. The US Society of Automotive Engineers (SAE) has classified the technology required for autonomous vehicles into six stages through the SAE J3016.
Level 0 is the stage in which the system simply warns or temporarily intervenes while driving, including autonomous emergency braking (AEB), blind spot detection (BSD), and lane detection warning (LDW). Level 1 is the stage in which the system supports steering or acceleration/deceleration, including lane keeping assistance (LKA) and adaptive cruise control (ACC). Level 2 is the partial automation stage where the system simultaneously supports steering and acceleration/deceleration under specific driving conditions. The highway driving assist (HDA) system is a representative The associate editor coordinating the review of this manuscript and approving it for publication was Michail Makridis. example that requires selective control of the driver. Level 3 is the conditional automation stage in which the system recognizes the driving environment and enables autonomous driving in a specific section of the road via vehicle control; however, driver control is required in the case of emergencies. Level 4 is the highly automated stage where the system has the ability to cope with emergencies by itself, but there are limitations in some specific road sections. Level 5 is a fully automated stage that enables autonomous driving on all roads under all conditions [1].
Currently, the technological development stage for the commercialization of autonomous driving level 3 has been achieved. In addition, autonomous driving technologies such as various sensors and vehicle controls, are being developed rapidly to commercialize autonomous driving level 4.
However, the legal and institutional bases and traffic safety facilities for safe driving on actual roads in autonomous driving mode are insufficient. Moreover, responding to unexpected situations in which other obstacles may block the autonomous driving sensor's operating range or braking distance, is physically insufficient and has limitations for traffic safety. Autonomous driving should be available for roads in which traffic constitutes both ordinary drivers and pedestrians, and the system must be able to recognize and judge surrounding vehicles, pedestrians, and objects using various sensors and high-precision maps. However, when autonomous driving can predict and prepare for various dangerous situations beyond the level of driving abilities of ordinary drivers and compete equally with skilled drivers, it can only satisfy the role and function of highly reliable autonomous driving. Therefore, the demonstration of autonomous driving capability evaluation is required and should be established from a human-centered perspective in terms of safety rather than stability.
Autonomous vehicles have significantly increased the complexity of the design of electric/electronic control systems. If an instantaneous malfunction occurs owing to the complexity of designing the system, a big problem, such as a traffic accident, occurs, and the malfunction must be prevented in advance.
An example of an accident caused by a malfunction is the T * * company accident in Japan in 2009, where the software controlling the system became a problem and an accident occurred. In the automobile industry, the international standard for automotive functional safety ''ISO 26262 -Road Vehicle -Functional Safety'' was established in 2011, based on the IEC 61508 international standard, which is the functional safety of electric/electronic/programmable electronic safety management systems. The 2nd edition was distributed through revision and supplementation in 2018 [2]. The purpose of ISO 26262 is to prevent risks arising from defects in systems and components, and to improve functional safety and reliability.
However, dangerous situations can occur even when the system or components are not defective. For example, there are no defects in the camera sensor, but the recognition function may be lost owing to sudden changes in illuminance. Consequently, malfunctions may occur and an unintended risk may also exist. To solve this problem, a new functional safety standard called ''ISO/PAS 21448 -SOTIF -Safety Of The Intended Functionality'' has been proposed [3].
Because autonomous driving artificial intelligence (AI) technologies rely entirely on road and traffic environment systems, both AI technology and road and traffic environments must be considered and evaluated for adequacy of driving ability for safety while coexisting with general road users in actual traffic environments. In addition, autonomous driving is required to prepare for future traffic safety issues, based on autonomous driving ability test evaluations. From a realistic perspective, the level of risk perception, compliance with laws, concessions, and ethical awareness within an acceptable range for road users based on the Road Traffic Act are important in addition to autonomous driving ability evaluations in terms of simple mobility; thus, a test evaluation system is needed for driving ability and behaviors in autonomous driving. However, driving ability evaluations and driver license systems for autonomous vehicles are insufficient in real-road driving, so only institutional bases for test operations and temporary driving permit systems are currently being implemented. Moreover, the current autonomous driving evaluation system focuses only on test evaluations for the structure, performance, and function of autonomous driving, and traffic safety for ordinary road users with autonomous driving is not considered. The autonomous driving ability evaluation method is based on the replacement of a series of processes that occur when a human is driving, and should be premised on the fact that people can empathize and socially accept conditions that are beyond those of human driving ability.
Thus, it is necessary to verify the various interactions that can occur in situations in which autonomous vehicles share operating spaces with numerous other objects, pedestrians, and general vehicles.
Until now, research trends in the safety evaluation of autonomous vehicles have been as follows.
Sung et al. [4] identified the current research trends of self-driving cars and investigated the recognition, judgment, and control of the core technology of self-driving car research. Yun [5] outlined the sensor convergence and control system, precision map-based self-driving car technology, and the related standardization progress. Ka et al. [6] analyzed domestic and foreign research trends in autonomous cooperative vehicle-only lanes among autonomous cooperative vehicle traffic operation strategies, and identified and analyzed effective strategies such as market penetration rates for the introduction of autonomous cooperative vehicles. Lee et al. [7] suggested the development direction of new ITS services and centers through research and analysis of the business status of next-generation intelligent transport systems (C-ITS) and self-driving technology research. Lee et al. [8] manufactured a platform capable of experimenting with Level 3 autonomous driving technology by mounting sensors such as radar, camera, and LiDAR in order to convert Level 2 vehicles that are mass-produced for control conversion research. Lee et al. [9] analyzed driver characteristics in level 3 autonomous vehicles and proposed necessary functions based on data obtained through tests for the general public for actual road driving. Ahn et al. [10] designed a safety concept and verified it through simulation to propose an evaluation item for evaluating the functional safety of the functional controller of the FSRA (full speed range ACC) system, a longitudinal autonomous support system, based on Euro NCAP's safety evaluation method, and NASS (National Automotive Sampling System) traffic accident DB. Shin and Kwon [11] constructed hypotheses and scenarios for the FOT of automated vehicles using V2X communication to analyze the results and effects of a field operational test (FOT), and Shin and Kwon [12] classified real-time control and post-processing data for efficient operation control and data analysis of autonomous vehicles in urban areas. Park and Son [13] proposed a baseline VOLUME 10, 2022 scenario for evaluating the safety of switching self-driving control rights for the five aspects of urgency, risk, time appropriateness, mental burden, and physical burden in a driving simulator.
Until now, research trends in test evaluation or distance measurement using cameras have been as follows.
Kim et al. [14] proposed a high-precision mapping method that classifies roads and background pixels using deep learning-based semantic segmentation algorithms in monocular camera images, and compared image analysis results and map information to predict the correct pose of the moving object. Kim et al. [15] used a small model vehicle camera. Kim and Suh [16] proposed a system that enables high-speed precise driving with only node unit recognition on a topological map without fast-cycle sensor data implemented using an inexpensive single-board computer. Kim et al. [17] proposed a system for detecting lanes through monocular cameras and detecting objects and estimating distances based on deep learning networks. Liang et al. [18] proposed a modified Velocity-Obstacle (VO) algorithm that calculates speed using probabilistic partial observation of the environment and searches for a robot as a target. Kim and Kim [19] proposed a visual mileage meter that can effectively track the real-time pose of a camera moving in 3D space from RGB-D input images. Ahn and Kwak [20] proposed an algorithm to recognize and track distant vehicles using a monocular camera installed in the center of the vehicle's front glass to operate unmanned autonomous vehicles in racing games. Bae and Lee [21] proposed a theoretical equation to calculate the distance to the vehicle ahead of the lane and the vanishing point detected in the image, the width of the lane, and geometric information. Bae and Lee [22] proposed a theoretical formula for measuring the distance to the lane using a monocular camera and conducted a real-time test based on the lanekeeping assistant system (LKAS) test scenario to verify the LKAS test evaluation method using a monocular camera. Lee et al. [23] mounted a dual camera composed of two general webcams on a vehicle to correct images and detect lanes, selected the optimal mounting position of the dual camera, and verified the calculation theory formula for an object ahead of the proposed straight and curved roads through a real-time test.
In addition, looking at the research trends in autonomous vehicles, Saeed et al. [24] established a classification of roadway infrastructure and discussed the challenges and opportunities associated with infrastructure preparation for autonomous vehicles. and Saeed studied the uncertainty in autonomous driving and developed a real options analysis framework along with project case studies to build facilities to improve network levels. Michelmore [26] developed a framework based on a state-of-the-art simulator to evaluate end-to-end Bayesian controllers. In addition to computing pointwise uncertainty measures that can be computed in real time and with statistical guarantees, also provided a method for estimating the probability that, given a scenario, the controller maintains car safety within a finite horizon. Cao et al. [27] presented the first study on the security issues of MSF-based perception in AD systems. Our attack is also stealthy, robust to victim positions, transferable across MSF algorithms, and physically realizable after being 3D-printed and captured by LiDAR and camera devices [27]. Erik [28] was studied as a simple but realistic model of an autonomous emergency brake (AEB) system.
Recent research trends include simulation-based autonomous vehicle evaluations, scenario creation, and testcase derivations. Research on safety evaluation theories and autonomous vehicle evaluation methods based on realvehicle tests is insufficient. Therefore, in the present study, a measurement method using a monocular camera and dual camera was used to evaluate the driving safety of autonomous vehicles. In addition, integrated scenarios are proposed, wherein the scenarios proceed continuously, and the precision of the autonomous vehicle safety evaluation method using cameras is verified via comparisons and analyses with the results from real vehicle tests.

II. THEORETICAL BACKGROUND A. TEST EVALUATION METHOD USING A MONOCULAR CAMERA
The test evaluation method using a monocular camera uses a theoretical equation to calculate the distances of the lead vehicle and the distances to the lane [21], [22].

1) ASSUMPTIONS AND CONDITIONS OF TEST AND EVALUATION METHOD USING MONOCULAR CAMERA
The assumptions and conditions of the test evaluation method using monocular cameras are as follows.
• The camera was installed at the midpoint of the vehicle width.
• The camera faced forward and was oriented parallel to the ground surface.
• The required rear-overhang value of the lead vehicle was known in advance.
• The hood of the test vehicle, lanes, rear tires of the lead vehicle, and the vanishing point were captured in the image obtained by the camera.

3) GEOMETRIC VARIABLE
The geometric variables related to the two vehicles in a lane are presented in Figures 2 and 3. Figure 2 shows the geometries of the camera-equipped vehicle and the lead vehicle, and Figure 3 shows the geometry of the cameraequipped vehicle and the lanes according to the heading angle ψ. Even if the slope of the test vehicle differed from that of the lead vehicle, the results of the theoretical formula and the real vehicle test, as well as the distance between the vehicles, were not affected up until a maximum slope of 5% (max. highway slope mandated by the highway design regulations of the Republic of Korea) In Figure 2, h 1 is the height from the ground to the camera, h 2 is the height from the hood to the camera, b and c are the distances from the camera to the hood and the front bumper, respectively, a is the distance from the front bumper to the spot hidden from the view by the hood, β is the angle to the spot hidden by the hood (with respect to the perpendicular to the ground), α is the angle between the lines joining the camera to both the spot hidden by the hood and the bottom of the rear tires of the lead vehicle. Further, l is the actual height of H 2 , l is the actual height of H 1 , d O,H is the rear-overhang of the lead vehicle, and d Image is the distance between the spot hidden by the front bumper and the rear tires of the lead vehicle.
In Figure 3, d left and d right are the distances from the left and right tires of the camera-equipped vehicle to the lane, respectively, d f.wh is the distance from the front tire to the front bumper of the vehicle, k is the distance from the endpoint of a to the lane, ψ is the heading angle between the vehicle and lane, θ is the camera angle of view, L is the lateral distance across the front view of the image with respect to the top of the hood.

4) FORMULATION
Equations (1) and (2) can be obtained using the aforementioned geometric relationships and the corresponding figures.
Equations (3)∼(6) can be obtained on the basis of the camera angle of view and mounting height.
The geometric relationships of the camera-equipped vehicle and lead vehicle can be used to derive Equations (7) and (8).
The geometric relationships between the camera-equipped vehicle and lane in terms of the heading angle can be used to derive Equations (9)- (12).
Equation (13) can be used to calculate the distance to the lead vehicle by applying Equations (1)- (12). Equations (14) and (15) can be used to determine the distances from the closest front tires to the left and right lanes, respectively. The following distance between the vehicles can be maintained via longitudinal control based on Equation (13), and lane keeping can be achieved via lateral control based on Equations (14), (15).

B. THEORETICAL EQUATION FOR DISTANCE MEASUREMENT USING DUAL CAMERAS 1) OPTIMIZATION OF THE MOUNTING POSITIONS OF DUAL CAMERAS
In a previous study [23], the optimal position of dual cameras for mounting on a vehicle was determined after image and focal length correction and lane detection. In addition, the distance of the front object for each mounting position of the dual camera was measured by image processing, and both the theoretical equation for measuring the distance to the front vehicle and the optimal position for the dual cameras were selected. The contents are detailed below. The value of the variable considers the specifications of the camera and vehicle. The camera heights were 30cm, 40cm, and 50cm. The minimum value was selected as 30cm because that the bumper height of a typical sedan is higher than 30cm, therefore, the camera cannot be physically mounted at a position lower than 30cm. The reason for selecting the maximum value of 50cm is that if the mounting height is greater than 50cm, the ground within 1m cannot be photographed owing to the vertical angle of view of the camera used in the test. The variables of the camera baseline were 10, 20, and 30cm. The minimum value was selected as 10cm because the distance between the left and right camera lenses cannot be smaller than 10cm because of the size of the camera used in the test. In addition, 20cm and 30cm were set to confirm the tendency for the value of the mounting interval. The camera angle variables were 3, 7, and 12 • . This method considers the position of the preceding vehicle and the ground captured in the image. For example, the minimum value is set to 3 • because the ground within 1m cannot be photographed when the value is less than 3 • because of the mounting height and vertical angle of view of the camera. In addition, the maximum value was set to 12 • because, when the mounting angle was greater than 12 • , the area occupied by the ground in the image increased, and the horizon was located at the top of the image, making it difficult to photograph a preceding vehicle located at a distance of 10m or more.
For, when the camera installation height was 40cm, the camera installation interval was 30cm, the camera shooting angle was 12 • , and the best results were derived, they were selected as the optimal location. The camera installation height was selected as 40cm because when the height increased from 30 to 40cm, the error rate decreased. When the height increased from 40 to 50cm, the error rate increased. The camera installation interval was selected as 30cm, because the error rate tended to decrease as the interval increased. Camera photography was performed at an angle 12 • because the error rate was likely to decrease as the value increased.

2) MEASUREMENT OF THE DISTANCE TO AN OBJECT IN FRONT OF THE VEHICLE USING DUAL CAMERAS
Dual cameras were installed collinearly such that the optical axes of the two cameras were parallel. The lenses were positioned at identical heights above the ground. The 3D coordinates of any object were calculated relative to the camera positions, based on the geometry and triangulation of the cameras depicted in Figure 4. This can be represented by Equation (16): where, X, Y, Z are the coordinates of the object, the local coordinate system with their origins at the center of the dual cameras, f is focal length, b is the interval, d is the disparity, xl, yl are the coordinates of the object in the left camera image plane, xr, yr are the coordinates of the object in the right camera image plane. The focal length is an essential parameter in the calculation of the Z-coordinates. However, a problem with the use of inexpensive webcams is that some manufacturers do not provide details such as focal length. Further errors originating from image correction necessitate accurate estimation of the focal length.
This can be achieved by employing curve fitting based on actual data. Based on the relationship between distance and disparity, where Z actual is calculated using Equation (17): where, Z actual is the distance to the object, α, β are the coefficients obtained via focal length correction. During testing, images of objects installed at intervals of 0.5m over the range of 1-5m were captured. In addition, the differences between the X-coordinates of each object captured by the two cameras were recorded. Then, α and β were evaluated by fitting the curve described by the differences calculated using the least-square method.
The Z-coordinate of an object in front of the vehicle was obtained using the coefficient α in Equation (17) (as evaluated via focal length correction) as the focal length and substituting it into Equation (16). However, during testing, the cameras were mounted at an angle θ, to capture closerange ground. That is, the optical axes of the cameras and the ground were not parallel during testing.
The Z-coordinate of the object relative to the position of the camera can be calculated by considering angle θ in Equation (18): where, X g , Y g , Z g are the coordinates of the object considering the angle of inclination of the mounted cameras, the local coordinate system with their origins at the center of the dual cameras, θ is the angle of inclination of the mounted cameras, h is the mounting height of the camera. On a straight road similar to that depicted in Figure 5, the calculation of the distance between the cameras and the object in front of the vehicle requires only an estimation of the longitudinal vertical distance. Therefore, Z g can be considered the distance between the cameras and the object in front of the vehicle.
On a curved road similar to that depicted in Figure 6, the radius of curvature of the road should be incorporated into the measurement of the distance to the object in front of the vehicle. Therefore, after calculating the vertical distance using the X-and Z-coordinates of the object, the distance  to the object in front of the vehicle was calculated by considering the radius of curvature.
The distance to the object in front of the vehicle can be represented by Equation (19).
where, chord is the vertical distance between the vehicle and object, X g1 , Z g1 are the x-and y-coordinates of the vehicle on the ground, respectively, X g2 , Z g2 are x-and y-coordinates of the object on the ground, respectively. The angle φ is subtended at the center of the curvature and the vertical distance from the camera position to the object in front of the vehicle. The angle φ can be calculated using Equation (20): where, φ is the angle subtended by the vehicle and the object at the center of curvature of the road and R is the radius of curvature of the road. The length of the arc of the circle corresponding to the aforementioned chord was calculated using Equation (21) VOLUME 10, 2022 using ϕ and R: where, arc is the distance between the vehicle and the object along the curved road. Theoretical equations were proposed for measuring the distance between a vehicle and an object in front of it on straight and curved roads. When the radius of curvature was 1293m, An error rate of at most 0.1% was observed. Therefore, the two proposed equations were integrated using 1293m as a threshold, and the distance to the object in front of the vehicle can be represented by Equation (22).
where, Z t is the distance between the vehicle and object in front of the vehicle.

C. MEASUREMENT OF THE DISTANCE TO LANE USING DUAL CAMERAS 1) DUAL CAMERAS IMAGE
In a previous study, image and focal length correction, and lane detection methods were developed. The vanishing point and center point required for the calculation can be obtained using the lane detected through the lane detection algorithm. Figure 7 shows the detected right and left lanes extending above the image, where the coordinates of the vanishing point A(x1, y1) and the coordinates of center point B(x2, y2) can be obtained. In addition, the coordinates of points with respect to a specific y-coordinate of the image can be obtained based on a curve equation corresponding to the detected lanes. 2) GEOMETRICAL VARIABLES OF VEHICLE Figure 9 shows the geometric composition of the dual cameras installed in the vehicle when viewed from the side.  Given the installation angle (α), installation height (h), and vertical field of view (θ v ) of the dual cameras, the shortest distance from the ground (d g ) that could be detected in the image could be obtained.

3) PROPOSED THEORETICAL EQUATION
As shown in Figure 7, using the coordinates of the vanishing point and the center point obtained through image processing, the heading angle (ψ) can be derived as shown in Equation (23).
where, ψ is the heading angle of the vehicle, x1 is the x-coordinate of the vanishing point, y1 is the y-coordinate of the vanishing point, x2 is the x-coordinate of the center point, y2 is the y-coordinate of the center point, f is the focal length As shown in Figure 8 and 9, a lane extension line can be generated, and the coordinates at the bottom of the image can be obtained. Using these values, the distance from the front-left wheel to the left lane (L lw ) can be derived as shown in Equation (24).
where, L lw is the distance from the optical axis of the camera to the left lane, x3 is the x-coordinate when the y-coordinate of the extension line of the left lane is I height , x4 is the x-coordinate when the y-coordinate of the extension line of the right lane is I height , I width is the width of the image, L w is the width of the lane As shown in Figure 10, considering the angle of installation of the camera, the shortest distance from the ground (d g ) that can be detected in the image can be derived as shown in Equation (25).
where, d g is the shortest distance from the ground that can be detected in the image, h is the installation height of the camera, α is installation angle of the camera, θ v is vertical field-of-view of the camera As shown in Figure 7 where, d ll is the distance calculated by the left camera from the left front wheel to the left lane of the vehicle, d lr is the distance calculated by the right camera from the left front wheel to the left lane of the vehicle, b is the interval between the cameras, C w is the overall width of the vehicle, C wh is the distance from the front of the vehicle to the center of the front wheel, d left is the distance from the left front wheel of the vehicle to the left lane.

D. TEST SCENARIO
In this study, the scenario under which real autonomous vehicle tests were conducted was as follows. Figure 11(a) shows integrated scenario 1 for response to a fixed target of the autonomous vehicle; it consists of straight-road following, emergency braking malfunction evaluation, crossroad (left-turn and right-turn), avoidance and braking, lane changing, and road-sign recognition scenarios. Figure 11(b) depicts integrated scenario 2 showing the response to a vehicle driving at a constant speed in front of the autonomous vehicle; it consists of straight-road following, emergency braking malfunction evaluation, crossroads (left-turn and right-turn), following, lane changing, and road sign recognition scenarios. Figure 11  The maximum speed for the real vehicle tests in the scenarios was set to a constant 40 km/h, which is the designated road speed limit for autonomous vehicles set by companies such as Google, and the scenarios were configured sequentially. Figure 12 presents different integration scenarios in detail. Figure 12(a) shows the own-lane following scenario, which is considered for both straight and curved roads. Figure 12(b) shows the emergency braking malfunction evaluation scenario used to determine whether there is a malfunction when the autonomous vehicle recognizes a stopped vehicle in the adjacent lane. Figure 12(c) shows the crossroad scenario (left and right turns) to determine whether the autonomous vehicle crosses the road safely by understanding the traffic lights and detecting surrounding obstacles. Figure 12(d) shows the braking and avoidance scenario to determine whether the autonomous vehicle recognizes obstacles at a certain distance and slows down or stops to avoid instead of emergency braking. Figure 12(e) shows the following scenario in which the autonomous vehicle trails a preceding vehicle moving at a constant speed. Figure 12(f) shows the lane cutting-out scenario to determine whether the autonomously driven car passes by a preceding vehicle that has moved to the next lane without misunderstanding. Figure 12(g) shows the lane cutting-in scenario for evaluating whether the autonomous vehicle recognizes a preceding vehicle in an adjacent lane and maintains a relative distance when the preceding vehicle cuts in front of the autonomous vehicle. Figure 12(h) shows the lane-changing scenario where the autonomous vehicle changes lanes on a straight road through a set route.

III. ACTUAL VEHICLE TEST A. TEST VEHICLE
In this study, the actual vehicle test was verified using an integrated scenario to compare the precision of the test evaluation method using a camera, and three vehicles were used in the test.
As shown in Figure 13, H * * 's Avante AD-based autonomous vehicle and H * * 's Veracruz were used as the primary target vehicles. In addition, the malfunction evaluation vehicle used R * * QM6. The vehicle specifications are listed in Table 2.
The autonomous vehicles used in the actual vehicle test were equipped with a DGPS, camera, LiDAR, etc., as illustrated in Figure 14, and their specifications are listed in Table 3.
To measure the camera data, an autonomous vehicle was equipped with a monocular camera and dual camera. The camera used was the L * * webcam, and the specifications of the webcam are summarized in table 4.

B. TEST LOCATION & ENVIRONMENT
The actual vehicle test was a driving test conducted by the Korea Intelligent Automotive Parts Promotion Institute, and an autonomous vehicle test road was used, as shown in Figure 16. It was confirmed that the average friction coefficient of the road surface was 1.08 using STANLEY LONDON's ''skid-resistance'' as a driving test, and it was judged that it was suitable for the test site because the actual road friction coefficient and road width were applied. This is illustrated in Figure 17.
To obtain objective data, the test scenario must be reproduced repeatedly in the same manner. Accordingly, the number of people conducting the actual test, the test equipment, and the control coding were maintained the same, and the test was repeated three times under the same environmental conditions. It was conducted under the test environment conditions summarized in table 5, and there were no changes in weather such as showers.   Figure 16-21 show the results of three repeated real vehicle tests conducted on KIAPI's autonomous vehicle driving test road, which are expressed in terms of velocity, longitudinal acceleration, lateral acceleration, heading angle, yaw rate, inter-vehicle distance, and distance to the lane. Figure 18 shows the real vehicle test results of integrated scenario 1 to evaluate the autonomous vehicle's response to a fixed target. The test results showed similar trends, although there were differences in time. It was confirmed that the distance to the lane diverges mainly because the lane is not recognized when passing through the intersection or making a U-turn.    Figure 19 shows the real vehicle test results of the integrated scenario 2 to evaluate the autonomous vehicle's response to a target driving at a constant speed. The test  results showed similar trends, although there were differences in time. The in-lane results diverged between 10 and 20s for the 3rd test. This corresponds to the section in which the autonomous vehicle turns right to enter the first section of the detailed scenario, such that the acceleration was affected by instantaneous lane divergence; however, analyzing the sections of the detailed scenario showed that the results had similar trends. Figure 20 shows the real vehicle test results of integrated scenario 3 to evaluate the autonomous vehicle response when the target driving at a constant speed cuts out to the next lane. The test results showed similar trends, although there were differences in time. Figure 21 shows the real vehicle test results of integrated scenario 4 to evaluate the autonomous vehicle responses when a target driving in the next lane cuts in. When the 2nd test was conducted for approximately 0 to 44s, it was seen that the speed was lower than those of the 1st and 3rd tests. This is the section in which the autonomous vehicle turns right to enter the first section of the detailed scenario, although the entrance time is twice that of other tests. However, analyses of the sections of the detailed scenario show that the results have similar trends.

B. ACTUAL VEHICLE TEST RESULTS USING MONOCULAR CAMERA
To utilize the test evaluation method using a monocular camera, a monocular camera was installed in an autonomous vehicle and three repeated tests were conducted under the same conditions to ensure reliability of the test results.
The relative distance from the preceding vehicle and the distance to the lane, which are important variables in the test evaluation method using a monocular camera, are shown in Figure 21 for each scenario. Figure 22(a)-(c) show the distances to the preceding vehicle and lane corresponding to integrated scenario 1, which evaluates the autonomous vehicle's response to a fixed target vehicle. For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 1, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the monocular camera. Although there were differences in time, the values of the three repeated tests showed similar trends. The distance to the lane was measured for the entire section of integrated scenario 1, and it was observed that the results diverged in sections where the lanes disappeared, such as intersections. Figure 22(d)-(f) show the distances to the preceding vehicle and the lane corresponding to integrated scenario 2, which evaluates the autonomous vehicle's response to a target driving at a constant speed in its own lane. For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 2, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the monocular camera. Although there were differences in time, the values of the three repeated tests showed similar trends. The distance to the lane was measured for the entire section of the integrated scenario 2, and it was VOLUME 10, 2022  observed that the results diverged in sections where the lanes disappeared, such as intersections. Figure 22(g)-(h) show the distances to the preceding vehicle and the lane corresponding to integrated scenario 3, which evaluates the autonomous vehicle's response when the target driving at a constant speed in its own lane cuts out to the next lane. For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 3, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the monocular camera. Although there were differences in time, the values of the three repeated tests showed similar trends. Figure 22(i)-(l) show the distances to the preceding vehicle and to the lane corresponding to integrated scenario 4, which evaluates the autonomous vehicle's responses when a target driving in the next lane cuts in to its own lane. For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 4, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the monocular camera. Although there were differences in time, the values of the three repeated tests showed similar trends. The distance to the lane was measured for the entire section of the integrated scenario 4, and it was observed that the results diverged in sections where the lanes disappeared, such as intersections.

C. ACTUAL VEHICLE TEST RESULTS USING DUAL CAMERA
To utilize the test evaluation method using a dual camera, a dual camera was installed in the autonomous vehicle at an optimal location selected in previous studies. To ensure reliability of the test results, three repetitions were conducted under the same conditions.  The relative distance from the preceding vehicle and distance to the lane, which are important variables of the test evaluation method using a dual camera, are shown in Figure 23 for each scenario.   For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 1, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the dual camera. Although there were differences in time, it was confirmed that the three repeated tests exhibited similar trends. The distance to the lane was measured for the entire section of integrated scenario 1, and it was observed that the results diverged in sections where the lanes disappeared, such as intersections. Figure 23(d)-(f) show the distances to the preceding vehicle and the lane corresponding to integrated scenario 2, which evaluates the autonomous vehicle's response to a target driving at a constant speed in its own lane. For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 2, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the monocular camera. Although there were differences in time, the values of the three repeated tests showed similar trends. The distance to the lane was measured for the entire section of the integrated scenario 2, and it was observed that the results diverged in sections where the lanes disappeared, such as intersections. Figure 23(g)-(h) show the distances to the preceding vehicle and the lane corresponding to integrated scenario 3, which evaluates the autonomous vehicle's response when the target driving at a constant speed in its own lane cuts out to the next lane. For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 3, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the monocular camera. Although there were differences in time, the values of the three repeated tests showed similar trends. Figure 23(i)-(l) show the distances to the preceding vehicle and to the lane corresponding to integrated scenario 4, which evaluates the autonomous vehicle's responses when a target driving in the next lane cuts in to its own lane. For the inter-vehicle distance from the preceding vehicle, the section corresponding to detailed scenario 4 of integrated scenario 4, in which the tests are conducted with the main target vehicle, was selected, and the time along the x-axis and distance range of the y-axis represents the relative distances based on the inter-vehicle graph to the preceding vehicle in its own lane appearing on the monocular camera. Although there were differences in time, the values of the three repeated tests showed similar trends. The distance to the lane was measured for the entire section of the integrated scenario 4, and it was observed that the results diverged in sections where the lanes disappeared, such as intersections.

D. COMPARATIVE ANSLYSIS OF THE RESULTS OF THE ACTUAL VEHICLE TEST
The inter-vehicle distance a between the autonomous and preceding vehicles through the real vehicle test based on the integrated scenarios was measured in the fourth detailed scenario of each integrated scenario. Figure 24 show the actual measured values for each scenario and each repeated test using both monocular and dual cameras. Table 6 compares and summarizes the results of integrated scenarios 1 to 4 for the values measured by the equipment installed in the autonomous vehicle and the results of the inter-vehicle distances using the monocular and dual cameras.
Comparing the measured values of integrated scenario 1 with those of the monocular camera, the minimum error rate was 0.002% in the 1st test with a 0.002m deviation, and the maximum error rate was 7.443% in the 2nd test with a 2.770m deviation. The average error rate was within 2.5% for all three test repetitions. Comparing the measured values of integrated scenario 1 with those of the dual camera, the minimum error rate was 0.006% in the 1st test with a 0.004m deviation, and the maximum error rate was 7.656% in the 2nd test with a 2.528m deviation. The average error rate was within 2.7% for all three test repetitions.
Comparing the measured values of integrated scenario 2 with those of the monocular camera, the minimum error rate was 0.135% in the 1st test with a 0.044m deviation, and the maximum error rate was 7.447% in the 2nd test with a 2.548m deviation. The average error rate was within 3% for all three test repetitions. Comparing the measured values of integrated scenario 2 with those of the dual camera, the minimum error rate was 0.001% in the 3rd test with a 3.0.E-04m deviation, and the maximum error rate was 6.693% in the 2nd test with a 3.595m deviation. The average error rate was within 2.7% for all three test repetitions.
Comparing the measured values of integrated scenario 3 with those of the monocular camera, the minimum error rate was 0.224% in the 1st test with a 0.094m deviation, and the maximum error rate was 7.464% in the 1st test with a 3.450m deviation. The average error rate was within 3% for all three test repetitions. Comparing the measured values of integrated scenario 3 with those of the dual camera, the minimum error rate was 0.006% in the 2nd test with a 0.002m deviation, and the maximum error rate was 8.174% in the 3rd test with a 3.767m deviation. The average error rate was within 2.7% for all three test repetitions.
Comparing the measured values of integrated scenario 4 with those of the monocular camera, the minimum error rate was 0.137% in the 2nd test with a 0.041m deviation, and the maximum error rate was 7.725% in the 3rd test with a 2.397m deviation. The average error rate was within 3.1% for all three test repetitions. Comparing the measured values of integrated scenario 4 with those of the dual camera, the minimum error rate was 0.002% in the 3rd test with a 0.001m deviation, and the maximum error rate was 7.201% in the 1 st test with a deviation of 1.352m. The average error rate was within 2.5% for all three test repetitions.
The inter-vehicle distances between the autonomous and preceding vehicles in the real vehicle tests based on the integrated scenarios were measured for all the integrated scenarios. Figure 25 shows the actual measured values for each scenario and repeated tests using the monocular and dual cameras to distinguish between the distances to the left and right lanes. Table 7 compares and summarizes the results of the distances to the lanes for each scenario, except for the divergence of the actual measured value and area where lane recognition was not possible when passing by a crossroad or making a U-turn.
For result of distance to the left-lane, comparing the measured values of integrated scenario 1 with those of the monocular camera, the minimum error rate was 0.014% in the 3rd test with a 2.64E-04m deviation, and the maximum error rate was 11.977% in the 3rd test with a 0.163m deviation. The average error rate was within 5.7% for all three test repetitions. For result of distance to the right-lane, the minimum error rate was 0.014% in the 3rd test with a  2.63E-04m deviation, and the maximum error rate was 11.989% in the 3rd test with a 0.177m deviation. The average error rate was within 5.9% for all three test repetitions.
For result of distance to the left-lane, comparing the measured values with those of the dual camera, the minimum error rate was 0.001% in the 3rd test with a 2.11.E-05m deviation, and the maximum error rate was 11.982% in the 1st test with a 0.226m deviation. The average error rate was within 5.3% for all three test repetitions. As result of distance to the rightlane, the minimum error rate was 0.001% in the 3rd test with a 1.91.E-05m deviation, and the maximum error rate was 11.997% in the 2nd test with a 0.226m deviation. The average error rate was within 5.4% for all three test repetitions.
For result of distance to the left-lane, comparing the measured values of integrated scenario 2 with those of the monocular camera, the minimum error rate was 0.006% in the 3rd test with a 1.10E-04m deviation, and the maximum error rate was 11.991% in the 1st test with a 0.247m deviation. The average error rate was within 4.7% for all three test repetitions. For result of distance to the right-lane, the minimum error rate was 0.006% in the 3rd test with a 1.14E-04m deviation, and the maximum error rate was 11.989% in the 1st test with a 0.177m deviation. The average error rate was within 4.8% for all three test repetitions. For result of distance to the left-lane, comparing the measured values with those of the dual camera, the minimum error rate was 0.002% in the 2nd test with a 2.89.E-05m deviation, and the maximum error rate was 11.993% in the 1st test with a 0.225m deviation. The average error rate was within 6.2% for all three test repetitions. For result of distance to the rightlane, the minimum error rate was 0.002% in the 2nd test with a 2.49.E-05m deviation, and the maximum error rate was 11.993% in the 1st test with a 0.225m deviation. The average error rate was within 6.2% for all three test repetitions.
For result of distance to the left-lane, comparing the measured values of integrated scenario 3 with those of the monocular camera, the minimum error rate was 0.002% in the 1st test with a 3.66E-05m deviation, and the maximum error rate was 11.988% in the 2nd test with a 0.158m deviation. The average error rate was within 4.8% for all three test repetitions. For result of distance to the right-lane, the minimum error rate was 0.003% in the 1st test with a 3.86E-05m deviation, and the maximum error rate was 11.987% in the 1st and 3rd test with a 0.225m deviation. The average error rate was within 4.8% for all three test repetitions. As result of distance to the left-lane, comparing the measured values with those of the dual camera, the minimum error rate was 0.004% in the 2nd test with an 8.15E-05m deviation, and the maximum error rate was 11.993% in the 1st test with a 0.175m deviation. The average error rate was within 5.3% for all three test repetitions. For result of distance to the rightlane, the minimum error rate was 0.005% in the 2nd test with a 8.15E-05m deviation, and the maximum error rate was 11.977% in the 3rd test with a 0.225m deviation. The average error rate was within 5.4% for all three test repetitions.
For result of distance to the left-lane, comparing the measured values of integrated scenario 4 with those of the monocular camera, the minimum error rate was 0.003% in the 1st test with a 4.85E-05m deviation, and the maximum error rate was 11.976% in the 1st test with a 0.204m deviation. The average error rate was within 4.3% for all three test repetitions. For result of distance to the right-lane, the minimum error rate was 0.004% in the 1st test with a 4.32E-05m deviation, and the maximum error rate was 11.987% in the 2nd and 3rd test with a 0.225m deviation. The average error rate was within 4.7% for all three test repetitions. For result of distance to the left-lane, comparing the measured values with those of the dual camera, the minimum error rate was 0.0011% in the 3rd test with a 1.76E-04m deviation, and the maximum error rate was 11.999% in the 2nd test with a 0.156m deviation. The average error rate was within 4.9% for all three test repetitions. As result of distance to the right-lane, the minimum error rate was 0.014% in the 2nd test with a 1.32E-04m deviation, and the maximum error rate was 11.952% in the 3rd test with a 0.244m deviation. The average error rate was within 5% for all three test repetitions.
It is judged that the large errors in the results of the monocular and dual cameras compared with the measured values are because the cameras are affected by some causes: recognizing lanes right after passing though the crossroad and completing a U-turn, changing lanes for avoidance in integrated scenario 1, changes in lane widths of the set routes of the integrated scenarios, routes without lanes, differences between the actual width and that recognized by the camera, and vibrations while driving.

E. ANALYSIS OF RESULTS & THRESHOLD
When comparing the result value using camera with the measured value using of the dynamic characteristics measuring equipment, the obtained result is as follows.
1) As a result of analyzing the inter-vehicle distance of the monocular camera, showed a tendency to recognize that the preceding vehicle was closer than it was. The distance to the lane referring to Table 8, an average error rate of 5.7% to the left lane and an average error rate of 5.9% to the right lane, the detection and distance measurement of the left lane were better.
2) As a result of analyzing the inter-vehicle distance of the dual camera, showed a tendency for a reference point of 50 m to recognize the preceding vehicle was closer when the real distance was less than 50m and to recognize it was father when the real distance was father than 50m. The distance to the lane referring to Table 8, the average error rate to the left and right lanes was 6.2%, and both lane detection and distance measurement were the same.
3) Table 9 summarizes the advantages and disadvantages of each camera. Referring to Table 9, since the monocular camera installed at a higher position than the dual camera, the range taken when shooting in front was wider, which was advantageous for lane detection, and the precision was somewhat superior to that of the dual camera. 4) Dual camera show the process of recognizing obstacles over 70m, the instantaneous error rate (maximum error rate) was about 0.45% larger than that of monocular cameras. However, in the dual cameras, since they are installed on the bumper in front of the vehicle, the accuracy of the inter-vehicle distance, such as avoidance and following functions, was relatively superior to that of monocular cameras. 5) Therefore, considered the advantages and disadvantages of each camera, Therefore, it is expected that each other's shortcomings can be supplemented if they are used at the same time rather than independent use of monocular cameras and dual cameras.
Through result analysis, the thresholds of this study are as follows.
1) In the case of the monocular camera, it was assumed that the rear overhang was known. When recognizing a vehicle through a camera, it was determined that the specifications of the vehicle provided by the manufacturer were known in advance.
2) In a study of the optimal mounting position of dual cameras, the average error rate was within 1%. However, the maximum error rate was 2.15%. The results of the real vehicle test were judged to reflect the influence of the errors. In addition, the results of the real vehicle test using the monocular camera are as follows: As a result of the intervehicle distance, the average error rate was 2.7%, and the maximum error rate was 7.725%. As a result of the distance to the lane, the average error rate was 5.9% and the maximum error rate was 11.993%. The results of the real vehicle test using the dual camera are as follows. As a result of the intervehicle distance, the average error rate was 2.4%, and the maximum error rate was 8.174%. As a result of the distance to the lane, the average error rate was 6.2% and the maximum error rate was 11.999%. These results are judged to have additional causes of errors owing to road surface vibrations and air resistance.
3) The real vehicle test was not conducted on the actual road, and it was due to various reasons such as the risk of traffic accidents.
4) The real vehicle test using a camera was conducted in consideration of clear weather, and tests for various environments were insufficient. 5) In addition to the scenarios conducted in this study, tests for various scenarios such as pedestrian scenarios were insufficient.

V. CONCLUSION
1) To compare the safety evaluation precisions of autonomous vehicles using cameras, the distance measurement method using monocular and dual cameras proposed in previous studies was used.
2) The integrated scenarios were proposed to assess the behaviors of autonomous vehicles and used in real vehicle tests.
3)As a result of analyzing the inter-vehicle distance through the results of the real vehicle test, the monocular camera showed a tendency to recognize that the preceding vehicle was closer than it was. In case of dual camera, showed a tendency for a reference point of 50 m to recognize the preceding vehicle was closer when the real distance was less than 50m and to recognize it was farther when the real distance was father than 50m. 4) As a result of analyzing the distance to the lane, the monocular camera had better left lane detection and distance measurement, and the dual camera had the same lane detection and distance measurement. 5) Through the comparison of the advantages and disadvantages of each camera, monocular cameras were advantageous for lane detection and distance measurement, and dual cameras were advantageous for inter-vehicle distance measurement. Therefore, Therefore, it is expected that each other's shortcomings can be supplemented if they are used at the same time rather than independent use of monocular cameras and dual cameras.
6) There is a disadvantage in that expensive equipment and experts are required to evaluate the autonomous vehicles, which is costly and time consuming. Based on these results, when using the monocular and dual cameras compared in this study, it is judged that the tendencies of autonomous vehicles can be evaluated using only the cameras and without skilled experts. It is also judged that useful trends can be identified in environments where real vehicle tests are impossible or for developing sensors for autonomous vehicles. The method using monocular and dual cameras is judged to enable reduction of the cost and time incurred during real vehicle tests. 7) In the future, it is judged that a study considering the shortcomings of generally pointed out cameras and additional tests on cameras other than the webcam used in this study are needed. In addition, it is judged that tests for various scenarios such as pedestrian scenarios and additional tests for tests considering various speeds and road surfaces are necessary.
An example of camera weakness. camera malfunction in bad weather conditions such as rain and snow; camera malfunctioning in situations such as light reflection; in the dark night the camera realize obstacles and lanes; when contaminants such as dust are attached to the camera; focusing according to the vehicle vibration; etc.