Effects of Sim2Real Image Translation via DCLGAN on Lane Keeping Assist System in CARLA Simulator

Autonomous vehicle (AV) simulation using a virtual environment has the advantage of being able to test algorithms in various scenarios with reduced resources. However, there may exist a visual gap between the virtual environment and the real-world. In this paper, in order to mitigate this gap, we trained Dual Contrastive Learning Generative Adversarial Networks (DCLGAN) to realistically convert the image of the CARLA simulator and then evaluated the effect of the Sim2Real conversion focusing on the lane keeping assist system (LKAS). Moreover, in order to avoid the case where the lane is translated distortedly by DCLGAN, we found the optimal training hyperparameters using feature similarity (FSIM). After training, we built a system that connected the CARLA simulator with DCLGAN and AV in real-time. As for the result, we collected data and analyzed them using the following four methods. First, image reality was measured with Fréchet Inception Distance (FID), which we quantitatively verified to reflect the lane characteristics. The CARLA images that passed through DCLGAN had smaller FID values than the original images. Second, lane segmentation accuracy through ENet-SAD was improved by DCLGAN. Third, in the curved route, the case of using DCLGAN drove closer to the center of the lane and had a high success rate. Lastly, in the straight route, DCLGAN improved lane restoring ability after deviating from the center of the lane as much as in reality. Consequently, it convinced that the proposed method could be applicable to mitigate the gap of simulation toward real-world.


I. INTRODUCTION
The autonomous driving industry has largely been expanding at a rapid pace. Autonomous driving has been proposed as a solution to traffic accidents that cause 1.25 million deaths worldwide every year [1]. However, in the United States alone, a total of 392 traffic accidents recently occurred even in one year due to Level 2 autonomous driving, and these reports utterly raised the need for securing the safety of autonomous driving technology [2].
The associate editor coordinating the review of this manuscript and approving it for publication was Razi Iqbal .
Since autonomous driving technology has been required to have diverse validation processes, autonomous driving simulation is drawing attention as a new testbed. Unlike reality, the simulation can implement complex situations, such as traffic accidents and specific weather, that are hard to reproduce in the real-world [3], [4], [5]. This technology gives abundant training datasets about abnormal situations and can potentially prevent traffic accidents more perfectly. Therefore, many studies have been conducted for modeling vehicles in simulation [6], [7], [8].
In order to adapt proven algorithms from virtual-world to real-world, reducing the gap between simulation and reality is an inevitable task. This gap can be caused by several factors such as the unreality of sensor models (e.g., camera, LiDAR, GPS, and IMU), mismatch of dynamic models, and measurement noise in the real environment. Among the factors, the camera sensor input images are the main cause of the gap because this sensor accounts for the largest portion of driving. The image gap which is a difference in visual realism tends to make driving results unreliable. Besides, mitigating this gap only by adjusting the simulation parameters can be almost impossible. Therefore, simulation developers and users often have used a high-level graphic engine to make the visual contents of the simulation look more realistic.
To make images from the two domains similar, Generative Adversarial Networks (GAN) [9] have been adopted. With the development of the GAN structure, the model has developed to be applied to various fields such as colorizing gray image [10] and restoring image quality [11]. Several studies have also been proposed to make the images of the virtual world close to reality [12], [13]. Although the study on improving the graphic quality of the virtual world is active, autonomous driving tests with these photorealistic techniques have not yet been well investigated so far.
The purpose of this paper is to show the possibility of applying GAN to the autonomous vehicle (AV) simulation not just translating images, but also testing driving results by using CARLA [14] simulator. As for the GAN, Dual Contrastive Learning Generative Adversarial Networks (DCLGAN) [15] was selected because DCLGAN was trained better than other unpaired image translation models such as CycleGAN [16] without overfitting even with our less diverse data. Nevertheless, to avoid the lane distortion that DCLGAN sometimes exhibits, we explored how to train in optimal conditions using feature similarity (FSIM).
For this purpose, the data were collected from the three domains: real-world, CARLA with DCLGAN (hereafter referred to as CwD), and CARLA without DCLGAN (hereafter referred to as Cw/oD). The real-world data were obtained from the Korea Intelligent Automotive Parts Promotion Institute (KIAPI) and the virtual-world data were obtained from a virtual map created identically to KIAPI. The types of data were camera images and GPS data. To show the DCLGAN contributed to making more realistic driving results, a method to quantify the Sim2Real gap is needed. There are several original contributions of this paper as following.
First of all, Fréchet Inception distance (FID) [17], which calculates differences in density of two distributions of image sets in the diverse environment, was used to measure the realism of the images of the CARLA simulator. After determining whether FID was suitable for measuring this image gap, we measured the FID of the images converted by DCLGAN. These converted images were also used to measure the accuracy of the lane detection model, an important part of LKAS. We compared CwD, Cw/oD, and real-world lane segmentation accuracy. Furthermore, we compared how much the GPS trajectory of the vehicle driving the virtual KIAPI map deviated from the center of the lane with and without DCLGAN. The ability of these systems to recover from lane departure on a straight line was also measured. The last two results have a differentiated meaning in that they are the driving test, not an image task. Consequently, we found that the above processes show that our proposed CwD method is more effective in practical use of autonomous driving simulation tests such as LKAS.

II. RELATED WORKS
CARLA [14] has been considered one of the reliable autonomous driving research simulator. Thus, we used the CARLA simulator for the implementation of a virtual-vehicle and building environments by using the functions followed. First, the provided ROS-bridge enables the connection between the real-vehicle and the virtual-vehicle. Second, CARLA allowed us to spawn various sensors and configure specific parameters such as the camera's FoV, resolution, and gamma. Lastly, target vehicles can be implemented through Unreal Engine 4 (UE4), the game engine of CARLA.
We imported a 3D model of Hyundai IONIQ electric vehicle to spawn sensors at the same position as the real-vehicle.
GAN consists of a generator that creates fake images following real images' distribution, and a discriminator that distinguishes between the fake images and the real images [9]. As the two modules compete repeatedly, GAN can generate 33916 VOLUME 11, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply. more realistic images. However, the GAN model has a limitation that it necessarily requires paired datasets for training. CycleGAN solves the problem by adding the reverse mapping that returns the fake image to the original image. Whereas cycle consistency loss has a drawback that the fake images have low diversity. This limited output problem of low diversity is well known as mode collapse. Contrastive Learning for Unpaired Image-to-Image Translation (CUT) and Dual Contrastive Learning GAN (DCLGAN) tried to deal with this issue through contrastive learning. Especially, Similarity Dual Contrastive Learning GAN (SIMDCL) with similarity loss added to DCLGAN suffers less from the mode collapse by allocating embeddings as much as the number of data domains [15]. Thus, we mainly used SIMDCL, considering the characteristics of our dataset, which has low diversity and a high risk of mode collapse.
FID is an evaluation metric for a generative model such as GAN, text-to-image, etc. The distance between the fake images and the real images is measured by comparing the extracted features from the pre-trained Inception v3 model with the Imagenet dataset [18]. In this paper, the distance is used to measure how realistic the generative model creates images. FID between true samples (T) and generated samples (G) is calculated as equation (1) using the mean of the variance of features (µ) and covariance ( ). Tr stands for the trace of the matrix.
ENet-SAD [19] has the characteristic of self-learning, which delivers and enhances the representation top-down by distilling the attention map of the lower layer. With this distillation technique, ENet-SAD can be computed faster without additional annotation or inference even with much fewer parameters compared to other CNN-based lane detection models such as SCNN. Since the IoU of the ground truth lane and lane segmentation prediction results were better than the previous models, our LKAS system uses ENet-SAD.
TuSimple [20] introduces the lane segmentation accuracy formula as equation (2). C clip is the number of true points in each clip and S clip is the number of the ground truth points in the clip. Thus, we used this evaluation method to compare accuracy in each domain.
Yurtsever et al. [12] proposed a rendering method that realistically synthesizes the background of CARLA using Conditional GAN and CycleGAN, while leaving objects of interest such as lanes and vehicles intact. In this way, distortion or disappearance of the object can be prevented, but it is not possible to completely translate the appearance of objects to accurately reflect the properties and patterns of each domain. The model presented by Richter et al. [13]  converts the entire image without such hallucinations by using the features of G-buffer information such as geometry, materials, lighting, and segmentation created during the game rendering process in the generator model. However, these studies do not confirm how realistically changed images actually affect the perception or control results of autonomous driving algorithms, or do not compare them with actual driving data. For this comparative study, we trained a GAN model with data that directly reflects the difference in characteristics between our simulation and real domain, and also dealt with the hallucination problem.

III. ENVIRONMENT SETTING
A. REAL-WORLD SETTING 1) SPECIFICATION Table 1 describes the sensors and hardware specifications and Fig. 2 shows the location of the sensors. Although the camera maximum resolution was 1632×1248 pixels, 808×620 was used by binning. The vehicle was the Hyundai IONIQ electric vehicle modified for applying the drive-bywire system.

2) LKAS ALGORITHM
We have autonomous driving algorithms and architecture that were certified by MOLIT (Ministry of Land, Infrastructure, and Transport) of the Republic of Korea. This certification required autonomous emergency braking (AEB), smart cruise control (SCC), and lane keeping assist system (LKAS). Among the technologies we have, only LKAS function was used in the experiment.
According to previous papers [21], [22], the path of autonomous vehicles was derived through camera, GPS, and IMU sensors. Three paths are obtained through each three sensors. Then, Optimal Path Tracker Selection picks the best path among the paths considering the instability of steering angle, GPS reliability, vision reliability, and road information.
In this study, we only used the path from the camera and designed this algorithm being worked in two stages. The first stage performed in deep learning PC installed on AV is lane segmentation by using the ENet-SAD network model. This ENet-SAD model was trained by datasets from both the CULANE dataset [23] and our dataset. The second stage is lane detection which transforms the segmented image to a bird's eye view through inverse perspective mapping (IPM), and then calculates lane center points by finding the fitting function that best describes the lane among linear, quadratic, VOLUME 11, 2023  and cubic least squares fitting [22]. After these two stages, the main PC installed in the AV calculates the steering angle value using the lane detection result and the pure pursuit algorithm, which enters the CARLA vehicle through the ROS topic. In this paper, the lane keeping algorithm was used to obtain data by running a real-vehicle on the KIAPI high-speed main circuit or by running a virtual-vehicle on the virtual KIAPI map.

B. CARLA SIMULATOR SETTING 1) CAMERA AND VEHICLE
CARLA simulator provides various sensors and attributes in the virtual environment. The attributes make it possible to set the sensor specifications. The performance of the camera sensor described in Table 1 was imported such as FoV or Gamma. Then, the camera was spawned according to Fig. 2 position. The CARLA image taken by spawned camera sensor and the real-world image can be also seen in Fig. 3.

2) MAP
To make a custom map, CARLA requires map geometry information in .fbx format and OpenDRIVE information in .xodr format. For these, we used RoadRunner recommended by CARLA as a map editor and OpenStreetMap (OSM) as  map data. OSM has latitude and longitude vector data and RoadRunner makes OSM possible to get both map geometry information and OpenDRIVE information.
The process of making the KIAPI map was as follows. First, as shown in Fig. 4(a), we dragged the desired area from the satellite map in OSM and export the file in .osm format. Second, as shown in Fig. 4(b), the latitude and longitude were corrected by comparing the OSM file exported from JOSM with the GPS file of actual measurement data. After that, the calibrated OSM file was imported into RoadRunner, and the RoadRunner file set the road width, lanes, and slope of the road according to the information announced in KIAPI [24]. Fig. 4(c) describes this process and Table 2 shows the specification of each parameter. Lastly, this file was exported in CARLA format such as .fbx and .xodr format files.

3) TIME
The experiment with the same time period as the real-world is not able to be performed because of the calculation time of GAN. As a solution, we used synchronous time. According to CARLA documentation, ''On synchronous mode, the server waits for a client tick, a ''ready to go'' message, before updating to the following simulation step'' [25]. It means that we can set the attribute to wait for a step until the vehicle control command arrives. Therefore, the client server would wait for a cycle described in the next section.

4) NETWORK CONNECTION BETWEEN CARLA AND AV
Three computers were used for the experiments, called the Main PC, Deep Learning PC (DL PC), and Simulation PC. As described in Table 1, Main PC decided on vehicle control such as steering angle value and throttle value. DL PC computed deep learning algorithms such as the LKAS algorithm, and the Simulation PC played CARLA simulation. Table 3 describes the ROS version for each computer. For connecting    calculates proper steer angle, throttle, and brake values. Then the function sends them to the Simulation PC for control of the vehicle.

5) TRAINING DCLGAN
We tried to reduce this Sim2Real gap with DCLGAN. The data of the real-world domain was collected from the high-speed circuit of KIAPI and the data of the CARLA domain was collected from the OSM map of KIAPI. Both were collected while driving at a speed of 30 km/h in the second lane. The CARLA domain contains 5498 images as train set and 611 images as test set, and the real-world domain contains 6929 images as train set and 770 images as the test set. In order to match the channels of the FLIR camera images used in the AV, the order of the CARLA image channels was changed from RGB to BGR.
The images used for training the model were loaded with a size of 600×600 and cropped to a size of 300×300. The deep learning network model was selected as SIMDCL and the rest of the options kept the DCLGAN default settings. Additionally, λ NCE,X , a hyperparameter of the loss function, was determined through the process as described in equations (3)- (5).
As shown in Fig. 6, if the image passes through the DCLGAN model, the lane in CARLA may not maintain its original shape and may be distorted enough to affect lane-keeping driving. This lane distortion occurs when the generator of the GAN does not fully learn the data distribution and is fixed only in a specific biased direction to generate an image [15]. Also, the imbalance between the straight line data and curved line data of the lane can be the cause.
In order to solve the problem of distortion of the lane shape that occurs when training with the default settings of the model, we tried to find the optimal hyperparameter set by adjusting the coefficients of the loss function (Hyperparameter tuning). Equation (3) shows the loss function of DCLGAN, and equation (4) shows the loss function of SimDCL [15]. G is the X domain(in our paper, Carla)→ Y domain(Real-world) generator, and F is the Y → X generator. D X distinguishes the generated X images from the real images, and D Y distinguishes the Y images. H X and H Y refer to vectors embedded by G encoder and H encoder, respectively. When converting X → Y or Y → X , it uses the normal GAN loss (L GAN ) for G or F. In addition, PatchNCE loss (L PatchNCE ) is used, which learns to maximize the similarity between a patch in the generated real-world image and its positive pair in the original CARLA image while minimizing the similarity between the real-world patch and negative CARLA pairs. Also, Identity loss (L identity ) is calculated to preserve the color of the existing image, and Similarity loss (L sim ) is calculated to measure whether the real image and the generated image have similar styles in the same domain. The default coefficients for each loss term are 1 for GAN , 2 for NCE, 10 for sim, and 1 for idt.
As in equation (5), the X part of the PatchNCEloss term is associated with a generator that converts X → Y . Where z s l is the query patch of the generated image, z s l is the positive patch of the original image, and z S\s l is the negative patches. It calculates the entropy-cross loss (l) for all layers and patches. If the query is closer to the positive paired patch and farther from the negative patches, it will try to retain more features of the CARLA patch. We checked whether better results that keep the direction or shape of the lane can be obtained by changing the coefficient such as λ NCE,X .
In this case, FSIM [26] was used to evaluate the degree of lane distortion of the generated image in the case of each adjusted hyperparameter. The edge map similarity between the original image and the fake image is measured from the phase congruency (PC) map extracted based on frequency information and the gradient magnitude (GM) map extracted based on the gradient of pixel values. The PC map is obtained by calculating the sum of vectors (local energy) consisting of  amplitude and phase from frequency information and dividing it by sum of the amplitudes, and the GM map can be obtained by convolving the image with a first-order differential edge detection mask. To compare the edge information of only lanes, as shown in Fig. 7, 808×245 sized images excluding the background and car bonnet were cropped from each 808×620 image of the CARLA test imageset, and the FSIM of each pair of the Cw/oD image and the CwD image was measured. A model with proper λ NCE,X that maintains the original lane shape or direction well can have a high FSIM value. The weight determined in this way is then used for FID measurement. The relationship and differences between FSIM and FID are shown in Fig. 8. FSIM can directly measure the similarity between two paired images, but since FID compares the distribution of images, it is necessary to compare image sets when measuring FID. Table 4 shows the average FSIM value in each λ NCE,X . In all cases, the weight that seems to have the least lane distortion among the weights trained more than 15 epochs and less than 60 epochs were selected (because the loss function has largely converged after epoch 15), and the average of the FSIM values before and after translation of 611 images of the CARLA domain test set was measured. For FSIM measurements, quality_metrics and image_similarity_measures Python packages were used [26]. λ NCE,X value 3, which has the highest FSIM value 0.451 and is judged to have the least distortion, was determined as the optimal condition.

IV. APPROACH METHOD A. FID VERIFICATION
It is necessary to guarantee whether the difference in characteristics of lanes can be quantitatively measured. When a CwD image is generated from each CARLA image, the FSIM is measured from each pair. FID is measured between unpaired CwD imageset and real-world imageset to measure how realistic the generated images are. As previously shown in Fig. 3, the lanes in the real-world are clearer and thicker than the ones in the virtual environment. To see if the FID used to quantify the Sim2Real difference reflects this well, it is necessary to test the FID by separately creating maps in which the characteristics of these lanes are clearly distinguished. These maps were created based on KIAPI map and varied the thickness, texture, length, and distance of lanes by RoadRunner. Fig. 9 shows these characteristic differences. For the texture, LaneMarking1, 2, and 3, which are lane materials provided by RoadRunner, were used. The material used in LaneMarking1 is the sharpest and 3 is the faintest. All other variables in RoadRunner were fixed except for lanes.
After that, the vehicle was summoned by the RoadRunner Scenario function, and then driven on the same fixed route on the KIAPI map. While the car was driving in the center of the road, 25-second videos were recorded at 30 FPS on each map with specific characteristics of the lane. By extracting each frame of the recorded video, about 700 images were created for each map, and FID operation was performed between the maps using this. The FID code used in this paper was written based on PyTorch and the FID's dimensionality of features was set to 2048 (default) [27]. To compare only the thickness of lanes, we fixed the lane material to LaneMarking1, the length and spacing of lanes to (10m, 10m), and increased the lane thickness from 0.125m to 0.15m, 0.175m, and 0.2m respectively. Also, to compare the texture, the thickness was fixed to 0.125m and the length and spacing of the lanes were fixed to (10m, 10m). To compare the length and spacing of the lanes, the thickness was fixed to 0.125m and the material to LaneMarking1. Table 5 shows the FID measurements between each map. The fact that the value increases toward the upper right of the table means that the FID actually appears larger as the characteristics of the lanes are quantitatively farther from each other. For example, the thickness changing result as shown in Table Table 5(c) were obtained for the length and spacing of the lanes. This can be explained that the Inception v3 model abstracts the characteristics such as thickness, texture, length, and spacing of lanes well and extracts them as features. This tendency confirms that FID is suitable as a numerical value to quantitatively measure the difference in characteristics of the lanes.

B. EXTRACTING GROUND TRUTH
To automatically obtain the ground truth data in CARLA simulator, it was required to perform several works. As shown in Fig. 10, our works had following three steps. First, we created the same map except for the shape of the line. This map's line was fully connected shown as Fig. 10 (a). Then, the vehicle ran in the line-connected map while recording its movement and collecting segmentation images. Second, the recording process was turned on in the original map, and then started to save the test images. Then, we found the specific pixel color belonging to the line and counted the number of points for each h_sample which is the height value that lanes can exist. Lastly, line order was designated.
This algorithm was implemented as described in Algorithm 1 only for the second road. When searching the lines based on height, the number of possible points in one height is 2, 3, or 4 (top red line of Fig. 10(b)). The two points mean line 2 and line 3 in order from the left (bottom red line of Fig. 10(b)). The three points mean line 2, line 3, and line 4 if a second point is on the right line based on the middle of the image (middle red line of Fig. 10(b)). Conversely, if the second point is on the left line, the three points mean line 1, line 2, and line 3. Lastly, the four points mean line 1, line 2, line 3, and line 4. Moreover, as shown in Fig. 11, this algorithm was successfully applied in the presence of diverse road conditions for the second lane (i.e., straight and curved lane).

C. COLLECTING GPS DATA
To compare the driving test results, we collected GPS data. The virtual vehicle was summoned to a map created    Fig. 12 shows how the proposed method is applied by using both test and simulation environments with respect to GPS measurement.

V. EXPERIMENTS AND RESULTS
To show the usefulness of DCLGAN in simulation driving tests, three methods were used to analyze data. First, FID thoroughly measured the reality of virtual-world images by calculating differences in the density of two distributions. FID showed DCLGAN made CARLA images to realistic. Second, lane segmentation performance, which was the important attribute for LKAS for autonomous vehicle, was measured and compared. These results show that ENet-SAD has a better understanding of images. Third, GPS data showed CwD images made driving results more realistic. We selected two road types: curved roads, and straight roads. Then, the curved road was analyzed by using Root Mean Square Error (RMSE) and the straight road was considered by the ability of lane restoring.

A. FID SCORE
Cw/oD2Real image gap was obtained by measuring the FID between 5498 CARLA images without DCLGAN and 6929 real-world images, and CwD2Real image gap was obtained by measuring the FID between 5298 images passed through DCLGAN and 6929 real-world images. Additionally, 4585 separate images captured independently were used for real to real comparison. This relationship is depicted in Fig. 13.
As a result of training by setting λ NCE,X to 3, the weight (epoch 26) obtained with FSIM value of 0.451 was used. An example of an image translated using this weight is shown in Fig. 14. The lanes of CARLA are inherently blurry and indistinguishable from the surrounding roads ( Fig. 14 (a)), whereas the lanes of the real-world domain are clear and distinctly visible in the distance (Fig. 14 (b), (c)). This DCLGAN weight translates the image by well reflecting the difference in lane characteristics between these domains without distorting the shape of the lane.     Table 6 is the FID measurement of the CARLA images before and after conversion using the corresponding weights.  Considering the lack of diversity in our dataset, it was also measured when the dimensionality of the feature was 64, 192, and 768. In all cases, the FID value of CwD was lower than that of Cw/oD. (e.g. 321.97 (Cw/oD) → 240.80 (CwD) at 2048 feature dimensionality) That is, as we saw in the previous section, DCLGAN changed the CARLA image characteristics such as road color and lane clarity to fit the real-world domain, and it was possible to quantitatively confirm the reduction of the gap with FID.

B. LANE SEGMENTATION ACCURACY
Lane segmentation accuracy in each domain was evaluated and followed by the TuSimple evaluation method. Prediction results were obtained using ENet-SAD used for LKAS mentioned in the previous section. As shown in Table 7, ENet-SAD shows better results in CwD(84.94%) than Cw/oD(82.85%). Fig. 15 describes specific flow.  Fig. 16 shows the failed points during Cw/oD. This result shows that DCLGAN made CARLA images look like ENet-SAD training image dataset. As the training dataset was collected from real-world, CwD become similar to the real image. This made ENet-SAD easier to understand the images.

C. GPS ORBIT 1) DIFFERENCE OF GPS ORBIT
After data collection, to utilize RMSE, we selected two road types (i.e., slightly curved road, and steep curved road) and two sections for each road type. Fig. 16(a)-(d) described each section respectively. Then, the RMSE between the ideal path connecting the central points of the road and each driving path was calculated. More specifically, there are two ways to calculate RMSE: distance east (x) and distance north (y). Table 8 shows the 10% trimmed mean of RMSE values of CwD and Cw/oD for each section. The experiment was performed with the desired speed set at 50km/h turning on LKAS until finishing one lap. Then we did this experiment 20 times.
As shown in Table 8, CwD tended to run closer to the center route than Cw/oD for the most part. According to section I and section IV, the steep curved roads, CwD was more stable. However, in section II and section III which are slightly curved roads, CwD and Cw/oD were almost the same. Moreover, CwD finished whole laps but Cw/oD failed seven times. There are many failures on the bottom right corner road in Fig. 16 (a) which was the most curved road in the case of Cw/oD. These results of RMSE and success rates mean that Cw/oD can cause bad driving results, especially on the steep curved road which is the most important part of the map.

2) LANE RESTORING
In this section, we tried to evaluate how well a vehicle can return when it abruptly maneuvers out of its lane. If CwD recognizes the lane better than Cw/oD, the ability to return to the existing lane within reasonable time should be better when it deviates from the lane. In common, if the vehicle starts from its own location and returns to the original lane within 2 minutes and continues to keep the lane stably, it was judged as a success.
The results are shown in Fig. 17 and Table 9. As shown in Table 9, Cw/oD failed when starting by moving 1.5 m, 1.4 m to the right, and 1.2 m, 1.3 m, 1.4 m, and 1.5 m to the left. However, CwD succeeded in all but the right 1.5m, left 1.3m, 1.4m, and 1.5m. As shown in Fig. 17, the path returning to the original lane was confirmed even under relatively extreme  conditions such as +1.4m and −1.3m unlike Cw/oD. Compared to Cw/oD, it can be seen that CwD has better restoring force to return to the center of the lane by better recognizing the lane even when it is above the lane away from the center of the lane.

VI. DISCUSSION AND CONCLUSION
Previous studies have shown ways to make virtual-world images close to the real-world images by improving GAN. However, these studies have not focused on the performance of GAN especially while driving test in simulation. In this paper, we evaluated the effect of translating simulation images into real image via DCLGAN on LKAS in CARLA simulation.
To measure the effect, we first found the well-trained DCLGAN weights while minimizing the distortion of the lanes using FSIM. Then FID quantitatively showed that virtual images can be more similar to reality through DCLGAN, especially in our KIAPI high-speed circulation environment. In addition, we made ENet-SAD better understand lane information of images by making the features of virtual images closer to the train set images of ENet-SAD by DCLGAN. In this process, we proposed an algorithm that automatically measures lane recognition accuracy in simulation. Finally, we compared the driving trajectory of critical parts through RMSE (x) and RMSE (y) and the ability of lane restoring.
These indicate that the driving result of CwD is more similar to reality. Thus, these results suggest that applying image translation on the simulation driving test can be essential and this visual gap can be reduced by using GAN.
Most notably, this is the first study to our knowledge to measure the performance of image translation on LKAS in autonomous simulation. Our results will make researchers in simulation field focus on image translation. Based on this study, our future work will concentrate on implementing scenarios for surrounding vehicles and testing autonomous driving algorithms such as LKAS and even AEB or SCC. In addition, a more accurate dynamic model will be applied through CARSIM with minimum gap of the simulation to real-world.