Augmented Reality-Based Navigation Using Deep Learning-Based Pedestrian and Personal Mobility User Recognition—A Comparative Evaluation for Driving Assistance

Recently, research on augmented reality-based head-up displays (AR-HUDs) for driving assistance has been widely conducted in the automotive industry. The disadvantage of having to look away from the road while driving can be compensated by using AR-HUD-based visualization instead of an auxiliary display on the central dashboard. As the number of personal mobility users on the road increases, and their moving speed is considerably faster than pedestrians, personal mobility makes it more difficult for the driver to cope with dangerous situations. However, there is little research work for considering personal mobility users for driving assistance. This study aims to enhance the driver’s situational awareness to respond to unexpected situation by providing driver assistance information on the AR-HUD by combining deep learning and AR. In particular, the deep learning-based anomaly detection method can recognize personal mobility users effectively. This study also investigates the driver’s understanding of the relationship between the amount of prioritized information provided to AR-HUD and situational cognitive ability. This understanding can be used to adjust the amount of information displayed on the AR-HUD to maintain drivers’ situational awareness. The proposed approach was evaluated through an online study. The results showed that the proposed deep learning-based AR-HUD system improved the driver’s situational awareness and showed advantages in driving assistance compared to the typical system.