Performance Analysis and Enhancement of Multi-Task Learning for Autonomous Driving

Various sensor-based visual recognition tasks are essential for autonomous driving to perceive the surrounding environment. Using cameras, tasks such as lane detection, object detection, and image segmentation are performed. In cases without LiDAR, tasks like 3D distance estimation, 3D object detection, 3D restoration, and SLAM are conducted. However, due to hardware constraints and the demand for rapid and safe driving, processing all these computations is challenging. In this study, a solution for achieving safe autonomous driving under given hardware conditions was investigated by considering the accuracy and processing speed of visual recognition tasks. Particularly, the use of Multi-Task Learning (MTL) to concurrently handle multiple tasks was analyzed, and the Multi-task Decision and Enhancement (MDE) algorithm was proposed to enhance task performance. Through the MDE algorithm, stability could be maintained while minimizing overall latency and adjusting weights for multiple task sets and shared backbone models. Experimental results showed that this strategy yielded a performance improvement of up to approximately 15-54% in processing time while maintaining the same level of accuracy.