Enhancing Autonomous Driving: A Low-Cost Monocular End-to-End Framework with Multi-Task Integration and Temporal Fusion

End to end autonomous driving system has rapidly progressed and garnered significant attention. Recently, multimodal fusion methods have surpassed vision-based methods in terms of performance. However, the multi-modal methods are both costly and time-confusion. This paper aims to propose a low-cost monocular end-to-end multi-task autonomous driving framework to address these challenges. In particular, to improve the performance of the vision-based model, our model considers three key aspects. First, to enhance the level of detail and visual representation of the surrounding environment, depth embeddings and semantic embeddings are utilized to assist in the perception of the Bird's Eye View(BEV) space. In addition, the motion queries are separated from the BEV queries and the novel transformer-based motion decoder is proposed to generate the output waypoints and target speed for motion planning. Subsequently, to optimize the utilization of the historic features, we introduced a streaming-based processing temporal module to fuse current and historic features. The performance of our proposed model is evaluated using the longest-06 and Town05-long benchmarks within the CARLA-0.9.10.1 simulator. The experiment results demonstrate that our proposed model performs state-of-the-art results on the longest-06 benchmark and promising results on the Town05-long benchmark even compared to these multi-modal sensor based models.