Cutransnet: Transformers to Make Strong Encoders for Multi-Task Vision Perception of Autonomous Driving

In autonomous driving, perception plays a critical role as it serves as a fundamental requirement for both planning and control. Currently, most perception tasks are processed independently, which requires designing multiple models and networks to handle multiple tasks. This division leads to multiple sub-tasks in real-world environments, making it difficult to ensure real-time performance and communication between tasks. The paper introduces an innovative neural network architecture termed CUTransNet, which presents a unified approach capable of simultaneously detecting drivable areas, lane markings, and traffic objects, achieving multi-task processing with a single model. To build a more robust feature encoder, we proposed the CUT module that combines the global context ability of Transformers. The module is integrated into the convolutional neural network’s backbone to compensate for the low-level visual clues lost by Transformers and achieve higher detection accuracy. Experimental results on BDD100K demonstrate that CUT model surpasses traditional multi-task networks in both task accuracy and computational efficiency while maintaining high real-time performance.