Self-Training Enhanced Multitask Network for 3-D Point-Level Hybrid Scene Understanding for Autonomous Vehicles

Recently, AI has gradually been integrated into various transportation tools. Additionally, precise environmental perception is key to ensuring the stable operation of autonomous vehicles. Although current research has started to focus on the processing of various environmental perception tasks with point-cloud data, existing studies can only achieve a “box-level” hybrid scene understanding, with limitations in recognizing more fine-grained “point-level” variations. Therefore, we propose a multitask network with hierarchical channel-attention gating” that can flexibly and simultaneously handle 3-D semantic segmentation and scene flow estimation within a single architecture to provide point-level hybrid scene understanding. In addition, we also identify the issue of underutilization of training data in existing studies. Due to the difficulty of point-cloud data labeling, some point-cloud data are left unlabeled to reduce the cost of labeling, but this also results in the underutilization of the unlabeled data. To efficiently incorporate unlabeled data into the learning process, we propose a strategy involving point-level self-training for a multitask network. This technique employs flow information to generate reliable pseudo-labels for unlabeled data, before integrating them into the training process to enhance model performance. The experimental outcomes on the Waymo open data set demonstrate the superiority of the proposed network.