Hybrid-Fusion Mamba for Multitask Point Cloud Learning With Visual Perception Sensors

Mobile robots and autonomous vehicles rely on 3-D point cloud technology for environmental perception, which often employ various visual perception sensors within their Internet of Things (IoT) systems to acquire point cloud information. Nevertheless, the obtained point cloud data is frequently incomplete, making it difficult to effectively perform perception tasks. Multitask point cloud learning networks can not only achieve point cloud reconstruction under high occlusion rates but also assist the system in accomplishing various point cloud tasks. However, existing multitask point cloud learning models based on advanced Transformer frameworks often suffer from quadratic complexity, limiting their performance in massive point cloud data learning. In this article, we propose a novel approach named Hybrid-Fusion Mamba (HFMamba), as a pretraining network model specifically designed for multitask point cloud learning. Compared to Transformer-based networks, the proposed HFMamba possesses linear complexity, which can significantly reduce IoT systems’ computational cost. HFMamba employs a unique first-layer feature fusion design that integrates point cloud features from three different perspectives, enabling it to capture deeper dependencies among the point clouds. Moreover, a hybrid scan strategy is proposed to separately scan the hidden states and residuals, aiming to model the sequence from different directions. Experimental results demonstrate that the proposed HFMamba network model outperforms many state-of-the-art methods without applying any serialization strategies to the original point cloud data. In particular, the proposed HFMambda approach achieves classification accuracies of 93.7% and 93.89% on ModelNet40 and ScanObjectNN, respectively.