CMI-Net: Cross-view Message Token Interaction Network for 3D Shape Recognition

Understanding 3D shapes is crucial across various fields in computer vision. View-based methods have shown remarkable performance in 3D shape recognition by leveraging pre-trained networks on extensive 2D image datasets while preserving stable multi-view structures. However, traditional approaches often extract view features independently before fusion, limiting each view's perception to its narrow domain and impeding the network's ability to capture global structures and inter-view dependencies. To address these issues, we propose a novel Cross-view Message Token Interaction Network (CMI-Net) that enhances contextual information exchange between views, effectively extending the receptive field throughout the entire process. Specifically, we introduce a Message (MSG) token for each view to aggregate intra-view contextual information. We establish two interaction mechanisms: the Anonymous Delivery Mechanism (ADM) and the Integrated Broadcast Mechanism (IBM), each optimized for different interaction efficiencies. The ADM promotes view-level interaction by randomly transferring MSG tokens to selected views, while the IBM aggregates and broadcasts tokens across all views, enabling shape-level interaction. This design ensures that each view benefits from a comprehensive global receptive field. Moreover, we have developed a Contextual Information-Guided Part Selection module to direct the network's attention to significant local features within each view. Finally, through multi-granularity feature fusion, CMI-Net improves the understanding of 3D shapes at the local patch, intra-view, and cross-view levels. Extensive experiments conducted on diverse datasets, including ModelNet40, FG3D, ScanObjectNN, and ShapeNet Core55, demonstrate that CMI-Net achieves state-of-the-art results in 3D shape classification and retrieval tasks. Additionally, visualizing feature variations during cross-view interactions enhances interpretability, providing deeper insights into the network's decision-making process.