A Transformer-based Framework for Visual Grounding on 3D Point Clouds

The task of 3D visual grounding involves the challenge of accurately locating an object within a 3D scene based on an accompanying textual description. This task marks an emerging frontier at the intersection of 3D vision and language, with diverse real-world applications. Notably, compared to visual grounding on images, visual grounding on point clouds is a more challenging task. In recent 3D visual grounding approaches, a common strategy involves adopting a two-stage grounding-by-detection paradigm, with an emphasis on using robust detectors and employing language encoding modules. In this work, rather than relying on different detectors, we focus on generating enriched object proposals that can be more effectively fused with distinct features extracted from textual descriptions. To achieve this, we present an attentive fusion module (AFM) for fusing two distinct feature spaces. we employ transformers to model complex relationships between objects in a scene and fuse these representations with text features. The conducted experiments demonstrate that the proposed framework outperforms the previous works on ScanRefer dataset in both Acc@0.25 and Acc@0.5 metrics.