Learning Transferable and Discriminative Representations for 2D Image-Based 3D Model Retrieval

Existing research on the 2D image-based 3D model retrieval task focuses on learning transferable representations directly to narrow the domain discrepancy. However, it is not easy to achieve in practice due to the significant variations across two domains. In addition, some methods design a domain discriminator to distinguish the feature arising from source or target domains for transferable feature representations learning, which will lead to an unexpected deterioration of the feature discriminability. To settle these problems, we propose jointly learning transferable and discriminative representations for 2D image-based 3D model retrieval. Specifically, we extract features from the 2D images and 3D models (described as multiple views) by CNN. Considering the difficulty of directly narrowing the discrepancy of two domains, we are prone to connect 2D image and 3D model domains to an intermediate domain, where the domain gap aims to be eliminated. However, the feature transferability does not denote well discriminability. Based on the batch spectral penalization (BSP) theory, the feature transferability is dominated by feature vectors with higher singular values, while the feature discriminability depends on more eigenvectors with lower singular values to convey rich discriminative structures. Therefore, we penalize the largest singular values so that the feature vectors with lower singular values are appropriately enhanced, thereby strengthening feature discriminability. A series of experiments on two challenging datasets, MI3DOR and MI3DOR-2, indicate that our method can significantly improve performance.