MV-C3D: A Spatial Correlated Multi-View 3D Convolutional Neural Networks

As the development of deep neural networks, 3D object recognition is becoming increasingly popular in the computer vision community. Many multi-view-based methods are proposed to improve the category recognition accuracy. These approaches mainly rely on multi-view images that are rendered with the whole circumference. In real-world applications, however, 3D objects are mostly observed from partial viewpoints in a less range. Therefore, we propose a multi-view-based 3D convolutional neural network that takes only part of contiguous multi-view images as input and can still maintain high accuracy. Moreover, our model takes these view images as a joint variable to better learn the spatially correlated features using 3D convolution and 3D max-pooling layers. The experimental results on ModelNet10 and ModelNet40 datasets show that our MV-C3D technique can achieve outstanding performance with multi-view images that are captured from partial angles with less range. The results on 3D-rotated real image dataset MIRO further demonstrate that MV-C3D is more adaptable in real-world scenarios. The classification accuracy can be further improved with the increasing number of view images.