WeLDCFNet: Convolutional Neural Network based on Wedgelet Filters and Learnt Deep Correlation Features for depth maps features extraction

With the emergence of depth sensors, extraction of depth maps features is becoming more and more solicited and prominent for several computer vision applications, such as gesture recognition, face recognition and segmentation. These applications can be more accurate thanks to the depth information that provides more precise separate foreground objects from background. In this paper, we propose an automatic depth maps features extraction model based on an optimized Convolutional Neural Network (CNN), trained on a mixture of depth maps and grayscale texture images. The CNN includes a first convolutional layer of pre-defined wedgelet filters, followed by a pre-trained VGG-19 neural network. Then, we opt for an image style classification based on Learnt Deep Correlation Features to capture features distinguishing depth maps from grayscale texture images of the training set. Experimental results demonstrate the potential effectiveness of the proposed wedgelet (We) and Learnt Deep Correlation Features (LDCF) based Network (WeLDCFNet) with a mean accuracy gain up to 32.77%, when compared to existing features extraction approaches. As our aim in this paper is depth maps features extraction and not the texture/depth classification itself, we propose, as a use case, to leverage the proposed WeLDCFNet for depth maps learned compression. If our model succeeds to extract depth features that make them distinguishable from texture, it would be useful to save the main compact depth information. Our WeLDCFNet based autoencoder, tailored to compression needs, shows competitive Rate/Distortion tradeoffs when compared to the latest depth maps compression standard.