ODMTCNet: An Interpretable Multiview Deep Neural Network Architecture for Feature Representation

Recently, deep cascade architecture-based algorithms have attracted wide attention and have been applied to numerous application domains successfully. Nevertheless, the black-box structure of such algorithms has always been considered the Achilles’ heel by the machine learning community. Moreover, due to its data-driven nature, the deep cascade architecture likely causes over-fitting problems when there is no sufficient data available. In order to solve these pressing issues, this work proposes a novel multiview deep neural network (DNN) model, namely, optimal discriminant multiview tensor convolutional network (ODMTCNet), which integrates statistics-guided optimization (SGO) principles with the DNN architecture. Specifically, a discriminant multiview tensor convolution strategy is proposed and integrated with a deep cascade architecture. Different from the traditional DNN models, the parameters of the convolutional layers in ODMTCNet are determined by solving SGO problems. Based on the SGO principles, the relation between the optimal performance and parameters (e.g., the number of convolutional filters) can be analytically predicted, with each layer generating justified knowledge representations. In addition, information quality (IQ) is adopted to further improve multiview feature representation. Because of its unique design, ODMTCNet is able to handle different types of features (e.g., raw, hand-crafted, prior knowledge-based, and DNN-generated features), forming a general platform for multiview feature representation. To validate the genericness and effectiveness of the ODMTCNet model, we conducted experiments on five datasets of different scales: The Olivetti Research Lab (ORL) database, the Facial Recognition Technology (FERET) database, the ETH-80 database, the Caltech 256 database, and the nanyang technological university (NTU) red green blue-depth (RGB+D) 120 database. Experimental results show the superiority of the presented solution over the state-of-the-art. Implementation codes will be made available in the final version.