Deep Ensemble Learning Model Based on Covariance Pooling of Multi-Layer CNN Features

Compared to standard deep convolutional neural networks (CNN) which include a global average pooling operator, second-order neural networks have a global covariance pooling operator which allows to capture richer statistics of CNN features. They have been shown to improve representation and generalization abilities. However, this covariance pooling is performed only on the deepest CNN feature maps. To benefit from different levels of abstraction, we propose to extend these models by using a multi-layer approach. In addition, to obtain better predictive performance, an end-to-end ensemble learning architecture is proposed. Experiments are conducted on four datasets and have confirmed the potential of the proposed model for various image processing applications such as remote sensing scene classification, indoor scene recognition and texture classification.