Convolutional, Extra-Trees and Multi layer Perceptron

In this paper, we propose a novel approach for building and initializing deep neural networks based on extremely randomized trees (extra-trees) an ensemble learning method for both classification and regression and feature extraction techniques. We use convolutional neural networks (CNNs), a family of modern deep learning models, extensively used in the area of computer vision and image classification, to improve the accuracy and generalization performance of classifiers. First, a CNN model is built to automatically extract multi-level features from the data. Second, a random forest obtains the structures of the trees. Finally, the neural networks (MLP) are built. This hybrid method combines two standard adaptive methods: decision trees and artificial neural networks. In this article, we illustrate the structure of the hybrid method, the problems occurring during the building of the model, and the solutions for these problems. The experimental results indicate that the proposed approach achieves consistently high performance for a variety of regression and classification tasks. These results should motivate further studies seeking to develop accurate and efficient tree-based models.