Reducing the Computational Complexity of Learning with Random Convolutional Features

In the last decade, there has been a surge of research interest in feature extraction using random sampling. These techniques are fast and scalable and, at the same time, have practical favorability in low-sample size and high-dimensional training data. Convolutional Kitchen Sinks-based methods are promising random feature extractors for time series data. Since these methods are data-independent, many of the extracted features are redundant. To address this problem, we propose a simple and efficient feature selection method based on knee/elbow detection in the curve of ordered coefficients in linear regression. Our empirical studies show that without significant loss in accuracy, the proposed feature selector, on average, prunes more than 84 percent of randomly generated features.