How the Representation Similarity Between Layers Affects the Random Vector Functional Link Neural Network?

Random vector functional link neural network (RVFLNN) is a fast learning model for single-hidden layer neural networks. The optimal output weight vector of RVFLNN is determined by the pseudo-inverse of the concatenation of the input matrix and enhancement matrix; however, how the representation similarity of the input layer and enhancement layer affects the random vector functional link neural network is still unknown in the literature. To this end, in this paper, we conduct a comprehensive investigation of the relationship between the input layer and enhancement layer. Moreover, by using the canonical correlation analysis (CCA) to define the statistical similarity of the input layer and the enhancement layer, we propose a pretraining method for RVFLNN. The main idea is to adjust the network weights connecting the input layer and enhancement layer to decrease the similarity between layers using the gradient method. Simulations are conducted on ten UCI classification datasets to verify the effectiveness of the proposed pretraining algorithms.