Sparse Broad Learning System via a Novel Competitive Swarm Optimizer

Broad Learning System is recently proposed intended as an alternative to deep learning neural networks. It has attracted a lot of attention due to its excellent performance on classification and regression problems. However, the typical BLS is based on grid search to determine the hidden layer nodes, which undoubtedly imposes a heavy training burden. On the other hand, BLS chooses to use sparse autoencoder to fine-tune the weights of input data to feature mapping nodes as a way to reduce the uncertainty caused by random mapping and extract sparse features. In contrast, this paper proposes a BLS that determines the hidden layer nodes and sparse hidden layer weights by solving a multi-objective optimization problem. Firstly, we propose a novel competitive swarm optimizer(NCSO) with embedded sparse operators, called S-NCSO. Second, a bi-objective optimization problem with an error cost and the l
o
norm of the hidden layer weights is solved by using S-NCSO to determine both the hidden layer nodes and the sparse hidden layer weights. Finally, experiments on multiple regression datasets show that the proposed method not only yields sparser weights and compact structures, but also greatly reduces the training time while improving the prediction accuracy.